Apache Spark is a framework for distributed computing. It provides one of the best mechanisms for distributing data across multiple machines in a cluster and performing computations on it. Spark achieves this by constructing data structures called RDDs (Resilient Distributed Datasets). RDDs allow data to be broken into disparate chunks and processed independently of one another. The individual chunks can
Read more...
When one looks at the amazing roster of talks for the Spark + AI Summit, what you don’t see is a lot of discussion on how to leverage object storage. On some level you would expect to —
Read more...
One of the key challenges in any digitization journey is the adoption of machine learning techniques. Given the explosion of tools and frameworks, it can be difficult
Read more...
When early object storage APIs were developed they focused on the efficient storage and retrieval of objects. Amazon’s success with S3 and its implementation of the robust S3 API quickly became the de facto standard for object storage in the cloud. MinIO, recognizing this, invested heavily in creating the most compliant implementation of the S3 API outside of Amazon.
Read more...Some our community members have repeatedly asked for Backblaze B2 Cloud Storage and MinIO integration [1] [https://github.com/minio/minio/issues/4072] . B2 is competitively priced and has a huge fan following. We also heard from Backblaze team that they are actively expanding their B2 cloud storage service. We added experimental support for Backblaze B2 backend in MinIO to
Read more...In this post, I explain how to use NGINX and NGINX Plus as a reverse proxy and load balancer for MinIO servers.
Read more...Applications today generate [https://blog.minio.io/object-storage-what-is-it-all-about-62920ca164ca#.qfa0ylbd1] more data than ever, and this upward trend is expected to keep up [https://www.emc.com/leadership/digital-universe/2014iview/executive-summary.htm] in foreseeable future. How do you handle this ever growing storage requirement of your application? A storage solution that can run where your application runs and can scale with
Read more...s3verify performs a series of API calls against an object storage server and checks the responses for AWS S3 signature version 4 compatibility. s3verify is licensed under Apache V2.0 license. Why we built s3verify Amazon Web Service’s (AWS) Simple Storage Service (S3) has emerged as the goto cloud storage, so any new competing service has a strong incentive
Read more...MinIO Client “mc” is a tool for Amazon S3 compatible cloud storage and POSIX compatible filesystems. It implements basic Unix commands such as ls, cp, cat and diff. mc is entirely written in Golang and released under Apache License, Version 2.0. Project is hosted at Github —https://github.com/minio/mc NAME: mc - MinIO Client for cloud storage
Read more...
Replication
Versioning
Observability
Encryption
Key Management Server
S3 Compatibility
Object Immutability
Catalog
Object Prompt
Identity + Access Mgt
Firewall
AIHub
Lifecycle
Cache
Events and Lambdas
Console
Erasure Code Calculator
Reference Hardware
Integrations with MinIO
Download MinIO Object Store
Events
