A collection of 32 posts tagged with "S3"
What Amazon Web Services has done is truly remarkable. Over the better part of fifteen years they have built a business with an annual run rate in excess of $45B. Underpinning that remarkable growth is storage – specifically S3, launched back in 2006. S3 revolutionized the storage industry and the S3 API is now the standard in the cloud, replacing the
Read more
As data has grown, so has the challenge associated with moving it. Indeed, the bandwidth costs to migrate a PB of data out of AWS would be more than keeping it there for years. Still, customers often need to move large amounts (100s of TBs up to PBs) with some frequency. Amazon knows this and has, in their intensely customer
Read more
The power of scale is well-documented in the world of business. Cloud providers - Amazon in particular - have amassed extraordinary scale in a very short period of time. The cloud providers are now using this scale to rearchitect how enterprises interact with their data. They are remaking the enterprise data landscape with two primary levers: price and performance. Let’
Read more
Apache Spark is a framework for distributed computing. It provides one of the best mechanisms for distributing data across multiple machines in a cluster and performing computations on it. Spark achieves this by constructing data structures called RDDs (Resilient Distributed Datasets). RDDs allow data to be broken into disparate chunks and processed independently of one another. The individual chunks can
Read more
When one looks at the amazing roster of talks for the Spark + AI Summit, what you don’t see is a lot of discussion on how to leverage object storage. On some level you would expect to —
Read more
One of the key challenges in any digitization journey is the adoption of machine learning techniques. Given the explosion of tools and frameworks, it can be difficult
Read more
When early object storage APIs were developed they focused on the efficient storage and retrieval of objects. Amazon’s success with S3 and its implementation of the robust S3 API quickly became the de facto standard for object storage in the cloud. MinIO, recognizing this, invested heavily in creating the most compliant implementation of the S3 API outside of Amazon.
Read moreSome our community members have repeatedly asked for Backblaze B2 Cloud Storage and MinIO integration [1] [https://github.com/minio/minio/issues/4072] . B2 is competitively priced and has a huge fan following. We also heard from Backblaze team that they are actively expanding their B2 cloud storage service. We added experimental support for Backblaze B2 backend in MinIO to
Read moreIn this post, I explain how to use NGINX and NGINX Plus as a reverse proxy and load balancer for MinIO servers.
Read moreApplications today generate [https://blog.minio.io/object-storage-what-is-it-all-about-62920ca164ca#.qfa0ylbd1] more data than ever, and this upward trend is expected to keep up [https://www.emc.com/leadership/digital-universe/2014iview/executive-summary.htm] in foreseeable future. How do you handle this ever growing storage requirement of your application? A storage solution that can run where your application runs and can scale with
Read mores3verify performs a series of API calls against an object storage server and checks the responses for AWS S3 signature version 4 compatibility. s3verify is licensed under Apache V2.0 license. Why we built s3verify Amazon Web Service’s (AWS) Simple Storage Service (S3) has emerged as the goto cloud storage, so any new competing service has a strong incentive
Read moreMinIO Client “mc” is a tool for Amazon S3 compatible cloud storage and POSIX compatible filesystems. It implements basic Unix commands such as ls, cp, cat and diff. mc is entirely written in Golang and released under Apache License, Version 2.0. Project is hosted at Github — https://github.com/minio/mc NAME: mc - MinIO Client for cloud storage
Read more
