All posts

The Cloud Has Picked a Winner; Now They Are Enforcing It

The Cloud Has Picked a Winner; Now They Are Enforcing It

The power of scale is well-documented in the world of business. Cloud providers - Amazon in particular - have amassed extraordinary scale in a very short period of time. The cloud providers are now using this scale to rearchitect how enterprises interact with their data. They are remaking the enterprise data landscape with two primary levers: price and performance. Let’

Read more...

Benchmarking MinIO vs. AWS S3 for Apache Spark

Benchmarking MinIO vs. AWS S3 for Apache Spark

Apache Spark is a framework for distributed computing. It provides one of the best mechanisms for distributing data across multiple machines in a cluster and performing computations on it. Spark achieves this by constructing data structures called RDDs (Resilient Distributed Datasets). RDDs allow data to be broken into disparate chunks and processed independently of one another. The individual chunks can

Read more...

Introducing Spark-Select for MinIO Data Lakes

Introducing Spark-Select for MinIO Data Lakes

When early object storage APIs were developed they focused on the efficient storage and retrieval of objects. Amazon’s success with S3 and its implementation of the robust S3 API quickly became the de facto standard for object storage in the cloud.MinIO, recognizing this, invested heavily in creating the most compliant implementation of the S3 API outside of Amazon.

Read more...

s3verify — A simple tool to verify AWS S3 API compatibility

s3verify performs a series of API calls against an object storage server and checks the responses for AWS S3 signature version 4 compatibility. s3verify is licensed under Apache V2.0 license.Why we built s3verifyAmazon Web Service’s (AWS) Simple Storage Service (S3) has emerged as the goto cloud storage, so any new competing service has a strong incentive to

Read more...

Go based Amazon S3 CLI

MinIO Client “mc” is a tool for Amazon S3 compatible cloud storage and POSIX compatible filesystems. It implements basic Unix commands such as ls, cp, cat and diff. mc is entirely written in Golang and released under Apache License, Version 2.0.Project is hosted at Github — https://github.com/minio/mcNAME:  mc - MinIO Client for cloud storage and

Read more...