Hybrid Cloud Object Storage
Baremetal
Reference Hardware
MinIO Optimizes Small Object Storage and Adds .tar Auto-Extraction
Cloud-native workloads push storage to the limit with many small files. MinIO combines metadata and data to optimize small file operations.
Read more...
All posts
The Trouble With Cassandra: Why It's a Poor Choice For a Metadata Database for Object Stores
Cassandra is a popular, tried-and-true NoSQL database that supports key-value wide-column tables. Like any powerful tool, Cassandra has its ideal use cases - in particular, Cassandra excels at supporting write-heavy workloads, while having limitations when supporting read-heavy workloads. Cassandra's eventual consistency model and lack of transactions, multi-table support like joins, subqueries can also limit its usefulness. However, using Cassandra as
Read more...Why Small Objects Are Such a Big Deal
Over the last decade or so, object storage use cases have evolved considerably as they replace traditional file and block use cases. Specifically the need to work with small data objects is becoming commonplace. Yes, there’s still plenty of large objects but small objects are becoming more prevalent than large for specific workloads and application environments. Traditional object storage
Read more...Impact of Intel vs. ARM CPU Performance for Object Storage
The recent announcement from AWS about the general availability of their new ARM-powered Graviton2 servers caused us to take another look at the performance of these ARM servers. In this blog post we describe the results which you may find surprising. Introduction MinIO is an Apache licensed, open source S3-compatible object storage server with a particular focus on high performance.
Read more...Accelerating aggregate MD5 hashing up to 800% with AVX512
IntroductionWhile MD5 hashing is no longer a good choice when considering a hash function, it is still being used in a great variety of applications. As such any performance improvements that can be made to the MD5 hashing speed are worth considering. Due to recent improvements in SIMD processing (AVX2 and especially AVX512) we are providing a Go md5-simd package
Read more...Breaking the HDFS Speed Barrier - a First for Object Storage
Few would argue with the statement that Hadoop HDFS is in decline. In fact, the HDFS part of the Hadoop ecosystem is in more than just decline - it is in freefall. At the time of its inception, it had a meaningful role to play as a high-throughput, fault-tolerant distributed file system. The secret sauce was data locality. By co-locating
Read more...Running Peta-Scale Spark Jobs on Object Storage Using S3 Select
When one looks at the amazing roster of talks for the Spark + AI Summit, what you don’t see is a lot of discussion on how to leverage object storage. On some level you would expect to —
Read more...Building an on-premise ML ecosystem with MinIO Powered by Presto, R and S3 Select Feature
One of the key challenges in any digitization journey is the adoption of machine learning techniques. Given the explosion of tools and frameworks, it can be difficult
Read more...Stream processing with Apache Flink and MinIO
How to use Apache Flink to build a private cloud data pipeline for a variety of use cases.
Read more...Modern Data Lake with MinIO : Part 2
In this post we’ll learn more about object storage, specifically Minio and then see how to connect Minio with tools like Apache Spark and Presto for analytics workloads.
Read more...