Performance - MinIO Blog

The Trouble With Cassandra: Why It's a Poor Choice For a Metadata Database for Object Stores

Jonathan Symonds Jonathan Symonds on Performance 17 January 2021

Cassandra is a popular, tried-and-true NoSQL database that supports key-value wide-column tables. Like any powerful tool, Cassandra has its ideal use cases - in particular, Cassandra excels at supporting write-heavy workloads, while having limitations when supporting read-heavy workloads. Cassandra's eventual consistency model and lack of transactions, multi-table support like joins, subqueries can also limit its usefulness. However, using Cassandra as

Why Small Objects Are Such a Big Deal

Jonathan Symonds Jonathan Symonds on Performance 22 December 2020

Over the last decade or so, object storage use cases have evolved considerably as they replace traditional file and block use cases. Specifically the need to work with small data objects is becoming commonplace. Yes, there’s still plenty of large objects but small objects are becoming more prevalent than large for specific workloads and application environments. Traditional object storage

Impact of Intel vs. ARM CPU Performance for Object Storage

Frank Wessels Frank Wessels on Intel 3 June 2020

The recent announcement from AWS about the general availability of their new ARM-powered Graviton2 servers caused us to take another look at the performance of these ARM servers. In this blog post we describe the results which you may find surprising. Introduction MinIO is an Apache licensed, open source S3-compatible object storage server with a particular focus on high performance.

Accelerating aggregate MD5 hashing up to 800% with AVX512

Frank Wessels Frank Wessels on High Performance 29 April 2020

IntroductionWhile MD5 hashing is no longer a good choice when considering a hash function, it is still being used in a great variety of applications. As such any performance improvements that can be made to the MD5 hashing speed are worth considering. Due to recent improvements in SIMD processing (AVX2 and especially AVX512) we are providing a Go md5-simd package

Breaking the HDFS Speed Barrier - a First for Object Storage

Sidhartha Mani Sidhartha Mani on Benchmarks 6 August 2019

Few would argue with the statement that Hadoop HDFS is in decline. In fact, the HDFS part of the Hadoop ecosystem is in more than just decline - it is in freefall. At the time of its inception, it had a meaningful role to play as a high-throughput, fault-tolerant distributed file system. The secret sauce was data locality. By co-locating

Running Peta-Scale Spark Jobs on Object Storage Using S3 Select

Frank Wessels Frank Wessels on Performance 13 May 2019

When one looks at the amazing roster of talks for the Spark + AI Summit, what you don’t see is a lot of discussion on how to leverage object storage. On some level you would expect to —

Building an on-premise ML ecosystem with MinIO Powered by Presto, R and S3 Select Feature

Ravishankar Nair Ravishankar Nair @passionbytes

on S3 7 May 2019

Building an on-premise ML ecosystem with MinIO Powered by Presto, R and S3 Select Feature

One of the key challenges in any digitization journey is the adoption of machine learning techniques. Given the explosion of tools and frameworks, it can be difficult

Stream processing with Apache Flink and MinIO

Nitish Tiwari Nitish Tiwari on Integrations 26 November 2018

How to use Apache Flink to build a private cloud data pipeline for a variety of use cases.

Modern Data Lake with MinIO : Part 2

Nitish Tiwari Nitish Tiwari on Performance 11 October 2018

In this post we’ll learn more about object storage, specifically Minio and then see how to connect Minio with tools like Apache Spark and Presto for analytics workloads.

Modern Data Lake with MinIO : Part 1

Nitish Tiwari Nitish Tiwari on Performance 9 October 2018

In the first part of this two post series, we’ll take a look at how object storage is different from other storage approaches and why it makes sense to leverage object storage like Minio for data lakes.