Bringing ARM into the AI Data Infrastructure Fold at MinIO Using SVE

Bringing ARM into the AI Data Infrastructure Fold at MinIO Using SVE

One of the reasons that MinIO is so performant is that we do the granular work that others will not or cannot. From SIMD acceleration to the AVX-512 optimizations we have done the hard stuff. Recent developments for the ARM CPU architecture, in particular Scalable Vector Extensions (SVE), presented us with the opportunity to deliver significant performance and efficiency gains

Read more...

Scaling up MinIO Internal Connectivity

Scaling up MinIO Internal Connectivity

A MinIO cluster operates as a uniform cluster. This means that any request must be seamlessly handled by any server. As a consequence, servers need to coordinate between themselves. This has so far been handled with traditional HTTP RPC requests - and this has served us well.  Whenever server A would like to call server B an HTTP request would

Read more...

Confluent Platform with MinIO Tiered Object Storage Throughput Benchmark

Confluent Platform with MinIO Tiered Object Storage Throughput Benchmark

Confluent, Intel and MinIO conducted benchmarking and certification testing for MinIO Tiered Object Storage for Kafka storage. This blog post describes the observations and results of testing MinIO object storage as a backend for the tiered storage feature of Confluent Platform 7.1.0 on servers equipped with third generation Intel Xeon Scalable processors. The scope of these tests was

Read more...

Western Digital OpenFlex Data24 Performance Testing

Western Digital OpenFlex Data24 Performance Testing

We recently benchmarked MinIO S3 performance on the Western Digital OpenFlex Data24 using the WARP benchmark tool. Western Digital’s OpenFlex Data24 [https://www.westerndigital.com/products/data-center-platforms/openflex-data24-nvme-of-platform] NVMe-oF Storage Platform extends the high performance of NVMe flash to shared storage. It provides low-latency sharing of NVMe SSDs over a high-performance Ethernet fabric to deliver similar performance to locally

Read more...

Impact of Intel vs. ARM CPU Performance for Object Storage

Impact of Intel vs. ARM CPU Performance for Object Storage

The recent announcement from AWS about the general availability of their new ARM-powered Graviton2 servers caused us to take another look at the performance of these ARM servers. In this blog post we describe the results which you may find surprising. Introduction MinIO [https://github.com/minio/minio] is an Apache licensed, open source S3-compatible object storage server with a

Read more...

Scaling MinIO: Benchmarking Performance From Terabytes to Petabytes

Scaling MinIO: Benchmarking Performance From Terabytes to Petabytes

MinIO provides the best-in-class performance as we have repeatedly shown in our previous benchmarks [https://min.io/resources/#benchmarks]. In those benchmarks, we chose the highest-end hardware and measured if MinIO could squeeze out every bit of the resources afforded it. This proved two key points: 1. Ensuring that MinIO utilizes the maximum possible CPU, Network, and Storage available. 2.

Read more...

Performance at Scale: MinIO Pushes Past 1.4 terabits per second with 256 NVMe Drives

The fact that MinIO is fast is not a secret. We routinely publish our benchmarks and have put out comparision work against HDFS [https://blog.min.io/hdfsbenchmark/] and AWS (Spark [https://blog.min.io/benchmarking-apache-spark-vs-aws-s3/] + Presto [https://blog.min.io/running-presto-on-minio-benchmarking-vs-aws-s3/]) in addition to our HDD [https://blog.min.io/s3-benchmark-using-hdd/] and NVMe [https://blog.min.io/s3-benchmark-minio-on-nvme/] numbers.

Read more...

Hadoop HDFS's Logical Successor

Hadoop HDFS's Logical Successor

The demise of Hadoop is probably overblown. It will not suddenly disappear from the enterprise landscape - there are simply too many clients, too much sunk investment for it to vanish into the night. What is not overblown is the fact that Hadoop, like countless technologies before it, is in secular, irreversible decline. There are a number of reasons but

Read more...