All posts

simdjson-go: Parsing gigabytes of JSON per second in Go

simdjson-go: Parsing gigabytes of JSON per second in Go

IntroductionJSON has established itself as the "lingua franca" of the web. As such the parsing performance of JSON is hugely important for many applications. Despite the simple and human-friendly nature of JSON, it is not a technically trivial format to parse at high speeds. Recently some new designs have been presented one of which is simdjson by Daniel Lemire and

Read more...

Scaling MinIO: Benchmarking Performance From Terabytes to Petabytes

Scaling MinIO: Benchmarking Performance From Terabytes to Petabytes

MinIO provides the best-in-class performance as we have repeatedly shown in our previous benchmarks. In those benchmarks, we chose the highest-end hardware and measured if MinIO could squeeze out every bit of the resources afforded it. This proved two key points: Ensuring that MinIO utilizes the maximum possible CPU, Network, and Storage available.Ensuring that MinIO is NOT the IO-bottleneck.

Read more...

Hadoop HDFS's Logical Successor

Hadoop HDFS's Logical Successor

The demise of Hadoop is probably overblown. It will not suddenly disappear from the enterprise landscape - there are simply too many clients, too much sunk investment for it to vanish into the night. What is not overblown is the fact that Hadoop, like countless technologies before it, is in secular, irreversible decline. There are a number of reasons but

Read more...

Benchmarking MinIO vs. AWS S3 for Apache Spark

Benchmarking MinIO vs. AWS S3 for Apache Spark

Apache Spark is a framework for distributed computing. It provides one of the best mechanisms for distributing data across multiple machines in a cluster and performing computations on it. Spark achieves this by constructing data structures called RDDs (Resilient Distributed Datasets). RDDs allow data to be broken into disparate chunks and processed independently of one another. The individual chunks can

Read more...

S3 Benchmark: MinIO on NVMe

S3 Benchmark: MinIO on NVMe

Well written software is fast software. When MinIO was conceived it was designed from scratch to be simple, to scale (because simple things scale better) and to be fast. Simplicity and scale have their own subjective and objective measures - but fast is generally a numbers game. When you take well-written, fast software and pair it with fast hardware the

Read more...

S3 Benchmark: MinIO on HDDs

S3 Benchmark: MinIO on HDDs

High performance object storage is one of the hotter topics in the enterprise today. On the one hand, object storage has become an indispensable part of the enterprise storage strategy (public or private cloud) - carrying the vast, vast majority of the enterprise burden when measured in TBs or PBs. On the other hand, object storage has traditionally served a

Read more...