MinIO Blog (Page 3)

High Performance Object Storage, Kubernetes + Why You Can't Containerize a Storage Appliance

Jonathan Symonds Jonathan Symonds on Kubernetes 13 February 2020

There are two forces that are fundamentally remaking the technology landscape today. One is Kubernetes and the other is high performance Object Storage. They are powering (or are shaped by, depending on your perspective) modern, data-rich applications that include AI/ML and application logs. Either way, modern applications need Kubernetes and Object Storage and Kubernetes and Object Storage owe their

simdjson-go: Parsing gigabytes of JSON per second in Go

Frank Wessels Frank Wessels on High Performance 11 February 2020

IntroductionJSON has established itself as the "lingua franca" of the web. As such the parsing performance of JSON is hugely important for many applications. Despite the simple and human-friendly nature of JSON, it is not a technically trivial format to parse at high speeds. Recently some new designs have been presented one of which is simdjson by Daniel Lemire and

Why the Healthcare Industry is Adopting High Performance Object Storage

Jonathan Symonds Jonathan Symonds on 10 February 2020

The healthcare industry is unique in so many ways. To begin with, it lacks a common objective function on which to optimize. This is because different healthcare organizations, both payers and providers, value different things. Profitability is not the universal goal - there are teaching hospitals, faith-based hospitals and not-for-profit insurance plans. Consider also that a large portion of the

Kubernetes Storage Patterns

Nitish Tiwari Nitish Tiwari on Kubernetes 4 February 2020

Kubernetes has fundamentally altered the traditional application development and deployment patterns. Application development teams can now develop, test and deploy their apps in days, across different environments, all within their Kubernetes clusters. Previous generations of technology typically took weeks if not months. This acceleration is possible due to the abstraction that Kubernetes brings to the table, i.e. it deals

Developer Friendly Kubernetes Storage

Daniel Valdivia Daniel Valdivia @dvaldivia

on Kubernetes 22 January 2020

Most developers don’t love managing infrastructure — including storage infrastructure. One of the great things about Kubernetes is the ability to simplify infrastructure management so that developers can focus on creating applications and let Kubernetes handle orchestration. Kubernetes does not natively manage storage infrastructure — but using MinIO and Kubernetes together can help provide the storage that applications need with the

Using mc to Migrate Data to/from AWS Snowball

Harshavardhana Harshavardhana on S3 15 January 2020

As data has grown, so has the challenge associated with moving it. Indeed, the bandwidth costs to migrate a PB of data out of AWS would be more than keeping it there for years. Still, customers often need to move large amounts (100s of TBs up to PBs) with some frequency. Amazon knows this and has, in their intensely customer

The Wisdom of Crowds: Distilling the Storage Newsletter 2020 Prediction Panel

Jonathan Symonds Jonathan Symonds on 14 January 2020

The hyper-connected Philipe Nicolas is the editor of the Storage Newsletter. Each year he asks the leaders in the industry to weigh in with their predictions on what will transpire in the coming year. The list of companies is long and the respondents are distinguished. So long in fact that, Phillipe has to break it into two lists (A-M, N-Z)

Relaxing the 32-Node Restriction on MinIO

Harshavardhana Harshavardhana on Modern Data Lakes 6 January 2020

MinIO previously imposed a much-discussed 32-node per cluster limit, which we lifted in December. There was no hard technical reason for the 32-node limit — you can use MinIO to create 1,000 node clusters. That reason we imposed the limit was to drive good architectural decisions. We learned, however, that some saw this as a limitation (indeed some competitors suggested

Scaling MinIO: Benchmarking Performance From Terabytes to Petabytes

Sidhartha Mani Sidhartha Mani on Benchmarks 17 December 2019

MinIO provides the best-in-class performance as we have repeatedly shown in our previous benchmarks. In those benchmarks, we chose the highest-end hardware and measured if MinIO could squeeze out every bit of the resources afforded it. This proved two key points: Ensuring that MinIO utilizes the maximum possible CPU, Network, and Storage available.Ensuring that MinIO is NOT the IO-bottleneck.

The State of Kubernetes: Five Takeaways from Kubecon

Jonathan Symonds Jonathan Symonds on Kubernetes 3 December 2019

This year’s Kubecon was the biggest yet on almost every vector: attendance, announcements and keynotes :). In between all of that were some stark themes on the state of the movement, the evolving vendor landscape and the challenges ahead. While there are some great summaries, we wanted to share our thoughts on what we saw and what we think the