MinIO Object Storage Running on the Google Cloud Platform

MinIO Object Storage Running on the Google Cloud Platform

As organizations organize themselves around data, they are becoming application-oriented. The modern application is a cloud native and data-centric application, and benefits from decoupled stateless, immutable services capable of exceptional performance and scale. While MinIO is available on every cloud — public, private, and edge, this post is focused on the Google Cloud Platform with an eye on why you need

Read more...

Architecting a Modern Data Lake

Architecting a Modern
Data Lake

Approximately 90% of all the data in the world is replicated data, with only 10% being genuine, new data. This has significant implications for an enterprise's data strategy — particularly when you consider the growth rates. For example, in 2020, the total amount of data generated and consumed was 64.2 zettabytes. In 2021, it was forecast that the overall amount

Read more...

Simplicity at Scale

Simplicity at Scale

Frank Wessels is a founder at Sneller and previously worked at MinIO. Introduction While at MinIO, perhaps the most important lesson that I learned was about simplicity. Simplicity is is paramount if you want to achieve (true) scalability. This permeated MinIO's architecture and software design from front to back and drove, amongst others, the important Sneller decisions that I will

Read more...

Using MinIO as external tables to extend Snowflake

Using MinIO as external tables to extend Snowflake

Snowflake is now the de facto standard for cloud data warehouse platforms. It is architected to support a variety of data tasks from pipelines, ETL, analytics and governance. Traditionally, all the data needed to be moved into Snowflake for an enterprise to take advantage of the Snowflake capabilities. Snowflake, however, has understood that enterprises want to integrate their data wherever

Read more...

Migrating MinIO Cluster Instances with Zero Downtime and Zero Data Loss

Migrating MinIO Cluster Instances with Zero Downtime and Zero Data Loss

With the advent of cloud computing, ephemeral compute instances have become ubiquitous. This introduces a whole set of challenges around managing the software, applying DevOps principles, addressing security vulnerabilities and ensuring automation. These are mission-critical in order to prevent data theft and service disruption. Addressing security vulnerabilities is particularly challenging as it frequently takes the form of updating and restarting

Read more...

Takeaways from Kubecon EU

Takeaways from Kubecon EU

Kubecon EU in Valencia, Spain was a welcome breath of fresh air for the Kubernetes community. While the Linux Foundation and the CNCF did great work in the pandemic, nothing is quite like seeing your “people” in person. That what Valencia delivered. I don’t know the attendance figures off the top of my head, but having been in Los

Read more...

Introducing DirectPV

Introducing DirectPV

DirectPV is a CSI driver for Direct Attached Storage. At the most basic level, it is a distributed persistent volume manager, and not a storage system like SAN or NAS. DirectPV is used to discover, format, mount, schedule and monitor drives across servers. Before we get into the architecture of DirectPV, let’s address why we needed to build it.

Read more...

Machine Learning Using H20, R and MinIO

Machine Learning Using H20, R and MinIO

I’ve been working with neural networks and machine learning since the late ‘80. Yes, I’m that old. The first product I bought was California Scientific Software BrainMaker Professional. I loved that product because it got me started with neural networks. I still have it: It was a 3 layer neural network product that came with source code in

Read more...