Digging Deeper into Iceberg: ACID Transactions on Tables

Digging Deeper into Iceberg: ACID Transactions on Tables

The Apache Iceberg data lake storage format enables ACID transactions on tables saved to MinIO. ACID transactions enable multiple users and services to concurrently and reliably add and remove records atomically. At the same time, queries are isolated to maintain read consistency against tables that are in the process of being altered. You can put MinIO and Iceberg, in conjunction

Read more...

Disaggregation, Analytical Engines and Starburst Trino

Disaggregation, Analytical Engines and Starburst Trino

With today’s announcement [https://www.prnewswire.com/news-releases/starburst-announces-new-product-release-which-extends-flexibility-when-building-data-lakehouse-architecture-301425279.html] of Starburst’s support for MinIO, it made sense to revisit the architectural trends that are becoming the standard for analytics workloads. Starburst provides a perfect example as we shall see shortly. The architecture follows the model of disaggregating storage and compute. Modern, high speed networks

Read more...

Kubernetes, Consistency and Commoditization - The Way of the Cloud

Kubernetes, Consistency and Commoditization - The Way of the Cloud

When you think about the cloud, it helps to think about the types of businesses that have been built with elastic compute, networking and storage as a foundational component and self-service/multi-tenancy as the vehicle for customer engagement. For the most part, those businesses succeeded at scaling by focusing their efforts on building their product, almost exclusively on a single

Read more...

Using MinIO for Know Your Customer Data at a Major Bank

Using MinIO for Know Your Customer Data at a Major Bank

Introduction Document management is a core requirement for all sorts of regulated institutions - finance, telecom, healthcare, government and others. These institutions need to manage and retain an ever growing number of documents and regulatory guidelines often require these documents to be stored for a very long term (7-10 years). Take for example, KYC (Know Your Customer) documents. Anyone starting

Read more...