The Apache Iceberg data lake storage format enables ACID transactions on tables saved to MinIO. ACID transactions enable multiple users and services to concurrently and reliably add and remove records atomically. At the same time, queries are isolated to maintain read consistency against tables that are in the process of being altered. You can put MinIO and Iceberg, in conjunction
Read more...
Learn how to build a multi-cloud data lake with the Delta open storage format and MinIO object storage.
Read more...
This post focuses on how Iceberg and MinIO complement each other and how various analytic frameworks (Spark, Flink, Trino, Dremio, and Snowflake) can leverage the two.
Read more...
Enabling and tuning transparent data compression in MinIO.
Read more...
Learn how to build a cloud-native analytics and visualization stack backed by MinIO.
Read more...
Large numbers of small files present big challenges for application performance.
Read more...
Working with a data lake of ZIP files? Learn how to download individual files from ZIP archives saved on MinIO.
Read more...
Stellar performance-at-scale, flexibility and consistency make object storage the best choice for cloud-native enterprises.
Read more...
Migrate data from HDFS to MinIO and enjoy the benefits of cloud-native architecture.
Read more...
With today’s announcement [https://www.prnewswire.com/news-releases/starburst-announces-new-product-release-which-extends-flexibility-when-building-data-lakehouse-architecture-301425279.html] of Starburst’s support for MinIO, it made sense to revisit the architectural trends that are becoming the standard for analytics workloads. Starburst provides a perfect example as we shall see shortly. The architecture follows the model of disaggregating storage and compute. Modern, high speed networks
Read more...
Take advantage of cloud native, Kubernetes-oriented, microservices-based architectures with object storage.
Read more...
How to pair fast and efficient search with high-performance Kubernetes-native object storage.
Read more...
When you think about the cloud, it helps to think about the types of businesses that have been built with elastic compute, networking and storage as a foundational component and self-service/multi-tenancy as the vehicle for customer engagement. For the most part, those businesses succeeded at scaling by focusing their efforts on building their product, almost exclusively on a single
Read more...
Introduction Document management is a core requirement for all sorts of regulated institutions - finance, telecom, healthcare, government and others. These institutions need to manage and retain an ever growing number of documents and regulatory guidelines often require these documents to be stored for a very long term (7-10 years). Take for example, KYC (Know Your Customer) documents. Anyone starting
Read more...
With the introduction of Apache Arrow, language-independent columnar memory format for flat and hierarchical data, organized for efficient analytic operations, MinIO data lakes can be much more powerful. This article explains how to make use of Apache Arrow by using ArrowRDD.
Read more...
Apache Nifi [https://nifi.apache.org/] is one of the most popular open source data flow engines available today. Nifi supports almost all the major enterprise data systems and allows users to create effective, fast, and scalable information flow systems. Creating data flow systems is simple with Nifi and there is a clear path to add support for systems not
Read more...MinIO previously imposed a much-discussed 32-node per cluster limit, which we lifted in December. There was no hard technical reason for the 32-node limit — you can use MinIO to create 1,000 node clusters. That reason we imposed the limit was to drive good architectural decisions. We learned, however, that some saw this as a limitation (indeed some competitors suggested
Read more...
The demise of Hadoop is probably overblown. It will not suddenly disappear from the enterprise landscape - there are simply too many clients, too much sunk investment for it to vanish into the night. What is not overblown is the fact that Hadoop, like countless technologies before it, is in secular, irreversible decline. There are a number of reasons but
Read more...
When early object storage APIs were developed they focused on the efficient storage and retrieval of objects. Amazon’s success with S3 and its implementation of the robust S3 API quickly became the de facto standard for object storage in the cloud. MinIO, recognizing this, invested heavily in creating the most compliant implementation of the S3 API outside of Amazon.
Read more...
In this post we’ll learn more about object storage, specifically Minio and then see how to connect Minio with tools like Apache Spark and Presto for analytics workloads.
Read more...
Hybrid Cloud Object Storage
Baremetal
Erasure Code Calculator
Reference Hardware
