MinIO Blog

From Data Swamps to Reliable Data Systems: How Iceberg Brought 40 Years of Database Wisdom to Data Lakes

From Data Swamps to Reliable Data Systems: How Iceberg Brought 40 Years of Database Wisdom to Data Lakes

The data lake was once heralded as the future, an infinitely scalable reservoir for all our raw data, promising to transform it into actionable insights. This was a logical progression from databases and data warehouses, each step driven by the increasing demand for scalability. Yet, in embracing the data lake's scale and flexibility, we overlooked a critical difference.

Read more

Turbocharged Storage: MinIO, KIOXIA, and AMD team up to take on AI

Turbocharged Storage: MinIO, KIOXIA, and AMD team up to take on AI

MinIO, the leader in high-performance AI storage, has once again raised the bar in the AI infrastructure industry with its groundbreaking MinIO AIStor platform. Leveraging next-generation AMD hardware, KIOXIA NVMe™ SSDs, and cutting-edge software optimizations, MinIO AIStor delivers unmatched performance, scalability, and efficiency for AI-driven and other data intensive workloads. Today, we are excited to share benchmark results that demonstrate

Read more

Software Defined Hardware: Supermicro's Rise Explained

Software Defined Hardware: Supermicro's Rise Explained

It is hard to believe that it was 13 years ago that Marc Andressen penned his famous blog entitled “Software is Eating the World.” In it he spoke of the disruption that modern software organizations were inflicting on traditional businesses. Thirteen years later, even in the face of stratospheric valuations for Nvidia, software continues to eat the world. The evidence

Read more

Mitigating Ransomware Attacks with Object Storage

Mitigating Ransomware Attacks with Object Storage

Ransomware attacks are nothing new. The first ransomware attack occurred 36 years ago in 1989, and it is known as the AIDS Trojan PC Cyborg Virus. Floppy disks infected with a Trojan virus were mailed to attendees of the World Health Organization’s AIDS conference and other individuals. The virus waited until the computer had been booted 90 times. Then

Read more

MLOps Architecture Guide for AI Infrastructure

MLOps Architecture Guide for AI Infrastructure

MLOps, short for Machine Learning Operations, is a set of practices and tools aimed at addressing the specific needs of engineers building models and moving them into production. Some organizations start off with a few homegrown tools that version datasets after each experiment and checkpoint models after every epoch of training. On the other hand, many organizations have chosen to

Read more

AI ML Architecture: Modern Datalake Reference Guide

AI ML Architecture: Modern Datalake Reference Guide

An abbreviated version of this post appeared on The New Stack on March 19th, 2024. In enterprise artificial intelligence, there are two main types of models: discriminative and generative. Discriminative models are used to classify or predict data, while generative models are used to create new data. Even though Generative AI has dominated the news of late, organizations are still

Read more

The Case for Native Iceberg Catalog APIs and Unified Governance in Object Storage

The Case for Native Iceberg Catalog APIs and Unified Governance in Object Storage

Apache Iceberg is significantly transforming modern data lakes. Its introduction to object storage platforms has been celebrated for delivering ACID transactions, strong schema evolution, and warehouse-like reliability to data lake architectures. The Iceberg Catalog API standard is crucial to this transformation, as it ensures that various tools can consistently discover tables and execute atomic transactions once a compliant catalog service

Read more

How a Global Content Leader Rebuilt Their Observability Architecture for Hyperscale

How a Global Content Leader Rebuilt Their Observability Architecture for Hyperscale

This is the story of how an enterprise cloud content platform architected one of the largest and most resilient SmartStore deployments in production using MinIO AIStor and Splunk SmartStore. One of the world’s largest content platforms faced a serious scaling issue. They were ingesting multiple terabytes of observability data per day, and that footprint was growing by multiple petabytes

Read more

AI-Powered Quality Control, Built for the Factory Floor

AI-Powered Quality Control, Built for the Factory Floor

How a global manufacturer is cutting inspection labor by 10x with anomaly detection and edge-native object storage Overview On factory floors around the world, visual inspection remains one of the most labor-intensive and error-prone steps in the manufacturing process. At one global consumer goods manufacturer, this challenge is being redefined with AI at the edge. By replacing manual spot checks

Read more

What Is Edge Storage? Benefits & Use Cases

What Is Edge Storage? Benefits & Use Cases

Edge computing is a hot topic and carries with it some confusion, particularly around storage. Handling data properly at the edge can ensure a scalable, cost-effective and secure infrastructure - but failing to set up the right architecture can lead to data loss, security vulnerabilities and sky-high costs related to the bandwidth needed to transfer data repeatedly to and from

Read more