Tl;dr: We train a small LLM to become good at reasoning with reinforcement learning (similar to the process that led to Deepseek R1) all against AIStor AIHub, an on-premises model repository. Based on the great GRPO demo by will brown. Motivation: A growing requirement for teams is the need for an organized, secure, "single source of truth"
Read more
In today’s AI-driven enterprise landscape, resource optimization has evolved from a desirable goal into an operational imperative. As organizations scale their artificial intelligence initiatives to meet rising demands for innovation, the efficient orchestration of compute resources directly shapes operational performance and model precision. The forthcoming integration of NVIDIA GPUDirect Storage (GDS) with MinIO AIStor is a co-engineered solution slated
Read more
The Arm architecture is revolutionizing the hyperscale cloud, propelled by its Total Cost of Ownership (TCO) advantages—lower power consumption and reduced cooling requirements—that enable sustainable, high-performance computing at scale. Industry leaders like AWS, Azure, and GCP are embracing Arm to drive their latest compute instances for AI training, harnessing its efficiency to meet the demands of data-intensive workloads.
Read more
Building upon AIStor's robust AI capabilities, MinIO's PromptObject has been enabling users to interact with their data through natural language queries as described here. PromptObject transforms how users interact with stored objects by allowing them to ask questions about their data's content and extract information using natural language—eliminating the need to write complex
Read more
Modern enterprises seeking to leverage AI capabilities often face a significant hurdle: the complex deployment and management of GPU infrastructure in their Kubernetes environments. MinIO's AIStor addresses this challenge head-on by integrating the NVIDIA GPU Operator, revolutionizing how organizations deploy and manage GPU resources for AI workloads. Through automated GPU setup, driver management, and resource optimization, this integration
Read more
MinLZ is a compression algorithm developed by MinIO. The main goal is to provide a format that offers the best-in-class compression while providing very fast decompression even with modest hardware.
Read more
MLflow Model Registry allows you to manage models that are destined for a production environment. This post picks up where my last post on MLflow Tracking left off. In my Tracking post I showed how to log parameters, metrics, artifacts, and models. If you have not read it, then give it a read when you get a chance. In this
Read more
A cybersecurity firm faced soaring cloud costs and performance bottlenecks with AWS S3 as their log data grew to a multi-exabyte scale. They adopted MinIO AIStor for high-performance, S3-compatible object storage, cutting costs and boosting efficiency.
Read more
Apache XTable enables interoperability between Iceberg, Hudi, and Delta Lake by translating metadata, eliminating the need for data migration. This makes it easier to adopt new query engines, optimize performance, and build a more flexible data architecture.
Read more
In several previous posts on MLOps tooling, I showed how many popular MLOps tools track metrics associated with model training experiments. I also showed how they use MinIO to store the unstructured data that is a part of the model training pipeline. However, a good MLOps tool should do more than manage your experiments, datasets, and models. It should be
Read more
In the rapidly-evolving world of artificial intelligence (AI) and machine learning (ML), speed and scalability are paramount. The ability to process massive amounts of data in real-time is a critical requirement for organizations looking to leverage AI/ML for competitive advantage. Whether it's training large machine learning models, running complex inference tasks, or scaling data pipelines, the performance
Read more
Choosing the right open table format—Apache Iceberg, Delta Lake, or Apache Hudi—can make or break your data lakehouse. This guide breaks down their strengths, how they integrate with object storage, and which one is best for AI, analytics, and real-time workloads.
Read more
In this post we look at how search, and specifically OpenSearch can help us identify patterns or see trends in our ever growing data.
Read more
This post first appeared on The New Stack on January 16th, 2025. Often, while accessing the legitimacy of a new technology receiving a lot of hype, studying existing core capabilities and history is helpful. If the new technology in question is not based on existing or imminent capabilities, we can label it as “hype” and move on. Another litmus test
Read more
2025 has inherited a slew of geopolitical concerns that started years ago. U.S. Foreign policy, U.S. - China Relations, China’s geopolitical maneuvers, Conflicts in the Middle East, Russian Ukraine war, and cybersecurity threats. Additionally, new leadership in the United States adds to the uncertainty created by these concerns. And, as if all this were not enough, the
Read more
dbt’s acquisition of SDF Labs reinforces a powerful trend: the modern data stack is open. Learn why this matters for performance, interoperability, and future-proofing your data strategy.
Read more
Running AIStor on OpenShift enables enterprises to achieve cloud-native elasticity on their hardware or cloud instance of choice, balance cost, capacity and performance.
Read more
The evolution of data roles never stops—first, we were all data scientists, then data engineers, and now, DataOps engineers. But is DataOps really new, or just a fresh take on the same mission: delivering business value through data?
Read more
Discover the power of Apache Iceberg and AIStor in transforming data lakehouses! From multi-engine compatibility to time travel, schema evolution, and blazing-fast performance, this guide dives deep into how Iceberg unlocks the full potential of modern AI and analytics workloads.
Read more
In November of 2023, Amazon announced the S3 Connector for PyTorch. The Amazon S3 Connector for PyTorch provides implementations of PyTorch's dataset primitives (Datasets and DataLoaders) that are purpose-built for S3 object storage. It supports map-style datasets for random data access patterns and iterable-style datasets for streaming sequential data access patterns. The S3 Connector for PyTorch also includes
Read more
