MinIO Blog

MinIO Blog

Parallel ML Experimentation leveraging MinIO & lakeFS

MinIO MinIO MinIO on AI/ML |
Parallel ML Experimentation leveraging MinIO & lakeFS

Introduction This post was written in collaboration with Iddo Avneri from lakeFS. Managing the growing complexity of ML models and the ever-increasing volume of data has become a daunting challenge for ML practitioners. Efficient data management and data version control are now critical aspects of successful ML workflows. In this blog post, we delve into the power of parallel ML

Read more

Get Started with MinIO on Red Hat OpenShift for a PoC

Get Started with MinIO on Red Hat OpenShift for a PoC

When we announced the availability of MinIO on Red Hat OpenShift, we didn’t anticipate that demand would be so great that we would someday write a series of blog posts about this powerful combination. This combination is being rapidly adopted due to the ubiquitous nature of on-prem cloud and the need of large organizations wanting to bring their data

Read more

Setting up a Development Machine with MLFlow and MinIO

Setting up a Development Machine with MLFlow and MinIO

About MLflow MLflow is an open-source platform designed to manage the complete machine learning lifecycle. Databricks created it as an internal project to address challenges faced in their own machine learning development and deployment processes. MLflow was later released as an open-source project in June 2018. As a tool for managing the complete lifecycle, MLflow contains the following components. * MLflow

Read more

Migrating from Hadoop to a Cloud-Ready Architecture for Data Analytics

Migrating from Hadoop to a Cloud-Ready Architecture for Data Analytics

This post was a collaboration between Kevin Lambrecht of UCE Systems and Raghav Karnam The cloud operating model and specifically Kubernetes have become the standard for large scale infrastructure today. More importantly, they are evolving at an exceptional pace with material impacts to data science, data analytics and AI/ML. This transition has a significant impact on the Hadoop ecosystem.

Read more

Leveraging Object Storage for Enterprise Legacy Data

AJ AJ on DevOps |
Leveraging Object Storage for Enterprise Legacy Data

MinIO is built with speed and resiliency at the forefront, regardless of the type of environment you choose to run it on. Whether it's multi cloud, bare metal, cloud instances or even on-premise, MinIO is designed to run on AWS, GCP, Azure, colocated bare metal servers and Kubernetes distributions such as Red Hat OpenShift. MinIO runs just as

Read more

The New Math on Backup and Replication

Ugur Tigli Ugur Tigli on BC/DR |
The New Math on Backup and Replication

The world of backup has entered a brave new world where traditional solutions still have utility but where the scale, speed of change and application landscape require different…radically different…approaches. This post seeks to lay out the challenges of this new world, where the line of demarcation exists and how to think about architecting a data protection framework that

Read more

Enhance Large Language Models Leveraging RAG and MinIO on cnvrg.io

MinIO MinIO MinIO on AI/ML |
Enhance Large Language Models Leveraging RAG and MinIO on cnvrg.io

This post was written in collaboration with Harinder Mashiana from cnvrg.io. Large language models (LLMs) have revolutionized the world of technology, offering powerful capabilities for text analysis, language translation, and chatbot interactions. The revolution will heavily impact businesses, according to OpenAI, approximately 80% of the U.S. workforce could have at least 10% of their work tasks affected by

Read more

Object Management for AI/ML

Object Management for AI/ML

Introduction In a few previous posts on AI/ML, I mentioned that one of the benefits of MinIO is that you have tools for Versioning, Lifecycle Management, Object Locking, Object Retention and Legal Holds. These capabilities have a variety of uses. You may need a simple way to keep track of training experiments. You could also use these features to

Read more

Putting a Filesystem on Top of an Object Store is a Bad Idea. Here is why.

Putting a Filesystem on Top of an Object Store is a Bad Idea. Here is why.

When purchasing storage, the emphasis is usually on media, but it may be even more important to consider access methods too. You will need to take storage protocols into account when designing and procuring infrastructure, especially when you leave legacy storage behind in order to migrate to cloud-native object storage. However, object storage relies on the S3 API for communications,

Read more

YouTube Summaries: Kubernetes and the MinIO Operator

YouTube Summaries: Kubernetes and the MinIO Operator

Our latest YouTube training series is all about the MinIO Operator, which brings native support for deploying and managing MinIO deployments (“MinIO Tenants”) on a Kubernetes cluster. MinIO’s Mike Johnson (aka MJ) brings us through the 10-part video series to set the foundation of understanding Kubernetes before focusing on installing and configuring the MinIO Operator for Kubernetes, which will

Read more

The Architect’s Guide to Storage for AI

The Architect’s Guide to Storage for AI

This post first appeared in The New Stack. Developers gravitate to technologies that are software defined, open source, cloud native and simple. That essentially defines object storage. Introduction Choosing the best storage for all phases of a machine learning (ML) project is critical. Research engineers need to create multiple versions of datasets and experiment with different model architectures. When a

Read more