MinIO Blog

AI/ML

A collection of 118 posts tagged with "AI/ML"

Hungry GPUs Need Fast Object Storage

Hungry GPUs Need Fast Object Storage

A chain is as strong as its weakest link - and your AI/ML infrastructure is only as fast as your slowest component. If you train machine learning models with GPUs, then your weak link may be your storage solution. The result is what I call the “Starving GPU Problem.”  The Starving GPU problem occurs when your network or your

Read more

The Strengths, Weaknesses and Dangers of LLMs

The Strengths, Weaknesses and Dangers of LLMs

Much has been said lately about the wonders of Large Language Models (LLMs). Most of these accolades are deserved. Ask ChatGPT to describe the General Theory of Relativity and you will get a very good (and accurate) answer. However, at the end of the day ChatGPT is still a computer program (as are all other LLMs) that is blindly executing

Read more

Distributed Training and Experiment Tracking with Ray Train, MLflow, and MinIO

Distributed Training and Experiment Tracking with Ray Train, MLflow, and MinIO

Over the past few months, I have written about a number of different technologies (Ray Data, Ray Train, and MLflow). I thought it would make sense to pull them all together and deliver an easy-to-understand recipe for distributed data preprocessing and distributed training using a production-ready MLOPs tool for tracking and model serving. This post integrates the code I presented

Read more

Distributed Training with Ray Train and MinIO

Distributed Training with Ray Train and MinIO

Most machine learning projects start off as a single-threaded proof of concept where each task is completed before the next task can begin. The single-threaded ML pipeline depicted below is an example. However, at some point, you will outgrow the pipeline shown above. This may be caused by datasets that no longer fit into the memory of a single process.

Read more

Data Science and AI with a SQL Server 2022 Data Lakehouse

Matt Sarrel Matt Sarrel Matt Sarrel @msarrel on SQL |
Data Science and AI with a SQL Server 2022 Data Lakehouse

Microsoft SQL Server 2022 is one of the most commonly implemented enterprise relational databases. Many of the world's most successful companies, regardless of vertical, have significant SQL Server deployments. Thousands of companies have relied on SQL Server for decades. Microsoft has made great strides over the past decade in embracing open-source and standards-compliant technologies. The result is that

Read more