Tl;dr: GET, PUT, PROMPT. It’s now possible to summarize, talk with, and ask questions about an object that is stored on MinIO with just natural language using the new PromptObject API. In this post, we explore a few use-cases of this new API along with code examples.
Motivation:
Object storage and the S3 API’s ubiquity can be
Read more
tl;dr:
In this post, we will explore four technical reasons why AI workloads rely on high performance object store.
1. No Limits on Unstructured Data
In the current paradigm of machine learning, performance and ability scales with compute, which is really a proxy for dataset size and model size (Scaling Laws for Neural Language Models, Kaplan et. al.). Over
Read more
Tl;dr:
In this post, we will use MinIO Bucket Notifications and Apache Tika, for document text extraction, which is at the heart of critical downstream tasks like Large Language Model (LLM) training and Retrieval Augmented Generation (RAG).
The Premise
Let’s say that I want to construct a dataset of text that I can then use to fine-tune an
Read more
Much has been said lately about the wonders of Large Language Models (LLMs). Most of these accolades are deserved. Ask ChatGPT to describe the General Theory of Relativity and you will get a very good (and accurate) answer. However, at the end of the day ChatGPT is still a computer program (as are all other LLMs) that is blindly executing
Read more
Tl;dr:
In this post, we will create a custom image dataset and then train a You-Only-Look-Once (YOLO) model for the ubiquitous task of object detection. We will then implement a system using MinIO Bucket Notifications that can automatically perform inference on a new image.
Introduction:
Computer vision remains an extremely compelling application of artificial intelligence. Whether it’s recognizing
Read more
Making the serving of your AI models more lightweight by leveraging the simplicity of MinIO’s object store.
tl;dr
MinIO object storage can be used as a ‘single source of truth’ for your machine learning models and, in turn, make serving with PyTorch Serve more efficient when managing changes to Large Language Models (LLMs). As always, sample code is
Read more