A collection of 120 posts tagged with "AI/ML"
An abbreviated version of this post appeared on The New Stack on March 19th, 2024. In enterprise artificial intelligence, there are two main types of models: discriminative and generative. Discriminative models are used to classify or predict data, while generative models are used to create new data. Even though Generative AI has dominated the news of late, organizations are still
Read more
MinIO’s high-performance object storage is key for AI innovation, offering scalability and integration for feature stores. Its capabilities enable seamless ML workflows, enhancing data management for AI development and deployment, impacting sectors like e-commerce and healthcare.
Read more
As the computing world has evolved and the price of DRAM has plummeted, we find that server configurations often come with 500GB or more of DRAM. When you are dealing with larger deployments, even those with ultra-dense NVMe drives, the number of servers multiplied by the DRAM on those servers can quickly add up – often to several TBs. That DRAM
Read more
Unstructured-IO, MinIO, & Weaviate redefine ETL, turning unstructured web data into actionable insights. This collaboration enhances data management, offering a robust solution for dynamic data transformation and analysis, marking a leap in how we process and leverage web-generated content.
Read more
Explore Langchain’s LLM Tool-Use and leverage Langgraph for monitoring MinIO’s S3 Object Store. This guide walks you through developing custom conversational AI agents and creating powerful OpenAI LLM chains for efficient data management and enhanced application functionality.
Read more
Explore the fusion of GitOps, MinIO, Weaviate, and Python in AI development for unparalleled automation and innovation. This combination offers a solid foundation for creating scalable, efficient, and automated AI solutions, propelling projects from concept to reality with ease.
Read more
This tutorial guides you through constructing robust data pipelines on the edge, ensuring flexibility and scalability. Learn to create, populate, and transform datasets seamlessly while prioritizing data privacy. Master the art of automation with MinIO's Python SDK.
Read more
Explore integrating MinIO with Weaviate using Docker Compose for AI-enhanced data management. Learn to back up Weaviate to MinIO S3 buckets, ensuring data integrity and scalability with practical Docker and Python examples. Streamline your AI-driven search and analysis with this robust setup.
Read more
Learn how to run Python stored procedures on SQL Server 2022.
Read more
Tl;dr: In this post, we will use MinIO Bucket Notifications and Apache Tika, for document text extraction, which is at the heart of critical downstream tasks like Large Language Model (LLM) training and Retrieval Augmented Generation (RAG). The Premise Let’s say that I want to construct a dataset of text that I can then use to fine-tune an
Read more
A chain is as strong as its weakest link - and your AI/ML infrastructure is only as fast as your slowest component. If you train machine learning models with GPUs, then your weak link may be your storage solution. The result is what I call the “Starving GPU Problem.” The Starving GPU problem occurs when your network or your
Read more
I suspect some folks will accuse me of clickbait titling. Others will say, that’s not really a reach - most folks will fail in their initial AI attempts but it doesn’t matter and the learnings are worth it. On some level both are right - but I think WHY enterprises will fail is worth exploration and may allow
Read more
Explore the synergy of MinIO, Langchain, and OpenAI in enhancing data storage and processing. This article illustrates MinIO’s integration for efficient document summarization using Langchain and OpenAI’s GPT, revolutionizing AI and ML data handling.
Read more
Explore the essential role of Data Engineers in unleashing the true power of AI! Data Engineers have a critical foundation in cleaning and structuring raw data for ML success. Learn why their expertise in data infrastructure, feature engineering, and pipeline optimization is indispensable.
Read more
Much has been said lately about the wonders of Large Language Models (LLMs). Most of these accolades are deserved. Ask ChatGPT to describe the General Theory of Relativity and you will get a very good (and accurate) answer. However, at the end of the day ChatGPT is still a computer program (as are all other LLMs) that is blindly executing
Read more
Explore the future of AI in an open-source landscape, challenging Big Tech's masked efforts. Learn how embracing extreme open innovation fosters collaboration, drives market growth, and sets the stage for an open-source AI data stack.
Read more
Joust against data complexity with LanceDB, a lightning-fast vector database optimized for AI/ML on the open-source Lance format. Teaming up with MinIO, it scales seamlessly, offering high-performance, cloud-native storage. Dive into the tutorial for a swift deployment.
Read more
Over the past few months, I have written about a number of different technologies (Ray Data, Ray Train, and MLflow). I thought it would make sense to pull them all together and deliver an easy-to-understand recipe for distributed data preprocessing and distributed training using a production-ready MLOPs tool for tracking and model serving. This post integrates the code I presented
Read more
Most machine learning projects start off as a single-threaded proof of concept where each task is completed before the next task can begin. The single-threaded ML pipeline depicted below is an example. However, at some point, you will outgrow the pipeline shown above. This may be caused by datasets that no longer fit into the memory of a single process.
Read more
The calendar year 2023 will be a meaningful one, perhaps one of the most meaningful ones, when the history of AI is written. It was, in essence, the big bang. It started in late 2022 with OpenAI’s ChatGPT but it was the response that was so breathtaking. Within months we had Meta’s LLaMA 2, Google’s Bard chatbot
Read more
