MLOps is to machine learning what DevOps is to traditional software development. Both are a set of practices and principles aimed at improving collaboration between engineering teams (the Dev or ML) and IT operations (Ops) teams. The goal is to streamline the development lifecycle, from planning and development to deployment and operations, using automation. One of the primary benefits of
Read more
In this tutorial, we'll deploy a cohesive system that allows distributed SQL querying across large datasets stored in Minio, with Trino leveraging metadata from Hive Metastore and table schemas from Redis.
Read more
Explore how Kubernetes v1.30 can enhance your MinIO deployment. Kubernetes v1.30 offers enhanced security, networking improvements upcoming features in beta. Consider upgrading for optimized, secure deployments for modern data workflows.
Read more
When a MinIO Modern Datalake deployment is extended by adding a new server pool, by default it does not rebalance objects. Lets dive deep and learn how to rebalance smoothly without affecting cluster operations.
Read more
Implementing KES within Kubernetes in a stateful configuration ensures the persistence of encryption keys through pod lifecycle events and restarts. This setup offers resilience especially in environments where relying on external KMS is not an option or preferred.
Read more
The phenomenon of the public cloud is difficult to get your arms around. Since AWS kicked it off early in the century it has grown and evolved into a modern computing platform - creating the cloud operating model as we know it. Ironically, this standardization around the cloud as an operating model is the one of the reasons that cloud
Read more
It is hard to believe that it was 13 years ago that Marc Andressen penned his famous blog entitled “Software is Eating the World.” In it he spoke of the disruption that modern software organizations were inflicting on traditional businesses. Thirteen years later, even in the face of stratospheric valuations for Nvidia, software continues to eat the world. The evidence
Read more
Discover how to seamlessly migrate from HDFS to modern object storage without ripping out all of your current systems. Learn valuable strategies to retain essential tools and modernize your infrastructure for AI/ML.
Read more
Delve into AI’s next frontier with MinIO S3 Object-Store and SDK, enhancing a Weaviate Retreival Augmented Generation (RAG) Pipeline for robust data management. Discover how to elevate efficiency in AI systems using LangChain, unlocking new dimensions in scalable AI solutions.
Read more
Discover RisingWave, an open-source streaming database revolutionizing data lakehouses. Built for speed and scalability, it empowers developers with SQL on streaming data. Unlock the potential of real-time analytics and scalable data processing for your AI initiatives.
Read more
Apache Arrow is an open-source columnar memory format that is vital for modern datalakes. This is because Arrow makes data processing swift and seamless across various systems. Arrow propels AI and analytics by enhancing interoperability and computational efficiency.
Read more
If you are implementing a generative AI solution using Large Language Models (LLMs), you should consider a strategy that uses Retrieval-Augmented Generation (RAG) to build contextually aware prompts for your LLM. An important process that occurs in the preproduction pipeline of a RAG-enabled LLM is the chunking of document text so that only the most relevant sections of a document
Read more
In mid-2024, creating an AI demo that impresses and excites can be easy. Take a strong developer, some clever prompt experimentation, and a few API calls to a powerful foundation model and you can often build a bespoke AI bot in an afternoon. Add in a library like langchain or llamaindex to augment your LLM with a bit of custom
Read more
The public cloud changed the way companies build, deploy and manage their applications - mostly for the better. As you’re getting started, the public cloud supplies the infrastructure, services, enablement and maintenance to be up and running quickly. It provides ultimate scalability, in almost unlimited fashion, up and down, to provide you with the necessary resources no matter the
Read more
Stories matter and customer stories are the best. The ones where they delivered jaw-dropping stats or overcame massive obstacles are the ones that garner the best headlines. They are also the ones that are the hardest to get published. We know, because we are going to share a few with you that we are tirelessly working to get published -
Read more
Explore the integration of SingleStore, a high-performance cloud-native database, with MinIO in the Modern Datalake Stack. This tutorial provides hands-on experience in data storage, processing, and querying, fostering experimentation and innovation in data management, analytics, and AI workloads.
Read more
Explore the exciting possibilities of leveraging MinIO and LangChain to create a robust and efficient agent capable of handling complex data processing tasks.
Read more
An abbreviated version of this post appeared on The New Stack on March 26th, 2024. Businesses aiming to maximize their data assets are adopting scalable, flexible, and unified data storage and analytics approaches. This trend is driven by enterprise architects tasked with crafting infrastructures that align with evolving business demands. A Modern Datalake architecture addresses this need by integrating the
Read more
It's been just over a week for me here at MinIO. The big takeaway from immersing myself in whiteboarding sessions, architecture reviews and customer calls is that the simplicity of the product is both its distinguishing feature and one of its most defining value drivers. This is particularly true at scale. The explosive growth in computing power due
Read more
Anyone who has worked in a team environment knows that every successful team has one go-to person—that special individual who can help you regardless of the nature of your problem. On a traditional software development team, this individual is an expert programmer and is also an expert in one other technology, which could be a database technology like Snowflake
Read more