Before diving into Amazon’s S3 Connector for PyTorch, it is worthwhile to introduce the problem it is intended to solve. Many AI models need to be trained on data that cannot fit into memory. Furthermore, many really interesting models being built for computer vision and generative AI use data that cannot even fit on the disk drive that comes
Read more
Object tags give you greater power. You now have the ability to categorize by up to ten dimensions. If you want to add the diagram to a project, then all you have to do is tag it appropriately.
Read more
One of the biggest challenges facing organizations today for AI and data management is access to reliable infrastructure and compute resources. The Intel Tiber Developer Cloud is purpose-built for engineers who need an environment for proof-of-concepts, experimentation, model training, and service deployments. Unlike other clouds, which can be unapproachable and complex, the Intel Tiber Developer Cloud is simple and easy
Read more
Parsec Labs is a company of engineers. Most have designed storage systems, been responsible for backups and replication, or worked in networking building switches. Founded in 2013, their Unified Data Mobility and Protection Appliance provides the most straightforward tools for migrating, replicating, and backing up data at scale. A Common Request As a one-time pre-sales engineer, Mark Clark, CEO of
Read more
AIStor is a foundational component for creating and executing complex data workflows. At the core of this event-driven functionality is MinIO bucket notifications using Kafka.
Read more
Everybody claims to be a software company these days. From the nearly decade old pronouncement by Marc Andressen that “Software Is Eating the World” to the push from Wall Street to produce recurring software revenue; the pressure is on to claim - at least - that you are a software company. This is obviously problematic for appliance vendors. Try as
Read more
Cloud-native AI/ML workloads push storage to the limit with many small files. AIStor combines metadata and data to optimize small file operations.
Read more
Unlock real-time analytics, scalability, and future-proof your data with a lakehouse. Hadoop can't keep up with AI, but a lakehouse blends the best of data lakes and warehouses. Get actionable tips to smoothly phase your migration with minimal disruption.
Read more
We have said it before, but it bears repeating. The cloud is an operating model - not a physical location. That is why you will find MinIO everywhere on the public cloud, on the private cloud, at the edge. We don’t differentiate and because we are cloud native we are cloud (location) agnostic. The public cloud has mindshare and
Read more
In this post we’ll show you how you visualize the cluster metrics in a web browser and also we’ll set up alerting so that when something like a drive needs to be replaced or drive runs out of space we can get alerted for it.
Read more
To ensure AI success, start by hiring a data engineer, not an AI/ML expert. Learn from our experience and find out why a strong data foundation—focused on object storage, data lakehouses, and optimized pipelines—is critical for scalable, efficient AI/ML workloads.
Read more
Tesla just open-sourced its Tesla Transport Protocol over Ethernet (TTPoE). Like Tesla’s earlier move to share EV patents, this opens the door to faster innovation through open-source contributions, proving once again that transparency drives progress.
Read more
Getting started with MinIO is easy—with a few simple commands and a small 100 MB binary, you can have a functional development environment up and running in no time. However, to harness the full capability of MinIO at production scale, professionals are encouraged to learn more about MinIO’s extensive capabilities. We are introducing the MinIO Learning Subscription, featuring
Read more
You’ve surely version controlled code in the past. But have you version controlled your data? Did you ever want to collaborate on large sets of data with various teams without committing a large chunk?
Read more
Enhance your AI workflows by combining MinIO’s scalable AIStor with Polars, a lightning-fast DataFrame library. Learn how this powerful duo accelerates data pipelines, handles massive datasets, and offers powerful performance and scale.
Read more
Dell generally focused on the filer game, but they dabble in object storage and have a very old offering, ECS. That makes sense, it was a step up from tape and wasn’t suited for dynamic workloads like HDFS modernization or database workloads. Needless to say, AI was out of the question. For a few years now, Dell has been
Read more
Faced with skyrocketing compute costs, MinIO data scientist Archana Vaidyanathan leveraged the power of the data lakehouse, which allows for flexible compute choices without overhauling storage. AIStor enhances this model, delivering speed, scalability, and cost savings.
Read more
Simplifying your data streaming architecture with WarpStream, a cloud-native, Kafka-compatible platform that cuts costs and complexity, recently acquired by Confluent. Paired with MinIO's high-performance object storage, it's a powerful alternative to Kafka for scalable, cost-effective streaming.
Read more
Large numbers of small files present big challenges for application performance.
Read more
Confluent's WarpStream acquisition highlights the future of data streaming built on object storage. WarpStream’s cloud-native design cuts costs by 85% over traditional Kafka. Believe the hype: object storage drives low-cost, scalable performance.
Read more
