Model Checkpointing using Amazon’s S3 Connector for PyTorch and MinIO

Model Checkpointing using Amazon’s S3 Connector for PyTorch and MinIO

In November of 2023, Amazon announced the S3 Connector for PyTorch. The Amazon S3 Connector for PyTorch provides implementations of PyTorch's dataset primitives (Datasets and DataLoaders) that are purpose-built for S3 object storage. It supports map-style datasets for random data access patterns and iterable-style datasets for streaming sequential data access patterns.  The S3 Connector for PyTorch also includes

Read more

AIStor on ROSA

AJ AJ on AIStor |
AIStor on ROSA

Lately, there has been a trend in the industry to bring data “closer” to home. The result is that  organizations now want to keep their data on servers that they own, in their own datacenter or at a colocation provider.

Read more

The Innovations from AWS re:Invent

The Innovations from AWS re:Invent

Earlier this month, Amazon held their re:Invent conference in Las Vegas, Nevada, from December 1st to 5th - a 5-day event.  If you have never been to a re:Invent conference, then the word that describes it best is “huge” - not just in terms of the number of attendees (60,000) but also the breadth of topics covered.

Read more

The Blog Year in Review: Top 10 for 2024

Sasha Wodtke Sasha Wodtke on |
The Blog Year in Review: Top 10 for 2024

2024 was a HUGE year for MinIO. Not only did we release AIStor, the most powerful version of MinIO to date, but we also attended 54 events, wrote 159 blogs, won over 10 awards and so much more. And all of it wouldn’t be possible without the support of our amazing MinIO community.  So, as a shoutout to you

Read more

Iterable-Style Datasets using Amazon’s S3 Connector for PyTorch and MinIO

Iterable-Style Datasets using Amazon’s S3 Connector for PyTorch and MinIO

In November of 2023 Amazon announced the S3 Connector for PyTorch. The Amazon S3 Connector for PyTorch provides implementations of PyTorch's dataset primitives (Datasets and DataLoaders) that are purpose-built for S3 object storage. It supports map-style datasets for random data access patterns and iterable-style datasets for streaming sequential data access patterns.  In a previous post, I introduced the

Read more

Revolutionizing Mobile Testing with Big Data and AI

Revolutionizing Mobile Testing with Big Data and AI

A mobile application is a company's brand available on demand. It is a window into any service or product an organization offers. At Kobiton, they understand this—it is their mission to improve mobile applications through testing.  Kobiton is a mobile testing platform that allows customers to perform manual and automated testing on real mobile devices from anywhere

Read more

MinIO’s S3 over RDMA Initiative: Setting New Standards in Object Storage for High-Speed AI Data Infrastructure

MinIO’s S3 over RDMA Initiative: Setting New Standards in Object Storage for High-Speed AI Data Infrastructure

As the demands of AI and machine learning continue to accelerate, data center networking is evolving rapidly to keep pace. For many enterprises, 400GbE and even 800GbE are becoming standard choices, driven by the need for high-speed, low-latency data transfer for AI workloads that are both data-intensive and time-sensitive. AI models for tasks like large language processing, real-time analytics, and

Read more