Integrations - MinIO Blog (Page 2)

How to Back Up with Restic and MinIO

on BC/DR 20 November 2023

Every system needs to be backed up because there are countless ways to lose local filesystem data and configurations. That loss can be devastating – potentially resulting in revenue loss, dissatisfied customers and even costly litigation. The statistics are pretty bleak – sixty percent of businesses that suffer a data loss event close within six months and ninety-three percent of companies that

MinIO and Quickwit

AJ AJ on DevOps 16 November 2023

In today’s post we’ll show you how to configure MinIO as a storage provider and metadata store for Quickwit.

Confluent Platform with MinIO Tiered Object Storage Throughput Benchmark

Matt Sarrel Matt Sarrel @msarrel

on Benchmarks 23 October 2023

Confluent Platform with MinIO Tiered Object Storage Throughput Benchmark

Confluent, Intel and MinIO conducted benchmarking and certification testing for MinIO Tiered Object Storage for Kafka storage. This blog post describes the observations and results of testing MinIO object storage as a backend for the tiered storage feature of Confluent Platform 7.1.0 on servers equipped with third generation Intel Xeon Scalable processors. The scope of these tests was

Integrating MinIO with Hugging Face Datasets

Keith Pijanowski Keith Pijanowski on AI/ML 23 October 2023

Hugging Face's DatasetDict class is a part of the Datasets library and is designed to make working with datasets destined for any model found on the Hugging Face Hub efficient. As the name implies, the DatasetDict class is a dictionary of datasets. The best way to understand objects created from this class is to look at a quick

MinIO's OpenID Connect Integration Explained

Aditya Manthramurthy Aditya Manthramurthy on Operator's Guide 19 October 2023

MinIO provides a flexible Identity and Access Management system that can be integrated with popular external identity providers. MinIO IAM is built with AWS IAM compatibility at its core - access is controlled by policies mirroring AWS' IAM policies. While AWS supports myriad ways to control access, including ACLs, Bucket Policies, etc, in the interest of simplicity, MinIO'

Building a Scalable, Data Sovereign National ID System

Brian Costa Brian Costa on Modern Data Lakes 19 October 2023

Some of the smartest minds in philanthropy are backing the concept of a simple yet powerful national ID system. The Bill and Melinda Gates Foundation, the Tata Trusts, the Omidyar Network and the Pratiksha Trust have all gotten involved with this movement because of its foundational capabilities for enabling a wide range of social programmes. They have put their resources

Percona Streaming Backup

AJ AJ on DevOps 17 October 2023

What is streaming mode? Essentially it allows you to backup with Percona xtraBackup without touching disk. When used alongside MinIO Jumbo, it is designed to upload and retrieve large objects from the MinIO cluster.

Creating an ML Scenario in SAP Data Intelligence Cloud to Read and Model Data in MinIO

Matt Sarrel Matt Sarrel @msarrel

on Modern Data Lakes 17 October 2023

Creating an ML Scenario in SAP Data Intelligence Cloud to Read and Model Data in MinIO

Enterprise customers use MinIO to build data lakehouses to store a wide variety of structured and unstructured data, and work with it using ML and analytics. Data flows into MinIO from across the enterprise and the S3 API allows applications, such as analytics and AI/ML to work with it. I previously blogged about building data pipelines with SAP Data

Simplify Data Pipelines

Satish Ramakrishnan Satish Ramakrishnan on Integrations 12 October 2023

With MinIO, enterprises are not forced to make a choice. They can literally use FTP and SFTP to move that data into an S3-like data store. It is the principle of AND not OR.

Streamlining Data Streaming: A Guide to WarpStream and MinIO

Brenna Buuck Brenna Buuck

on Operator's Guide 12 October 2023

Streamlining Data Streaming: A Guide to WarpStream and MinIO

Explore the next generation of data streaming with WarpStream and MinIO! While Apache Kafka has been the standard for streaming data, it may be time to consider a simpler, more cost-effective, and cloud-native solution.

Snapshot Backups for MongoDB Using MinIO

Brenna Buuck Brenna Buuck

on Databases 9 October 2023

Snapshot Backups for MongoDB Using MinIO

Explore how MongoDB's Ops Manager pairs with MinIO's high-performance object storage, creating a robust backup strategy for safeguarding MongoDB data. Discover the power of this combination and how it can transform your data management strategy.

Build Data Pipelines with SAP Data Intelligence Cloud, SAP HANA Cloud and MinIO

Matt Sarrel Matt Sarrel @msarrel

on Analytics 6 October 2023

Build Data Pipelines with SAP Data Intelligence Cloud, SAP HANA Cloud and MinIO

Tap into unlimited amounts of valuable enterprise data with SAP Cloud and MinIO.

Active-Active Example Using an Email Provider

AJ AJ on Architect's Guide 25 September 2023

Email is the ultimate performance-at-scale use case as it generally only goes up in terms of data volume. Further, the more data that’s stored, the more valuable the data becomes. MinIO’s multi-site active-active replication focuses on keeping the cluster in top performance.

Databases for an Object Storage Centric World

Brenna Buuck Brenna Buuck

on Databases 18 September 2023

Databases for an Object Storage Centric World

Object storage is the primary storage solution for OLAP databases. This survey highlights major database players that have embraced this movement.

Using OpenObserve with MinIO

AJ AJ on Integrations 12 September 2023

OpenObserve is an open-source observability platform designed to streamline the monitoring of logs, metrics, and traces.

The Disruptive Nature of Data Lakehouses

Keith Pijanowski Keith Pijanowski on Apache Iceberg 12 September 2023

Introduction In 1997, Clayton Christensen, in his book The Innovator’s Dilemma, identified a pattern of innovation that tracked the capabilities, cost, and adoption by market segment between an incumbent and a new entrant. He labeled this pattern “Disruptive Innovation.” Not every successful product is disruptive - even if it causes well-established businesses to lose market share or even fail

Building a Data Lakehouse using Apache Iceberg and MinIO

Keith Pijanowski Keith Pijanowski on AI/ML 31 August 2023

Introduction In a previous post, I provided an introduction to Apache Iceberg and showed how it uses MinIO for storage. I also showed how to set up a development machine. To do this, I used Docker Compose to install an Apache Spark container as the processing engine, a REST catalog, and MinIO for storage. I concluded with a very simple

Oracle RMAN to MinIO Backup

AJ AJ on Integrations 29 August 2023

The Oracle Secure Backup (OSB) cloud module allows you to back up your Oracle Database to a MinIO bucket. It leverages RMAN’s encryption to ensure the security of the database backup.

A Developer’s Introduction to Apache Iceberg using MinIO

Keith Pijanowski Keith Pijanowski on AI/ML 24 August 2023

Introduction Open Table Formats (OTFs) are a phenomenon in the data analytics world that has been gaining momentum recently. The promise of OTFs is as a solution that leverages distributed computing and distributed object stores to provide capabilities that exceed what is possible with a Data Warehouse. The open aspect of these formats gives organizations options when it comes to

Storage Infrastructure for Automating Configuration Management with Salt and Puppet

AJ AJ on DevOps 22 August 2023

Globally there has been a shift to bring applications closer to home. Enterprises want more control of their data and have had enough of paying egress fees to the public cloud to get access to their own data. Besides cost, there is also the matter of security, or lack thereof, when resources are shared with unknown organizations. Vulnerabilities can trickle