Using Apache Airflow with MinIO

MinIO is the perfect companion for Airflow because of its industry-leading performance and scalability, which puts every data-intensive workload within reach.
Read more...MinIO is the perfect companion for Airflow because of its industry-leading performance and scalability, which puts every data-intensive workload within reach.
Read more...Together, MinIO and Commvault provide a software-defined and hardware agnostic backup, restore and replication system that is both simple and powerful at the same time.
Read more...While the growth trajectory for data in general is extraordinary, it is the growth of log files that really stand out. As the heartbeat of digital enterprise, these files contain a remarkable amount of intelligence - across a stunning range, from security to customer behavior to operational performance. The growth of log files, however, presents particular challenges for the enterprise.
Read more...In this tutorial, we’ll bake the MinIO binary and the service dependencies, such as the username and group required for MinIO. We are baking this instead of frying because these will remain the same no matter how we configure and launch our image.
Read more...In this post, we will learn the process of gathering metrics from a Python application integrated with OpenTelemetry, storing them in Prometheus, and visualizing the data in a Grafana dashboard. A great tutorial by AJ from MinIO.
Read more...Observability can be roughly divided into three sub-categories: logging, metrics, and tracing. In this blog post we’ll show you how simple it is to get set up with tracing in your new or existing MinIO application.
Read more...MinIO is the perfect companion for Elasticsearch because of its industry-leading performance and scalability. MinIO’s combination of scalability and high-performance puts every data-intensive workload, not just Elasticsearch, within reach.
Read more...We’ll go over how to set up the required infrastructure for integrating GitHub Enterprise packages and actions to use MinIO as a backend. At a high level we’ll need running instances of MinIO and GitHub Enterprise.
Read more...Software-defined MinIO, with its flexibility and rich set of cloud-native integrations, can be deployed in single or multi-tenant modes, and this post is designed to help you determine the appropriate architecture for your deployment.
Read more...Learn how to monitor and alert on metrics using Grafana Mimir and save data to MinIO for persistence.
Read more...Encrypting network traffic is low-hanging fruit when securing IT infrastructure. MinIO follows a pragmatic approach when it comes to TLS. It has to be secure, it has to be performant and it has to be simple. Things that matter In almost all cases, there are just a couple of things we need to take into consideration: * The TLS version. * The
Read more...LXD is a next generation system container and virtual machine manager for Linux systems from Canonical Ltd. LXD lets you manage your containers with a simple command line tool or via a REST API. LXMIN (lex-min) is a simple backup and restore tool for LXD instances (containers or virtual machines) using MinIO object storage. It provides both a command line
Read more...Install and configure Loki, Promtail, Grafana and MinIO to quickly get started working with log data.
Read more...Take advantage of MinIO’s performance, data protection and S3 API functionality for log management and analysis.
Read more...While users take it for granted that infrastructure is available, you know that metrics, monitoring and alerting are critical to operations.
Read more...In 2022, every business must take precautions to protect against the serious threat of ransomware. In 2020 and 2021, we saw an increase in the number and severity of ransomware attacks, and 2022 promises to continue this fearsome trend. According to BlackFog [https://www.blackfog.com/2021-ransomware-attack-report/], in 2021 there was a record total of 292 reported ransomware attacks. Of
Read more...Learn how to secure data in transit, data at-rest and establish role-based access control policies in the first of a series of blog posts about securing MinIO.
Read more...With today’s announcement [https://www.prnewswire.com/news-releases/starburst-announces-new-product-release-which-extends-flexibility-when-building-data-lakehouse-architecture-301425279.html] of Starburst’s support for MinIO, it made sense to revisit the architectural trends that are becoming the standard for analytics workloads. Starburst provides a perfect example as we shall see shortly. The architecture follows the model of disaggregating storage and compute. Modern, high speed networks
Read more...We took MinIO’s partnership with Veeam up a notch today with the addition of Veeam Object Storage with Immutability badge. Already a member of the Veeam-Ready [https://www.veeam.com/ready.html] program, the additional badge puts MinIO in rarefied air - less than a quarter of the object storage vendors have been able to add this distinction. The
Read more...Apache Nifi [https://nifi.apache.org/] is one of the most popular open source data flow engines available today. Nifi supports almost all the major enterprise data systems and allows users to create effective, fast, and scalable information flow systems. Creating data flow systems is simple with Nifi and there is a clear path to add support for systems not
Read more...