MinIO as Helm Chart Repository

If you are part of a team running infrastructure whether it is DevOps, SRE or Systems Engineer its paramount to ensure you keep tech debt to a minimum. In this case you want to ensure the number of supporting systems in your infrastructure such as Databases, Cache Systems, Messaging Queues, Log Aggregators, Monitoring Systems, Application Performance Monitoring systems and I’m sure I’m missing a few more here do not add to the overall complexity of managing the infrastructure.

As an Engineer you always want to ensure the infrastructure you are setting up is something that could be used by multiple teams for multiple applications and projects. If you are picking a database do your due diligence and pick one that could serve most, if not all, of your applications. Having too many of these disparate database servers running alongside each other will add to the complexity of installing them, updating, maintenance, backups, testing the backups, monitoring, among other things that really takes up a lot of the resources.

Same is true with storage systems. One of things you want to make sure is that the storage systems for backups and DR are not used just for those purposes but the other way around. Meaning, you should have storage systems that support object storage, DB external tables storage, Metrics and Logs Storage, Configuration Management data, AI/ML data in Data Lakes, Big Data such as Spark among countless other use cases. This way you have a storage infrastructure that is not only resilient, but also scalable because there are many different teams and applications that are depending on Storage more than ever. While the overhead of managing myriad pieces of infrastructure is now minimized, it becomes critical to ensure the underlying infrastructure powering them is also scalable, reliable and performant. MinIO fits the bill for more than several of these use cases because of its industry-leading performance and scalability. MinIO is capable of tremendous performance – we’ve benchmarked it at 325 GiB/s (349 GB/s) on GETs and 165 GiB/s (177 GB/s) on PUTs with just 32 nodes of off-the-shelf NVMe SSDs – and is used to build data lakes/lake houses and analytics and AI/ML workloads. With MinIO playing a critical role in storage infrastructure, it's important to collect, monitor and analyze performance and usage metrics. 

MinIO is also a major proponent of Kubernetes, because of its open source nature and the vast resources available in the way applications are deployed. Just as MinIO can be deployed anywhere, it was a no-brainer why we wanted to have more ways to deploy MinIO via containers. From the get go MinIO supported the Docker way of deployment which is the easiest way to get up and running. You might also know we maintain and distribute our own Helm Chart to deploy the MinIO Operator and other components. This helps us deploy MinIO alongside any Kubernetes Application whether they use Helm or Not.

But did you wonder how those charts actually get deployed? Today we’ll show you how you can use your MinIO cluster (on-prem, in the cloud, or on Kubernetes) to be used as a Helm Chart repository.

Install MinIO

Before we set up and configure the Helm repository please ensure you have a working MinIO cluster which you are already using for existing data. If you just want to test this out first please follow the instructions below to spin up a MinIO container quickly.

We’ll bring up a MinIO node with 4 disks. MinIO runs anywhere - physical, virtual or containers -  and in this overview, we will use containers created using Docker.

For the 4 disks, create directories on the host for minio:

Launch the Docker container with the following specifications for the MinIO node:

The above will launch a MinIO service in Docker with the console port listening on 20091 on the host. It will also mount the local directories we created as volumes in the container and this is where MinIO will store its data. You can access your MinIO service via http://localhost:20091.

If you see 4 Online that means you’ve successfully set up the MinIO node with 4 drives.

Go to the browser to load the MinIO console using http://localhost:20091, log in using minioadmin and minioadmin for username and password respectively. Click on the Create Bucket button and create testbucket123.

Installing Helm

Another similarity Helm shares with MinIO is its single binary. You don’t need to install multiple files or dependencies in different locations dealing with path issues. You simple run the command below and you’ll be able to get going

$ curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3
$ chmod 700 get_helm.sh
$ ./get_helm.sh

Run Helm repo update to make sure all is working well

helm repo update

Configuring MinIO with Helm

In order to set up Helm as a repository in MinIO, we’ll need to create a separate bucket. You can name this as helm-repo or use the testbucket123 we used earlier. Ensure this bucket is public and accessible over http/https without any API or libraries.

In this bucket we’ll need to create an Index.yaml. Think of it as the index.html on a website. It is the first page that the browser looks for when accessing a website. Similarly, when the helm command is run it will look for the index.yaml at the root of the MinIO bucket which will have a list of packages that are available in that bucket which Helm can use. Think of it as a combination of index.html and the contents of a Site Map page so you would know exactly where you want to go all the time. This is the same way Helm package manager uses the index.yaml in the MinIO bucket.

Now let's go ahead and set up the repo.

Create an index.yaml with the following contents

apiVersion: v1
entries:
  minio:
  - apiVersion: v1
    created: 2017-06-15T17:48:36.895822482Z
    description: Distributed object storage server built for cloud applications and
      devops.
    digest: 75ff1e3d779d8937cff57c28a102da97a520245d50e22c1a2763cbea064a76cd
    home: https://minio.io
    icon: https://www.minio.io/img/logo_160x160.png
    keywords:
    - storage
    - object-storage
    - S3
    maintainers:
    - email: hello@acale.ph
      name: Acaleph
    - email: hello@minio.io
      name: Minio
    name: minio
    sources:
    - https://github.com/minio/minio
    urls:
    - https://play.minio.io:9000/minio-helm/minio-0.1.2.tgz
    version: 0.1.2

Upload the index.yaml to the root of the bucket

mc cp ./index.yaml myminio/testbucket123

Upload the chart’s tarball to the bucket root as well

mc cp minio-0.1.2.tgz myminio/testbucket123

Add the MinIO bucket public http endpoint 

helm repo add myrepo http://localhost:20091/testbucket123

The moment of truth! Try install the Helm package from the MinIO bucket you added as a Helm repo

helm install myrepo/minio

Single Pane of Glass

This term is generally used when describing monitoring. But it can also be used to explain the state of your infrastructure. By having yet another useful way to use your MinIO clusters, you have automatically reduced the tech debt of your team and the larger organization. By using MinIO you can also leverage our SUBNET support portal where a team of engineers who actually write the core codebase of MinIO are available to help you tackle some of the cluster’s important tasks such as architecting the initial set up, developing a DR plan and scaling the cluster as the need for more data, applications and use cases increases.

For more information, ask our experts using the live chat at the bottom right of the blog to learn more about the SUBNET experience or email us at hello@min.io.