MinIO private cloud on Pivotal Container Service

Pivotal Container Service (PKS) is the latest offering from Pivotal and VMware, in the enterprise container orchestration space. With PKS, operators can provision, operate and manage enterprise-grade Kubernetes clusters using Pivotal Cloud Foundry (PCF) on their own infrastructure (public or private cloud). Along with Kubernetes provisioning, PKS also offers user and resource management features to allow fine grained control over infrastructure resources.

Minio, the leading container object storage platform fits very well onto the Kubernetes deployment model. You can spin up multiple instances per user, or even create a federated deployment where all the buckets on all Minio instances share a single namespace. Refer to this link for more details on deploying Minio on Kubernetes.

With PKS in the picture providing cluster management and orchestration, it is now straightforward to create multiple Kubernetes clusters and deploy Minio instances as and when needed to scale based on your storage needs.

Admin has access to PCF Ops Manager and they create Kubernetes clusters for departments / users. Each cluster has predefined hardware resources (CPU and Drives) specified in the PKS cluster plan. As a cluster gets created, users can access their cluster using kubectl . This point onwards, the flow is same as other Kubernetes deployments.

In this post, we’ll see how to setup PKS on GCP and then create a new Kubernetes cluster. Finally we’ll see how to deploy Minio on the cluster.

Setup PKS

Detailed documentation is available on Pivotal website about

• Preparing to install PKS on GCP
• Installing and configuring PKS

Once you’re done with above steps, you’ll need to configure PKS API access using the steps mentioned here.

Now that you have PKS users created, you should be able to create your own cluster. However there are a few steps before you can do that.

Create Kubernetes Cluster

PKS needs an external-hostname to create a new Kubernetes cluster. This is the endpoint where Kubernetes cluster can be accessed. The hostname can be an IP or FQDN (fully-qualified domain name).

To set it up on GCP, you’ll need to create a load-balancer based on a public IP. Here is how you do it

1. Login to GCP console, select the relevant project and go to Network Services -> Load Balancing .

2. Click Create Load Balancer , then click the Start Configuration button under TCP Loading Balancing . Retain the default settings on the page that appears.

3. Click Continue , fill in a load balancer name and then click on Backend Configuration . Select the region that you’ve been using until now.

4. Click on the Frontend Configuration and fill in the details like so:

Remember to use Create IP Address under IP field. Then click create to launch the load balancer. Save the public IP address in this load balancer. We’ll use this IP address as the external-hostname .

5. Next, you’ll need to create a firewall. Click on VPC Network -> Firewall rules .

6. Now try creating the Kubernetes cluster. You should be logged in as PKS admin before attempting this. The external-hostname is the public IP address of the load balancer we created in step 4. I chose the plan small based on the plans I created while installing PKS. Make sure to choose the relevant plan, then wait until the cluster gets created. You can check the status using pks cluster <cluster-name> .

7. You now need to update the cluster’s master VM created by PKS with load-balancer details we created in step 4. Go to Compute Engine -> VM Instancesand identify the new VM; it should have a tag job: master and shouldn’t be in use by any load-balancer. Once identified, click on edit button and add a network tag. Tag to be added is the name of load-balancer created in step 4. For example if the load-balancer name is pks-cluster1 , add a tag pks-cluster1 in Network Tags field.

8. Now, go to Network services -> Load balancing . Click the edit button of the load balancer you created earlier. Then click Backend Configuration, Select existing instance and then select the VM instance you identified in the previous step.

9. Finally, confirm if your cluster is available using the command pks clusters . Once confirmed, use the command pks get-credentials <cluster-name> to make sure kubectl is configured to access the cluster <cluster-name> .

You’re now all set to access your Kubernetes cluster.

Deploy Minio on Kubernetes cluster

Once you have kubectl configured to access your cluster, it is simple to deploy Minio server. You can either use the MinIO Operator or our official Helm chart to deploy Minio on your cluster.

Recap

In this post, we learned about Pivotal Container Service deployment and how to use the pks command line tool to create and manage Kubernetes clusters. We also saw how to deploy Minio once your PKS Kubernetes cluster is set up and running.

While you’re at it, help us understand your use case and how we can help you better!

Fill out our “Best of Minio” deployment form (takes less than a minute), and get a chance to be featured on the Minio website and showcase your Minio private cloud design to Minio community.