Fast, Scalable and Immutable Object Storage for Commvault

Matt Sarrel -

As a foundational technology of the cloud, object storage is gaining popularity as a backup target because it is reliable, scalable and immutable. Object storage enables cloud providers and enterprises to scale up in a distributed manner using commodity hardware. All major backup software providers have embraced object storage because it enables cost-effective and reliable backup storage that scales to meet the demands of today’s data-intensive environments while also meeting data security and protection requirements. NAS, SAN and other backup appliances cannot provide the same high availability, high durability and storage efficiency for backups that object storage can.

We recently teamed up with Commvault to demonstrate that MinIO and Commvault together are a fast, secure and scalable backup, restore and replication solution. MinIO is fully compatible with Commvault as a backup target, enabling backup, restore and replication of Microsoft Office 365, Sharepoint, Outlook and Exchange, Oracle databases, SAP HANA, VMware ESXi VMs, and Salesforce – just to name a few.

As the highest performing object storage on the planet (349 GB/s GET and 177 GB/s PUT on 32 nodes of NVMe), MinIO propels Commvault to previously inconceivable backup and restore speeds. Decreasing backup and restore windows increases productivity and simplifies operations – both are wins in today’s 24/7 enterprise.

MinIO writes metadata atomically along with the object data, eliminating the need for an external metadata database. Backups are written to MinIO as objects and are therefore immutable. All objects written to MinIO are protected with inline erasure-code, bitrot hashing and encryption.

As you can see, MinIO and Commvault provide all the speed and scale you’ll ever need for enterprise backup. What follows is a functional walkthrough of setting up MinIO and Commvault together so you can experience it yourself.

How to Configure Commvault to Back up MinIO Object Storage

This tutorial will show you how to use Commvault to back up data stored in MinIO. Commvault has the same functionality for MinIO as it does for Amazon S3 (Simple Storage Service). We will need to install MinIO, create at least one bucket and record the S3 compatible account credentials whose buckets will be backed up using Commvault. We also require a Linux or Windows machine to act as the access node (proxy) through which Commvault communicates to the S3 Interface and backs up the bucket data.

Prerequisites:

Download and install MinIO. Record the access key and secret key.

Download and move MinIO binary to an executable location

wget https://dl.min.io/server/minio/release/linux-amd64/archive/minio_20221126224332.0.0_amd64.deb -O minio.deb
sudo dpkg -i minio.deb

Download and install MinIO Client.

Install the MinIO client as well

wget https://dl.min.io/client/mc/release/linux-amd64/mc
chmod +x mc
mv mc /usr/local/bin/mc

You must already have the Commvault Command Center installed. For reference, please consult the Quick Start Guide to get started, and make sure you size your physical server or VM for the “All in One” configuration described in System Requirements.

You will also need to install the Cloud Apps agent on the server. From Command Center, navigate to Storage -> Servers -> then click the actions button (3 dots on right) and choose Add software

Choose the Cloud Apps agent, hit OK and then Install

Wait for the install job to complete. You can check status under Jobs.

The next step is to set up object storage backup. Go to Guided Setup -> Protect -> More (on the lower right or your screen)

Then click on Object storage. If the Object storage option isn’t available, please enable it to be visible.

Once the Object storage screen opens, click Add object storage on the top right of the list.

Configure the Add Object Storage screen for Amazon S3, and use Access and Secret Keys for authorization. Add the Host URL as the storage endpoint of the MinIO server that will be your backup source. For Access Node, the server where you installed the Cloud Apps agent should be listed in the dropdown.

This will create the object storage client within the default content group, which will back up all buckets. After creating and configuring the S3 object storage client, you can create content groups with bucket-specific rules for greater customization.

To create a new content group, go to the Commvault Command Center and from the navigation pane on the left, select Protect > Object storage. When the Object storage page appears, click the object storage repository that you want to add content to, then click Configuration, and then Add content group on the upper right.

Then you can create custom inclusion rules as needed for objects and buckets. In the Add content group dialog box, type a name for the content group in the Name box. Then, from the backup Plan list, select a backup plan to use for this content group. Finally, in the Backup Content path box, type a path or browse to the content that you want to include in the content group. Click Save.

How to Back Up MinIO Object Storage with Commvault

Note: remember to add or change data in the bucket between incremental backups or there wont be data to backup.

You will want to use a decent amount of data to get a good test, and have at least a few GB changed between incremental backups.

The Commvault S3 agent allows you to run full, incremental and synthetic full backup operations for the default and user-defined content groups.

Let’s verify that you can back up data to your preferred type of media.

Full and Incremental Backup

From Commvault Command Center, navigate to Object storage and select the MinIO endpoint you wish to back up.

Then, at the bottom of the screen, right click the content group you want to back up, then select Backup. At the next screen, Select backup level, choose the Full or Incremental backup option as you desire.

You’ll see a message that the backup has started and a Job ID has been assigned. Click the Job ID to monitor the job progress.

Multi Stream Backup

You can improve performance by using more parallel streams for backup operations. You want a minimum of two streams, probably between five and 10 for better performance. Navigate to Protect -> Object Storage -> Object Storage Client -> Content Group Detail Page. In the General box, you’ll find a setting for Number of data backup streams that you can increase.

Verify Backups and Restore Data

Making backups is only half the battle, verifying backups and restoring data is the other half.

You have three options for restoring the backup we just created:

Let’s walk through restoring data backed up from one MinIO deployment to another.

From the navigation pane in Commvault Command Center, go to Protect -> Object storage. Right-click on the object storage repository that you want to restore, then click Restore.

From the Backup content page, select the backup(s) you want to restore, then click Restore. You can drill-down to select previous backups.

Specify the following configuration information when the Restore options dialog box appears:

On the Out of place tab, specify the following information:

  • Cloud target: Type the path to the target or select a previously entered target from the drop-down list.
  • No of streams: Type the number of streams to use for the restore operation. For this exercise, the default (10) will be fine.
  • Destination path: Type the full restore location path, or select the path by clicking Browse.
  • Restore ACLs: Select to restore the ACLs (optional).

Then, select one of the following:

  • Overwrite files unconditionally
  • Overwrite files only if the backed up file is newer

Then click submit to run the restore job.

You can verify that the job is running by using the navigation pane to click Jobs -> Active jobs.

Commvault and MinIO for Fast, Flexible Backup and Restore

Performance is critical when it comes to backup and restore operations. Commvault, with the ability to backup to and from object storage, benefits when paired with scalable, speedy and straightforward MinIO.

Together, MinIO and Commvault provide a software-defined and hardware agnostic backup, restore and replication system that is both simple and powerful at the same time. Simple in terms of architecture and operation, yet powerful, performant and scalable enough to meet even the most stringent backup requirements. The result is that Commvault customers using MinIO as a backup target achieve enormous savings and total flexibility to meet their backup requirements.

Commvault users should try out this architecture – it will modernize their application stack, improve performance, deliver greater scale and improve the bottom line on backup.

Download MinIO and take it for a test drive. Reach out to us on Slack with questions or comments.