Every system needs to be backed up because there are countless ways to lose local filesystem data and configurations. That loss can be devastating – potentially resulting in revenue loss, dissatisfied customers and even costly litigation. The statistics are pretty bleak – sixty percent of businesses that suffer a data loss event close within six months and ninety-three percent of companies that lose their data centers for more than ten days file for bankruptcy within a year.
One staggering statistic is that ninety-six percent of companies do not back up user workstations. This blog post shows you how to back up quickly, easily and unobtrusively using the combination of Restic and MinIO.
High-performance MinIO supports every workload you could think of running on it. MinIO is the standard for building on-premise data lakehouses and exposing them to AI/ML and analytics workloads. Because of its industry-leading performance, scalability, S3-API compatibility and durability, MinIO is frequently used as a backup target. Enterprises have left traditional tape backups behind and rely on MinIO for fast backups and restores. No more managing, rotating and searching for tape cartridges – MinIO houses all your primary data and backups, accelerating backup and restore times while eliminating backup corruption.
MinIO not only accelerates backups, it simplifies backups and that goes a long way towards increasing automation and reducing complexity when compared to tape. It's easy to predict and control workloads with capacity-based pricing and the ability to scale non-disruptively without limit – while also providing enterprise features like cloud-native security and access control and active-active replication. Data stored in MinIO is protected as MinIO encrypts objects at the storage layer with Server-Side Encryption (SSE). MinIO does this with extreme efficiency – benchmarks show that MinIO is capable of encrypting/decrypting at close to wire speed.
Restic is a modern backup program that can back up from Linux, BSD, Mac and Microsoft Windows to a number of different storage types, including S3, on-premise and in the cloud. Restic pairs well with MinIO because it is open source, straightforward to deploy and use, easily automatable and both are written in speedy Go.
Read on to learn more about backing up servers and workstations with Restic and using MinIO as a backup target.
How to Back Up with Restic to MinIO
This tutorial will show you how to use Restic to back up data to MinIO. This is an easy-to-use and powerful way to back up enterprise data. Restic has the same functionality for MinIO as it does for Amazon S3, plus MinIO has the added value of also serving as an SFTP server for Restic and more. We will need to install MinIO and record the S3-compatible account credentials. Then we'll install Restic, configure MinIO as a new repository and make a backup.
MinIO Client (mc) is required to access the MinIO Server. Here’s how to install mc locally.
Make note of the credentials and S3 endpoint displayed the first time you run MinIO Server because you will need them to configure your Restic repository.
Like MinIO, Restic is straightforward to install.
$ sudo apt update $ sudo apt install restic
Update Restic prior to use
$ sudo restic self-update … successfully updated restic to version 0.16.2
Configure Restic for Your MinIO Repository
Restic saves backups to local or remote repositories. We're going to use MinIO as an S3-compatible repository.
Let's create the MinIO repository. Set the variables for authentication that we obtained when installing MinIO.
$ export AWS_ACCESS_KEY_ID=<your-minio-access-key> $ export AWS_SECRET_ACCESS_KEY=<your-minio-secret-access-key>
Initialize your Restic repository
$ restic -r s3:http://your-minio-ip-address:9000/restic init
To make automated backups, you can set the repository using the
RESTIC_REPOSITORY environment variable or specify the repository using the flag
You can put all the environment variables in a file
$ vim ~/.restic-env
Add the following variables:
export AWS_ACCESS_KEY_ID=<your-minio-access-key> export AWS_SECRET_ACCESS_KEY=<your-minio-secret-access-key> export RESTIC_REPOSITORY="s3:http://your-minio-ip-address:9000/restic" export RESTIC_PASSWORD="your-strong-repository-password"
Save the file and then source it
$ source ~/.restic-env
Finally, verify the repository
$ echo $RESTIC_REPOSITORY s3:http://your-minio-ip-address:9000/restic
Now you can easily initialize the repository with
$ restic init
You now have a
restic bucket on your MinIO instance.
After the repository is initialized, we're ready to make a backup. Find a local directory with data files to back up. In my case, I'm using the
~/openlake directory on my workstation as an example.
## not using restic-env file $ restic -r s3:http://10.0.0.10:9000/restic --verbose backup ~/openlake ## using restic-env file $ restic --verbose backup ~/openlake open repository repository 914b1ddc opened (version 2, compression level auto) … processed 97 files, 111.876 MiB in 0:02 snapshot 1c334016 saved
That was fast!
In Restic backups are called snapshots. You can list all of your snapshots with
$ restic snapshots repository 914b1ddc opened (version 2, compression level auto) ID Time Host Tags Paths ----------------------------------------------------------------------------- 1c334016 2023-11-16 15:10:48 MDS-MINIO /home/msarrel/openlake ----------------------------------------------------------------------------- 1 snapshots
We see that the
restic bucket contents have been updated:
Our recent backup is in the
snapshots bucket. You can see the object name is the same as the snapshot ID.
Make another backup of the same directory on your local machine as before.
You now have 2 snapshots of the same data in the repository.
diff command to see the difference between two snapshots.
$ restic diff 1c334016 d7044fee repository 914b1ddc opened (version 2, compression level auto) comparing snapshot 1c334016 to d7044fee: [0:00] 100.00% 1 / 1 index files loaded Files: 0 new, 0 removed, 0 changed Dirs: 0 new, 0 removed Others: 0 new, 0 removed Data Blobs: 0 new, 0 removed Tree Blobs: 0 new, 0 removed Added: 0 B Removed: 0 B
Hopefully, you won't need to use this command very much, but if you do you'll be happy that it is highly configurable.
To restore a specific snapshot to a specific target:
$ restic restore 1c334016 --target /tmp/restore-openlake repository 914b1ddc opened (version 2, compression level auto) restoring <Snapshot 1c334016 of [/home/msarrel/openlake] at 2023-11-16 15:10:48.12843098 -0800 PST by msarrel@MDS-MINIO> to /tmp/restore-openlake Summary: Restored 134 files/dirs (111.876 MiB) in 0:00
We can verify this by listing the contents of the target directory.
To restore from the most recent snapshot:
$ restic restore latest --target /tmp/restore-new-openlake repository 914b1ddc opened (version 2, compression level auto) restoring <Snapshot d7044fee of [/home/msarrel/openlake] at 2023-11-16 15:24:38.906919692 -0800 PST by msarrel@MDS-MINIO> to /tmp/restore-new-openlake Summary: Restored 134 files/dirs (111.876 MiB) in 0:00
We're big fans of automation at MinIO, and Restic backups can easily be automated as
cron jobs. We already took the first step when creating the
restic-env file to store variables.
$ crontab -e
Enter the following, customizing for your environment:
30 * * * * ./home/msarrel/.restic-env; restic backup -q /home/msarrel/openlake; restic forget -q --keep-last 2 --prune
- 30 * * * * runs the backup every thirty minutes of every hour, day, month and day of the week
- ./home/msarrel/.restic-env; is where environment variables are declared
- restic backup -q /home/msarrel/openlake; is the backup command
- restic forget -q --keep-last 2 --prune used to maintain the running archive of snapshots. Here we're pruning every snapshot except the last two.
Simple Speedy Backup with Restic and MinIO
This blog post showed you how to use Restic to back up a Linux system to MinIO using the S3 API. We only walked through some basic commands as a demonstration. Please see Introduction — restic 0.16.2 documentation for more details, including encryption, tuning backup parameters and setting restore options.
The combination of Restic and MinIO yields a fast, efficient, secure and automatable backup solution. With a little scripting, you can easily protect data from workstations and servers across your organization.
How and what are you backing up to MinIO? Join our community slack to share tips and tricks for making the most of your MinIO deployment.