MinIO Gateway Migration

We have deprecated MinIO Gateway. The product was initially developed to allow customers to use the S3 API to work with backends, such as NFS, Azure Blob and HDFS, that would not otherwise support it. The MinIO Gateway primarily acted as a translation layer for S3 API between the client and the backend so each backend could be written to in its native format.

MinIO Gateway was never intended to be a permanent solution, instead it was a stop-gap for those who were not on S3-compatible APIs yet so they could easily and reliably transition away from legacy storage solutions. The S3 API is ubiquitous (thanks in part to MinIO Gateway), but if we were to continue developing the MinIO Gateway, we would simply be perpetuating older technologies that are neither high-performance nor cloud-native. Also, addressing the ongoing technical challenges required to maintain MinIO Gateway for each backend are time and resource intensive so it makes much more sense to deprecate it entirely.

How to Migrate

In this blog post, we’ll show you how you can migrate your existing MinIO Gateway setup to a MinIO Server deployment. To avoid confusion between the old and new setups, let’s name them appropriately:

• old_gateway
• new_node

Let’s first go ahead and deploy a new_node single drive instance of MinIO separate from where the old_gateway is installed. Please follow the instructions provided in our earlier blog post, Configuring MinIO with SystemD.

Next let’s install mc admin

$ curl https://dl.min.io/client/mc/release/linux-amd64/mc \
--create-dirs \
-o $HOME/minio-binaries/mc

$ chmod +x $HOME/minio-binaries/mc

$ export PATH=$PATH:$HOME/minio-binaries/

$ mc --help

Set two alias for mc admin for old_gateway and new_node

$ mc alias set old_gateway <OLD_GATEWAY_ADDRESS> <OLD_GATEWAY_USER> <OLD_GATEWAY_PASS>

$ mc alias set new_node <NEW_NODE_ADDRESS> <NEW_NODE_USER> <NEW_NODE_PASS>

Confirm alias has been properly set for new_node

$ mc alias list old_gateway
$ mc alias list new_node

If you are using in fs mode, export and import configuration from old_gateway to new_node

$ mc admin config export old_gateway > config.txt
$ mc admin config import new_node < config.txt

While this will allow you to migrate any setting you set with mc admin config set, it will not migrate environment variables, because those can be set in a variety of ways and there is no automated way to migrate them. Instead, we recommend checking the following two places:

First, grep for any environment variables set on old_gateway and duplicate that environment variable on new_node.

$ env | grep MINIO_

Second, if old_gateway is a systemd install, check the following file and migrate its settings to new_node in the same location.

$ cat /etc/default/minio

Restart MinIO on new_node after importing old_gateway’s config, env vars and defaults:

$ systemctl restart minio

If you are using in fs mode, export bucket metadata settings from old_gateway to new_node

$ mc admin cluster bucket export old_gateway

The above command will generate a cluster-metadata.zip file. Import metadata settings form this file

$ mc admin cluster bucket import new_node cluster-metadata.zip

If you are using in fs mode, export IAM settings from old_gateway to new_node

$ mc admin cluster iam export old_gateway

The above command will create a file called old_gateway-iam-info.zip. Import IAM settings from this file

$ mc admin cluster iam import new_node old_gateway-iam-info.zip

Last but not least, let's migrate the actual data from old_gateway to new_node. Stop any further writes on old_gateway to avoid sync issues before proceeding to the next step.

Create a bucket on new_node called new-bucket

$ mc mb new_node/new-bucket

Use the mc mirror command to migrate data, assuming the old bucket’s name is old-bucket (replace if this is not the case)

$ mc mirror --preserve --watch old_gateway/old-bucket new_node/new-bucket

Depending on the amount of data to mirror, this can take time; please wait until it completes.

Once mirroring is complete, you are ready to use your new MinIO deployment. Be sure to verify that you have successfully duplicated the environmental variables, configurations and other settings from old_gateway.

Once you’ve verified the new_node is operating like the old_gateway, you can update your applications that we asked earlier to stop writing to old_gateway and can now start writing to the new_node. This can also include cleaning up the resources of the old_gateway setup and any remaining backends.

MinIO Gateway - Going, Going, Gone!

As you can see, migrating from a MinIO Gateway to a MinIO Server running as a single node or set of distributed nodes is straightforward. In fact, it’s so fast and easy that there is no reason not to migrate off of Gateway, which has already been removed from the MinIO codebase as of the latest release. You can check the documentation for further information on migration.

If you have any questions on the MinIO Gateway migration be sure to reach out to us on Slack!