Using MinIO for AWS Local Zones

This post has been updated at AWS's request to remove the "AWS recommends" language.

Earlier this year, Amazon Web Services announced it was expanding a relatively new product called Local Zones.

AWS Local Zones recreate much of AWS (compute, storage, database, and other select services) closer to large population, industry, and IT centers. This improves performance, reduces latency, and cuts down on bandwidth costs. The expansion added another 30 or so cities to the 16 US cities that were already in play. The use cases vary, however, for high throughput, low-latency applications like streaming, simulation, video games, AI/ML; these local zones push deeper into the “on prem” wallet share. There are also data locality regulations but this is not the primary driver.

Why are we writing about an AWS product? Well that’s an interesting question….

It turns out that of the catalog of services currently offered by AWS Local Zones, S3 isn’t one of them. While they may (okay, probably will) fix this over time — they have written a tutorial outlining how you can run MinIO on EC2 and EBS in place of S3.

Let’s repeat that: if object storage is a requirement for your Local Zone instance, MinIO is going to be a great option.

To say we are flattered would be an understatement. We have tremendous admiration for the team at AWS, what they have built, how they have changed the industry, how they made object storage primary storage and how they never stop thinking about the customer. Because of this customer obsession, they are comfortable with customers using MinIO. It only increases our esteem. Other companies would not have said anything.

The case for MinIO is laid out in a superb blog post by Leonardo Solano. Leonardo is a Senior Hybrid Cloud Solution Architect with the Cloud Acceleration Team in Latin America (Bogota). The post is in Spanish so we will pluck out the high points.

Leonardo lays out the case (courtesy of Google Translate) at the beginning:

“S3 object storage is natively located in AWS Regions, but not in Local Zones (at the time of publishing this blogpost). For use in Local Zones, it is possible to use third-party software solutions available on the AWS Marketplace, such as the one presented in this blog post. Hybrid Cloud Object Storage is a high-performance distributed object storage solution that can be deployed on EC2 instances with EBS volumes and is directly maintained and supported by MinIO.”

Given MinIO is the only object store in the AWS Marketplace (and Google Cloud and Azure), it makes sense that the team would use us for this tutorial.

Here is the architecture diagram:

Solano goes on to describe the environment:

“In this architecture, the Distributed Mode of MinIO will be deployed on EC2 instances, within an AWS Local Zone. This architecture will have 2 nodes in the MinIO cluster and each node will have 4 GP2 type 8GB EBS volumes…In order to prepare the environment, it is essential to have a VPC with Subnets extended to a Local Zone. For this example, there is a VPC in the region on which the Local Zone depends with a CIDR of 10.0.0.0/16.”

Since it would not make sense to simply translate the entire post (although for those deeply interested it is actually quite easy given all the screenshots are in English), the summary of what he details is as follows:

1. Create two EC2 instances of type t3.medium in the AWS console with at least 4 EBS volumes in addition to the boot volume.
2. Access each of the instances via SSH and modify the /etc/hosts file to add the IP addresses of each host in the cluster (MinIO requirement).
3. Creating the mount directories for each of the drives to be used by MinIO to store the objects, format them, mount the drives, and grant read and write permissions on the mount folders.
4. Start the MinIO service on the two nodes using parameters described in the post.
5. Configure an Application Load Balancer or ALB and create a target group to allow front-end balancing. Leonardo also details how to create another target-group for the MinIO API access.
6. How to use object storage via the web console and via the AWS CLI for S3, to verify the solution works.
   

Everything the developer needs to get this going is in the post.

On a larger note, we think this is very exciting and want to encourage our joint customers to jump in here. This extends a theme we have written about extensively — that of MinIO everywhere — any cloud, any where. AWS local zones effectively expand the reach of the cloud — closer to the edge. Some of the customers quoted in the launch PR include Netflix, SuperCell, FOX, Edgegap, Ubitus, JamKazam, Masomo and Couchbase — all of which can take advantage of MinIO for their object storage needs.

Again, we are deeply flattered that AWS thinks this highly of us. If you have questions, don’t hesitate to ask them on general Slack. If you become a marketplace customer, those will be answered directly by our engineers under a strict SLA through the SUBNET portal.