The Long Term Costs of Object Storage in the Public Cloud

The promise/allure of the public cloud is based on the concept that it is elastic. One can, with little effort, scale up workloads and, if desired, scale down those same workloads. We have written on this subject before - from the perspective of what workloads to consider as you evaluate what to take to the public cloud and what to repatriate to the private cloud.

We have not, however, taken a hard look at the costs associated with the two options. Our position can be related as an analogy. The public cloud is like a nice hotel. Plenty of amenities, secure, spacious etc. It is priced like a nice hotel too. As a result, people don’t live in nice hotels - they stay there for a period of time to achieve a certain objective (business trip, vacation) because it gets too expensive otherwise.

We wanted to quantify this objectively so we rolled up our sleeves, made some reasonable assumptions and wrote this post. Here we go:

If we assume that we will be using object storage in the cloud and want to understand the costs of deploying a PB of data (200TB active, 800TB inactive, with a Read:Write ratio of 80:20) we can compute what this storage would cost on various public clouds.

AWS S3 storage tiers costs are as follows (See https://aws.amazon.com/s3/pricing/). This should not be a surprise as prices have not changed in the last four years (but costs have :).

• S3 Standard storage the first 50TB costs $0.023 per month per GB, the next 450TB costs $0.22 per month per GB, and all storage over 500TB costs $0.021 per month per GB. So 1PB of S3 Standard storage would cost $1.2K/month for the 1st 50TB, $9.9K/month for the next 450 TB and $10.3K/month for the next 500TB or $21.6K/month for 1PB of data.
• S3 Intelligent Tiering storage costs the same as S3 Standard for frequently active data and $0.0125 for infrequently accessed data plus an $0.0025 per 1000 Objects fee for management. So assuming 200TB:800TB active:inactive the active data would cost $1.2K/month for 50TB and $3.3K/month for the 150TB and $10.0K for the inactive 800TB. To that, we must add object management costs. It’s not unusual to see PB data warehouse with billions of objects/files. So if we assume 1B objects (average ~1MB/object) in our PB of data object management costs would be $2.5K per month. Totaling all that together we have an overall storage and object management costs of ~$17.0K per month for our 1PB of data.
  

It is true one could do a blend of standard S3 plus S3 Glacier or S3 Glacier Deep Archive, but that doesn’t really get to the apples to apples comparison we are seeking. So for the purposes of this we will consider the first two.

Although Azure and GCP don’t have exactly equivalent tiers of storage, if we just focus on S3 Standard equivalents then Azure Blob and GCP cloud storage costs are:

• Azure (Hot) Blob storage costs a flat $0.0184 per GB per month or $18.4K per 1PB per month, or about 85% of AWS S3 standard costs.
• GCP us-central cloud storage costs are $0.020 per GB per month or $20K for 1PB per month or about 93% of what AWS S3 standard costs.
  

There are some additional costs that are associated with per 1,000 S3 operations. But we estimate in the scheme of things they don’t add more than $300/month to the above costs. As such we will ignore these costs here.

What if we wanted to copy the data out of the cloud?

Of course none of the above cloud storage costs takes into consideration any egress charges which for AWS and GCP are $0.09 to $0.05/GB per month and $0.12/GB, respectively. So if you wanted to move the active 200TB of your 1PB of data out of the (AWS or GCP) cloud each month you would need to add it would cost you another ~$14K/month (on average) with AWS S3 and $24K/month with GCP.

For Azure they don’t appear to have a standard egress charge but rather charge per operation and bandwidth used. We would guess (although we haven’t verified this) that the costs would be comparable to AWS S3 standard egress charges.

Of course there would be no direct charge to move the 1PB of data center data around. You would incur bandwidth costs depending on where you moved it. But the server costs are already accounted for in the costs above.

The Private Cloud/OnPrem Equivalents: Software

First off, you need equivalent software. MinIO offers that (and more) with its S3 compatible, feature rich object storage suite. It is literally a private cloud, drop-in equivalent for AWS. While MinIO is open source, it does offer two tiers for the MinIO Subscription Network. SUBNET, as we call it, combines a commercial license with 24/7/365 direct-to-engineer support, security and resilience audits and other diagnostic technologies that effectively insure production deployments of our software.

The Standard tier is priced at .01 per GB per month and the Enterprise tier is priced at .02 per GB per month.

For our PB of data, that equates to 10K and 20K per month respectively. Needless to say, there is the open source option as well, which would be appropriate if your data was not mission critical. That cost is zero.

There are no egress costs. There are no per object management costs.

For the purposes of this exercise let’s choose the middle - the Standard plan at $10K per month for software costs.

The Private Cloud/OnPrem Equivalents: Hardware

While MinIO can run on a range of hardware from Raspberry Pis to IBM Power, we wanted to target dense JBODs for the purposes of our analysis.

It just so happens that through our partnership with Seagate, we have publishable pricing for a 1TB configuration of the Exos AP 2U12 (Dual AP) with two Intel Silver Xeon CPUs. It has

60 drives at 16 TB per drive delivering .96 PB raw capacity and .72 actual capacity. This assumes erasure coding factor of .75. The price for that hardware is a very reasonable $70K. Let’s choose a three year amortization schedule on that hardware to determine a monthly per GB cost. At 36 months our monthly cost for the hardware works out to $1,510 per month or $0.0015 GB/month.

Interested in more? Jump over to the Reference Hardware page to play with the calculator to see other capacities.

Don’t Forget the Data Center Costs

For a data center deployment, we would need to add administration, rack, space, power and cooling costs. We could go into great detail here to determine each of these costs but in general according to the US Chamber of Commerce data center space can be had for about $1305/NRSF (net rentable square feet) in CapEx and another $112/NRSF ( ~8.6% of CapEx) in OpEx or a total of ~$1.4K/NRSF annual cost or an extra $116 per month.

Totaling Up the Private Cloud

The cost to run your own, 1 PB private cloud, with state of the art hardware, 24/7 direct-to-engineer support, panic button access and annual performance reviews is $11,510 per month. Let’s circle back and compare that to what we calculated above.

Summarizing the One Year Costs

At $11,510 MinIO and Seagate represent the best economics - by a considerable amount. The combination is 47% less expensive per month than standard S3 and 33% less than S3 with Intelligent Tiering.

These calculations ignore AWS egress costs - which would make MinIO and Seagate less than half as expensive as Standard S3.

MinIO and Seagate are also 38% less than Azure’s comparable option and 43% less than Google Cloud Platform’s comparable offering.

The breakeven point for building your own private cloud vs. S3 Standard comes in at seven months. Thought of another way - for the price of 1PB on S3 Standard you could have almost 1.5 PBs of MinIO and Seagate.

The overall point is not to compete on price.

We think it is a poor metric when thinking about object storage. What we seek to detail here, is that, over time, you will get better performance, superior security, more control and additional flexibility by going on prem - without sacrificing anything on cost (indeed you gain economic advantage). That is why many innovative enterprises are engaged in large scale repatriation strategies, because they realize the cloud is here to stay and they have a choice about what cloud that is - and they are picking the private cloud.

Feel free to disagree with us. You can reach out at hello@min.io.

Feel free also to put us to the test.  You can contact a Seagate rep from the calculator and download our software here. If you need a little help, join the 9,700 members of the Slack channel.