What it means to provide true cloud native storage

For Cloud Native, Architecture Matters

The term ‘cloud native’ is widely used in technical circles but doesn’t have a particularly clear definition. The confusion lies in the fact that being ‘cloud native’ has little to do with the environment your application is deployed to—the term is equally applicable to on-premise or the public cloud. Rather, the term refers to application and architecture characteristics.

Evidence can be found in the Cloud Native Computing Foundation’s most recent definition of the term:

Cloud native technologies empower organizations to build and run scalable applications in modern, dynamic environments such as public, private, and hybrid clouds. Containers, service meshes, microservices, immutable infrastructure, and declarative APIs exemplify this approach.

These techniques enable loosely coupled systems that are resilient, manageable, and observable. Combined with robust automation, they allow engineers to make high-impact changes frequently and predictably with minimal toil.

What is Cloud Native Storage?

So what does it mean for storage to be cloud native? In a nutshell, it has to behave in the same dynamic, API-driven way as everything else in the cloud native ecosystem. The two storage-specific questions that determine a storage system's Cloud Native credibility. They are:

It is purpose-built for Kubernetes
It is compatible with Amazon’s S3 API

Let’s take a look at these criteria in more detail.

Cloud Native Storage is Object Storage

Modern application architecture is based on object storage, and S3 is, by default, the API language of the cloud. Because object storage is the only kind of storage that is designed to handle the amount of data the cloud native applications generate, at a price companies can afford and at the speed users expect - it dominates modern storage. Other attributes of object storage include a superior approach to distribution, better resiliency and higher availability than competing block or file storage.

These are the attributes expected by cloud native applications.

The Amazon S3 API is the defacto standard for object storage to the point where every object storage software vendor claims compatibility. That said, AWS S3 compatibility is effectively binary. It works in all cases or it doesn’t.

That means is that there are hundreds, perhaps thousands of corner cases where what you expect to happen does not. This is an area where Open Source software has a significant advantage. Given the size and diversity of the applications, operating systems and hardware architecture they have seen most of corner cases.

As an application creator this matters. You will need to test your applications against those vendors. Open Source makes it easy to assess vendor claims and determine what platform performs against your applications. If it passes, it is likely cloud native. If it doesn’t, it isn’t.

Cloud Native Storage: Kubernetes Native

The second criteria for being ‘cloud native’ is to use a distributed architecture that leverages an external container orchestration platform. This means being Kubernetes native is a defining attribute of cloud native storage. Kubernetes is already the clear industry winner when it comes to container orchestration, so being built to work with Kubernetes is essential for a storage solution to be considered cloud native.

What exactly does it mean for a storage to be Kubernetes native? We think there are six main criteria.

1. Let Kubernetes orchestrate storage nodes

Kubernetes is a powerful orchestrator, and can be leveraged to handle both compute and storage orchestration. A truly cloud native storage option like MinIO integrates with Kubernetes, allowing operators to manage storage with the Kubernetes interface and Kubernetes to handle everything from provisioning to volume placement.

2. Highly multi-tenant

Multi-tenancy allows multiple customers to use a single instance of an application, and when implemented correctly can reduce operational overhead, decrease costs and reduce complexity, especially at scale. However, it also requires strict resource isolation, so that multiple users can access either compute or storage resources without impacting other users. A truly cloud native storage solution will provide sufficient resource isolation to make multi-tenant architecture secure, highly available and performant.

What this means in the object storage world is that the Kubernetes infrastructure needs to isolate and manage storage tenants. If Kubernetes isn’t managing the underlying infrastructure then it is not truly cloud native. This disqualifies those appliance vendors with CSI or Operator integration.

3. Lightweight

Multi-tenancy isn’t possible unless the storage system is extremely lightweight and able to be packaged with the application stack. If the storage system takes too much resources or contains too many APIs, it won’t be possible to pack many tenants on the same infrastructure.

4. Scalable

Scalability is one of the key attributes of cloud native storage systems. Kubernetes advantage is that it has proved itself at scale. Kubernetes can also be used to manage storage scaling, but only if the underlying storage system integrates with Kubernetes and hands off provisioning and decommissioning capabilities to Kubernetes.

5. API-driven

One of the core tenets of Kubernetes and cloud native systems in general is to manage as much as possible through automation. For a storage system to be cloud native in the truest sense, it must integrate with Kubernetes through APIs and allow for dynamic, API-driven orchestration.

6. User Space

The final consideration is perhaps the most difficult. For an object storage solution to be cloud native it has to run entirely in the user space with no kernel dependencies. This is not how most object storage systems have been built, particularly the hardware appliances. Nonetheless, if you want to containerize your storage and deploy it on any Kubernetes cluster, you have to abide by these constraints. By definition, this means solutions that require kernel patching or have specialized hardware won’t be cloud native.

Conclusion

While simple on some level, these two criteria to claim “cloud-native” status are actually quite difficult in practice. Public cloud object storage vendors do quite well against them - indeed one would expect them to given that Google is the source of Kubernetes and Amazon is the source of S3. Private cloud object storage vendors have a much more difficult time passing these tests. While some claim S3 compatibility, closer inspection reveals otherwise. For the vast majority of legacy vendors, Kubernetes is simply not in their DNA and often not even on the roadmap. They are stuck and no amount of marketing will help.

MinIO, on the other hand, was built specifically for cloud native workloads. Built in the last four years, it was designed with Kubernetes in mind and speaks Kubernetes’s language. It is compatible with S3, but also can be used with Google, Azure or private clouds, making multi-cloud and hybrid cloud possible.

High performance, cloud native object storage is the only way to get the performance, reliability and scalability needed in cloud native applications.

See for yourself by downloading MinIO now. As a 100% open source solution, you will get our latest and greatest with nothing held back. We are also 100% transparent on how we price our subscriptions - and how that pricing is designed for multi-petabyte scale. If you want to do a little research first, check out our Docs or hang around our Slack Channel to see what’s getting people's attention.