Secure, Cloud-Native Backups at Scale with Kasten and MinIO

Gaurav Rishi, Kasten; Kris Inapurapu, MinIO

Kubernetes has fundamentally changed the way applications are architected, built and managed. The implications are widespread, but one area that requires a significant overhaul is backup and disaster recovery. In the Kubernetes world, applications are composed of microservices and short-lived containers, requiring a modern, application-centric approach that can scale affordably and securely.

MinIO and Kasten have teamed to bring a truly cloud native approach to this mission critical problem. Together, they offer seamless protection for cloud-native applications. This post describes this solution and elaborates on the benefits associated with a software-based, application-centric approach capable of massive scale and secure, performant operations.

Let's start with a summary of the two software components.

MinIO is a high-performance, Amazon S3 compatible object storage system.

With more than 375M Docker pulls it has become the standard for private cloud Kubernetes deployments. Designed from its inception to address data-rich, cloud-native workloads its performance characteristics make the ideal choice for a broad range of applications ranging from AI/ML/Advanced Analytics to fast snapshot and backup use cases. Because of these characteristics, more than half of the Fortune 500 run MinIO including 84 of the top 100.

The Kasten K10 data management software platform has been purpose-built for Kubernetes. K10’s application-centric approach and deep integrations with relational and NoSQL databases, storage systems, and Kubernetes distributions provide for backup/restore and mobility of your entire Kubernetes application. K10, with operational simplicity as a core tenet, makes Kubernetes application mobility and backup as easy as 1-2-3.

KUBERNETES BACKUP CHALLENGES

Supporting Kubernetes application backups is a critical need for organizations looking to benefit from all the agility and scale benefits that cloud-native technologies and operations provide. The Kubernetes world has different requirements and considerations, however, and ops teams need to consider the following when looking to backup from this environment:

Application as the Atomic Unit - the application must include the state that spans across storage volumes, databases (NoSQL/Relational) as well as configuration data included in Kubernetes objects such as configmaps and secrets.

Snapshots are not Backups - While snapshots are very useful for fast recovery the snapshot data is typically stored alongside the primary data. This lack of fault isolation does not protect data against loss. You need true backups by exporting these snapshots to an independent object store that can also provide cost effective long term data retention.

Application Portability - a cloud-native backup solution needs to offer capabilities that allow for the portability of applications across clusters, regions, and even clouds with diverse infrastructure and Kubernetes environments.

Cloud-Native Scale - Cloud-native applications requirements from a scale perspective have dramatically increased vs. hypervisor based applications. Some of the underlying reasons include an explosion in application components (e.g., ConfigMaps, secrets, etc), dynamic autoscaling (clusters and applications), and polyglot persistence (viz. Multiple databases used by a single cloud-native application).

Cloud-Native Security - In addition to scale, security in the cloud-native environment is of paramount importance where the data needs to be encrypted end-to-end with customer managed keys. Operations team must be able to offer self service portals that have natively integrated authentication and RBAC. Last but not least, in this era of growing cyberattacks, the solution must allow for a quick recovery from ransomware attacks.

BENEFITS OF KASTEN K10 AND MINIO

Because MinIO and Kasten K10 are software-defined, application centric and built for the Kubernetes world - they solve the aforementioned challenges elegantly using the following approach.

Treat the application as the operational unit. Kasten’s data management solution works with an entire application and not just the infrastructure or primary storage layers. This balances the often competing needs of operations and development teams in cloud-native environments. Operations teams can scale by ensuring business policy compliance at the application level instead of having to think about the hundreds of components that make up a modern app. Development teams can retain the application level controls they need without slowing them down.

Create scalable and resilient backups. Kasten K10 integrates with the MinIO Cloud-Native Object Store so that your applications can be stored as a true backup in a fault-domain that is separated from primary storage and has the cost efficiencies to afford long term retention. Not only is the data efficiently transferred by K10 using techniques like dedup and change-block-tracking, MinIO has the ability to read/write at speeds in excess of 170 GB/s in a single 32 node cluster, making for very efficient backup and recovery operations. Data resiliency is achieved since all of the I/O is committed synchronously with inline erasure-code, and encryption built into the solution.
Software Defined Scale. Because MinIO and Kasten K10 are software-defined solutions, when deployed together they offer unparalleled flexibility to design systems that deliver performance at scale and with commodity hardware pricing. This, alongside the simplicity of the software, ensures superior TCO for data protection initiatives.
Comprehensive Information Lifecycle Management. Modern enterprises have to contend with modern challenges when it comes to information lifecycle management. Using Kasten K10’s rich automation policies and MinIO’s immutability and versioning capabilities, these enterprises can have fine grained control over their data without introducing complexity.
Application portability across clusters, regions, clouds. Kubernetes has brought the promise and benefits of application portability to the cloud-native ecosystem. However, to truly enable application portability across critical data management use cases including restore, clone, disaster recovery and migrate we need to handle transformations across infrastructure (e.g., spinning disk to SSD), Kubernetes environments (e.g., OpenShift v.311 to OpenShift v4.x), and application specifications (e.g., DNS name changes). Kasten K10’s Application Transformation Engine does exactly that and empowers operations teams to provide application portability benefits to these applications. Leveraging MinIO’s object storage integrations with K10, allows operations teams to also export data across non-federated clusters at massive scale giving them complete freedom of choice.
Security and automation built in. Kasten K10 and MinIO offer a production-ready solution with robust operations-specific capabilities, including everything from global visibility, monitoring, alerting, and auditing, to features such as end-end encryption, compliance, RBAC, and deep data services integration. K10 for example, supports trusted root certificates for CA so that SSL certification remains enabled even when operating in on-premises environments. Additionally, these capabilities work at scale, not just across diverse environments, but also with PBs of data that is compressed and deduplicated. Policy-driven automation capabilities let you set up custom and default policies to meet both your container storage and data management needs. The policies provide automated enforcement to help meet your SLA’s across thousands of applications.

This brings together the best of Kubernetes from MinIO object storage and cloud-native data management from Kasten. Enterprises can confidently backup and restore cloud-native applications at scale and then protect them on an on-going basis with Kasten’s policy-based approach to automation. MinIO and Kasten are both software solutions and provide the flexibility of choosing the most effective COTS hardware and or cloud infrastructure. This solution enables use cases including:

Easy backup/restore for your entire application stack to make it easy to “reset” your application to a good known state
Disaster recovery of your applications in another cluster, region, or cloud
Application portability across unfederated clusters in a secure and scalable manner
On the MinIO side:

As always we encourage you to try it out for yourself. You can download a full-featured and free edition of Kasten K10 here. You can download MinIO here.