How MinIO AIStor helped transform a global digital payments provider’s S3 private cloud

This global digital payments provider has provided online payment systems for nearly 30 years, seeing its highest rates of growth over the past ten years.  Its user base expanded from over 150 million to more than 400 million customers worldwide which boosted its total payment volume from $200 billion to more than $1.5 trillion. 

This led to their data storage growing exponentially from petabytes to over an exabyte today. The company started its journey utilizing scale-up storage architectures, with 80% of data stored on block systems and 20% stored on file systems. This balance shifted as the company moved to distributed systems and object storage. With unstructured data driving the rapid growth, around 20% of the company’s data is now stored in object storage, with significant growth expected in coming quarters. 

This growth created a number of environmental challenges that made modernizing the company’s on-premises infrastructure a top organizational priority, and led to them turning to MinIO – a trusted technology partner for more than seven years – to help transform its data platform. 

Analyzing the Environment

Customer business processes require the generation of monthly statements, weekly transaction records, balance details and other recurring reports for nearly half a billion merchants within the first first days of each month. Merchants rely on these reports to estimate earnings, revenue, expenses, even inventory. Each report is a small PDF file that must then be stored long term for compliance reasons, which quickly adds up to billions and billions of files.

The Challenge: Legacy object-gateway solution could not meet SLAs

Our customer was using a legacy object storage system as their object storage platform in their on-premises data centers. This legacy solution could no longer meet the demands of the company’s exponential data growth — especially with billions of small files from merchant reporting stressing its limits. Replication was extremely slow and complex, and the legacy solution struggled to keep up with existing workloads. This poor performance led to merchant reporting delivery delays of 7-10 days on average - causing our customer to miss their SLAs.

This challenge was complicated further by a six-month deadline to exit a shared data center, migrate the data, and rebuild critical systems elsewhere. This large-scale migration required replication of data between sites even as new data kept pouring in; however, the performance constraints of the legacy solution made timely replication impossible. The initial copy alone was projected to take more than six months, an unacceptable timeline that would result in high cost data center fees and contract extensions. 

As a result, our customer initiated a high-urgency search for a modern object storage solution to replace the legacy platform, something capable of addressing their immediate business challenges around reporting and site migration while future-proofing the environment to manage data seamlessly across their hybrid cloud.

The Solution: Object-native S3 hybrid cloud storage, architected to scale 

AIStor emerged as the ideal solution after a detailed Proof of Concept. With AIStor, our customer was able to efficiently and effectively address all of their pressing business challenges by implementing a unified namespace, with AIStor as the underlying S3 hybrid cloud storage platform.

The AIStor architecture is purpose built to handle small files at massive scale without the overhead and inefficiency of a shared metadata layer, enabling scale out or scale linearly. After migrating from the legacy storage platform to AIStor, our customer is now able to generate merchant reports within the first two days of each month, eliminating missed SLAs and improving customer satisfaction (a 70-80% reduction in time).

AIStor was also able to help this organization transform their data lakehouse from proprietary technology to an open table format and store their data in Iceberg, enabling them to leverage best-in-class open source tools and other key observability capabilities on top.

Finally, AIStor provides the elasticity required to quickly spin up GPUs and VMs when they need burst capacity in the cloud, making data readily available to data scientists. This, in turn, gives them the control to then place data on premise or in the cloud in compliance with internal policies.