Using MinIO for Know Your Customer Data at a Major Bank

Using MinIO for Know Your Customer Data at a Major Bank

Introduction Document management is a core requirement for all sorts of regulated institutions - finance, telecom, healthcare, government and others. These institutions need to manage and retain an ever growing number of documents and regulatory guidelines often require these documents to be stored for a very long term (7-10 years). Take for example, KYC (Know Your Customer) documents. Anyone starting

Read more...

Kubernetes Storage Patterns

Kubernetes Storage Patterns

Kubernetes has fundamentally altered  the traditional application development and deployment patterns. Application development teams can now develop, test and deploy their apps in days, across different environments, all within their Kubernetes clusters. Previous generations of technology typically took weeks if not months. This acceleration is possible due to the abstraction that Kubernetes brings to the table, i.e. it deals

Read more...

Introducing Spark-Select for MinIO Data Lakes

Introducing Spark-Select for MinIO Data Lakes

When early object storage APIs were developed they focused on the efficient storage and retrieval of objects. Amazon’s success with S3 and its implementation of the robust S3 API quickly became the de facto standard for object storage in the cloud. MinIO, recognizing this, invested heavily in creating the most compliant implementation of the S3 API outside of Amazon.

Read more...

MinIO updates from KubeCon

There has been a lot going on MinIO server development front. We recently added support for * Disk Caching [https://docs.minio.io/docs/minio-disk-cache-guide] * Large/Petascale buckets [https://docs.minio.io/docs/minio-large-bucket-support-quickstart-guide] * Storage Classes [https://github.com/minio/minio/tree/master/docs/erasure/storage-class] While features like MinIO bucket federation [https://github.com/minio/minio/pull/5501], revamped MinIO

Read more...

MinIO, the ZFS of cloud storage

ZFS is best known for abstracting away the physical storage device boundaries by pooling them together. ZFS completely removed the need to manually handle physical storage or worry about their individual capacities. ZFS is also a pioneer in its ability to detect data corruption and recover if data redundancy is available. However, as we already discussed [https://blog.minio.io/

Read more...

IoT data storage and analysis with Fluentd, MinIO and Apache Spark

IoT data storage and analysis with Fluentd, MinIO and Apache Spark

One of the major requirements for success with IoT strategy is the ability to store and analyze device and sensor data. As IoT brings thousands of devices online everyday, the data being generated by all these devices combined is reaching staggering levels. > Storing the IoT data in a scalable yet cost effective manner, while being able to analyze it easily

Read more...

Containerized data analytics at scale, with MinIO and Pachyderm

Containers running on orchestration platforms like Kubernetes, Docker Swarm, DC/OS et al. offer powerful, versatile ways to deploy applications. Containers let you deploy isolated application instances, and you can launch multiple such instances to scale up your load serving capacity. You don’t even need to worry about individual server capacities and scheduling thanks to orchestration tool, which provide

Read more...

Scalable Genomics Data Processing Pipeline with Alluxio, Apache Mesos, and MinIO

This is a guest blog from our friends at Guardant Health [http://www.guardanthealth.com/]. Guardant Health is the world leader in comprehensive liquid biopsy. Oncologists order our blood test to help determine if their advanced cancer patients are eligible for certain drugs that target specific genomic alterations in tumour DNA. Each test produces huge amounts of genomic data that

Read more...

ToolsLib switches to MinIO for scale and performance

ToolsLib recently switched to a brand new dashboard [https://toolslib.net/blog/viewpost/2017/02/28/54-welcome-new-dashboard/]. However, behind the scenes, there was another switch happening. We used block based filesystem for underlying storage. However, as we grew, the filesystem was proving difficult to scale. So, we were looking to move from block based storage to an object store system,

Read more...

Deploy MinIO cloud storage to Mesosphere DC/OS

Container orchestration is gaining traction as the default way to deploy applications. Developers are architecting their modern applications from the ground-up to run in containers, which enables faster deployment and more resilience. Even legacy applications are adopting containers in every way they can to access these advantages. Of the many characteristics that make an application container ready, the way it

Read more...