MinIO updates from KubeCon

There has been a lot going on MinIO server development front. We recently added support for * Disk Caching [https://docs.minio.io/docs/minio-disk-cache-guide] * Large/Petascale buckets [https://docs.minio.io/docs/minio-large-bucket-support-quickstart-guide] * Storage Classes [https://github.com/minio/minio/tree/master/docs/erasure/storage-class] While features like MinIO bucket federation [https://github.com/minio/minio/pull/5501], revamped MinIO

Read more...

MinIO, the ZFS of cloud storage

ZFS is best known for abstracting away the physical storage device boundaries by pooling them together. ZFS completely removed the need to manually handle physical storage or worry about their individual capacities. ZFS is also a pioneer in its ability to detect data corruption and recover if data redundancy is available. However, as we already discussed [https://blog.minio.io/

Read more...

IoT data storage and analysis with Fluentd, MinIO and Apache Spark

IoT data storage and analysis with Fluentd, MinIO and Apache Spark

One of the major requirements for success with IoT strategy is the ability to store and analyze device and sensor data. As IoT brings thousands of devices online everyday, the data being generated by all these devices combined is reaching staggering levels. > Storing the IoT data in a scalable yet cost effective manner, while being able to analyze it easily

Read more...

Debugging Go Routine leaks

Before beginning with Debugging Goroutines leaks, let me give a small introduction of some fundamentals which will give you a broader perspective of the problem. Concurrent Programming. * Concurrent programming deals with concurrent execution of a program wherein multiple sequential streams of execution run at a time resulting in faster execution of your computations. * It helps in better utilization of multicore

Read more...

Fast hashing in Golang using BLAKE2

In this blog we would like to present an optimized implementation, blake2b-simd [https://github.com/minio/blake2b-simd] , in pure Go for the BLAKE2 [https://blake2.net/] hashing algorithm that takes advantage of SIMD instructions. It gives up to a 4x speed increase over the (non-assembly) Go implementation and can achieve hashing speeds close to 1 GB/sec per core on

Read more...

Object storage in practice: Creating a reliable data store

Object storage in practice: Creating a reliable data store

In my previous post [https://blog.minio.io/object-storage-what-is-it-all-about-62920ca164ca#.bcz4d4nnd] we learnt the why and what of object storage. Specifically, we learnt why at all a new storage paradigm is required and what it does to alleviate modern unstructured data problems. We saw how object storage lets you access objects at application layer with simple API calls over HTTP(s)

Read more...

Microstorage for Microservices

Microstorage for Microservices

One of the fundamental requirements of microservice is that application containers become stateless. However the states still need to be stored elsewhere, as in databases, object storage, session files, cookies, cache, etc. Microstorage is emerging as a new architecture to address the storage scalability for microservices. Microstorage inherits the idea from microservices that “only small things scale”. Scale-up is not

Read more...