Few would argue with the statement that Hadoop HDFS is in decline. In fact, the HDFS part of the Hadoop ecosystem is in more than just decline - it is in freefall. At the time of its inception, it had a meaningful role to play as a high-throughput, fault-tolerant distributed file system. The secret sauce was data locality. By co-locating
Read more...
When one looks at the amazing roster of talks for the Spark + AI Summit, what you don’t see is a lot of discussion on how to leverage object storage. On some level you would expect to —
Read more...
One of the key challenges in any digitization journey is the adoption of machine learning techniques. Given the explosion of tools and frameworks, it can be difficult
Read more...
How to use Apache Flink to build a private cloud data pipeline for a variety of use cases.
Read more...
In this post we’ll learn more about object storage, specifically Minio and then see how to connect Minio with tools like Apache Spark and Presto for analytics workloads.
Read more...
In the first part of this two post series, we’ll take a look at how object storage is different from other storage approaches and why it makes sense to leverage object storage like Minio for data lakes.
Read more...
In this post, we learn about Pivotal Container Service deployment and how to use the pks command line tool to create and manage Kubernetes clusters. We also saw how to deploy Minio once your PKS Kubernetes cluster is set up and running.
Read more...
Minio now has built-in support for exporting Prometheus compatible data on an unauthenticated endpoint. This enables Prometheus monitoring for Minio server deployments without sharing server credentials and eliminates the need to run an external Prometheus exporter.
Read more...There has been a lot going on MinIO server development front. We recently added support for Disk CachingLarge/Petascale bucketsStorage ClassesWhile features like MinIO bucket federation, revamped MinIO browser UX, Prometheus integration, etc are being actively worked upon and will be available soon. But this doesn’t stop us from keeping out eyes, ears and hearts close to the MinIO
Read more...ZFS is best known for abstracting away the physical storage device boundaries by pooling them together. ZFS completely removed the need to manually handle physical storage or worry about their individual capacities. ZFS is also a pioneer in its ability to detect data corruption and recover if data redundancy is available. However, as we already discussed, traditional filesystems can’t
Read more...
