ToolsLib switches to MinIO for scale and performance

ToolsLib recently switched to a brand new dashboard. However, behind the scenes, there was another switch happening. We used block based filesystem for underlying storage. However, as we grew, the filesystem was proving difficult to scale.

So, we were looking to move from block based storage to an object store system, when we found MinIO. After initial analysis, MinIO looked perfect for our requirements and we switched to MinIO.

Not only the switch was smooth, team is very happy with the new storage system. We thought this was a great success story for open source software and needed to be shared with our community.

ToolsLib

ToolsLib offers software hosting and related project management services for developers. ToolsLib enables easy access to daily use software via simple download and publishing options.

Situation

We used to store all of our unstructured data on a block based filesystem. While the filesystem served us well initially, as we grew it was proving difficult to scale. Also, the block based filesystem was difficult to synchronize in a reliable way between several servers, creating a bottleneck in our overall operations.

Solution

Object storage is more suited to organize our data, a bucket represents a project which stores several files — it was much easier for them to manage using this abstraction than using traditional folders.

Currently, we have deployed a single MinIO server with 16GB RAM, running two SSDs in RAID1 configuration. MinIO server already hosts several GB of data, and manages between 10 to 50 requests per second. Since the software stack is in PHP, we use the AWS PHP SDK. The upload is using multipart with a filesize up to 1.5GB and a stream on the download side.

Some of very popular software in our library create huge request peak in very few seconds. To handle that load efficiently and monitor it pro-actively we usemc events and mc watch.

The migration has allowed us to simplify our storage management, while giving flexibility to develop services.

To learn about other projects/companies that use MinIO, check out Awesome MinIO.


While you’re at it, help us understand your use case and how we can help you better! Fill out our best of MinIO deployment form (takes less than a minute), and get a chance to be featured on the MinIO website and showcase your MinIO private cloud design to MinIO community.

This post was originally published on ToolsLib blog, it is reproduced on MinIO blog with proper permissions.