Prefix vs Folder

In a traditional POSIX system, you are probably familiar with data access functions that can read and access a folder in the traditional sense. The issue with POSIX is it was never really designed to work with very large files over the network. The speed of POSIX based systems decreases as the concurrent demand increases, at which point it doesn’t have the ability to scale and meet these demands. POSIX file permissions are also primitive and cannot handle identity and management policies. Any mutations to a file will not show up until it's been committed, so when files are accessed concurrently, users will not see these modifications. When large organizations need to store and access oceans of data for deep learning, AI, and other data intensive use cases, they often look for data stores that are scalable and use RESTful API concepts which are the hallmarks of the cloud they are used to and love.

So how does MinIO differentiate itself? Why has MinIO become the storage darling for AI and ML workloads? If I have to pick one sentence, it's: Simplicity and Ease of Use. Out-of-the-box MinIO comes with enterprise features such as object level encryption, versioning and immutability – these simply have no equivalent in the POSIX world and nothing is able to translate them. MinIO secures objects with encryption at rest and in-transit, combined with Policy Based Access Control (PBAC) to regulate access and erasure coding to protect data integrity. You will achieve the best performance possible, regardless of where you run MinIO, because it takes advantage of underlying hardware to deliver the greatest possible performance.

In order to achieve this level of performance, scalability and feature richness, MinIO stores the objects in a unique way, let's dive in.

So is it a Prefix or a Folder?

In MinIO object store land everything is stored in a flat structure. The reason for this is that the hierarchical nature of the POSIX file system is one of its disadvantages. MinIO does away with it by keeping everything flat. But most of us using applications that aren’t necessarily cloud native have been used to having a structure, mainly folders, as a way to organize the data put inside them.

So to cater to everyone, MinIO uses “folders” as a means to organize the objects put inside it. A folder is nothing more than an object ending with a trailing slash object_name/. It has no size, no content and no metadata of any sort to show. It's simply a way to group objects together in a reasonable way for users to know where their objects are located.

So then, what is a prefix? A string of folders is what makes a prefix (and sometimes called path). When you combine prefixes (string of folders) and the object name, that makes it an object key. This is best illustrated with an example.

Let's say you have the following objects

 The first “folder” you see myminio/ is actually the alias name, which you can add using the command below.

So if you have another MinIO cluster, you would add another alias like so

The bucket name is vacation-photos/. This is where all the objects are actually stored. Within the bucket, folders can be created to organize data. In the above example, cancun/2024/ is the folder where the objects my-hotel.jpg, my-car.jpg and my-boat.jpg are stored.

When you combine the ALIAS + BUCKET NAME + FOLDERS that becomes the prefix myminio/vacation-photos/2024/cancun/ where multiple objects can be stored. When you add the object name my-hotel.jpg the entire string becomes a key like so myminio/vacation-photos/2024/cancun/my-hotel.jpg.

Internally MinIO will make any object that ends with / a folder-type object and you can add more objects that end with / for example 2024/, 2023/ etc. But please note you cannot have a bucket inside a bucket. For example inside vacation-photos/ you cannot have a myotherbucket/ like below. You would not see it listed as a bucket within MinIO.

It needs to be like this instead

So did you get the hang of the differences between what makes a prefix a prefix and what makes a folder a folder and an object an object in MinIO land?

POP QUIZ!!!

I know, I hated these back in high school too, but trust me, this one is much easier if I explained things well above. So it's on me whether you do well or not on this quiz.

The answers for each of these is in a text that matches the page background color. If you highlight below each of the questions you will see the answers. Try to see how many of these you get correct.

Note: If highlighting doesn't work, copy the entire content below to a text based notepad to reveal the answers.

Let's say you have a bucket with just one object at the below location

• Which one is the prefix?

• Which one is the alias?

• Which one is the bucket name?

• Which one is the object name?

• Which folder name is the object inside?

• Which one is the key name?

• How many folders in total does the bucket have and what are their names?

• How many folders can you create inside a folder?

• How many buckets can you create inside a bucket?

How many did you get right without highlighting? I hope all of them.

I heard you like POSIX

As you can see Folders, Prefixes and Paths are essentially a mechanism to organize your objects inside MinIO. It's a construct of a folder-type object that allows you to get a sense of where your data is located when you want to view your objects. Moreover, these folders and prefixes can be used to create IAM rules and policies to ensure you don’t have duplicated roles per object and can apply your settings to a whole swath of prefixes in one go.

If you have any questions on Prefixes or Folders, be sure to reach out to us on Slack! or email us at hello@min.io.