MinIO Fan-Out Feature for Time Shift Buffering

Satish Ramakrishnan -

MinIO has developed into a core building block for the media and entertainment industry. With a customer roster that includes the leading cable company, the biggest streaming company and dozens of companies up and down the stack we have added a number of different features in recent quarters. One of those is called the fan out feature and it is a regulatory requirement to implement time shift buffering (which is what happens when you rewind live tv a few seconds or minutes).

Specifically, Fan-Out addresses the legal copyright requirement for content distributors to ensure that every unit of recording be mapped to an object on the storage system and each unit needs to be copied a predetermined number of times. The predetermined number of copies are known as fanout.

This stems from a 2008 U.S. Court of Appeals for the Second Circuit case between the Cartoon network and Cablevision which ultimately ruled that Cablevision's proposed cloud-based DVR system did not infringe upon copyright holders' rights. The court held that because each playback transmission was made to a single subscriber using a single unique copy, made by that subscriber, each playback transmission was a private performance and did not infringe on the right of public performance.

As operators developed the feature of the 30 minute buffer (you can automatically go back up to 30 minutes on any show you have watched for 30 minutes, otherwise n minutes where n is less than 30 minutes) - the same requirement came into play.

The implications of the fanout requirement can vary depending on the scale of the service. For a small service, the fanout requirement might be in the hundreds or thousands. For a large, national service, the fanout requirement could be in the millions. The key is that you must make your copies from the source that is coming in. You want a single request to potentially call tens of thousands of objects. The network implications of having tens of thousands of requests simultaneously are not feasible.

MinIO has this covered in a new API that follows the traditional PutObject approach. Here, instead of writing a single object from a single stream, multiple objects are written, defined via a list of PutObjectFanOutRequest. Each segment (two seconds, two minutes, twenty-two minutes etc.) comes to MinIO with a list of buckets (with corresponding slash prefixes) where it needs to reside.


To use MinIO's fanout feature, you first need to create a fanout config file. This file is a YAML file that specifies the buckets that you want to upload the object to. The file should have the following format:

buckets:
  - bucket_name: my-bucket
    url: https://my-minio.example.com/my-bucket
  - bucket_name: your-bucket
    url: https://your-minio.example.com/your-bucket

Each entry in PutObjectFanOutRequest carries an object keyname and its relevant metadata if any.

Key is mandatory, the rest of the other options in PutObjectFanOutRequest are optional.

The fanout API will automatically upload the object to the specified buckets. The object will be uploaded with the same name and content as the original object.

In addition to adhering to the regulatory/copyright requirement, the fanout API saves massive amounts of time by automating the process of uploading objects to multiple buckets. While this is not a redundancy approach (there are far more efficient ways of achieving that) it does ensure that the  data is always available in multiple locations.

MinIO's fanout feature:

  • The fanout config file can specify any number of buckets.
  • The fanout config file can specify the access keys and secret keys for each bucket.
  • The fanout config file can specify the region for each bucket.
  • The fanout config file can specify the endpoint for each bucket.
  • Allow read of specific copy
  • Allow delete of specific copy
  • Allow delete of all copies
  • Allow copy of specific copy
  • Allow listing of copies
  • Allow bucket listing of fanout objects

At this point the fanout feature is only available through the API but will be available in the CLI and Console in the not too distant future. It should be noted that the amount of fan out each node (or server) can handle is dependent on its hardware capabilities.

Summary

The fanout feature is not particularly complex but it is required in certain situations and MinIO has it covered and can solve this elegantly. Feel free to try it out and if you find an alternative use case be sure to drop us a note at hello@min.io and we will let folks know.