Composite Manifest Support for Rough Cut Editing scenarios in SSME

We released the Beta 2 for the IIS Smooth Streaming Player Development Kit (SSPDK) that contains the SmoothStreamingMediaElement (SSME) interface. With this release we added a fun new feature. We refer to this as Composite Manifest Support for Rough Cut Editing. The idea is simple:

I have many clips but want to create a composite clip taking relevant portions from these clips.

I have a large clip from an NBA game, I want to create a highlight for this game.

Now, let’s expand this simple idea. Not only do I want the above, I also want the ability to present this as a single stream to the user. This means the user can fast forward, rewind across the entire highlights and seek to any points in the highlight. In addition, I can add markers and present useful information to the user watching the stream. Well, this is exactly what we enabled with the Beta 2 release of SSPDK.

Composite Manifest

At the heart of this feature are small updates to the existing IIS Smooth Streaming Client manifest structure and we this new manifest a composite stream manifest (.csm). If you have used Smooth Streaming, you might already be familiar with the client manifest structure. Here is what it looks like before this feature:

If you notice carefully, all we have done is changed the relationship above to: SmoothStreamingMedia contains Clip and Clip contains StreamIndex. Finally StreamIndex contains QualityLevel and ‘c’ elements.

While doing this we defined a new element called Clip which has attributes called Url, ClipBegin and ClipEnd.

Url – this specifies the Url to the original source manifest from which these clips were cut. The value is exactly as you would set on the SmoothStreamingSource property in SSME (e.g., http://abcxyz.com/sample.ism/Manifest).

ClipBegin – This specifies the time in nanoseconds where to begin the playback for the clip.

ClipEnd – This specifies time in nanoseconds where to end to end the playback for the clip.

You will also notice that ‘c’ elements are still present in this composite manifest. This is done so that this manifest is self sufficient and you don’t need to download the source manifest. ‘c’ elements are essentially chunk timestamps for the FMP4 chunks the client will download.

Note: Not all ‘c’ elements are included in this manifest. The idea is to include only the ones needed to represent the section of the clip. In this case times are close to the ClipBegin and ClipEnd. They may not exactly match in timestamps as chunks are 2 seconds apart typically while the Clip is cut at finer granularity.

Multiple Clip Elements

In the section above, we saw how to compose a single clip. However, composite manifest promises to have multiple such clips stitched together. All you need to do is to have multiple such Clip elements and that takes care of that. Here is how an example of composite manifest with two clips will will look like:

With this, you can now upload this manifest along with the original manifests and content used by them to a HTTP accessible location. Next, set the SmoothStreamingSource property to the URL to this composite manifest and SSME should take care of the rest. Here is how it will look like in XAML:

We will have more detailed documentation up on iis.net very soon.In the meantime, let me know if you have any questions.

2 Comments

Using a composite manifest, I received an error message saying 50 consecutive chunks failed to download.
I used Fiddler to inspect HTTP traffic and it was requesting /QualityLevels({bitrate})/Fragments(video={start time}).
Obviously, it should be requesting /path/to/files/Filename.ism/QualityLevels({bitrate})/Fragments(video={start time}).

For anyone who encounters the same problem, I resolved the issue by specifying the complete URL in the Url-attribute of every StreamIndex, for example:

Not sure if this is how they are intended to work. Looking at the provided sample in the article, I would expect to get similar errors as stated above.

I was just wondering if DRM is supported with composite manifests. That would be most useful. I've tried creating a CSM containing a protection header under the "Clip" tag, but with no success (I copied the asset's ismc protection header).