Multimedia

Develop an app that’s ready to help users both create and play back exciting media experiences. This is now easier to do in Windows 8.1, with new properties for capturing media in different environments, a simple way to add playback controls to apps that use XAMLExtensible Application Markup Language (XAML), and native support for Media Source Extensions (MSE) for HTML5 playback.

Video processing feature updates

Windows 8.1 introduces some new updates and support related to video formats, codecs, and processing.

Support for CFF

Microsoft Media Foundation added support for Common File Format (CFF) to its MP4/fMP4 implementations. CFF is an audio/video container format based on the ISO Base Media File Format.

New XVP mode

The Media Foundation Transcode Video Processor (XVP) added the MrfCrf444 mode, which always runs in software and does not use DirectX Video Acceleration (DXVA) hardware acceleration. This ensures consistent results across platforms, independent of underlying hardware. Because the MrfCrf444 mode executes entirely in software, the transcode operation may take longer and use more power. This new mode forces the use of larger filters, uses full precision for color conversions, prefers higher-quality color formats, and enables a high quality deinterlacer.

No code example is currently available or this language may not be supported.

Support for extended range YUV in Media Foundation

In Windows 8, the Media Foundation pipeline defaulted to studio swing YUV range [Y: 16-235] and did not provide the ability to signal if the input was in extended range YUV [Y: 0-255]. The Media Foundation pipeline now has the ability to signal extended range to the graphics processing unit (GPU) hardware. This makes the entire Media Foundation pipeline, including the MJPEG decoder, extended-range aware.

Media Stream Source

Windows 8.1 introduces a new generic media source for Windows Store apps called Media Stream Source (MSS). The APIs for this technology are in the Windows.Media.Core namespace, and the main media-source class is MediaStreamSource. MSS enables apps to send compressed and uncompressed audio and video samples to the media pipeline for playback, transcoding, and streaming. Media samples can be dynamically generated by the app, or de-multiplexed from a stream or files. This flexibility enables apps to more easily extend platform support for new media formats or solve complex problems, such as adaptive streaming or implementing a custom media source in managed code.

The MediaStreamSource API are very similar to the Microsoft Silverlight APIs of the same name. So developers who are familiar with the Silverlight versions of these APIs can apply much of their knowledge to build Windows Store apps using MSS.

Here are some of the main MSS APIs. The order reflects the basic flow of how MSS functions. You'll notice that the MediaStreamSource sends request objects to the app through event arguments. These request objects enable the app to interact with the MediaStreamSource and pass data back to it.

Represents a request from the MediaStreamSource for a new media sample. Setting the Sample property to the new MediaStreamSample triggers the MediaStreamSource to retrieve the media sample and continue processing the media data.

A key scenario for photo sequence mode is getting photos in the past—that is, before the user takes a photo. You can use PhotoCapturedEventArgs.CaptureTimeOffset to tell whether a frame was in the future or past. A value greater than 0 indicates the future and less than 0 indicates the past.

If your app wants to set how many frames it gets per second, it can use LowLagPhotoSequence.PhotosPerSecondLimit. This can be useful in situations where the sensor on the device can handle 30 frames per second (fps), but the app needs only 4 fps.

Torch Mode

You can find out whether a device supports this control by checking the TorchControl.Supported property. If it's supported, you can use Torch Mode in capture apps, or in non-capture apps to do things like brighten a room.

Flash Mode

You can find out whether a device supports this control by checking the FlashControl.Supported property. If it's supported, you can use Flash Mode in a photo app to handle low-light conditions or for RedEyeReduction.

ISO Mode

ISO Mode enables your app to manage the ISO speed settings on a device. In low-light conditions, you may prefer a higher noise level in return for higher overall brightness. The ISO speed control lets your app adjust this tradeoff. You configure this mode by using IsoSpeedControl, which you access through the MediaCapture.VideoDeviceController property.

HTML5 video and audio playback

Windows 8.1 introduces a number of updates to audio and video media playback, including support for the World Wide Web Consortium (W3C) HTML5 Media Source Extensions (MSE), support for the W3C HTML5 Encrypted Media Extensions (EME), and updates to the XAML MediaElement control.

Media Source Extensions (MSE) for HTML5 video and audio.

HTML5 Media Source Extensions (MSE) is a W3C standard that extends the video and audio elements to enable dynamically changing the media stream without the use of plug-ins. This supports scenarios like adaptive media streaming, live streaming, splicing videos, and video editing.

Custom adaptive streaming solutions can be built on MSE. Adaptive streaming solutions dynamically change the bit rate of the media stream in order to provide the highest quality streaming experience. They attempt to determine the optimal bit rate based on constraints such as network bandwidth and CPU use. For example, if network bandwidth drops, you can use MSE to switch the video stream to a lower bit rate. This would reduce the quality of the image, but would enable the video to continue playing without interruption.

One use for dynamically splicing videos together is to insert ads during media playback. This means you do not have to put ad clips in the media files beforehand, but can instead insert a clip in the media stream during playback.

Play To feature updates

Windows 8.1 introduces a number of updates to improve the Play To experience.

Programmatically launch the Play To UI

New UI guidance for Windows 8.1 encourages app developers to provide an in-app button, using a standard Play To glyph, to programmatically display the Play To flyout, and not to require the user to bring up the UI by swiping the Devices charm. Use ShowPlayToUI to display the flyout.

Play To by reference

Play To by reference enables webpages and Windows Store apps to play DRM-protected content. The app specifies a preferred Uniform Resource Identifier (URI) to be sent to the Play To target device. This URI can point to a cloud-based media server. If the preferredSourceUri is present, the Play To target device can stream content directly from the cloud, and that content can be DRM protected. (Contrast the alternative—streaming content from the Windows 8.1 machine—where the content must be unprotected.)

No code example is currently available or this language may not be supported.

Adobe Flash audio and video support

Play To now supports playing unprotected Adobe Flash audio and video from Internet Explorer in the new Windows UI on a Digital Media Receiver (DMR).

Support for Media Source Extensions

Media Source Extensions (MSE) is a new W3C standard that will become the backbone of high-quality, premium, power-efficient, feature-length video delivery using only a web browser. MSE does not require a plug-in. Play To now supports playing unprotected MSE content from Windows Store app using JavaScript and websites in Internet Explorer in the Windows UI.

Video to audio-only devices

The Play To UI now supports selecting an audio-only receiver as a remote endpoint for video content. The video content is demixed and just the audio component is sent to the device.

Media transport controls UI

The media transport controls UI, the UI you see when you press volume up or down, can now be used as a remote control for playback on a Play To device.

The transport controls can be displayed and used to control local or remote playback while the source app is in the background or while the device screen is locked. All Windows Store apps get a default implementation that includes:

Hooking up basic transport controls, such as play, pause, and seek

The app name in place of song title

The app icon in place of album art

Your app can improve on the default implementation by:

Hooking up advanced transport controls, such as next-track and previous-track functionality

Providing the song title

Providing the album art

Providing the album title

Devices charm UI updates

The Devices charm is now organized hierarchically and puts devices in these categories:

Play

Print

Project

Tap & Send (if the device hardware supports it)

This helps reduce confusion for users about what they can expect to find in the Devices charm. Even when there are no devices currently available, these sections are visible.

Digital Media Receiver persistence

The most recently selected Digital Media Receiver, along with its connection status, is now highlighted on the Device charm.

Improved connection/disconnection flow, and better error reporting

The Play To UI now supports an improved connection and disconnection flow.

While a Play To connection is being established with a device, a new UI shows the message "Connecting" and the Play To flyout remains visible until the connection has been made. If the connection fails, a connection error is reported to the source app.

The Disconnect button has been removed. Instead, to end remote playback and resume playback locally, users now select This PC, which is listed alongside other devices in the Play To UI.

Show:
Inherited
Protected

Was this page helpful?

Your feedback about this content is important.Let us know what you think.