Streaming Multimedia on BlackBerry

Summary

This article applies to the following:

BlackBerry® Device Software 4.2.1 and later

Details

This article describes how to create a solid streaming media application and how to use the StreamingPlayer class, which is an open-source application programming interface (API) containing classes and methods that support streaming media.

Basic Streaming Media

There are two methods that can be used to stream media.

Streaming media over RTSP

Real Time Streaming Protocol (RTSP) was first implemented in BlackBerry Device Software 4.3 for BlackBerry smartphones that operate on Code Division Multiple Access (CDMA) networks. RTSP is now supported by all BlackBerry smartphones running BlackBerry Device Software 4.5 and later. There are two ways to stream media content over RTSP.

Using the browser (starts the Media application on the BlackBerry smartphone)String url = "rtsp://mystreamingserver.com/001.aac";Browser.getDefaultSession().displayPage(url);

Using the Mobile Media API that is specified in JSR 135 (plays within an application)String url = "rtsp://mystreamingserver.com/001.aac";Player player = Manager.createPlayer(url);player.start();

Streaming media over HTTP

Hypertext Transfer Protocol (HTTP) is a protocol that can stream media content and provide real-time playback. Like RTSP, there are two ways to stream media content over HTTP.

Using the browser (starts the Media application on the BlackBerry smartphone)

String url = "http://mystreamingserver.com/001.mp3";

Browser.getDefaultSession().displayPage(url);

Using the Mobile Media API that is specified in JSR 135 (plays within an application)

String url = "http://mystreamingserver.com/001.mp3";

Player player = Manager.createPlayer(url);player.start();

Advantages of streaming media over HTTP

API calls are simple to use.

This method hides many complex details.

Limitations of streaming media over HTTP

Raw data in the buffer cannot be accessed.

The data flow from the buffer to the media player cannot be controlled.

There is no seek() support prior to BlackBerry Device Software 5.0.

There is no access to the raw data stream.

Advanced streaming media

The Mobile Media API (JSR 135) defines advanced APIs that you can use to control media streaming and allow flexibility in development. These APIs can be found in the javax.microedition.media package.

Components of advanced streaming media

The following diagram illustrates the different components and classes that are involved in streaming media and the data flow between a remote media source and the media player.

Source Media

The media file that is located on a content server.

Download Thread

A thread that connects to the source media and downloads the media file to the media buffer.

Media Buffer

A circular byte buffer for storing downloaded data until it is sent to media player for playback.

SourceStream

An instance of the javax.microedition.media.protocol.SourceStream class. The SourceStream acts as an input stream for the media player by providing methods such as read() and seek().

DataSource

An instance of the javax.microedition.media.protocol.DataSource. DataSource class, and performs the following procedure:

Open a connection to the remote source media.

Start the download thread.

Initialize and return SourceStream objects to play.

Close the connection.

Player

An instance of javax.microedition.media.Player. The Player object plays media content as the content is downloaded. You can initialize a Playerobject by invoking

javax.microedition.media.Manager.createPlayer(DataSource source).

Implementation

The code sample that is included this article shows a complete implementation of all the components and classes that are mentioned. The code sample also includes a StreamingPlayer class and a StreamingPlayerListener interface that you can use as an API or extend to meet special use cases.

Implementing the DataSource

DataSource is an abstract class and has six methods. Each of the following methods is invoked by the Player object:

public void connect()

Opens connections and input/output (I/O) streams

public void disconnect()

Closes connections, I/O streams, and so on

public String getContentType()

Returns the content type of the source media

public SourceStream[] getStreams()

Returns SourceStream objects

public void start()

Starts the download thread and begins downloading data

public void stop()

Stops downloading data

Implementing the SourceStream

public ContentDescriptor getContentDescriptor()

Returns a ContentDescriptor object based on the content type of the media

public long getContentLength()

Returns the content length.

public int getSeekType()

Returns RANDOM_ACCESSIBLE

Allows the Player to seek to nearby locations from the current position. SourceStream RANDOM_ACCESSIBLE must be set because video files can have audio and video at different offsets for each frame and the BlackBerry Device Software requires both for smooth playback. If -1 is returned by getContentLength(), the seek calls will only be to a previous position in the media file.

public int read(byte[] b, int off, int len)

Sends data to the Player.

Blocks or resumes the feed,depending on the status of the buffer

public long seek(long where)

Called by the Player to seek to a point between 0 and the content length

Implementation varies based the following connection types:

HTTP streaming

Uses this method only if seek() is feasible.

seek() point is within the buffered data. Otherwise, return the current position.

FileConnection streaming

Uses this method with mark(), reset() and skip()

public long tell()

Returns the current position in the stream

How to handle user-initiated seek()

There are two types of user-initiated seek calls. In both cases, Player.setMediaTime(long microseconds) must be invoked, which in turn invokesSourceStream.seek(long position).

Seek calls within available data

Invoke seek() within the buffer using mark(), reset() or skip()

Seek calls beyond the data that is available

Block the feed to the Player.

Stop downloading data.

Close the HTTP connection.

Reopen the connection using the HTTP range header to get data from the seek position.

Start downloading using the new connection.

Resume the feed to the Player when buffer contains enough information.

Stream control

Because media is streaming over a wireless network, there must be a mechanism to stop data feed to the Player when the buffer runs out of space due to of network delay. There must also be a way to determine how much of the buffer should be filled before restarting or resuming playback. Downloaded data in the buffer must be held as long as possible to allow for backward seek calls made by the Player, but when buffer is full the data must be discarded to download more data. You can use the following variables to control the feed to the Player:

initialBufferDetermines how much data to buffer before playback starts for the first time. This variable can be determined dynamically by the following criteria:

Connection type

Bit rate

Bandwidth and latency

restartThresholdDetermines when to resume the data feed to the Player based on the data that is available in the buffer. This variable can be determined dynamically by the following criteria:

Connection type

Bit rate

Bandwidth and latency

bufferCapacityDetermines the size of the buffer where data is stored. This variable can be determined dynamically by the following criteria:

Connection type

Bit rate

Bandwidth and latency

If a buffer is too small, it can frequently run out of space.

If a buffer too big, it can waste system resources.

bufferLeakSizeVideo playback requires the Player to seek back from the current position because audio and video data for the same frame are at different offsets. To accommodate this situation, downloaded data is stored until the Player is at the end of the buffer and the buffer is full. This method requires that more data must be downloaded for the playback to continue. In this situation, the oldest data is discarded first to create space for downloading more data.bufferLeakSize determines how much data is discarded each time this occurs.

Creating a Player from a DataSource

To create a Player from a DataSource, use the following code:

Player player = Manager.createPlayer(new MyDataSource());player.start();

Using the right transport method

Wi-Fi® technology is the preferred transport because it is fast and free. If a Wi-Fi connection becomes available during a streaming session, it is recommended that the Player dynamically switch to the Wi-Fi connection after prompting the BlackBerry smartphone user to do so. If a Wi-Fi connection is not available, transport methods that are used by wireless service providers, such as Wireless Application Protocol (WAP/WAP2) or Transmission Control Protocol (TCP), are recommended. Using the BlackBerry® Mobile Data System (BlackBerry MDS) or the BlackBerry® Internet Service is not recommended because media streaming results in large data usage through these transport methods. Both the BlackBerry MDS and BlackBerry Internet Service also result in added latency because the data travels through the BlackBerry® Infrastructure.

Downloads

A complete implementation of a streaming media application is available to download. This example application uses the StreamingPlayer class andStreamingPlayerListener class as libraries, and can be used in your application to stream media over HTTP. This application will only play the media files that are supported by the target BlackBerry smartphone. Make sure that you type the web page and content type of your media file on the Options screen of the application before pressing play.

Note: The API needs an implementation of a thread safe Circular Byte Buffer. You can copy the implementation as CircularByteBuffer.java to rimx.media.streaming package and then complile the project. For an example, click here.

Refer to attachment section below to download the example application, including the libraries.