Speed Control; Trick Play

This feature allows you to control playing speed, to speed-up or slow-down the media. The ability to change playing speed is called, Trick Play. This feature is available in the MediaElement control through a new property called PlaybackRate. This property takes a decimal value ranges from -8, -4, -2, … to …, 0.5, 1.2, 1.4, 2, 4, 8, and the default is 1. The downside of the Beta version is that it doesn’t have a pitch correction, means that when you change the rate to a value other than the default (1) you wouldn’t have any sound. And because of a bug found in the Beta version, you must set this property in code, if you set it in XAML will reset at application start to the default value.

The following are three examples of changing the playing speed of a media element:

Low-Latency Sounds

In the past, you had to create a wave file parser that reads file headers to determine if this is a wave file, and after recognizing a wave file, the parser reads file contents and encode them in order for your application to be able to play it. That changed completely in Silverlight 5. Today, you have native support for raw sounds data (i.e. low-latency sounds.) You can now play wave files and wave data directly in your application without the need to encode them.

This feature is available through a new API based on the XNA Framework. It uses the same classes and functions available in XNA. The main class you have is SoundEffect (Microsoft.Xna.Framework.Audio) that you can use it to play wave files and raw data. You have all the control over the media played; you can set volume, pitch, and stereo space. And you can create a more sophisticated class of SoundEffect called SoundEffectInstance that represents an instance of that sound file. This instance can be played repeatedly, and allows you to have all the control like SoundEffect and more, as we’ll see in the examples.

Finally, keep in mind that Silverlight 5 has the same limitations as XNA; it allows only for 8/16bit PCM, mono/stereo, 22.5/44.1/48khz wave files.

The following code loads the SoundEffect class with a wave file available in application resources:

After you have the SoundEffect at your hands, you can call the Play() method to play it. This method have two overrides, one can accept no parameters in which it plays the media normally with the default volume and no changes in the pitch or the stereo space. The other override accepts three values:

volume:
Ranges from 0.0f to 1.0f.

pitch:
Ranges from -1.0f to 1.0f. Default is 0f.

pan:
Controls the stereo space. Ranges from -1.0f to 1.0f. Default is 0f.

The following are some examples of playing sound files using SoundEffect:

Summary

While Silverlight 5 manages battery better, it also has improved media performance by depending more on the GPU for media decoding. And it allows now for high definition 1080p and H.264 videos. In addition, Silverlight 5 has introduced remote control support. And finally, the greatest features are its support for raw wave files and the ability to control playing speed.

Silverlight 5

The first thing to know about Silverlight 5 is that it’s currently in beta, means that it definitely has bugs, and it also not yet feature-complete; some features are going to be shipped later in the final version. In addition, being in beta means that no go-live license available, this is for internal use and testing only so do not install on user’s machine.

Worth mentioning that Silverlight 5 was first announced at PDC 2010 conference (October 2010) and the beta version was shipped later at MIX11 (April 2011,) and the final version will be released soon this year.

Goal

Goals of the version 5 include:

Improving performance

Getting closer to WPF

Enabling line-of-business scenarios

Better development experience

Improvements

Silverlight 5 has come with lots of improvements and changes, and those improvements can be divided into several areas:

Overview

Stephen Hawking is one of the most famous people using speech synthesis to communicate

In this article we are going to explore the Speech API library that’s part of the TTS SDK that helps you reading text and speaking it. We’re going to see how to do it programmatically using C# and VB.NET and how to make use of LINQ to make it more interesting. The last part of this article talks about…… won’t tell you more, let’s see!

Introduction

The Speech API library that we are going to use today is represented by the file sapi.dll which’s located in %windir%System32SpeechCommon. This library is not part of the .NET BCL and it’s not even a .NET library, so we’ll use Interoperability to communicate with it (don’t worry, using Visual Studio it’s just a matter of adding a reference to the application.)

Implementation

In this example, we are going to create a Console application that reads text from the user and speaks it. To complete this example, follow these steps:

As an example, we’ll create a simple application that reads user inputs and speaks it. Follow these steps:

If you are using Visual Studio 2010 and .NET 4.0 and the application failed to run because of Interop problems, try disabling Interop Type Embedding feature from the properties on the reference SpeechLib.dll.

Building Talking Strings

Next, we’ll make small modifications to the code above to provide an easy way to speak a given System.String. We’ll make use of the Extension Methods feature of LINQ to add the Speak() method created earlier to the System.String. Try the following code:

I Love YOU ♥

Let’s make it more interesting. We are going to code a VBScript file that says “I Love YOU” when you call it. To complete this example, these steps:

Open Notepad.

Write the following code:

CreateObject("SAPI.SpVoice").Speak "I love YOU!"

Of course, CreateObject() is used to create a new instance of an object resides in a given library. SAPI is the name of the Speech API library stored in Windows Registry. SpVoice is the class name.

Save the file as ‘love.vbs’ (you can use any name you like, just preserve the vbs extension.)

Now open the file and listen, who is telling that he loves you!

Microsoft Speech API has many voices; two of them are Microsoft Sam (male), the default for Windows XP and Windows 2000, and Microsoft Ann (female), the default for Windows Vista and Windows 7. Read more about Microsoft TTS voices here.

After we received your feedbacks and comments about th.e article, we decided to add a small appendix to the end of the article about setting information (volume, channel, sampling rate, etc.) to a MCI device (a Waveform device of course.)

Like anything else in MCI, you can set device information using a MCI command (string/numeric), and this time it’s the MCI_SET command.

This command is used to set information about a specific device. This command requires an input parameter of the MCI_SET_PARMS structure. However, that input parameter might have extended members for specific devices. Because we are concentrating of Waveform devices, so we are going to use the MCI_WAVE_SET_PARMS structure that contains the extended members for our device and is defined as following:

This structure contains all and every little piece of information that can be set to a device. I expect that you read the main article and you are already familiar with members like dwCallback (other members are self-explanatory) that we have talked about many times, and you are fine too with function calls and setting up input parameters, so I won’t get into the discussion of the structure or how you are going to use that command. However, if you need more help setting up the input parameters for the structure, you should take a brief look at the MCI_WAVE_SET_PARMS Structure documentation in the MSDN.

As you know, the MCI_WAVE_SET_PARMS unmanaged structure can be marshaled in C# as following:

Congratulations! You did set the device information! So how to get them back?

This can be done through the MCI_STATUS (discussed earlier) by setting up the MCI_STATUS_ITEM flag and setting the query item to the required information you need to query about (like MCI_DGV_STATUS_VOLUME to query about volume.)

Overview

This writing will focus on how you can record sound from an input device and how you can play sound files using MCI (Media Control Interface) in C and C#.

This writing does not involve a discussion or even an introduction to MCI. Instead, it provides technical discussion of what we will need to use to record and to play sound files. If you need an introduction to MCI refer to the MSDN documentation.

We will begin by a discussion to types and functions required to accomplish our tasks. Then we will look on how you can utilize those types and functions in your C or C# application.

Our demonstration examples in this writing will be in C. In the last section we will have a look at .NET and C#. Besides this, there are sample applications written by C and C# attached with the article.

Introduction

Actually, there are many ways to record and play sound files. However, in this writing we will focus on MCI (Media Control Interface) as it is one of the high-level easy-to-use interfaces for controlling all media types.

You can control a multimedia hardware using MCI by either sending commands like Windows messages or by sending string messages to the MCI. The second approach is preferable in scripting languages. As we are concentrating on languages C and C#, we will consider the first approach only.

Because .NET Framework does not include classes for handling MCI (or multimedia at all,) we will need to dig into Windows SDK, specifically WinMM.dll (Windows Multimedia Library) that resides in the system directory.

In order to access this library from your C# application, you will need to create your own marshaling types and PInvoke (Platform Invocation) methods to allow your managed code to communicate with the unmanaged server. That is what the last section of this article is devoted for.

To access this library from your C application, just reference the library object file WinMM.lib in your project. Notice that all of the functions and structures related to the MCI are prefixed with mci.

In the next sections we will talk about the functions and structures that we need to be aware of before digging into coding.

Sending MCI Commands

The key function to Windows multimedia programming using MCI is mciSendCommand(). This function sends a command to the MCI. As we have said, you can program MCI using one of two approaches, you can send Windows-messages-like commands, and you can also send string messages to the MCI. The key function that we will use to send MCI commands is mciSendCommand(). To send string messages, you can use the function mciSendMessage() which is not covered here.

IDDevice:
The ID of the device to which to receive the command. For example, the ID of the input device when recording or the output device if playing. As you know, many devices could be connected to the PC. You can forget about this argument and just pass the function 0 to direct the message to the default device (selected in the Sound applet in the Control Panel.)

uMsg:
The message (command) to be sent. Common messages are covered in the next few sections.

fdwCommand:
Flags (options) of the message. Every message has its own options. However, all messages share the flags MCI_WAIT, MCI_NOTIFY, and MCI_TEST (covered soon.)

dwParam:
A structure contains the specific parameter for the command message.

As a result, every command (message) has its name, flags, and structure parameter.

This function returns 0 if succeeded or an error code otherwise.

The Wait, Notify, and Test Flags

Common flags for all MCI messages are, MCI_WAIT, MCI_NOTIFY, and MCI_TEST.

The Wait flag, MCI_WAIT, indicates that the message should be processed synchronously; means that the function would not return until message processing finishes. For example, if you send a play message with the MCI_WAIT, your application would be suspended until the entire file finishes. Therefore, you should not use MCI_WAIT for play or record messages.

The Notify flag, MCI_NOTIFY, is the reverse of the Wait flag. It indicates that the message should be processed asynchronously; means that the function would return immediately and does not wait for the completion of the message. When the processing of the message finishes, it sends a MM_MCINOTIFY message to the windows specified in the dwCallback member of the message parameter. This flag should be used for play and record messages.

The wParam value of the MM_MCINOTIFY message is usually set to either MCI_NOTIFY_SUCCESSFUL to indicate command success or MCI_NOTIFY_FAILURE otherwise.

The Test flag, MCI_TEST, checks if the device can process this message, it does not process the message. The function would return 0 if the device can process the message or non-zero otherwise. You will use this flag in rare cases only.

Keep in mind that you should choose only one of the three flags, you cannot combine them.

If you didn’t specify any of these three flags, the call would be treated asynchronously and you will not be notified when it completes.

Handling MCI Errors

Every MCI command could succeed or fail. If the command succeeded, mciSendCommand() returns 0 (FALSE/false.) Otherwise, it returns the error code.

MCI defines error codes as constants that are prefixed with MCIERR_ like MCIERR_DEVICE_NOT_INSTALLED and MCIERR_FILE_NOT_FOUND (names are self-explanatory.) You can get the friendly error message of the code using the function mciGetErrorString(). The definition of this function is as follows:

Actually, you will make use of only the third and fourth members, lpstrDeviceType and lpstrElementName, of the structure when you open a device. lpstrDeviceType determines the type of the device (digital-audio, digital-video, etc.) that will be used. In our example that records and plays sound files, we will set this member to €œwaveaudio€ to indicate that we are going to work with waveform (WAV) data.

lpstrElementName on the other hand, should be set to an empty string (that is €œ€) if you are opening an input device for recording. If you want to play a file, set this member to the full path of that file.

Common flags of the command MCI_OPEN are:

The Wait, Notify, and Test flags:
The Wait command is usually used for MCI_OPEN.

MCI_OPEN_ELEMENT:
Mandatory. The lpstrDeviceType of the MCI_OPEN_PARMS is set. It is set to €œwaveaudio€ for WAV data.

MCI_OPEN_TYPE:
Mandatory. The lpstrElementType of the MCI_OPEN_PARMS is set. It is set to an empty string if recording or a path to a file if you want to play it.

You will always combine the flags MCI_WAIT, MCI_OPEN_ELEMENT, and MCI_OPEN_TYPE for the MCI_OPEN command.

When the function returns, the wDeviceID member of the structure is set to the ID of the device opened. You should keep this ID for future calls on that device until you close it using the close command.

The following C code shows how you can open an input device for recording:

Starting Recording

After you have opened the input device, you can order the MCI to start the recording process using the command MCI_RECORD. This command requires an opened input device and a parameter of type MCI_RECORD_PARMS. The definition of this structure is as following:

dwCallback:
The window handle that should be called after the processing of the command finishes if MCI_NOTIFY is specified in command flags.

dwFrom:
Indicates the position of the buffer (in thousands of a second) to start recording from. In most cases this would be set to zero.

dwTo:
Indicates the position of the buffer to stop recording when it is reached. Unless you want to record for a given period, this member should be zero.

Common flags for MCI_RECORD are:

MCI_WAIT, MCI_NOTIFY, and MCI_TEST:
Usually you will use the MCI_NOTIFY flag. If so, you should handle the MM_MCINOTIFY message.

MCI_FROM:
Set if you used the dwFrom member of the parameter.

MCI_TO:
Set if you used the dwTo member of the parameter.

If you want to record for a specific period, set dwTo member of the structure to that specific period (in thousands of seconds) and combine your flags with MCI_TO. When this period ends, MCI automatically stops the recording process and it sends a MM_MCINOTIFY message if you have set MCI_NOTIFY in the flags.

Resuming Recording

To resume after pausing, you can send a MCI_RESUME command to MCI. This command is very similar to the MCI_PAUSE command. It accepts the same parameter and the same flags. The example is the same, just change command name.

Stopping Recording

After you finish recording you will need to stop the recording process. To accomplish this, pass the MCI_STOP command along with the device ID and its parameter to the MCI.

This command is the same as MCI_PAUSE and MCI_RESUME. It accepts the same flags and the same parameters, and the example is identical too, just change the command name.

Retrieving Buffer Length

How long have you been recording? This could be easily answered with the MCI_STATUS command. This command queries MCI and retrieves information about the current session.

MCI_STATUS accepts a structure parameter of type MCI_STATUS_PARMS which is defined as following:

dwReturn:
After returning from the call, it should contains the return value of the query.

dwItem:
The item to query about.

dwTracks:
Length or number of tracks (specific to some query items.)

Common flags that MCI_STATUS accepts:

The Wait, Notify, and Test flags:
You will usually use the Wait flag.

MCI_STATUS_ITEM:
Mandatory in most cases. Indicates that the dwItem of the structure is set.

If you want to query MCI about specific information, pass the MCI_STATUS command to the MCI along with MCI_STATUS_ITEM and MCI_WAIT flags, and set the dwItem field of the parameter to one of the following values (some are for output devices):

MCI_STATUS_LENGTH:
Retrieves the total length of the buffer (in thousands of a second.)

MCI_STATUS_MODE:
Retrieves current mode of the device which can be one of the following values (names are self-explanatory):

MCI_MODE_NOT_READY

MCI_MODE_PAUSE

MCI_MODE_PLAY

MCI_MODE_STOP

MCI_MODE_RECORD

MCI_MODE_SEEK

MCI_STATUS_NUMBER_OF_TRACKS:
Retrieves the total number of tracks.

MCI_STATUS_POSITION:
Retrieves current position (in thousands of a second.)

MCI_STATUS_READY:
Returns TRUE (true in C#) if the device is ready or FALSE (false in C#) otherwise.

When the function returns, the dwReturn field of the structure should contain the result of the query item selected.

Playing a Sound File

Keeping in mind what you have learnt from the previous section would be very helpful in our discussion on how to play sound files using MCI.

Actually, the same rules and commands applied in the previous section, will be applied here.

To play a sound file on MCI, you should follow these steps:

Load the audio file. This automatically opens the output device.

Order the MCI to start the playing process.

Close the opened device when you finish.

Again, you should check whether an error occurred or not after each call to mciSendCommand().

Loading the File

As you know, to open a multimedia device, you pass MCI the MCI_OPEN command. To specify a file to be loaded, specify the full path of the file in the lpstrElementName field of the MCI_OPEN_PARMS structure.

The following C example shows how you can open an output device and load a file for playing:

At first sight, you notice that it is identical to the MCI_RECORD_PARMS. Actually, you are right. This structure defines the same members as the MCI_RECORD_PARMS structure. In addition, the description of the members is the same.

dwFrom specifies the position (in thousands of a second) where to start playing. dwTo on the other hand specifies the position (in thousands of a second too) where to end playing. If you set dwFrom, you will need to set the flag MCI_FROM. Conversely, if you set dwTo, you will need to set the flag MCI_TO. If you need to play the file from the start to the end, leave both members and do not specify either MCI_FROM or MCI_TO.

You will most likely combine MCI_PLAY flags with the Notify flag, MCI_NOTIFY, to allow the code to continue execution while the file plays. If you did so, your application would receive MM_MCINOTIFY message to the window specified in the dwCallback member of the parameter structure.

Pausing

As you know, you can pause the playback using the MCI_PAUSE command which accepts the MCI_GENERIC_PARMS parameter.

Resuming

To resume the playback after pausing, you can use the MCI_RESUME command discussed before.

Retrieving Current Position

To retrieve the current position of the file, pass the MCI_STATUS command along with its flags (MCI_WAIT and MCI_STATUS_ITEM) and its parameter (MCI_STATUS_PARMS) to the MCI. Do not forget to set dwItem member of the structure to MCI_STATUS_POSITION to retrieve the current position (in thousands of a second) in the dwReturn member of the structure. The following C example demonstrates this:

Receiving MCI_MMNOTIFY

In C, you can handle the MCI_MMNOTIFY message in the way you handle any other message. Just add the handler to the window procedure (WndProc) function.

In .NET, you do not have a WndProc function. However, .NET emulates this function using the protected Control.WndProc() function. You can override this function in your form and do the required processing. The following example demonstrates this:

Appendix A: Setting Device Information

After we received your feedbacks and comments about the article, we decided to add a small appendix to the end of the article about setting information (volume, channel, sampling rate, etc.) to a MCI device (a Waveform device of course.)

Like anything else in MCI, you can set device information using a MCI command (string/numeric), and this time it’s the MCI_SET command.

This command is used to set information about a specific device. This command requires an input parameter of the MCI_SET_PARMS structure. However, that input parameter might have extended members for specific devices. Because we are concentrating of Waveform devices, so we are going to use the MCI_WAVE_SET_PARMS structure that contains the extended members for our device and is defined as following:

This structure contains all and every little piece of information that can be set to a device. I expect that you read the main article and you are already familiar with members like dwCallback (other members are self-explanatory) that we have talked about many times, and you are fine too with function calls and setting up input parameters, so I won’t get into the discussion of the structure or how you are going to use that command. However, if you need more help setting up the input parameters for the structure, you should take a brief look at the MCI_WAVE_SET_PARMS Structure documentation in the MSDN.

As you know, the MCI_WAVE_SET_PARMS unmanaged structure can be marshaled in C# as following:

Congratulations! You did set the device information! So how to get them back?

This can be done through the MCI_STATUS (discussed earlier) by setting up the MCI_STATUS_ITEM flag and setting the query item to the required information you need to query about (like MCI_DGV_STATUS_VOLUME to query about volume.)