I'm not sure if this issue is with the Flash Player when encoding the stream or with the Flash Media Server that is saving the stream so I'll be posting in both forums.

The problem is when publishing an audio and video stream from Flash Player 11 (Windows, FF) to the Flash Server 4.5 using the H.264 video codec. The audio portion of the recorded stream doesn't appear to be playable by anything and FFMPEG cannot process the audio stream. Here are the differences in the stream:

When using Sorenson as the video codec and recording the stream (flv) from the Flash server, the information about the audio stream from FFMPEG is this:

Stream #0.0: Audio: nellymoser, 22050 Hz, mono, s16

This clip can be played and processed by various players and FFMPEG with no problem.

When using H.264 as the video codec and recording the stream (f4v) from the Flash server, the information about the audio stream from FFMPEG is this:

Stream #0.1(eng): Audio: nmos / 0x736F6D6E, 22050 Hz, 1 channels

This clip does not play the audio and cannot be understood by FFMPEG.

I tried using the Speex codec as well and I have a similar issue as above (it doesn't play the audio stream and FFMPEG can't understand it).

Note the version of FFMPEG I'm using supports both Nellymoser and Speex so there should not be an issue.

1) Any idea why the codec appears different and won't play.

2) How to achieve the goal: send H.264 video stream with audio from the Flash Player and have a recorded stream with playable audio.

After you record the stream to Flash Media Server, process it with the F4V Post Processor tool to play it back on the desktop or using progressive download (you can stream it without using the Post Processor):

Even after running it through the post processor, the audio does not play. Read this thread (the posts in the middle discuss the audio problem, specifically no audio after running through the post processor): http://forums.adobe.com/message/3677448#3677448

It does not play using progessive download, won't play in any players, and can't be recognized by FFMPEG.

I'm not sure if this is an issue with the Flash Player when encoding it, or the Flash Media Server when saving it, but the fact you can see the same audio codec appears differently to FFMPEG so something is going on. This is a major problem!

My FP11 H264 recording content plays fine, when live and as vod content via FMS. However nothing recognises it like, media info, mp4info, mpui or FFmpeg when inspected and even VLC Player can not play it. FLVCheck says its ok. MP4UI did say the atom size was too small, so I can only think FP11 is not creating the container within the MP4 spec.

I would like this raised as a bug! Point me to the correct url please.

So did you do what Jay said and it now can be processed? So Jay you are saying instead of saving it as MP4, I should tell FMS to save the file as an .flv, then run it through FFMPEG to convert it to a standard MP4? Sounds strange, but if it works I'll try anything because as of now there is no way to play those files without sending them through the server.

The AAC encoder in FFMPEG (FAAC) is terrible, and tends to result in poor quality at low bitrates. If you're transcoding to AAC, you'll likely want to use 128kbps or higher. If you workflow and endpoint targets allow for it, I'd go with MP3 audio.

Thanks for letting me know. I'm already using that combination (H.264/MP3) with the clips generated by the Flash Media Live Encoder so I'm good there. The problem of course is there is no way to encode MP3 audio in FP11 along with the H.264 video. For regular voice, do you find Nellymoser or Speex preferable to start with (knowing it's going to end up as MP3 in the end)? I've always just used Nellymoser. Obviously if MP3 was available, this wouldn't be an issue.

From a quality perspective, I prefer Nellymoser. Speex encoding in FP is limited to 16kHz, where Nellymoser is capable of 44.1khz. For voice, the difference isn't all that noticeable, but if there's any music involved, you'll find that the sounds is a bit dull when using speex, as the higher frequencies aren't represented.

Also, if your application involves multiway communication and you want to take advantage of AEC, your only option is Speex, as AEC doesn't work with Nellymoser.

From a licensing standpoint, Speex is the clear winner. To be completely legal, you'd need a license from Nellymoser to decode the audio on the server side using FFMPEG, and that license is $7500 per year per server the last time I checked. Speex on the other hand is not patent encumbered, and (AFAIK) requires no licensing.

That said, if you're in a position where codec licensing is an issue, you'll want to look into an MP3 encoding license for your implementation as well.

As I mentioned before, I've been using Wowza lately (and h.264/speex to flv works fine there), so perhaps this is just something that FMS won't do. Maybe someone from the FMS engineering team will have suggestions about how to record a file that anything other than FMS can read.

So from your experience you wouldn't say it's an issue with how the player is encoding it, but rather how FMS is saving it on the back end? At least determining that would help isolate where to go for help (although I don't see any other suggestions coming from employees in this thread yet).

I know this thread has moved beyond the F4V Post Processor (thanks for all the info, Jay), but I wanted to let you know that the audio bug mentioned in the previous thread has been fixed. Unfortunately, I didn't grab the fix build number and I don't have access to the bug base anymore, but maybe someone from Adobe could grab it for us.

I think what Jody was getting a was the inclusion of ulaw and alaw (pcm) audio in the flashplayer. The trouble there is, unless you're going to use a really low sample rate, the bitrate required for pcm is going to be very high, and (if I'm not mistaken), the flashplayer will not perform AEC for pcm.

@AdrianWilkinsUK - I would expect that you can't play those files outside of FMS without transcoding. Speex and nellymoser aren't mp4 compliant. They -might- play in the flashplayer (haven't tried that... but maybe the FP eng team has something in place there), but other programs won't.

If you check, you'll see in my other thread http://forums.adobe.com/thread/921501 an employee claims this was fixed with the latest version of FP11. However, I tested it and the problem still persists.

I have the feeling this is an FMS issue. When I play the H.264/speex FLV recorded through Wowza in flashplayer 11.0.1.152, it plays back just fine for me, both streaming from Wowza, and via progressive download.

Thank you for the info. Apparently this is a bug in the Post Processor tool. The responsible team is aware of it and they are working on a fix. Of course I can't tell you when it will be fixed or released but it is in the works.

that was in August and nothing since. It seems a fairly major bug that you can't have audio in your saved video. I can't find anywhere to contact that team for an update, and the problem seems to have been forgotten or ignored. If anyone from Adobe is looking in, an update would be appreciated.

When encoding live in H264 with the FP 11 then playing it live on another client (through FMS r) the video sometimes (1/3 of times) has major skips or jerks. It's almost as if the video goes back to some older keyframes and then returns to live. Same experience with non-h264 produces smooth video.

If the server (FMS ) records the video, the replay looks smooth, which is strange because I would have assumed the FP was not encoding well with H264.

@arnold.25: yes i am also facing similar issues. when i googled for it, found out that its because of either the buffer or bandwidth causing the issue. so need to set some buffertime on your playback client end or the bandwidth should be sufficeint enough to play the stream. but when i tried the first method it intriduced a lot of delay and the second method is expensive. so i am in search of some other solutions for this.

Thanks for your time. I could resolve this at my end. It was nothing but the swf version mismatch on my compiler-settings. I was using the Flash player 11.2 and the compilet option was setting -swf-version to 14. So H264 encoding had this jerking issue. But once give the appropriate version # it got resolved. I got the version detatils from here