Hi!Last night I broadcasted signal grabbed from my soundcard's line-in. I used OddCast+IceCast. OddCast was set to save archive of stream to disk. After hour IceCast crashed resulting in OddCast reconnect. After the broadcast I found two dumped OGG files, but first of them had no OGG container header (no "vorbis" text present in header). Second file is OK. If I'm getting it right, Vorbis within Ogg container has the following format:

If the stream has no header then you are probably not going to be able to salvage it. The header is the most important part of the stream due to the fact that it contains all of the setup information including codebooks, granulepos, vendor string, etc. Try running the file through ogginfo.exe and see what it tells you about the file.

A lot of people who are streaming from an icecast or oddcast server have this problem. Basically what I am going to tell you to do is attempt to use vcut.exe to try and cut the stream at specific granulepos. Seeking with an Ogg container is relative to wherever the granulepos is set within a page. You might be able to salvage or whatever is left of it if you truncate it down past a specific position. This is the specific problem that of people who streaming with Vorbis complain about. Why was the Ogg bitstream designed with all of the codebooks and other specific packet information at the start of the stream? the logical and physical bitstream structure though was very clever. I can't think of anything else , although that might be worth a shot.

Your post indicates you know how the headers should be at the start of the file. ie Main header, comment header, code book header.

In order to replace the headers you need to know what specifically they were... this might be a bit of trial and error, however there's a fairly good chance that they were a fairly standard configuration. There's a few things that can vary... the stuff in the first header is fairly likely to be easy to guess, ie it's probably 44100 stereo, grab the first page from some other vorbis file that is the same. The comment header is irrelevant, you can put any old comment header in there, you can always use a tool to change the comment header later.

As to the third header, this will depend on the encoder settings use to produce the file. You may be lucky and it's a fairly "default" encode, in which case you will probably be able to cut this header from a file that was encoded in a similar way, ie with the same encoder settings. There's a fixed set of codebooks, if this file was made by something other than the reference encoder, you may have to do a lot of trial and error to find one that will work.

Basically, what i would do is... get a hex editor, make a backup copy of your file. Then from the start of the file, look forward until you see the first OggS, if there's one right at the start, do nothing, if not, cut everything away upto the first one... that small bit of data is not likely to be salvagable.

Now you have a file with a clean page start. Now try inserting the headers... this will be the first 2 or sometimes 3 pages you will have to do some mental decoding to see this, though if the third page has granule pos 0, it has the continuation flag set, has only packet and it's fairly small, chances are this 3rd page is part of the headers. Cut this section from some other file, which you beleive was similarly encoded.

Save it into it's own file, then make another copy of this little file.

Now look in your broken file, look at the serial number field, now, in your little file with the headers, you need to edit the serial number field. Now you have a problem, the checksum on the header page will be wrong. There is no easy way around this unless you can do some programming to recalculate what it should be. Most player will reject the file if the checksums are incorrect.

However, i know my directshow filters have a bug and they don't check the checksum... so at this point you could maybe try and play the file, and you will at least know if you guessed the right headers, and then you can worry about fixing the checksum later.

On the downside, my filters scan the whole file at load time, so if there is something funky later on in the file... or if this file is chained (which my codecs don't handle yet), it may barf right there. A workaround, is to put it on the web, and use my filter to load url..., since it does not scan the whole file in a network stream, so it will play (assuming the headers are right) until it reaches something that is not right, and then either stop unexpectedly, or possibly crash

Once you know you have he right headers, then you fix the header page checksums... and possibly fix up any garbage that is at the end of the file.

So, all in all, it's almost definately possible to save this file... but it requires some knowledge of the bitstream, and some programming skill, there is nothing automated that i know of that will fix this, and it's unlikely there ever will be, because there's no easy way to guess which headers should be used. Though someone with a very in depth knowledge of the vorbis packets could possibly make some reasonably educated guesses.

So ask yourself... how bad do you want to save this file... and how much time are you willing to expend to do so.

If you send me the file, use www.yousendit.com and enter my email address ogg@illiminable.com after you upload it there, it will send me a link to download it from the yousendit server. I will try to have a look at it, but i can't make any promises that i will actually be able to fix it, or that i will be able to get around to quickly, so if you what i wrote above sounds like you can do it, then i'd suggest you do it, as you are obviously far more motivated to recover this file than me.

In order for this to even be possible i would need as much information about what settings you think it was encded at, also another file (probably another file from this ssame stream) that you suspect was encoded the same way, what version of icecast/vorbis encoder/other software used etc. Even in the best case this is time consuming. You should hope that it was made with the reference encoder(ie libvorbis_, if it was made with any other encoder (ie AoTuV) it may not be possible.

Havn't tried Avi-Mux but I'd be verysurprised if it could generate proper headers out of nothing

commenting on post #6:

One could also encode another sample (of the same format) with the same settings (including the serial number -- it can be forced afaik). Since the serial number is now the same you can just try to copy the first OggS pages (upto the setup header) and place'em in front of your recorded stream. No manual CRC calculation necessary.

Why was Ogg container designed so complex? The inability to cut Vorbis streams is a serious disadvantage compared to MPEG formats. Would it seriously hurt quality if each Ogg frame was self sufficient? To minimize overhead frames (Ogg pages?) could simply be longer.

Unlike 4 Bytes MPEG frame header, Vorbis stream header is ~2 kB long, so including it in every frame would really be an overhead. I'm not such an expert to say why, it might contain some dynamic codebooks or whatever.

The claim about "inability to cut Vorbis streams" is obviously false. All what is needed is that programs which save Ogg Vorbis streams save them properly. I don't know why it was broken in this case regardless of the crash, Ogg is supposed to be written linearly.

However I wonder why nobody suggested to the OP to take the headers from the second file from the same session, which was fine. The headers would be the same because it was apparently one encoding session.

I have the same problem as stick. I used wget to grab a chunk of a stream, but no tool I can find will let me cut out the interesting bits. They all crash or hang. foobar2000 plays the file beautifully.