I encoded a wav-file (Eurythmics' Sweet Dreams) to ogg using Oggenc RC3 -q 5.0, then decoded it to wav. The decoded Ogg was then inverse-mixed (using cool edit) with the original wav file, then compressed with Flac.

Originally posted by JohnV Umm difference signal and decoded ogg mixed together should amount the original.

Do you really think that the mixing, the decoding and the additional mixing are accurate enough to give you a WAV with the same CRC as the original? I have to see proof before i believe that.

QUOTE

The idea is that you can first download the ogg, if you later want to make it lossless you'll save few MBs worth of downloading by downloading the recovery file instead of original lossless.

Okay, but you could as well download the file off WinMX in 128 kbit MP3 'quality' first, listen to it, and if you like it, download the full FLAC (from whereever) and delete the MP3. Cause if you don't like the downloaded OGG, why would you wanna keep it?

I think this is an awesome development if in fact it would result in identically CRCd files.... when WAVpack came out with this feature a few weeks ago, I tried it out and thought it was an awesome idea, but wished it could be implemented in a codec better tuned for lossy compression - wavpack's lossy 320 kbps, although I didn't do extensive listening tests, I would imagine would be nowhere near the quality of ogg, mpc, or even mp3, as it has not undergone the extensive tuning those formats have.

For those who don't see the point of this, the reason I think its useful is because I want to encode my entire CD collection, but the current state of audiocoding is in such flux right now, I am having a hard time deciding on a format... do I go with MPC for quality, but then have to transcode to mp3 for hardware players? Do I go with ogg and hope that hardware player support that has been talked about actually comes through? If so, do I go ahead and encode now, or wait for 1.0? A feature such as this solves this problem, as I could encode in ogg for now, burn all the correction files to backups, so I have something to listen to for now, without requiring the massive disk space that would be required to store them all lossless on my HD... but then when the state of audiocoding settles down and I decide on a final format, I can restore the original waves and reencode without worrying about quality loss from transcoding.

If you inverse-mix the OGG decoded and original, it'll give you mostly fuzz, which these lossless coders really dont handle good. If there would be some lossless encoder dedicated to compress these fuzz files, you'd achieve a higher ratio.

I still have a hard time understanding the philosophy behind this all, Randum.

Are you telling me that you prefer a backup with two seperate files, of which a) the Ogg file will possibly become worthless when you settle on a different format/version in the future, b) the FLAC file _alone_ is already worthless, and c) where you can't re-encode without taking both files and joining them first... over a backup with one lossless FLAC copy and one small file for your hard disk, encoded with what you like best at this very moment, and where you can easily re-encode from the FLAC files off your CDs, when - for instance - Ogg Vorbis 1.0 is available?

Well, the more I think about it, the less convinced I am that madah's proposed method would actually result in bit-identical original wav's... however, if it could be made to work, I still think it would be very useful. CiTay, your suggestion that for my purposes I should just encode 2 versions, a lossy and a lossless, and burn the lossless to backup... that would, of course, work for my purposes, it would just bug me that I'm using backup media unnecesssarily for redundant data. With wavpack, the correction files are significantly smaller than the entire lossless file, so it would increase the number of backups I could fit on a CDR (or whatever backup medium I use)... if the same could be said of the hybrid ogg method (or... if it does work, it could be equally applicable to mpc) you would have the same benfit, just with better sounding lossy files. Essentially the only reason to do your way over mine/madah's would be all the additional steps required for the process of generating and compressing the correction files... which is of course a totally valid reason - I would never consider using this method unless someone (me?) took the time to write an automated tool to do this kind of thing in a simple one step frontend.

Originally posted by CiTay Do you really think that the mixing, the decoding and the additional mixing are accurate enough to give you a WAV with the same CRC as the original? I have to see proof before i believe that.

Create lossy file - Create difference file lossy/original - Compress difference file using a lossless scheme.
With this it is possible to get a bit accurate reproduction of the original. Interesting idea yet I do not see the benefits.

The size difference of the two lossless files in this case is only little more than 12%. If you want to backup for later reencode then why not just "waste" those 12% and have files you can reuse? Imagine: you decide to settle on ogg 1.0 but maybe someday you will want to reencode that also. So you need to create another set of recovery files etc. Even if automated this is still time and media consuming. Or another one: you loose your lossy files (hdd crash, accidental deletion...) and think: if only I had not saved those 12% I could still listen to the music today.

Perhaps you will be better off using general purpose compression since, as Tom pointed out, you are not necessarily encoding tonal data.

Originally posted by Gecko Or another one: you loose your lossy files (hdd crash, accidental deletion...) and think: if only I had not saved those 12% I could still listen to the music today.

Perhaps you will be better off using general purpose compression since, as Tom pointed out, you are not necessarily encoding tonal data.

Yes, I thought of this too, and I think its the most convincing argument against this method - and why I'm probably not going to actually do it. Still, I just think the idea has a coolness factor to it

I think that FLAC (and other lossless compressors) would probably do pretty well on the difference files. They are still 16-bit stereo audio and all those programs are adaptive to some extent, although a little alternate tuning might help. I'm sure they would still compress much better than a general purpose compressor.

However, if someone wants to use this method, they should store the decoder executable along with the ogg files. Different versions of the decoder would very likely produce slightly different decoded versions (just like different MP3 decoders). While these differences would probably not be audible, they would cause the "recovery" file to no longer work exactly the same (i.e. bye-bye lossless).

There are two situations: a) I want to encode an album where I have the original CD. b) I want to encode an album where I will not have the original CD in the future.

For case a), there is no need to create a recovery file and burn it to disk. Just re-rip the original CD and re-encode.

For case b), I pick a lossy format that offers significant headroom and just bite the bullet and realize it will be a permanently lossy copy. For these situations, I currently use "mppenc --xtreme --nmt 16 --tmn 32". Avg bitrate ranges from 275-325kbps...a filesize that I can manage with current disk space realities. It will be extremely rare if I can tell these MPCs apart from the WAVs.

For case b), if in the future I realize that I shouldn't have used a lossy format and wished I had the original WAVs again to re-encode, I could always buy the CD. If this CD was known to be OOP/rare/etc, I would have used a lossless format like LPAC or MAC.

You're neglecting a couple of scenarios. The whole point of this is to not HAVE to 'bite the bullet'.... as stated, the reason I want to do this is so that I can have small file sizes on my hard drive, yet still be able to reencode them from the original wav's at a later time should either a) a new audio codec comes out that I like better, or b) I hear artifacts in my lossy versions, and decide that I should have encoded at a higher bitrate.

Of course, there are other reasons which have been pointed out not to do it this way, most notably the fact that just storing complete, lossless versions rather than correction files would take only a bit more space, and serve the dual purpose of avoiding 'bullet-biting' and serve as an actual backup should you have a drive crash (or brain fart) and lose your lossy versions. This, however, is the rationale for wanting this kind of functionality in the first place.

Sometimes we need to take a step back and ask "what exactly are we doing?"

There are two things we must consider: diminishing returns and opportunity costs.

Tonight I encoded Live's A Distance To Here album in MAC's Normal mode and the average bitrate came to 988kbps. This is simply unacceptable compression from a practicality standpoint; not that it's MAC's fault, but lossless compression does not pay for its benefits, IMO. The same album encoded with my MPC settings required 284kbps, less than 1/3 of the lossless size. I simply cannot tell the MPC files from the originals. It's more than "good enough"...my ears think they are flawless.

If my original album were destroyed and I could never, ever get another original copy, I probably wouldn't "care" from an audio standpoint. What am I losing? Some wispy fine detail -60dB down in level that my ears can't hear anyway? Oh well, life goes on.

I personally believe you can get "perfect sound" from existing lossy codecs. It's all a matter of bitrate. Can any human tell the difference between a 450kbps MPC/Ogg file and a 900kbps lossless? I don't think so. That's diminishing returns.

We also have time limitations. Time IS money, to use a oft-used phrase. To go through this effort of making two files with two separate encoders, then burning one of them to media that can only handle 700MB (just a few albums worth)...forget it. In 10-15 years, I will probably rebuy my albums all over again in 24-bit, 96KHz format, so really, I think you can really go overboard with perfectionist tendencies. I found an encoder and a command switch combo that I'm very happy with and I am sticking to it.

From an academic standpoint, this lossy+FLAC exercise might be interesting, but it's not practical nor particularly necessary for "normal listening" needs.

One more thing, correct me if i have got this idea all wrong, but you'd store the 'lossless' file on CD for example and the ogg's on your harddrive and combine them again to create the original wav so you save a few megs on the media. What happens if you lose all your ogg's (recovery files) in the event of a hard disk faliure, won't the archived files then be of no use because they can't be recovered??

Sorry if i got this idea all wrong, seems i may have done since this hasn't been mentioned yet, but if it is correct then it's an important factor, you would infact need to bck up the ogg onto the CD or other medium as well!

Just a quick question to mithrandir. Is your modified mpc command line "mppenc --xtreme --nmt 16 --tmn 32" better than the standard --insane ? And if so why didn't you use a modified version of insane. I ask this because your average bitrate seemed to be a lot higher than I get with insane and 0.90s.

What happens if you lose all your ogg's (recovery files) in the event of a hard disk faliure, won't the archived files then be of no use because they can't be recovered??

Yes you're right. I personally would never use this for backup purposes, unless both the ogg and flac are stored. But storing the original flac alone will result in less space so this is useless...

I do find the idea of a lossy+recovery combo very interesting though...

It would better be used for other purposes, for example: You have a link to an insane 1400 kbps Ogg (let's say it's lossless). You download only so you get 128 kbps bitrate. Later you find you'd like to have better quality, so you download a difference-stream of 64 kbps, so total bitrate of the file will be 192 kbps. And if you want full quality, you only have to download 1208 kbps (1400-192) compared to the full file.

Maybe Ogg bitrate-coupling will work this way, or is it only capable of scaling the bitrate down?

Originally posted by Timmy The Turtle Is your modified mpc command line "mppenc --xtreme --nmt 16 --tmn 32" better than the standard --insane ? And if so why didn't you use a modified version of insane. I ask this because your average bitrate seemed to be a lot higher than I get with insane and 0.90s.

As I learned from Dibrom in another thread, --xtreme sounds better than --insane at equal bitrates. This is because --xtreme uses the ATH whereas --insane encodes the full 22.05KHz bandwidth, whether your ears can hear all of it or not. Therefore, --insane "wastes" bits that could otherwise be applied to other parts of the audible frequency spectrum. That's why I modified --xtreme rather than --insane: optimization.

--xtreme produces smaller files than --insane, so generally --insane sounds "better" by default. However, the --nmt and --tmn switches allow you to increase headroom beyond a preset's default setting (thereby eating up a lot more bits). Indeed, --xtreme --nmt 16 --tmn 32 produces files that are larger than --insane. That's because --insane's default nmt and tmn settings are lower (i.e. less sensitive, requiring less bits).

Yes, my settings produce files that are measurably better than --insane. It uses more bits and allocates them more optimally. But that's not to say that --insane is inferior from an audible perspective. I'm not claiming that I can hear a difference between --insane and my settings...I use these settings because I don't mind the larger files and know that I have files that are well-suited for transcoding, should the future need arise.

lossy+recovery is not useless. There are definitely uses for this. Wavpack, for example has an implementation of this. The uses of this idea has been discussed before too. I admit it is not that useful compared to many other things, definitely not to the point of being crucial, but it is useful nevertheless.

@Benjamin
What you described is a perfect optimal implementation of bitrate peeling, but unfortunately it won't be that simple. You can't peel a -q5 vorbis stream down to -q3 bitrate and expect it to sound as good as a -q3 encode, but it won't be that far behind. The way I understand how bitrate peeling works is that the bits in the frames are packed so that the most significant information are packed at the start of the frame and the least significant information at the end of the frame. When you need to peel, you can chop off or truncate the end of the frame and lose only the least significant information, and the stream plays back without any problems.