Near-lossless / lossy FLAC

Ah, I see. Well, it might be interesting to try. It sounds like you're tempted to do it!

Josh PM'd me to point out that the FLAC frame/block size can be set from the command line using the -b command. I've just tried it, and it works as expected: with the MATLAB code changed to use 1024 sample blocks, it seems I get better compression, but I need to try more samples. On the one I tried (41_30sec) this shaved another 20% off, though that sample encodes 3% more efficiently in 1024 sample blocks than 4096 blocks anyway.

Cool!. Could you clarify the "Notes" paragraph in the frame header section, please? What blocksizes are allowed if it's a variable length block stream? I'd use "1000-1111 : 256 * (2^(n-8)) samples" but it looks like they are not allowed.

all that convoluted logic will be going away with the next version of FLAC thankfully. I'll publish the details with the next release of FLAC (hopefully no later than july)

Near-lossless / lossy FLAC

I tried all the samples you provided, and couldn't abx them except for the second half of furious:With my first trial I got at 4/4 with furious, but I missed several times with the following guesses.With my second trial I got at 6/6, then 7/8, finally 8/10. Not a totally convincing result but maybe enough to show that furious lossy isn't totally transparent.This is pretty similar to what I have learnt from wavPack lossy behavior for furious.Anyway the difference is so subtle I can't really describe it. Just a minimal lack of precision may be. Not serious at all.

Can you provide Atem-lied, herding_calls, trumpet, harp40_1 please? These and all the other samples in the presumably more efficient short block version?

Near-lossless / lossy FLAC

Using a two-part file extension (e.g. .lossy.flac) should solve the compatibility problem, at the expense of longer filenames. Proper tagging is needed, of course, as filenames alone cannot be trusted.

Near-lossless / lossy FLAC

I tried all the samples you provided, and couldn't abx them except for the second half of furious:With my first trial I got at 4/4 with furious, but I missed several times with the following guesses.With my second trial I got at 6/6, then 7/8, finally 8/10. Not a totally convincing result but maybe enough to show that furious lossy isn't totally transparent.This is pretty similar to what I have learnt from wavPack lossy behavior for furious.Anyway the difference is so subtle I can't really describe it. Just a minimal lack of precision may be. Not serious at all.

Thank you for ABXing halb27.

I thought I could ABX the background noise at the end of Furious, but then failed. I'm not sure if I'm imagining it, or if it's nearly audible.

If you're in the mood to play, please try the attached files. I've played with the thresholding. I've also (intentionally) broken the lossy part by dithering the LSB itself so you can't cheat and look at the FLAC bitrate!

At least one of these files has more noise (so it's not as hard a job as it looks). At least one has less noise. So you should be able to ABX at least one, and maybe cannot ABX at least one. See what you think.

If you do have time to ABX, please decide upon the number of ABX tests before you start, and stick to that. As you probably know, selecting results or re-starting messes up the statistics.

Of course anyone is free to try.

Quote

Can you provide Atem-lied, herding_calls, trumpet, harp40_1 please? These and all the other samples in the presumably more efficient short block version?

ADDED:I forgot badvilbel. Can you provide badvilbel please?

I'll do those as time permits. If I get chance before you've responded, I'll post them at the default settings. However, if your next response suggests I need to reduce the noise addition slightly, then I'll post them with a less aggressive setting.

Near-lossless / lossy FLAC

BTW I don't concentrate on the background noise but on the accuracy of the 'main signal' in the second half of the track.

As for abxing if the question is 'is track X transparent?' I always allow for a second trial in case I have the impression that there is a difference and get at a result like 4/4 before I start to go wrong (due to possibly fatigueness). The number of guesses is fixed before a test (usually 10 guesses, sometimes , but in case I don't see what to concentrate on I often give up within the first guesses (usually with several wrong guesses at that time).Of course my insisting on going through the test also depends on previous experience with the sample. From furious I know it's a serious problem for wavPack lossy and I have an idea what to look for. I'm also more emotionally engaged cause this is one of the more serious problems to wavPack lossy - I can imagine to get a similar kind of music in real life encoding, and it's not just a tiny amount of altered or increased hiss/noise but inaccuracy - though in your case very tiny as well.

Near-lossless / lossy FLAC

As for abxing if the question is 'is track X transparent?' I always allow for a second trial in case I have the impression that there is a difference and get at a result like 4/4 before I start to go wrong (due to possibly fatigueness). The number of guesses is fixed before a test (usually 10 guesses, sometimes , but in case I don't see what to concentrate on I often give up within the first guesses (usually with several wrong guesses at that time).

I'm sure an ABX / statistics guru will be along in a minute to explain exactly why that alters the statistics. I just recall that it does, and in a way that makes it much harder to hit a given level of confidence.

I think you could say something like "I will do 8, and they will not count, then I will do 16, and they will count" if you stuck to it.

Of course, you can always listen carefully, and A/B (not X!) until you believe you hear something. Then, and only then, do the pre-decided number of ABX trials. Then there's no question.

Near-lossless / lossy FLAC

I may be misunderstanding something, but: why linking this to FLAC at all?I mean: this is a "sound simplifier", so to speak, so it's output could be very well fed to pretty any lossless (or even non lossless, even if this does a lot of less sense) coder, right?

Near-lossless / lossy FLAC

My suggestion to further prevent confusion whether a FLAC file is from a lossless or lossy source:Introduce a flag inside the FLAC (meta)data that indicate whether a file has a lossy or lossless source.

An user should be able to set that flag at encoding time only. It should be noneditable and unerasable inside the FLAC file (hex editor might be an execption).

Near-lossless / lossy FLAC

It was originally designed to specifically exploit FLAC's wasted_bits handling for compression gain.

Exactly.

However, it turns out you can use it with wavpack lossless too, by using the --blocksize=1024 switch to force wavpack to use 1024 sample blocks. (or 4096 sample blocks for the examples I provided on the previous page). Thanks to David Bryant for providing this, and other useful tips via email (I will reply properly David).

Near-lossless / lossy FLAC

... I think you could say something like "I will do 8, and they will not count, then I will do 16, and they will count" if you stuck to it. ...

Hm... it's not like this: I say in advance I'll do 10 guesses with each trial in order to call two tracks abxable.What shall I do in a situation when I have the impression (which doesn't count in the end but I can't ignore it) that there are audible differences, and this is backed up by the first guesses where I score 4/4? If after that I fail what does that mean? Failure can be due to the tracks not being able to abx, but also due to fatigueness or overconfidence according to the first results. Certainly this means I'm not very good at abxing, but with tracks hard to abx it happens to be like this - I can't change it. So what shall I do in such a situation? My solution is: I do a second trial and try harder. Can't see a better procedure. Taking the result of the first trial as the abxing result isn't the better alternative to me in case there's a suspicion that the tracks are abxable.Sure if I allow for a second trial I could also allow for a third one, and so on. I see the point. But that's theory cause things are clear after the second trial.

Near-lossless / lossy FLAC

It was originally designed to specifically exploit FLAC's wasted_bits handling for compression gain.

Exactly.

However, it turns out you can use it with wavpack lossless too, by using the --blocksize=1024 switch to force wavpack to use 1024 sample blocks. (or 4096 sample blocks for the examples I provided on the previous page). Thanks to David Bryant for providing this, and other useful tips via email (I will reply properly David).

Cheers,David.

Exactly. What I was meaning (sorry, my English isn't very good) is that the "simplification" made by your tool can enable similar gain in compression ratio with other similar tools. For example (using one of the posted sample):

Maybe two hours from now a new Lossless compressor will surface that enjoy an even higher gain than FLAC. In the end, I don't see the link between a generic tool that "simplify" a sound file, and a particular encoder (for witch the action of the tool may result especially beneficial).

On the same way, I don't see the point about flaggin in some way a lossless encoded file that had as a input a WAV file altered in some way.Even without using this tool, original files can be altered in a number of ways (badly equalized, or they can contains click/pop, can have some noise reduction effects applied, etc.)...

Near-lossless / lossy FLAC

...Exactly. What I was meaning (sorry, my English isn't very good) is that the "simplification" made by your tool can enable similar gain in compression ratio with other similar tools. For example (using one of the posted sample):

Maybe two hours from now a new Lossless compressor will surface that enjoy an even higher gain than FLAC. In the end, I don't see the link between a generic tool that "simplify" a sound file, and a particular encoder (for witch the action of the tool may result especially beneficial). ...

For optimum performance the intenal frame size of the encoder has to be taken into account when using the preprocessor. I suppose you haven't done this for TAK?

Possibly i will have to add an option to manually set the frame size for TAK files to get most out of the preprocessor. While TAK partitions each of it's fixed size frames into up to 5 variable size sub frames to adapt to signal changes, the "wasted bits" options works on the whole frame, which is too big (more than 4000 samples) to work well with the preprocessor.

Near-lossless / lossy FLAC

So, how soon before an executable version of SoundSimplifier™ ( ) is released to an expectant HA community? Inquiring (impatient) minds want to know!

I like the name!

I'm not keeping it back. I've attached the latest MATLAB script which I'm using to generate these samples. It executes very well if you have MATLAB!(Though you need lots of memory for normal sized audio files, since there's no buffering, and you'll need to change waveread to wavread and wavewrite to wavwrite).

As for an efficient C/C++ implementation which could be compiled - that's beyond me.

I think we need some more listening tests before anyone puts that much effort in, but the job is open to anyone who wants it!

Near-lossless / lossy FLAC

...Exactly. What I was meaning (sorry, my English isn't very good) is that the "simplification" made by your tool can enable similar gain in compression ratio with other similar tools. For example (using one of the posted sample):

Maybe two hours from now a new Lossless compressor will surface that enjoy an even higher gain than FLAC. In the end, I don't see the link between a generic tool that "simplify" a sound file, and a particular encoder (for witch the action of the tool may result especially beneficial). ...

For optimum performance the intenal frame size of the encoder has to be taken into account when using the preprocessor. I suppose you haven't done this for TAK?...

I was curious...I dowloaded "01_41_30sec_lossy" from here. Then i compressed it with TAK's default frame size and then with a frame size of 4096 Bytes. Results:

Near-lossless / lossy FLAC

For optimum performance the intenal frame size of the encoder has to be taken into account when using the preprocessor. I suppose you haven't done this for TAK?

Right. I have just compressed the two files "on a rush" to check if - as I supposed - the action of the SoundSimplifier™ would be interesting also for other encoders.And it was, so much, as your results show even more. Thanks for taking the time for the "optimized" encoding.