The FLAC support isn't released yet because its brand new, hardly tested code. I'd really prefer people not continue publishing pre-releases on the web where the people downloading them will have no idea where to contact the developers, won't necessarily expect the software not to eat their babies, etc.

I'd really prefer people not continue publishing pre-releases on the web where the people downloading them will have no idea where to contact the developers, won't necessarily expect the software not to eat their babies, etc.

I can certainly understand the concern that too many people are using bleeding-edge git snapshots compiled by random guys from the forums rather than using stable releases from solidly-trusted sources. That's only appropriate for people who are willing to deal with git breakage and get seriously involved in the testing/development process. To avoid headaches it'd be much better for people who expect the software to "just work" to be using stable binaries compiled by people who are absolutely sure to know what they're doing. It'd also help if people who are willing to get casually involved in the testing/feedback process were using better-trusted binaries.

But telling other people not to post prerelease binaries is not the way to fix this problem. Every open-source project has the potential for this kind of problem, but almost all of them successfully avoid it without trying to discourage people from building and distributing their software. Why is it that Opus has this problem while others don't?

The binaries linked from opus-codec.org predate the 1.0 release. This doesn't just mean that people who download from there are missing out on a fair number of bugfixes and a few improvements; it also means people will abandon using trusted sources for binaries. And many of the other sources which are distributing binaries are distributing git snapshots and/or "optimized" binaries that are actually slower than the gcc compile.

For the better part of a year people at HA and elsewhere have been told that the stable version is mediocre at music but exp/master is much better. While it's true that if the settings are held constant the development branch gives better encodes of tonal samples, it's still unclear how much improvement there is when it's constrained to give the same average bitrate over a large collection of music. I trust that the continuing refinements and fixes will make the final 1.1 release a substantial improvement. But in the meantime 1.0.x is not as bad as it's made out to be - a few trained listeners managing to ABX something doesn't necessarily mean a broad part of the populace at large will give it a low MOS - and there are substantial benefits to using a known-good stable version with predictable behavior. For instance, in one somewhat recent development build I found there are bitrates where the mode switching kinda thrashes around on speech input, and it's very distracting to have the bandwidth and character of your audio changing like that, especially in an audiobook. (From commit messages I suspect that may have been fixed since then, but I haven't tested much.) I'm also not convinced that the development version's attempt to provide higher bandwidth at lower bitrates than the stable version while in LP/hybrid modes is well-tuned yet even when it doesn't thrash around.

During earlier phases of development, I suspect most of the people who were distributing binaries to those willing to get casually involved in testing Opus were distributing development releases, which happened about once a month, and not random git tags. Only those who were actually ready to deal with git breakage were using the bleeding edge. But your development branch has now seen 9 months of vigorous development without releasing a single alpha or even just choosing appropriate snapshots to distribute. Anyone who's casually interested in trying out the hyped improvements is going to end up using some random guy's random git snapshot build.

If opus-codec.org provided up-to-date stable binaries, alpha source releases, and some kind of very visible recommendation about who should be using what, I think that would start to address the problem much more effectively. The reason that major open source projects don't usually have tremendous support hassles with naive users using random "OMG 0PT1M!Z3D!!!1" builds and random git snapshots isn't that such builds don't exist (au contraire!) or that they tell people to quit visibly publishing them, it's because the official releases handily outcompete such builds. They are better-publicized, better-trusted, bring users improvements on an appropriate schedule (freshly-baked rather than half-baked or stale), and generally deliver a better user experience.

But telling other people not to post prerelease binaries is not the way to fix this problem. Every open-source project has the potential for this kind of problem, but almost all of them successfully avoid it without trying to discourage people from building and distributing their software. Why is it that Opus has this problem while others don't?

I can simply stop posting code in public which I don't believe is ready for consumption by non developers. Is this the outcome you want? I don't mean to say that in a threatening way but it will be the natural outcome of the approach you're taking with me. Many projects develop with a model where things just appear when complete. It has its advantages and disadvantages.

If people who casually want to help need binaries— we're available almost all the time via IRC and email. Finding out that someone is posting binaries in yet another thread that I've missed on HA after someone shows up with some weird files complaining about bugs that never made it into a release is just really unwelcome. It's frustrating, it slows development, and it shifts the balance towards not posting code until things are complete and tested.

These claims are simply untrue. The binaries have not been neglected, the only binaries available from the site are the stable ones so its hard for me to find support for your undersold claims, and we are not behind our historic release cadence especially considering in the past we were developing the format, and now the format is complete and there have been far fewer reasons to cut releases.

People shouldn't expect releases more than a couple times per year. If they're interested in participating in development (e.g. by testing current development work) then I welcome them to start building their own from git (knowing what they'll get) or hopping on IRC and asking for binaries if they're unable to build themselves. The only thing I'm unhappy about is random development versions, often with incorrect versioning information, being promoted on HA as "the thing to use" to people who aren't interested in participating in the development, especially when that promotion is happening absent any objective testing results which shows them to be better.

QUOTE

This doesn't just mean that people who download from there are missing out on a fair number of bugfixes and a few improvements; it also means people will abandon using trusted sources for binaries. And many of the other sources which are distributing binaries are distributing git snapshots and/or "optimized" binaries that are actually slower than the gcc compile.

There are not missing out on a "fair number of bugfixes and improvements"— The 1.0 opus library itself was not functionally changed for 1.0. They are missing some build system tweaks which are irrelevant for the binaries and a version string.

And yes, I tagged opus-tools 0.1.6 on Saturday knowing that we probably wouldn't have binaries up until mid week. Normally I prefer that the source release predates binaries a little bit simply to get a slow roll deployment, but in this case tagging 0.1.6 also was delaying merging flac (since it wasn't worth the extra confusion of branching for it). I thought this was a good decision, but I now regret it because you're accusing me of "neglecting the official binaries" as a result.

QUOTE

For the better part of a year people at HA and elsewhere have been told that the stable version is mediocre at music but exp/master is much better.

I have responded this specifically to people complaining about surprisingly poor results on highly tonal samples, including people who weren't even trying to do critical listening (at least at lower rates). The issues are not some subtle ABX thing requiring trained listeners— in the prior HA 64k listening test _every_ listener slammed the couple really tonal samples. People get asked to use exp_analysis branches when they're doing comparisons and complaining about tonal samples because further reports of issues there simple aren't informative.

QUOTE

and there are substantial benefits to using a known-good stable version with predictable behavior. For instance, in one somewhat recent development build I found there are bitrates where the mode switching kinda thrashes around on speech input,

Yes, and we don't post binaries of development versions for that reason. I hand out one off builds to people who are actually reporting behavior and ask them for results. I don't want people who aren't paying attention ending up with the experimental stuff, especially if they don't have an established channel to report issues they find with it. And yet you seem to be yelling at me here for asking other people not to do it either.

QUOTE

Only those who were actually ready to deal with git breakage were using the bleeding edge. But your development branch has now seen 9 months of vigorous development without releasing a single alpha or ev1en just choosing appropriate snapshots to distribute.

Most of that time was R&D work, not engineering production time... and concurrently the stable branches have been developing and maturing. There will probably be a 1.1 alpha in the next week or two. You might as well say that we had 3 years of development without a release since we've certainly had things take that long between research and publication.

QUOTE

If opus-codec.org provided up-to-date stable binaries, alpha source releases, and some kind of very visible recommendation about who should be using what, I think that would start to address the problem much more effectively. The reason that major open source projects don't usually have tremendous support hassles with naive users using random "OMG 0PT1M!Z3D!!!1" builds

The website _does_ and we haven't generally had problems with it outside of hydrogen audio, or even too much of one on HA. HA has the additional problem that newest posts bump the threads, so unless there is a new release every day it can't hope to consistently out-compete people posting.

I can simply stop posting code in public which I don't believe is ready for consumption by non developers. Is this the outcome you want?

Obviously, no. Yes, some cathedral-style projects have private mailing lists, private repositories, and only occasional public code drops. But lots of big projects develop with all their code in readily available git repositories and don't have this problem; even though they do have people building and publicizing random snapshots with crazy optimization options, they don't have to go trying to persuade every single person to take down those binaries to avoid support nightmares. That sounds kinda like running around trying to exterminate malaria with one flyswatter. They somehow manage to keep non-testers using approved releases and testers using builds appropriate to their degree of involvement (nightly/"blessed snapshot"/alpha-beta for different kinds of tester) rather than having lots of folks in both categories using random and possibly broken builds.

QUOTE

If people who casually want to help need binaries— we're available almost all the time via IRC and email.

It is difficult to make that scale as interest in testing increases.

QUOTE

Finding out that someone is posting binaries in yet another thread that I've missed on HA after someone shows up with some weird files complaining about bugs that never made it into a release is just really unwelcome. It's frustrating, it slows development, and it shifts the balance towards not posting code until things are complete and tested.

Please understand I'm not trying to pester you about this and I'm not "yelling at you here." I'm just thinking there's got to be a way you can avoid precisely the kind of fruitless and frustrating game of whack-a-mole that you've just described, and trying to bring up a few thoughts about why it might be happening and what might help.

I was just saying these are possible reasons. If these don't help diagnose the problem, fine; let's find a better diagnosis. Of course some other reasons include that Opus has a good build system, is relatively easy to build, and at this point has an early adopter audience more ready to either tinker around with experimental builds or set up their own build environment. Something like OpenOffice is unlikely to have the same kinds of troubles because you have to be a little odd to want to be a bleeding-edge early adopter with your office software and you have to be a level 9 dark wizard to get the thing to build. But I don't think that's the whole explanation and I think there have got to be some reasons that one can do something about.

QUOTE

There are not missing out on a "fair number of bugfixes and improvements"— The 1.0 opus library itself was not functionally changed for 1.0. They are missing some build system tweaks which are irrelevant for the binaries and a version string.

Well, when I said "a fair number of bugfixes and improvements" I wasn't talking about the trivial diff between -rc and 1.0.1, I was talking about more recent improvements. And of course a few days' delay in putting up binaries for 1.0.2 is reasonable, and "neglecting" was probably the wrong word to use. But when you didn't update the binaries for 1.0.1, though I and anybody else who'd glanced at the commit logs knew nothing substantive had changed, there was confusion, disappointment, and demands for custom-rolled builds, because most people didn't know. A single phrase in the release announcement would have helped with PR, or even just rerolling the build with the new version string for PR's sake wouldn't have hurt. (Most major projects rebuild their software for official releases even if nothing changed from the last RC, for precisely this reason. It may seem silly but it saves communications hassles.) When you posted 1.0.2's release announcement without updated binaries, giving some kind of indication that updated binaries are on the way might have helped. Again, it's about building the perception that this is the up-to-date source to rely on for binaries.

QUOTE

QUOTE

For the better part of a year people at HA and elsewhere have been told that the stable version is mediocre at music but exp/master is much better.

I have responded this specifically to people complaining about surprisingly poor results on highly tonal samples, including people who weren't even trying to do critical listening (at least at lower rates). The issues are not some subtle ABX thing requiring trained listeners— in the prior HA 64k listening test _every_ listener slammed the couple really tonal samples. People get asked to use exp_analysis branches when they're doing comparisons and complaining about tonal samples because further reports of issues there simple aren't informative.

Well, at 64kbps they're certainly not always subtle. But from my admittedly limited experience I don't think it takes all that much higher a bitrate target before they do generally become subtle. I imagine that most people using it for music around here are using 96kbps and up, are reasonably well served by the stable version, and would do well to stick with that for now if they want something that "just works." Unfortunately I think we have people seeing "Oh wow, somebody ABXed a sample at 160kbps and was told to use an experimental version, I guess I'd better use the experimental versions too." This is a communication problem; people haven't heard the sales pitch for the stable version enough- "its results may not always be as good as exp's, but it gives good results in these situations, still quite decent results in these other situations, and unlike exp it is 99% certified non-baby-eating."

QUOTE

Most of that time was R&D work, not engineering production time

Sure, exp is R&D, but it hasn't been the kind of blue-sky R&D which you don't want anybody trying to test. Binaries of it have been mysteriously floating around here, often with your blessing, since at least May. Many projects would have called something in that time frame an alpha. ("If it compiles, it is good; if it boots up, it is perfect.") Whatever your thoughts about more-official development releases/snapshots, I still think that generally trying to keep the development versions hush-hush but handing out builds on IRC or in forum posts is actually part of the problem. With no prominent official source to go to for information about the development builds or for binaries, people rely on hearsay and on whoever has posted the latest random build. If you were publishing alphas or snapshots or something, you could have better control of what most people are testing and you could include a very visible recommendation that people not ready to participate in the testing/feedback process stick to the stable versions, a reminder to prospective testers about what to expect and how to appropriately report issues and results, etc.

Even if you prefer not to publicize development and testing like that on opus-codec.org you could do more about that here at HA.

QUOTE

HA has the additional problem that newest posts bump the threads, so unless there is a new release every day it can't hope to consistently out-compete people posting.

I don't think people here, especially those not really prepared to deal with testing and breakage, will blindly hop on the latest random-joe build if there's communication to help them understand better. A couple suggestions about that communication here at HA: I think there's enough Opus traffic that it might be worth asking the HA admins about a dedicated Opus subforum, and you could pin the above-mentioned "very visible recommendation to non-testers and reminders to testers" there, along with a link to a build you feel is worth testing. That would avoid the problem that you've said most of this before but it gets buried in the threads. Stuff like this could go on the HA wiki as well (the Opus page is currently a stub and unlike other codecs there's not yet a page talking about recommended encoder, encoder settings/bitrates, etc) and I'd be happy to help with that.

I was just saying these are possible reasons. If these don't help diagnose the problem, fine; let's find a better diagnosis.

Generally speaking when you find yourself trying to lecture a group of people running a project about how their project is run, and you realize that you actually have no idea whats going on, its a good indication that you are not helping and should reconsider your decision to continue talking.

@saratoga: Your advice might be true, but definitely does not solve the problem.

@NullC: Do you need to re-read the three clauses of the BSD License, in which you are distributing the specification, the reference implementation and the "revised implementation"? (whatever that is)

Reading your post I really wondered if you were regretting releasing the code as open source. Does that really do a favour to the codec image?

Do you really think that Opus is that much different than LAME or Vorbis? Do you realize that here at hydrogenaudio, people have modified and improved the original implementations, making changes that have moved the codec to new standards? ( Original Dibrom tunings that created alt presets on LAME, Garf Vorbis tunings, aotuv Vorbis tunings, and recently halb27 tunings on LAME VBR).

Do you really have a reason to (almost) censor the curiosity and testing intentions of the people over here? Have we made and explicit recommendation to use builds from here against the official build?

As jensend has tried to explain to you, if anything, there's a communication problem over here between you and us. Not the builds.

You told a person that wanted to solve a problem (NullC being angry because HydrogenAudio distributed binaries of Opus directly from the git repository), to shut up because he wasn't helping. (More or less).

QUOTE (saratoga @ Dec 12 2012, 20:26)

Why would someone need to re-read the BSD license exactly? This question doesn't really make sense.

Do you need to re-read it too?

CODE

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:...Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

There is no restriction on who can build that binary, there is no restriction on distributing that binary, and, obviously, there is no restriction in telling other people about the binary.

That's why I ended the post saying that If he's open to talk about it, we might find a common point that we can both agree. As it is, it gives a bad looking for an open source project to limit what you can do with the source (more so being BSD license, not GPL).

That's why I ended the post saying that If he's open to talk about it, we might find a common point that we can both agree. As it is, it gives a bad looking for an open source project to limit what you can do with the source (more so being BSD license, not GPL).

IF someone is distributing open source software that those who wrote it don't think is adequately stable, any resulting bad vibes shouldn't fall on the developers. I did pick up a dev release, but figure if it doesn't fix the problem I had with the old stable release, I'll discard it and wait, no hard feelings.

NullC isn't trying to tell people that they have no legal right to make their own builds from random and distribute them, he's trying to dissuade them from overpublicizing such builds, making too-strong of claims for them, etc.

While it's true that this could imaginably ruffle some feathers, I don't think his attempts at such persuasion have thus far harmed Opus's image in the slightest. The primary reasons I think such an approach is problematic are that it is ineffective and that it doesn't scale. If it were done in a heavy-handed way it could produce effects like what JAZ is talking about, but I have absolutely zero reason to think Xiph etc folks would take such an approach.

I'm sorry if portions of my initial post gave any other impression due to unfortunate phrasing, and I feel people have missed the point of what I was actually trying to say.

I still think that simple steps could be taken to better convince "I need it to just work" type users to stick with the official stable builds. I think simple steps could be taken to provide prospective testers with visible and trusted sources of test builds as well as basic information about testing (e.g. disclaimers, where to provide feedback and what kind of feedback is helpful). I think such steps would largely eliminate the frustrations NullC is facing. Third parties would continue to build random git snapshots, and some people would continue to choose to use those builds, but the danger that regular users or casual testers would end up using those builds without knowing what they're doing or based on misinformation would be greatly reduced.

Maybe having a page visible on the website with nightlies or cherry-picked snapshot builds or regularly-released alphas or whatever is repulsive to Opus devs' sense of "engineering production values" regardless of what kind of disclaimers, encouragement that regular users use the stable releases instead, and instructions for testers accompany the builds on that page. But if the only "approved" source for such builds is an unpublicized expectation that anyone who wants to test but doesn't have a proper build environment will beg for builds on IRC or the list, then people will instead continue to grab whatever random sources of binaries they can find.

I would again advocate a pinned thread, preferably in a new Opus subforum, for that purpose esp. if they'd rather not post that on the website. (Opus is now most of the traffic in "Other Lossy Codecs," and I'd hope HA admins would be receptive to giving it its own subforum.)

Here is a build of Opus Tools 0.1.6. This is current Git, ICL 12.1 32 bit SSE2. In order to get this to compile, it's necessary to disable all of 'resample_sse.h' except the xmmintrin.h include, otherwise the compiler throws errors. Compiler SSE2 optimisations are included'

I've just added an ICL12.1 Win32 SSE2 compile of opusfile based on current Git. This is completely untested as I don't know what to test it against, but if someone knows what to do with it and can verify whether it works, it may be useful!! The zip includes the .dll, .exp and .lib files.

Please be gentle .But this latest opus release.Does it improve the sound quality when I encode from flac to Opus with foobar? I am really happy with the quality but obviously always looking for improvements and appreciate all the hard work what goes into these projects.Many thanks .