This summary provides an overview of the things that we talked about
during the KDE2.2 multimedia meeting which took place on IRC (2001-03-06).
If you want to read more than the summary, the full IRC logfile ishere.

Participants (in no particular order) are listed at the bottom.

Overview

Somehow, it could have been called theaRts development meeting.
Maybe that was a result of talking about the aRts TODO list "first"
(because it was the only document which was available before the meeting),
and then getting to nothing else. Maybe it was a result of me moderating ;).
And finally, maybe it was because aRts is a very central component of the
KDE multimedia development, and thus working on KDE multimedia almost
always implies somehow getting in touch with aRts.

Anyway, the following will try to give the idea what we talked about, and
try to reconstruct the tasks that were assigned.

aRts Core/SoundServer

Threading

Error handling

SoundServer userfriendliness

CSL/C API

KMedia2

Gapless playing

PlayObjects for custom data streams

X11 Embedding

Kaboodle

GUIs for aRts objects

Mixer issues

Conclusion

aRts Core/SoundServer

Threading

Threading: mpeglib is threaded, decoders are threaded, the first point is
about making aRts threadsafe enough to develop threaded playobjects and such.
It's mostly done anyway, so it will be in KDE2.2 (libmcop-mt).

Error handling

We discussed error handling in MCOP apps. The problem is that currently when
artsd dies apps segfault unpredictable. This is usually since when artsd died,
remote operations returning object references return null pointers. Calling

soundserver.outstack().insertTop(someEffect,"name");

will return an "StereoEffectStack::null()" as outstack(), and calling a
method on that will segfault. It should be possible with some coding to
create a callback once proxies (soundserver) become invalid, so that the
applications can be terminated more or less gracefully before they come
in the situation that they can crash.

Tobi volunteered to care about this.

General userfriendliness

Jeff Tranter volunteered to care about the following two TODO items:

add kmessagebox style output of error messages (often, people
don't know why artsd is not running, does not start, doesn't like
their soundcard,...)

add -S option (to change the autosuspend timeout)

Note: Actually, it seems he has already completed this. ;)

CSL/C API

Providing a compatible way to do sound based on aRts in KDE/Gnome would be
nice. It seems that having a pure C implementation for the CLIENT is a good
idea for Gnome. CSL intends to provide that. This doesn't mean that CSL is
replacing the aRts C API (it will be kept for compat reasons but might end
up being a thin CSL wrapper one day).

I am working on CSL a bit, and maybe it will be more ready/popular soon.

KMedia2

Gapless playing

When playing more than one file with noatun there is a gap of 0.5 seconds
between the current and the next file. We came to the conclusion that noatun
is causing this by polling when a file is done. Luckily for noatun, there
are change notifications in aRts, so that it should be possible to find out
when a file is done using these.

That way, no gap should occur. Noatun will need to implement an MCOP object
(and be a server) to do this.

Charles Samuels volunteered to care about this.

PlayObjects for custom data streams

Arts::PlayObject has as interface for specifying what to play a

bool loadMedia(string filename)

method. So PlayObjects always play local files, and nothing else. Thats
suboptimal, because you can't for instance get an mp3 from a web server
(via KIO) and play it. The ideal way would be to use KIO, in a KDE process
(for instance noatun), and stream the data to the playobject, like

Probably using the MCOP streams will offer the best integration/performance.
They are also used by artscat and artsdsp (and the C API) so they are quite
tested. The only problem will be seeking in a total asynchronous deliver/send
model.

It's a bit undecided who wants to do this. Neil Stevens and/or Charles Samules
might be the developers who are most interested in getting this done.

X11 Embedding

Currently, noatun will open an extra window when playing videos. The optimal
solution would be that noatun embeds the video image.

Neil Stevens volunteered to take care about this.

Kaboodle

There was a bit of talk about moving kaboodle to kdemultimedia. Kaboodle is
an alternative media player which is like noatun based on the KMedia2
architecture. It has two differences to noatun: it's small (as lightweight
as possible), and it can embed itself into konqueror.

Nobody was against that, except making it default is probably not such a
good idea (as noatun is by definition more powerful).

It might also be nice to support reading HTML-pages with embedded wav files
and things like that.

Note: Charles told me noatun should be competitive in speed with kadaboodle
after the recent changes (i.e. kdeinit support).

GUIs for aRts objects

Often, aRts objects (like the freeverb effect) will need a GUI. The GUI should
be sharable between artscontrol and noatun (and other possible aRts aware
apps). An important design restrictions to meet is, that it should be possible
to keep effect GUIs toolkit independant.

So the idea for KDE2.2 might be having a "factory interface", which provides
a way to obtain a GUI for a given effect (like Arts::Synth_FREEVERB). This
factory might

autogenerate a GUI out of hints

get the GUI implemented in terms of Arts::Widget, Arts::Poti and such

get the completely implemented GUI

For the first two cases, this will be toolkit independant.
Arts::Widget, Arts::Panel, Arts::Poti and so on are interfaces.

There is a "Requires=kde_gui" line in the implementation (.mcopclass file),
so the Requires/Provides mechanism of MCOP will be able to select a fitting
one for each application (i.e. Gtk version for Gtk apps). The GUI will be
built inside the client application and thus be able to interact with the
application toolkit in a natural way.

Of course, the whole GUI issue will not only be useful for effects, but also
for PlayObjects (configure interpolation quality of the mikmod .xm engine),
Instruments, and all other kinds of objects.

I hope to get that done until KDE2.2.

Mixer issues

There was quite some discussion about mixers. The facts until now are:

kmix is about the hardware mixer only;

aRts doesn't know anything about hardware mixers at all;

artscontrol controls only what artsd does, which is among other things,
controlling the output effect stack, midi instruments and so on; and

it might be nice to have aRts simulate a multichannel software mixer with
effects.

Stefan Schimanski is working on giving kmix pluggable architecture, that
will work on the base of the Qt object model and dynamic loading, so it
might be possible to easily plug aRts into kmix. A "one mixer for everything"
approach might be more useful from a user-point-of-view than the current split.

Still, the general problem might be that aRts itself has no real
"abstraction" for mixers in the IDL. There is the audiomanager thing that
assumes the following:

there are apps that dynamically come and go

there are destinations (in form of busses) where audio might get played or
recorded

you can assign apps to destinations

On the other hand, one possible kmix model for apps would be dynamically
appearing channels per app, i.e. a new quake channel would pop up once you
start quake (whereas the current aRts idea would be more like: there are
eight static audio outputs, audio01 .. audio08, and once you start quake,
you can assign it to one of these channels).

It might be nice to add an IDL based abstraction for "mixers", which will
work the same for hardware and software mixers. This would provide things
like network transparent (and language independant) mixer control. Maybe
it would also make a good addition to the "AudioManager" style mixer
interface we have now.

This abstraction would also make it possible to have different clients
access the mixer abstraction in a unified way. I.e. a Gtk frontend to
the mixer would use the same software/hardware mixer than the KDE frontend.
Turning the volume in the KDE frontend would make the sliders move int he
Gtk frontend as well.

Entierly getting rid of artscontrol might not be the way to go, as there
are midi related tasks in artscontrol which really don't belong into a
mixer. But somehow unifying mixing would be "user-friendlier" than what
we currently have.

aRts model is powerful enough to provide a multichannel, effect based
mixer as software (and thats little code), and it might be possible to add
that to artscontrol. Assignment of programs to channels is trivial. You
can have equalizers (Synth_STD_EQUALIZER), per-channel-effects and so on.
It will work on all applications. It should be implementable with not too
much work (less than 1000 LOC).

Whatever, this is a bit of a are research topic ;), and different things
need to be done.

Stefan Schimanski is working on the kmix plugin architecture (1).
Stefan Gehn wanted to have a look at the GUI side of building a mixer (4).
I can help with getting the necessary backend work done for (4).

Conclusion

From the aRts point of view, the soundserver architecture and the core MCOP
technology seem solid, so what we do here is mostly polishing.

Then, there are services basing on the MCOP/IDL model. The most important
service here is KMedia2, which allows playing arbitary media. The current
IDL interfaces are good for many simple tasks, but more needs to be added
for more complex tasks.

Finally, there are areas which are not yet standarized in any way by the
IDL interfaces. We talked about GUI and mixer, but there are others where
ino work has been done yet (like hard disk recording or video filtering),
and others where more work needs to be done (like midi).

And of course, thats just one side of viewing things. Existing apps, such as
kmix, kmid(i) and games often do not take fully advantage of what aRts can
provide, or have goals/directions in which they develop as well. Sometimes,
integration (as for libkmid or a timidity PlayObject) might be a good idea,
sometimes, adding a bit of "interoperability code" with aRts might be the
right way to go.

The title is "Why not DCOP" but it discusses CORBA some as well (aRts originally used CORBA, but switched). All the reasons seem well thought-out and very logical. One very good reason cited there is speed: By some measures, MCOP is 6-8 times as fast as CORBA (and 4 times as fast as DCOP). MCOP's architecture is also much better suited to some of the tasks that aRts does. (after all, it was designed specifically for multimedia)

I don't think he's trolling, he's just uninformed. There is some misconception amongst some gnome-centric people I think that orbit is faster. There were also reports to mailing lists (i.e. mandrake) for the kde2.1 beta's that aRts took up a lot of CPU resources.

Where I work there is a very debian/gnome centric group. When I asked many people why they don't use KDE most replied that it was b/c KDE was inferior. However, the people that have used KDE only ever used the first version of KDE1. Once I introduced konqi to some of them they've been changing their opinion.

aRts takes up a moderate amount of resources. Here on my P3 500 MHz, average total CPU usage when playing an MP3 with Noatun is 3 to 3.5%. Doing nothing, it's about 1%.

When I kill artsd, CPU usage goes down to practically 0. Playing an MP3 with XMMS takes around 0.3-0.5%. I measured this using KPM.

So yes, aRts does take more CPU time than other sound players, but it has many more capabilities as well. Also, aRts is still in development, of course.

I wouldn't worry too much about people with slow computers. I think that probably the CPU time taken by aRts depends more on the soundcard than your processor speed, and therefore it would remain nearly constant no matter how fast your computer (because all soundcards play at the same rate). I could be wrong, though :-)

I'm I understanding this correctly, that CSL is the GNOME sound architecture? Does this mean that GNOME apps (like everybuddy) might one day happily play their notifications and sounds while running in KDE?

Will there really be *no* gap when playing consecutive songs, or will the gap simply be (much) smaller than it is now? I ask because I can't imagine that the gap could be completely eliminated without loading at least some of the second file *before* the first one has finished.

Needless to say, when listening to ripped CDs (ex, a live performance) it would be nice to have absolutely no gap. :)

while noatun does great, I would like to state here that
open source projects for video playing have been
dissappointing. There are too many different efforts that
do not even coordinate.
result? none of them can play all the available video formats.
Why not work together to give us a player that will play avi, mpeg 1/2/4, divx and asf?
Noatun is cool but, it too can not play most of the stuff out there.
For now you need all of them. xine, mplay, avifile,mtv, xanim????
Total Chaos!!!!!!

Actually - if you really want to play stuff like DivX files - try Xine. I tried avifile, MPlayer and xine - xine beats all of them, and version 0.4 is multithreaded - so it takes ~30% of my CPU. I can live with that ;)

I think it would be cool for aRts to support multi-channel hardware mixing. Nothing would have to change in the abstraction layer between the client apps and artsd, just the way artsd handles /dev/dsp. Several of the OSS drivers support opening /dev/dsp multiple times and pushing several, independant, streams into it. Other drivers support /dev/dsp{0,1,2,...}. I guess this is a "It would be nice" type of thing, to have the final output streams mixed in hardware (at no CPU cost, and higher quality), especially considering that it will take a much higher load to cause the sound to break up. Anyway I'll take a look at the aRts homepage and see if anyone thinks this is a good idea.

I have had problems with large files (or streaming from XMMS). After 20-30 mins, the CPU usage goes skyhigh, and finally causes Noatun or Xmms to jitter the sound. Also, artsd becomes highly unstable and easly crashes when some other sound is sent to it from licq (eg). This problem has been there since 2.0. Is it being addressed ?

I agree. artsd is extremely flakey when the audio device is being used by other technologies. It doesn't deal with the situation gracefully and causes audio to stop working altogether until it has been HUPped - and sometimes killed altogether. To be honest, I often find it better just to kill artsd completely and just use XMMS with the standard kernel sound drivers etc. I do hope I won't have to do this for much longer.

For Video, I would suggest a non-KDE program, Broadcast 2000, located at http://heroines.sourceforge.net/bcast2000.php3. To give you an idea of its quality, it is also sold packaged with hardware to small TV stations as a turnkey solution. Unfortunantly, it *does* require the latest and greatest kernel and such - it really does push the latest multimedia features in Linux. Unfortunantly, until the Linux video system matures and works its way into distros a bit more, it will be difficult for a simple package install... but it does exist.

KDE uses the standard X DnD protocol (that's why it interoperates just fine with GTK/motif/etc apps). I can drag thing from konqi to xchat, xmms, etc with no problems whatsoever.

If Broadcast 2000 doesn't do drag-n-drop from konqi, it's not because it's not a KDE app, it's because the author doesn't support drag-n-drop period :-). He certainly could, and then it would work from gmc, mozilla, others as well.

I have TONS of stability problems. After playing a few mp3's I end up having to kill noatun and the 3 copies of arts it leaves around. I was so annoyed I almost switched back to GNOME. Then I decided to try XMMS (using artsdsp xmmms -p). Since then I've had NO problems (even when playing system sounds while the mp3 is playing).

Anyone else get this?

Rimmer

PS - also I think the default skin should be smaller... just my 2 cents

I'm a recent convert from Gnome 1.0 to kde 2.1 and i think kde is definitely the nicest desktop i've used. but:

I have similar problems. first of all, if i start arts via the artswrapper (has to be done as root), i cant get it to suid to something else (or i didnt find how to do it in the docs).
So, i let arts start under my userid when i start kde, which of course means it doesnt have real-time priority. When I use either noatun or xmms, (arts or oss driver) the sound gets messed up (cracks, pops, pauses). This even happens when kde is starting up.. the kde sound that plays usually gets garbled.
So, i jack up the buffering on artsd and on the arts driver for xmms which causes my entire system to go berzerk: crashing kde before i get a chance to check my resource usage. it seems the system can't handle the load (K6-2 450 196MB ram, RH7.0).

now, i'm trying artsdsp xmms -p and it's doing much better, though i've gotten a couple of "pops" in the sound. I'm not picky about the sound, but if windows2000 lets me play winamp and get email notification sounds at the same time w/o any pops, i'd hope kde could do the same.

* actually, as i was getting ready to post this, i started having some major problems with sound. arts randomly shot up its resource usage to 68% of system resources, and the sound broke up into a million pieces. all i did was submit a form in opera.

I have a fairly fast computer (Thunderbird 1100) and I have problems with the sound in both GNOME and KDE. This sound is fine (when playing an mp3) until I try to move a large file (like a kernel tarball) or open several apps at once. Then I get lots of static and broken sounds. Maybe Linux just sucks... anytime I move a large file (or something similar like untar a kernel) the system becomes completely unresponsive. It doesn't help that GNOME and KDE both use over 80 MB of memory on my system.

You should apply the linux low-latency patches to the kernel. In fact the Linux kernel has been getting _worse and worse_, not better, when it comes to kernel latency, which is a common cause of audio-dropout. Basically, I'm with you on this one, Linux sound (and multimedia presentation) just sucks, at the kernel level, due to poor (from the perspective of multimedia needs) task scheduling. Even the low-latency patches, though they offer a HUGE improvement to real-time scheduled tasks, do not fully solve the problems with Linux and latency (for example, they have no effect on non-real-time scheduled tasks, which, admittedly, by definition, don't need to be as responsive) but the real problem is basically that the Linux kernel is designed to be a server kernel, so that if ever there is a trade-off between responsiveness or raw speed, raw speed always wins. This is unfortunate, as OS's such as QNX and BeOS show that a responsive kernel can lead to a nicer experience for the user, and a more suitable platform for content production, as well as at least the -impression- that the computer is fast. I'd love to see a new scheduler in Linux, one with things like extrememly low-latency real-time, smoother scheduling in general, and priority inheritance. In fact, in an ideal world, you'd see two different kernels out there, which both export a perfectly compatible common API, but "tuned" very differently -- one with a scheduler tuned for fast response, and fast thread creation/destruction (like BeOS), the other tuned for raw speed with more emphesis on processes than threads (for big servers). So that way if you are basically a desktop user (or gamer, artist) you'd use the BeOS-like kernel, but if you are building a server, scientific work-horse, or database back-end, you'd use the current Linux kernel. In 2.4.2, I've seen reports of 400 (FOUR HUNDRED!!!) millisecond latencies ("reponse" times). FOUR HUNDRED! We're almost entering Window's 95 territory there! Well, not that bad, but!!!!

Although enabling DMA might help in these cases, there are still situations in which this problem persists. I am using a USB harddisk, and when I copy large files to or from this device my system becomes unresponsive after a while and sometimes even hangs! Especially when there's another process doing intensive I/O (in my case, grabbing video from a TV capture card) the kernel tends to lock up hard after a while.

Since this problem appears after transferring about the same amount of data as there is free memory in my system, it seems as if the kernel has trouble flushing its write-behind cache on slow devices. I'm no kernel expert but as far as I can see this problem is most likely not USB driver related (I've used both uhci.o and usb-uhci.o, both express the same problem).