If anybody is interested on this subject, please give me some feedback, so that I know what is more interesting to be published (focusing on installation processes, the various kind of programs, tips about EQing, plugins, mastering, etc).
Please, don't write about application bugs on this thread. Open a new one or write directly to Gentoo bugzilla.

UPDATED:
14th May 2006: English translation (it will have the priority over the original Italian one). Added some programs to be tested to the list.
15th May 2006: Relased under Creative Commons Attribution Sharealike 2.5, added the section about audio useflags (thx evermind)
16th May 2006: 2.6.x Kernel configuration section has been almost rewritten. Thanks to Tom aka Gimpel for letting me merge here his guide. Added some observations from Dominique and Gimpel about useflags, KDE (but will have to introduce a new section for DE, when I'll start a "tips" section) , latency, RT.
22nd May 2006: Added some little notes to test if jackd is working

Sorry if it sounds harsh to someone, but I think it sounds pretty fair and straight forward.

#####################

Introduction

This HOWTO would be a (little?) overview about how it could be possible to use Linux (I'll focus on Gentoo Linux) as an Hard Disk Recording/Mastering Workstation.
I hope, after finishing that, to be able to give some misc tips, for example about how to use your Gentoo box as part of your multi-effects rack for live performances and stuff like that (any suggestions are VERY appreciated!)

I know very well there's A LOT of work to be done to reach the quality of the ultra-expensive MAc or Win professional softwares (and their quality continues to increase...), but you'll see that our penguin is already able to let us produce decent audio productions (demos, simple soundscores, etc)

Before starting I also want to warn you that most of the software I'll talk about are still in beta (or even alpha) versions... So keep in mind they could crash or even lead your whole system to a crash.

PREREQUISITES:
- I'll suppose your workstation has a working audio card, with working ALSA drivers.
This HOWTO has been written for Gentoo-Linux with 2.6 kernel, but could be easy adapted to other distros with different kernels (I'll discuss, for example, how to configure a 2.4 kernel, that were better than early 2.6 kernels regarding latency)
- I HIGLY reccomend you to use the unstable (~arch) versions of all the programs I'll discuss here.
So be always ready to run a

Code:

nano /etc/portage/package.keywords

to add the application you want to install.
I know that won't ensure you a 100% reliability, but the new features (an example was the introduction in some programs of DSSI, VST support, etc) introduced by the latest versions are often too much important if you want to be productive.

Another good reason to do that is that, if you find something wrong, you can make a good bug report that will help the whole community (the number of musicians doesn't seem to be too high, in fact)

INSTALLATION OF EVERMIND's OVERLAY
What is this?
It's a portage Overlay with tweaked and more updated versions of most Linux Audio-related programs. Here you'll find almost everything you need for at least a simple home-recording (or, off course, studio-recording, if you will go deeper with the subject)

If what I wrote above semms voodoo-magic, take a look here, so you'll be able to manage your overlays, if you already have more than one.
- To update the overlay, you just have to do:

Code:

cd /usr/local/portage-proaudio
svn up

Note: Sometimes it may happen you obtain error messages like "digest verification failed". That happens when someone forgets to re-digest ebuilds when they submit them. To solve that (supposing your package name is foo), a

Code:

cd /directory/of/the/ebuild/foo
ebuild foo.ebuild digest

should be enough to solve the problem.

Preliminary steps:
What follows is necessary to prepare your system for audio; follow the 2.4 or the 2.6 guide, depending on the kernel you use

- Instrucions for 2.4 Kernels (OBSOLETE)
Before compiling it, we need to make the following modifications:

Because it's a good idea to use Jack (and all the applications that depend on it) as normal user, but we don't want to lose real time scheduling advantages, we need to activate "POSIX draft capabilities" (called, more commonly, caps).
That's not difficult, we just need to modify a configuration file.

If you don't like to edit files by hand, you can try this patch: http://nospoof.de/linux/patches/posix-draft-capabilities.patch
(the link should be correct, but I've never tried it)
IMPORTANT: Every time we install/reinstall a new/different version of kernel sources, we need to remember to make this operation manually, or the new kernel won't be "caps-enabled", OK?

- Instructions for 2.6 Kernels(WARNING: THIS PART OF THE GUIDE MUST BE UPDATED, because realtime-lsm is discouraged. The recomended method is to use sys-libs/pam with rt-limits activated. It can be found on the overlay. Documentation about that will come soon.
Before starting this important part, I want to highlight you can make it in 3 ways, depending on how much important is audio for your environment. Take a look at this post by Gimpel and MetaSymbol for further information.

Model 1: " Want to start a synth from time to time"
Standard Gentoo kernel with PAM from Rui (also available in evermind's overlay)
Model 2 "Want to work with MusE or Rosegarden from time to time"
RT-sources kernel and PAM
Model 3 " I want make some fat production / live performing with many native FX and soundgenerators with less then 10ms audiolatency"
RT-sources kernel and realtime-lsm (WARNING: RT-LSM can maybe crash your system)

Note: To avoid bad missunderstandings, you should remember what follows:
realtime-lsm is nothing else but a linux security module (LSM) which makes it possible to grant users access to the realtime capabilities of the Linux kernel. It works on any kernel. This is going to be replaced by PAM's pam_limits.so which uses the kernel's RLIMITS feature to allow users access to the realtime capabilities in a more secure way than realtime-lsm does.
realtime-preemption is a feature provided by a kernel patch from Ingo Molnar. It makes the kernel fully preemtible which reduces latencies, especially on audio tasks. A kernel patched with the realtime-preemption patch is called RT-kernel. This has to do with CPU scheduling, but nothing with user permissions.
Note2: Kernel preemption gives some little overhead (reducing throughput), but the benefits for audio-production are greater than the drawbacks (for a musician, lower latency and NO xruns are more important than having that 1% more CPU power).

RT-Kernel... Why?This is an update of the HowTo from gimpel (gimpel.funpic.de), translated by oc2pus. Updated by BB Codes and gentoo'ified by gimpel again

The main goal in audio-processing is the latency. The latency is the delay beetween signal-creating and reaching the processing in the system (e.g time between pressing a key on the keyboard and hearing the tone). The standard kernels have the disadvantage that the processing is slower (~11ms) as in Windows or using a MAC . The faster the system, means the lower the latency, the more things can happen simular on a the system (e.g. more synths ..) Ingo Molnar, a RedHat developer and some others have started a project to minimize the latency in Linux: Realtime Preemption. By using excellent audio-hardware the JACK demon (Jack Audio Connection Kit) reach latency < 1ms, similar to the MacOS X coreaudio system. For interested people follows a little step-by-step HowTo to patch a vanilla kernel with Realtime Preemption stuff and how to compile and install this new kernel.

Note: This is an experimental kernel.
From the developers view the rt-lsm is deprecated. It's preferred to use PAM for SCHED_FIFO priorities. This will be explained later on (TODO)
Sample kernel is the stable linux-2.6.16.

Obtain the kernel-source and necessary patches

Choice 1: The simple approach
Evermind's overlay provides an ebuild for rt-sources. So to get the latest kernel source including the -rt patch, just type:

Note: this is an example, the version of the -rt patch changes almost daily. So just take a look at http://people.redhat.com/mingo/realtime-preempt, choose the latest version and replace -rt22 with the current version throughout the whole guide.
Unpack and patch kernel

The IDE sub-system can be a bottleneck if not properly configured. As pointed out by Con Kolivas in his Audio Hints, you should set the following config option for your ATA drive (relevant on and off options are showed):

The latest config option is just an example and, almost certainly, you have to change it to reflect your hardware (again lspci -v will tell you which chip-set you have).
- Make sure you get no overhead by debug features (Note: need to look at a -rt kernel config for a full list)

What is this? Working with experimental audio applications (especially in realtime mode) the system can occasionally become unresponsive to the user input. The Magic Keys permit to do some emergency tasks with some particular key combinations. For example, to sync the hard drive, umount it and reboot the system you can hit these keys: ALT+SysRq+s, ALT+SysRq+u, ALT+SysRq+b. The SysRq key is often indicated as Print Screen on most keyboards.
Another useful shortcut is Alt+SysRq+n that changes all the realtime tasks to normal priority (this works in a vanilla kernel, does not work using the realtime-preemption patch).
See the kernel documentation (Documentation/sysrq.txt in the kernel's sources dir) for a more complete description.

Important note about ALSA
DON'T use alsa-modules included on your kernel sources. Just follow the official Gentoo guide (that suggests to use the alsa-driver package). You'll find it here: qui.
Why?
If you have a professional audio-card, maybe you'll need the package

Code:

alsa-firmware

whose version must be the same of the installed alsa-driver or you may have problems.
Alsa modules included on your kernel sources may be older than current alsa-driver (and alsa-firmware).
If you decided to use the modules included in your kernel source anyway, remember that ALSA should have RTC support too. To have it, go to your kernel configuration menu and set it as follows

A note for users with a standard Gentoo desktop kernel (e.g. gentoo-sources):
If you decided to not use rt-sources you'll be missing some of the options above. Your kernel will not be patched to use realtime-lsm, so this is what you need to have it working:
- Go back to the kernel configuration menu

It's VERY IMPORTANT that CONFIG_SECURITY_CAPABILITIES is compiled as a module, or the patch WILL NOT WORK.
Install realtime-lsm, after adding it to /etc/portage/package.keywords. Do this BEFORE the make modules_install!

Code:

emerge realtime-lsm

Then follow all the other realtime-lsm related instructions. There are some other interesting options you can try, but I wont discuss them here. simply take a look in /usr/portage/distfiles/realtime-lsm-number-version, or read jack FAQ

Sound Server (jack) configuration

Jack is an audio-server made to work with low-latency. Unlike other famous sound-servers (Arts, ESD, Gstreamer, etc), it has some features (and we'll discover some of them, such low latency, flexible audio routing between applications, AUDIO/MIDI syncronization during this HOWTO) that make it a core compoent of a Linux DAW.

- Installation of jack (note we're using the overlay version, not the portage one)
Once installed the new kernel, rebooted, etc etc, we're ready to install Jack; remember we'll need the following USEFLAGS (add them to make.conf manually or with ufed or remember to specify them before every emerge)

Code:

USE=+jack,+jack-tmpfs

If you're using a 2.4 kernel, use the "+caps" USEFLAG too.
now let's install jack and its dependencies:

Code:

emerge jack-audio-connection-kit

I repeat once again it's better to use the latest unstable (~arch) version... But it's up to you to decide.

If you find too many xruns when working with audio-programs, you could add to try these lines to your /etc/fstab

Note for 2.6.x Kernels: use jackd instead of jackstart or it'll give you an error message (it'll say you need capabilities anabled... but if you folowed the guide, now you're using a perfectly configured 2.6 kernel! )

Now, if you want, you can use jack from commandline (for example to launch it on boot time), so read carefully

Code:

man jackd

that can avoid you some future headcaches.

But, believe me... it's not too much comfortable!
That's the reason why I suggest you to use qjackctl:

Code:

emerge qjackctl

This is a frontend to control Jack and its connections between programs. But we'll see that in the following part of the guide.

The only thing I can tell you, for now, is that it must be executed BEFORE other audio apps (or they won't work.)

INSTALLATION OF THE MOST IMPORTANT PACKAGES FROM THE OVERLAY:

Description of audio-related USEFLAGS:(note that some are introduced by the overlay)

caps: You need this useflag to give realtime privileges to user applications. It's needed for 2.4 kernels, so donìt use it if you have a 2.6.x one, since it's deprecated.
dssi: DSSI is a "temporary" API for wirtual instruments with a GUI. Without going deep in details, I'll just tell you there are some nice virtual synths using it and that there's an extra-plugin that lets you use VSTi and VST plugins under rosegarden (or other DSSI-compatible hosts). Take a look at http://dssi.sourceforge.net/ for further informations.
ladcca: Don't use it if you plan to use the newest lash useflag. Read what I wrote about lash useflag for explanations.
lash: Almost all the newest audio applications now use Lash, instead of Ladcca. After recommending you to take a look at their web sites, I also suggest to remove ladcca (emerge -C ladcca) and re-emerging your (~arch) version of wkeybd, jack-rack and museseq so they'll use Lash libraries.
jack: Jack Audio Connection Kit Support. You absolutly need it to work with most pro audio applications.
jack-tmpfs: Gives tmpfs support to jack (for better performances). You should have tmpfs support enabled in your kernel too. (need confirmation of this too)
jackmidi: Adds MIDI API for the Jack Audio Connection Kit (JACK). That gives MIDI support to Jack. Homepagevst: Gives VST (Steinberg's Virtual Studio Technology) to audio applications that support fst, dssi or vstserver.

NOTE: Consider to use /etc/portage/package.use to choose which useflags go with each program you install. Remember in your make.conf there should be only system-wide useflags!
NOTE: Museseq may not compile with fst-1.7, so, if it gives you problems, try to install it with a

Code:

USE=-vst emerge museseq

VST SupportWhat is a VST plugin? Without being too much technical, I can say VST is a plugin format used by most Win/Mac audio-related applications. It allows you to use inside your program (called host) real time effects, virtual synths and other things. If you are interested and want a more detailed explanation, take a look here.
Do I really need that stuff? I hate Wincows, Micro$oft and Macintrash! Yes. You need it. Just take a look at http://kvraudio.com to see how many tons of free plugins (some are really really good!) you can have access to.
How could it be? Sounds like magic; I suppose I didn't understand anything
There are at least 2 systems to use VST plugins under Linux:
-dssi-vst: it's a dssi plugin (so the host you're going to use must supporte DSSI!) that can make use of VST plugins. To do that, it uses Wine, DSSI headers and Steinberg VST SDK headers (VST protocol, in fact, was created by Steinberg)
-fst: it's a server that, with the combination of Wine, Jack and Steinberg VST SDK, allows you to use VST plugins under Linux
-Vst server. This should be the first program who gave VST support to Linux. I did never understand how to use it, so if you know how to, please write me a note, so I can add it to this documentation.
OK, it sounds cool. What do I have to do? First go here http://www.steinberg.net/532+M52087573ab0.html, read the agreement and download the 2.3 version of the SDK (or a following one, depending on what the overlay will require by the time you read this). If the link doesn't work anymore, go to Google to find the vstsdk2.3.zip file (it's part of the vst_sdk2_3.zip archive).
Remember that YOU CANNOT REDISTRIBUTE IT, SO READ CAREFULLY THE AGREEMENT I told you above. If you don't like it, simply don't use it (and you won't have VST support).

Once you found vstsdk2.3.zip (good luck), copy it in

Code:

/usr/portage/distfiles

Now add to your

Code:

/etc/make.conf

the following (new) useflags:

Code:

dssi vst jackmidi

(I know you just need the vst USEFLAG, but I prefer to set them all at this stage). This should be enough to let you have that support in Ardour, rosegarden and other important applications.

Rosegarden (sequencer) installation
With this one we can start to be a bit productive... Rosegarden is an audio and MIDI sequencer, based on KDElibs+QT. In the beginning it was written only to manage/edit MIDI files, but now audio support has been integrated (OK... it cannot be compared to Cubase or Sonar... But... It's growing quite well). It also have a good score editor, that allows you to export files to Lilypond and nice editing features.
A little note about KDE: even if KDE is very memory-hungry, once loaded it runs quite fast (comparable, if not even better than other "light" Desktop Environments). If you are lacking RAM, consider to use another Window Manager

So, if you want to start using it, you just need to do an

Code:

USE=+jack +dssi emerge rosegarden

(remember I suggest you to use the latest ~arch version).
On the overlay you'll also find rosegarden-cvs. Take a look at it too, because audio support is getting better and better in newest relases

Remember that, if you want audio support, you have to run jackd BEFORE Rosegarden. It would be a good idea to run it through qjackctl.

Ardour Installation (multi-track audio editor)
If you need VST support, you just have to do a

Code:

USE=vst emerge ardour-cvs

If that gives you problems, try to emerge ardour instead of ardour-cvs (I prefer the cvs version).
You can also try to emerge ardour2-cvs if you want a bunch of new features.

Interesting Virtual synths
zynaddsubfx: Real time Virtual Analog synth
fluidsynth: MIDI virtual Synth based on SoundFont. You can use it either via qsynth (a nice QT GUI) or via his DSSI version, called fluidsynth-dssi
hexter: A Yamaha DX7 reprodution via DSSI. It can also import your old Yamaha DX5 and DX7 patches. Take a look if you need a nice FM synth
hydrogen: A very good drum machine
ams: Alsa Modular Synth (still have to try it)
amsynth: another virtual synth that seems to sound very nice (I found filters to be a bit "cold", but I still have to experiment well with it)
Csound: As the homepage says: "Csound is a sound and music synthesis system, providing facilities for composition and performance over a wide range of platforms". Ehm... Still have to try it too

Rezound (audio editor) installation
If you need to cut&paste samples, resample, apply offline effects, remove noise, to put quickly your latest demo on a CD, converting it to mp3, ogg and other things, a sequencer couldn't be enough... You need an audio editor!
Lots of people use Audacity, because it's muti-platform, it's well known, etc etc. Well, I really hate it. That's why I suggest to try rezound. First because I find its GUI is more intuitive, second because it can use jack as an output.

To install it, a

Code:

USE=+jack emerge rezound

should be enough. Feel free to add other USEFLAGS you may need (emerge -pv rezound to have a complete list).
Once again, I recommend to put rezound and its deps to /etc/portage/package.keywords so you can use the latest version.

Jamin installation
Jamin is a "simple" mastering suite, made by a multi-band compressor and a 30-bands EQ. As all the other mastering suites, it requires you know very well what you're doing.
You can install it with a simple

Code:

emerge jamin-cvs

(or jamin, if you don't want the cvs version).
Remember that mastering is the final stage of the audio-production process, so, when you route the signal via qjackctl, insert jamin right before the ouutput you want to send audio to. (Jamin is used as an audio effect, so it must be used in real time with jack! you cannot import, for example, wave files on it to modofy them!)

Mastering is a very important aspect of music production, because it decides how your music will sound when played all around the world (from the ultra-expensive stero to the cheapest walkman or mp3 player).
That's why you need a studio and amplification system made just for mastering. If you don't have it... Well... You better give your precious single to a studio specialized in mastering

TODO:
- Adding at least a part of, this, and this guide, tweaking them to work with Gentoo and the overlay (adding an rt-sources HOWTO?).
- A "technical" Tip section, with OS-related tips (HD tips and more kernel hacks are the high priority)
- tips on EQ, mixing, mastering, automation, syncinc apps
- an introduction to the main effects used during HD recording
(flangers, reverbs, panning) and how/when to use them
(help appreciated)
- To insert other programs, after testing them: Wired (DAW with native Akai support), Seq24 (light sequencer for live performances), Bristol (moog emulator, etc), Freecycle (sample slicer)
- To give more space to free/opensource synths than to VST ones
- Case studies?

I preferred to group all the information about useflags in another section, called USEFLAGS, so I can add a description for the other missing ones or the new ones that will be introduced.

Please, take a look, first because I'm not too much sure of what I wrote about and second because maybe I'm missing some other ones.
Thank you_________________In the end we only see to change light to dark dark to light light to dark dark to light.

one thing i didn't get is why you think kernel preepmtion decreases realtime performance.
Ingo's patch makes the kernel even fully preemptible. That _massivley_ reduces latency, which is absolutely preferrable for heavy audio usage. the lower latency, the better the realtime performace.
or did i just get you wrong at that one?_________________http://proaudio.tuxfamily.org/wiki - pro-audio software overlay

About the note on kernel preemption support that decreases realtime performances for Sound Server (Jack), I would say at it must be a missunderstanding. What I have see in my system is at full (realtime) preemption support greatly increase the realtime performances of jack. I am able to use my Audigy at 2.67msec latency and around 5% processor use. If I decrease the latency to 1.33 msec, the processor use go up to around 8.5%, but the sound of aeolus, that use a lot of processing power, is not good anymore, because it doesn't have enough time to do its calculation.

Another issue is the desktop performances. It can appen when jack is runing with other sound apps and the processor is under heavy load, when switching from one program to another at the desktop seem like freezing a few seconds before switching. EDIT: At the same time, jack and the audio apps are just doing their job and it is what we want at the first place. End Edit. It doesn't append often but can append. Here, if you don't like it or don't want it, you have to try another jack setting. Increasing the latency will decrease the processor use of jack, and more power will be left for the desktop. If it doesn't work, I believe at you have to change the machine against a more powerfull one, or use less programs at the same time.

A sound card as an Audigy can save a lot of processor power, even it is not a so good card for audio pro. It have a decent soundfont synth, and you can use it at the place of a soft synth. I don't know how it work when used with a professionel card as an Hamerfall in the same box, and it would be nice if someone that have a such setup can rapport how it is working._________________[[[ To any NSA and FBI agents reading that text: please consider ]]]
[[[ whether defending the US Constitution against all enemies, ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

About Rosegarden and the kdelibs+QT, I would say at, if kde is ressources hungry, the last version (>= kde 3.4.1) are much better and stable as ever. It was a bug (lack in memory) with kde <~= 3.4.0 causing the desktop to become very slow and unresponsive when the uptime was greater as around one or 2 days, but it have been solved with 3.4.1, maybe before.

The main problem with kde is at it take time to load and at you must have a lot of RAM memory (At least 256MB for a base desktop use, It can work with less but you will get a lot of swapping.). But after at kde is loaded in memory, it have exactly the same performances as a light desktop as fluxbox. I run test on my system with kde, fluxbox, jack and some audio apps. Beside the load time, the result was exactly the same.

I have 1.5GB RAM in my box and can use the synth of my audigy and a soft synth with 2 big soundfonts, rosegarden and ardour, with kde and without problems.

# # # # # # # #

Harddisk:

As we are talking about realtime recording, the harddisk is a very important isuue. I have see many folks raporting problems with PCI harddisk. In most case, it was not an hdparm issue, but a controller one. The transfert rate of a modern PCI system is fast enough to do such work, but if you have the system partition and the recording partition on the same PCI controller, the controller will slow down the HDD when accessing simultaneously to the system and the recording partitions.

That implies at, for multitrack audio recording to work well with PCI, the recording partition MUST be alone on its controller (read alone on the cable coming from the controller. It is 2 PCI cables, each cable have it own controller). The system MUST be on a partition on the other controller.

SCSI and Firewire doesn't have this problem and are better suited for a such work. SCSI is best but expensive. It is even PCI to Firewire adapter that you can plug on a PCI hdd if you want to use an existing PCI hdd with a Firewire controller. Most cheap usb hdd have a PCI drive inside the box and it can be cheaper to buy a such drive and a PCI to Firewire controller.

See here for more on this hdd issue._________________[[[ To any NSA and FBI agents reading that text: please consider ]]]
[[[ whether defending the US Constitution against all enemies, ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

About kernel preemption:
Kernel preemption freezes processes after a certain amount of time, to let other processes to be executed. That gives the feeling of a more responsive system, for a normal desktop use, by decreasing latencies, even under heavy load.
The drawback of this system is that it generates a little overhead and decreases thorughput (that's why it's discouraged for a server use).
This was the reason why I wrote what you read on the HOWTO.

Maybe a Real Time Scheduler without preemption is better (even if it could give some lag when switching between applications)? Anybody has experience on it? Do you have any documentation about Real-Time fully preemption? Is it worth to go deep with this subject or I better remove the observations above from the HOWTO?

@Gimpel and Dominique: Thank you for the precious feedback. I'll integrate your observations on the guide before tomorrow. But before that I would like to understand well that preemption issue._________________In the end we only see to change light to dark dark to light light to dark dark to light.

Most distribution's kernel are optimized for server and network use, eventualy for the desktop. Suse is typiquely in the first case, debian and gentoo have a greater choice of kernels and are more desktop friendly.

With a kernel as the one in Suse, to install a multimedia kernel (read rt with the lsm module and some boot scripts to fix the priorities) make a really big difference even for the desktop. On gentoo, the desktop is working the same with or without rt kernel. It can freeze sometime a few seconds, as I allready wrote about.

But with a multimedia kernel, you don't only have a full preemption, but it is scripts that fix the priority of both the hardware and the software.
For the hard, the rtc get the higgest priority, followed by the sound card(s), and followed by the other hardwares.
For the software, users in the audio group get a higger priority as the other tasks.
EDIT: By that way, tasks with higger priorities will be preempted-executed before the other. END-EDIT

A server is generally a very simple system but very performing. In an audio workstation, we want a very complicated system to perform as good as possible. This is archieved by fixing the priorities. It is a little overhead with preemption, but you will win a lot more if the priorities are well fixed. You can read the first gentoo version of those scripts here. They are now in the rtirq package and you must install it and start /etc/init.d/rtirq at boot time. I have it in the default run level. Those scripts are not needed with a pam rt-limits kernel, but it would be interesting to compare the performances of the 2 ways to archieve real time operations in the same system doing the same work. I prefferre real work test over benchmark. A benchmark will give you a good idée, but a real work test will tell you exactly what append under such conditions.

I am trying to install and test a pam rtlimits system just now, but I am at the begining of the process. More later on that._________________[[[ To any NSA and FBI agents reading that text: please consider ]]]
[[[ whether defending the US Constitution against all enemies, ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

Last edited by Dominique_71 on Tue May 16, 2006 2:00 pm; edited 1 time in total

Maybe a Real Time Scheduler without preemption is better (even if it could give some lag when switching between applications)? Anybody has experience on it? Do you have any documentation about Real-Time fully preemption? Is it worth to go deep with this subject or I better remove the observations above from the HOWTO?

uhmm.. see the jacklab example on the LAC: they ran 7 mixed VST/linux native softsynths, a guitarproccessor, and two instances of sooperlooper simultanously for some hours with only 9 xruns all in all @ 11ms latency - using a realtime-preemptible kernel, and no problem with desktop _interactivity_ - using a dual head system and E17. what do you expect more? that's awesome - just perfect.

IRQ latency is all about preemption, so a non-reemptible realtime kernel sounds like an absurdum
Note we do not speak of "desktop latency" which i think should be called desktop interactivity actually.

Ouch! I have made some test with a 2.6.16.16 vanilla kernel and pam. The good news is at I get the same low latency as with a rt-lsm kernel, and a lower cpu usage. But the bad news is at one time, jackd was stopping the sound without xrun when shutting down oofice. It was only xmms runing with jack. Another time, with jack, aeolus and aplaymidi, a processor load less as 20%, the whole system was freezing to death at the start of aplaymidi.

It remind me old problems when using jack as root in the past... I don't like it at all!_________________[[[ To any NSA and FBI agents reading that text: please consider ]]]
[[[ whether defending the US Constitution against all enemies, ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

Allright!
Updated the guide with a more detailed and correct 2.6 kernel configuration for audio. I just need to know that thing about alsa rtc... Take a look and let me know.

@Dominique: Do you want I include the link for your rt-scripts? Should I use your guide and include it in the HOWTO (considering your scripts are in the audio overlay)? Or you think that what is written right now is enough, until when we'll have a working rtlimits pam?

Should I write something more about HD performances too? Kernel configuration and the suggestion of verifying that DMA is enabled with hdparm is enough?_________________In the end we only see to change light to dark dark to light light to dark dark to light.

Allright!
Updated the guide with a more detailed and correct 2.6 kernel configuration for audio. I just need to know that thing about alsa rtc... Take a look and let me know.

i have to admit that i have no idea, because i never used alsa-driver package, but allways add the latest alsa to my kernel directly from the hg alsa repo. But i think it gets compiled and installed too, search modprobe -l|grep rtc ..

Quote:

Should I write something more about HD performances too? Kernel configuration and the suggestion of verifying that DMA is enabled with hdparm is enough?

Yeah DMA is quite important i would say.

btw i added RLIMITS to the kernel guide on the wiki, with and without PAM (both is possible)_________________http://proaudio.tuxfamily.org/wiki - pro-audio software overlay

As with the Alsa driver. I would recommand to use the ones in the kernel, and install the alsa-driver package only if you need a most up-to-date version of the drivers.

About the realtime-lsm module, a gentoo kernel's user will not have full preemption, but with the realtime-lsm module, it will be able to use jack as normal user and with realtime privilege. Look here at gimpel post. (For what I know, it is the same with pam that will work with any kernel.)_________________[[[ To any NSA and FBI agents reading that text: please consider ]]]
[[[ whether defending the US Constitution against all enemies, ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

You are right. It must be 2 more files: /etc/init.d/realtime and /etc/default/realtime. They load the realtime-lsm module and fix rt priority for the audio group. Mine are here. I took them from agnula and just changed the line "PARAMETERS="gid=18"" to have gentoo's audio group.

I was thinking at they was in the rtirq package. It would be nice to add those files.

Edit: The 2 rtirq scripts in the rtirq package have some modifications to do them best suited to gentoo. So it is best to use them as the ones on the morph-sources thread. But the 2 realtime scripts must run just out of the box. Maybe just trim it to have /etc/conf.d/realtime at the place of /etc/default /realtime._________________[[[ To any NSA and FBI agents reading that text: please consider ]]]
[[[ whether defending the US Constitution against all enemies, ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

As with the Alsa driver. I would recommand to use the ones in the kernel, and install the alsa-driver package only if you need a most up-to-date version of the drivers.

Uhm... I don't know. I think it's better to follow step-by-step the official gentoo guide to Alsa. I say that because it could be a pain otherwise, if your soundcard needs alsa-firmware or other external packages that rely on a particular alsa-driver version (or you own a very new soundcard whose source code has not been integrated in the official kernel sources).

Quote:

About the realtime-lsm module, a gentoo kernel's user will not have full preemption, but with the realtime-lsm module, it will be able to use jack as normal user and with realtime privilege. Look here at gimpel post. (For what I know, it is the same with pam that will work with any kernel.)

Yes, I explained that not very well... I'll update it on the next howto relase

About the "easy-and-fast" way to have RT working, I'll update it when the script will be (re?)included in the overlay and everything will be working without too much pain

Another question: about kernel performance, I wanted to introduce the tips of this post. Some cannot obviusly be introduced in RT-kernel, but others (IRQ balancing, Use register arguments, Message Signaled Interrupts, Use 4Kb for kernel stacks instead of 8Kb and the Device Drivers part) should be available... I read a bit of documentation and it seems they should be OK for audio. What do you think about?_________________In the end we only see to change light to dark dark to light light to dark dark to light.

Should I write something more about HD performances too? Kernel configuration and the suggestion of verifying that DMA is enabled with hdparm is enough?

On the agnula users email list, it is many more problem with bad PCI partitioning (system and recording on the same partition) as hdparm issue. So, you must write something about this.

In fact, it is more to say, because the file system in use have an impact on the performances of the hdd. I have read a lot on this issue. It is not easy to find its way out because I found contradictory repports. Non journaled files system access 2 times when recording a data, one time to write the data and one time to write the inode. With journaled FS, it is more complicated because some are writing 2 times, one for the data and one for the journal, but other write all on the fly because the journal is "interleaved" with the data.

A general good reading is the system requirments on the ardour website. Follow the links too.

A good option to have in fstab at least for the recording partition is noatime. As exemple, something like

Code:

/dev/hda7 /home ext3 defaults,noatime 1 2

With noatime, the system will not write the last access time of the files it is working with. It can greatly improve the overall performance. Read kron olivas tips. He said to avoid reiser4 because it can do high troughput at the expance of high latency. He preffer reiser3 or ext3.

* ext3's default journaling mode is slower than those from XFS, JFS or reiserfs, because it's safer. When ext3 is going to write some metadata to the journal, it takes care of writting to the disk the data associated to that metadata. XFS and JFS journaling modes do *not* care about this, neither they should, journaling was designed to keep filesystem integrity intact, not data, ext3 does it as an "extra", and it's slower because of that. But if you want to do a fair comparison, you should use the "data=writeback" mount option, which makes ext3 behave like xfs and jfs WRT to journaling. Reiserfs default journaling mode is like XFS/JFS, but you can make it behave like the ext3 default option with "data=ordered"

That implies at data=ordered must be used if the recording partition is ext3.

In fact, it is so many things to say about hdd and file systems at it can be worth a thread or a howto just for it._________________[[[ To any NSA and FBI agents reading that text: please consider ]]]
[[[ whether defending the US Constitution against all enemies, ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

As with the Alsa driver. I would recommand to use the ones in the kernel, and install the alsa-driver package only if you need a most up-to-date version of the drivers.

Uhm... I don't know. I think it's better to follow step-by-step the official gentoo guide to Alsa. I say that because it could be a pain otherwise, if your soundcard needs alsa-firmware or other external packages that rely on a particular alsa-driver version (or you own a very new soundcard whose source code has not been integrated in the official kernel sources).

1. Use ALSA provided by your kernel. This is the preferred/recommended method.
2. Use Gentoo's media-sound/alsa-driver package.

In fact, the alsa-driver package is usefull only if you have a very recent hardware and it will be included faster in alsa as in the kernel, or if you have some kind of problem with the drivers in the kernel. Those problems can be not working hardware, but more often "small" things as wrong mixer control._________________[[[ To any NSA and FBI agents reading that text: please consider ]]]
[[[ whether defending the US Constitution against all enemies, ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

A kernel option that can be usefull to know for a newbie, even if it is not multimedia related is
Power management options -> APM -> <M> APM + [*] RTC stores time in GMT (It is the only option I have in this submenu)

The RTC manage the clock on the motherboard. If you are using only one OS, the only requirement is at the bios and the OS use the same reference. It can be GMT or local time. But if you have 2 or more OS in the same box, you have to use GMT in the bios and with each OS, otherwise you will get in trouble when your local time is changing from summer to winter time. Each OS will change the clock in the RTC and you will get a wrong time. By using GMT, the time in the RTC on the motherboard will never change, and the OSs will just interpret it to get the current time. (Linux use GMT as default, but some distros use local time as default. windows use local time but it can be changed somewhere in the control panel.)_________________[[[ To any NSA and FBI agents reading that text: please consider ]]]
[[[ whether defending the US Constitution against all enemies, ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

It appear at I have to files related to the realtime lsm, /etc/modules.autoload.d/kernel-2.6 with a line

Code:

realtime

. It does at the realtime-lsm module will be loaded at boot time.
/etc/modules.d/realtime with a line

Code:

options realtime gid=18

It will grant the users in the audio group realtime privilege. So in fact, it is no need for the 2 scripts above with thise files. The only advantage to have thoses scripts is to automaticaly install this module and reduce the user interractions. The rtirq scripts are still needed.

EDIT: I am testing my system with PAM, and with or without the realtime lsm module. More on that later. For the moment, it appear to me att the realtime script must be started by hand (as exemple from /etc/conf.d/local.start) to get it to work. So we need to modify those files to get the normal gentoo init behaviour, or use /etc/modules.d/realtime and /etc/modules.autoload.d/kernel-2.6. It is needed to run

Code:

update-modules

after each modification in those files and to reboot the system._________________[[[ To any NSA and FBI agents reading that text: please consider ]]]
[[[ whether defending the US Constitution against all enemies, ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

First, I have not been able to test a non pam system, because emerge will automatiquely upgrade the needed packages when installing pam, but will not downgrade those packages when desinstalling pam. If you try to desinstall pam, you will just get a non working system because it will not be possible to login.

The good news is at it is a few weeks ago at I instaled pam and was working on a pam-rt limits + realtime lsm system, and that without any trouble.

The bad news, is at a pam-rtlimits system will not work well on a non rt kernel. So if you want to use both rt and non rt kernels, just don't install pam. I am not sure if a rt kernel with a pam non rt-limits will work with the realtime-lsm module. I believe at it should work, but I have not try it. And I have no idée which one from pam and rt-lsm will take the advantage over the other.

The pam in portage (x86) is not rt-limits, so you have to install pam from the audio-pro overlay that is rt-limits enabled.

Installation of a pam system:
Put pam in your USE flags in make.conf (Ihave it and it work, I have not done an emerge world with the new use flag, so I don't think at it is necessary because emerge install all the dependencies)

Code:

emerge -av pam

be sure at the pam version is from the audio-pro overlay. (pam-0.78-r6 at that time of writing)
and install the rt-sources and compile the kernel as explained in this howto. (It is no need to recompile an existing rt kernel.)
Don't install rtirq and don't modify /etc/modules.d/realtime and /etc/modules.autoload.d/kernel-2.6 to load the realtime module.

If you want the realtime module on a pam system (or on a non pam system):

Code:

emerge -v rtirq

Code:

rc-update add rtirq default

modify /etc/modules.d/realtime and /etc/modules.autoload.d/kernel-2.6 to load the realtime module and

Code:

update-modules

Test
I have done a few fast test on 3 rt kernels. It is a 2.6.14.2 of my own, a 2.6.16-rt13 and a 2.6.16-rt22, each 2 from this overlay.
Be aware at this test is not scientific and must be repeted a lot of time and extended with much more cpu intensive testing. But I got an idée. And it is not bad. First, the result:

My system is a PIV 1.8GHz with 1.5GB RAM. The jack settings was 64 frames, 48kHz and 2 periods = 2.67 mseq with an Audigy2.

The 2.6.14 have worst results in most cases, so the 2.6.16 kernels are better. More test needed to be sure if this is real or just statistic.

The 2.6.16-rt13 is sometime better as the 2.6.16-rt22, sometime worst. So more test is needed. The different timer frequencies seem not have a big impact or no impact at all. I think at test with very high CPU power is needed to see if it is a difference.

The higgest CPU usage (between () for aeolus but not otherwise) was obtained when starting oofice. I was using kde and get no xrun at all. That is a very good news. Both the rt-lsm and pam seem to work just fine.

It is almost impossible to intepret the differences between pam + lsm and only pam. More test needed. In fact, they seem to behave the same. So it is a question of choice to know which one you want to use.

It would be interresting to compare with a rt-lsm only system, but I have no partition left to install one more system in my box. I can only say at, before the pam installation, the behaviuor of my system was so like now, at I was sure at I was using only the realtime-lsm module. It is only when begining this test and removing pam from my system at I understood at pam was not only installed, but working too.

I get no trouble at all, and that before the pam installation with only a rt kernel and the realtime lsm, after the pam installation and the use of both pam and the rt-lsm with a realtime kernel, and now with only pam and a rt kernel.

But simple things that do the job work generally better as complicated ones, so I would recommand to use the rt-lsm module or pam-rt limits, but not both._________________[[[ To any NSA and FBI agents reading that text: please consider ]]]
[[[ whether defending the US Constitution against all enemies, ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

I found a difference between the two 2.6.16 kernels. The one with 250Hz timer clock have a more responsive desktop as the one with 1000Hz timer clock.
The main difference is when fsck is working, the one with 1000 Hz timer clock is almost unusable and it is best to take a break (the sound still work fine), when the other is still usable. But even for a normal use, it is a little difference. Not always, but often.

So, to set 250 Hz timer clock can be a mean on a slow machine to archieve a rt kernel with a more responsive desktop (and maybe a few more xruns at high processor load). But be aware at at least old rosegarden versions will not work with that._________________[[[ To any NSA and FBI agents reading that text: please consider ]]]
[[[ whether defending the US Constitution against all enemies, ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

i also did some tests yesterday, running 3 softsynths, hydrogen and jackrack with some effects for over half an hour, using the Juli@ in 32bit/48kHz, while recompiling my gcc against updated glibc () - desktop was perfectly usable, and 1 xrun all in all. ok, the PORTAGE_NICENESS was set to 10.
kernel runs with 1000Hz internal timer freq, all of rt's debugging and statistics stuff turned off, 4k stacks enabled. ah and RLIMITS, no rt-lsm._________________http://proaudio.tuxfamily.org/wiki - pro-audio software overlay