Thursday, 11 August 2011

3.0.0-ck1 and BFS for 3.0.0

Note no packages this time because the current distributions get really confused thanks to the new numbering and may not boot.

Nauru was a very depressing place indeed, really living up to the expectations I had based on what I knew of the history of it. Goddamn we (collectively, the developed world) screwed that country up big...

Thank you! It's nice to have bfs on 3.0.x. I was using 3.0 for a while for the improvements to btrfs performance but I just couldn't stand the terrible responsiveness when compiling or compressing stuff in the background. I actually went back to 2.6.39-ck2 and just dealt with the slower btrfs performance. Yes, bfs is really that much better compared to the stock scheduler.

@ck, I speculated in your previous blog, this BFS for linux-3.0 is difficult, because of a few new flags in the official scheduler which is for Linux Containers. Did you implement all this in BFS also?

As per usual BFS practice, I... ignored features that are absolutely meaningless, useless, and more importantly inappropriate on the desktop. So - no, there is no support for any new features, just the same reliable predictable simple BFS which just plain works. I spent some time updating the sched domains code to be in line with the latest mainline and deleted stuff which is irrelevant to BFS which is why it's even smaller.

I was streaming video to internet other day, and the stream lagged really horribly using stock 3.0 kernel, so I had to go back to my 2.6-ck kernel.

Thanks for this, btw are you doing BFQ too?The stock IO scheduler locks completely my system when moving from HDD to HDD, haven't tested on 3.0 however. BFQ Fixed all these problems and more in the past.

@ anon:A few days ago I asked Paolo Valente (BFQ's developer) about it and he said he's "trying to find the time" to port it to 3.0.Guess we'll have to wait until a "3.0" appears in http://algo.ing.unimo.it/people/paolo/disk_sched/patches/or perhaps an announcement here: http://groups.google.com/group/zen_kernel

Well one doesn't have to look at benchmarks to verify that linux kernels are gettings worse by the month. Each new release is slower than the previous one, there are months (years?) old unfixed power problems and other issues, but people insist on working on numa stuff instead of fixing existing problems for everyday use. But stuff that actually make it work better - like bfs or bfq - is excluded from the official kernel. Bah...

Maybe it's a good idea to create a version control repo for BFS, even if version control isn't the way CK works. I say this because BFS right now pretty much depends on one person. A proper "project" might result in a proper community of developers around BFS over time. It really might be worth the effort to start a project on something like SourceForge or whatever.

One small bug I noticed and it's pretty recent, I don't know exactly when it started happening (because I wasn't using BFS for some time), but it happens on 3.0 and BFS 0.406.

With VDPAU (nvidia) video decoding, when resizing video from fullscreen back to windowed by double clicking the video in smplayer, the transition is incredibly sluggish. It takes maybe 1.5 seconds do it, and you can see parts of the GUI appearing sporadically before it resizes (the video slider etc), and in the process video stops running (annoying when you have seperate sound track running synchronized elsewhere, don't ask why :D). This doesn't happen on CFS.

@CK BFS breaks tuxonice resume from hibernation when LZO or LZF compression is applied. This started from 2.6.39.3 onwards. Stock kernels with just tuxonice work fine. Related discussions can be found at the Archlinux and Gentoo forums from users of the pf-patchset (ck+tuxonice+bfq). See https://bbs.archlinux.org/viewtopic.php?pid=963183#p963183 and below for more.

About tuxonice: That's a real shame. It's enough hard work for me to keep in sync with the mainline changes. It's nigh on impossible for me to try and keep in sync with all other out-of-mainline patches (yes I do see the irony given -ck is out-of-mainline too).

Well freezing all the tasks and putting the scheduler in a state where the suspend process can proceed is EXTREMELY non-trivial, so just about any sort of interaction can happen that breaks suspend / resume. Last I saw, the compression is done in weird ways like disabling interrupts. Just about anything bad might happen like that :(

@CK: do you have any info on two problems I posted above? VDPAU is kinda closed source and all, it depends solely on nvidia's ability to code it right so it works with BFS. But why would Openarena stutter? As stated above, I'm experiencing small mini-freezes every minute or so.

If you could do something for either of these bugs that would be awesome :)

@CK: I am using -ck (Archlinux AUR), but there is no BFQ for 3.0 yet (it's commented out), so for now -ck includes only BFS. I am not using ondemand power governor or compressed ram cache, pretty much default mainline kernel with BFS and it's recommended settings (ticks off, preempt on, timer 1000).

P.S. Do you by any chance have a nvidia card capable of vdpau so you could either try to reproduce the vdpau bug or openarena bug with nvidia blob drivers?

I tested pure BFS without the extra -ck patches. Both bugs are still there, although it may be that openarena freezes are less often, and when they do happen the freeze time is a bit shorter (I think).

Sorry, I mixed up things in above posts, I though -ck includes BFQ by default. I now see that -ck is in fact BFS but with lots of mini-patches. Anyway, I tested both and its roughly the same.

One more thing I noticed about the vdpau bug, when moving smplayer window around, CPU goes insane and movements are VERY choppy.

Getting a lot of '/usr/src/linux-3.0/usr/include/linux/sched.h:xxxx: userspace cannot reference function or variable defined in the kernel' errors with 'make headers_check'. It does not happen with the stock 3.0.0 kernel.

Also got the error:/usr/src/linux-3.0/usr/include/linux/sched.h:1016: included file 'asm-x86/current.h' is not exported

What the heck are you talking about? Linux only runs on 32 and 64 bit platforms and long and int are the same size on 32bits. They're only different on 64 bit platforms and there is zero speed advantage to int over long, just slight size difference.

"Int is the most natural integer for a given arch and is tied to word."

Not really. Neither the C nor C++ standards say anything about it. As a result, it's compiler-specific as well as system specific. GCC on Intel x86-64 has a 32-bit int while the native word size is 64-bit (long). Other compilers and GCC on other architectures can (and do) behave differently.

@other anonymous: are you really such a moron to think you deserve an apology, or that he was actually apologizing for something that is your bullshit view of what's important that he rightfully doesn't care about?