Derek Glidden gave an exploit to make his machine entirely unresponsive for
several minutes. He pinned this behavior to the VM in 2.4, and added that he
felt there were still serious problems before the 2.4 VM could be considered
stable. Others confirmed the behavior, and there was a long discussion. And
one point Miles Lane said,
"Mike and others, I am
getting tired of your comments. Sheesh. The various developers who actually
work on the VM have already acknowledged the issues and are exploring fixes,
including at least one patch that already exists. It seems clear that the
uproar from the people who are having trouble with the new VM's handling
of swap space have been heard and folks are going to fix these problems.
It may not happen today or tomorrow, but soon. What the heck else do you
want?"
Derek replied that his original post had been intended to provide
some useful data-points, and Miles replied,
"Actually,
I think your original message was useful. It has spurred a reevaluation of
some design assumptions implicit in the VM in the 2.4 series and has also
surfaced some bugs. It was not you who I felt was sending enflammatory
remarks, it was the folks who have been bellyaching about the current swap
disk space requirements without offering any new information to help developers
remedy the situation. So, thanks for bringing the topic up. :-)"

Linus Torvalds also replied to Derek's original post:

Now, this may well be true, but what you actually demonstrated is that
"swapoff()" is extremely (and I mean _EXTREMELY_) inefficient, to the
point that it can certainly be called broken.
It got worse in 2.4.x not so much due to any generic VM worseness, as
due to the fact that the much more persistent swap cache behaviour in
2.4.x just exposes the fundamental inefficiencies of "swapoff()" more
clearly. I don't think the swapoff() algorithm itself has changed, it's
just that the algorithm was always exponential, I think (and because of
the persistent swap cache, the "n" in the algorithm became much bigger).
So this is really a separate problem from the general VM balancing
issues. Go and look at the "try_to_unuse()" logic, and wince.
I'd love to have somebody look a bit more at swap-off. It may well be,
for example, that swap-off does not correctly notice dead swap-pages at
all - somebody should verify that it doesn't try to read in and
"try_to_unuse()" dead swap entries. That would make the inefficiency
show up even more clearly.

Felix von Leitner reported that some of the kernel headers would cause
userland libraries such as dietlibc that depended on them, to export the
wrong API. He pointed out a simple fix, but David S. Miller replied,
"Don't use kernel headers for userspace. Kernel headers
and user headers are distinctly different namespaces, and you have pointed
out only one of many places where we use different names/structures/etc. for
some kernel networking headers vs. what userspace wants."
Felix von
Leitner complained,
"What choice do I have?
Duplicate everything and then be out of sync when the specs change?"
To
which Linus Torvalds replied:

Yes.

Even more preferably - write user-space headers that have _only_ the
minimum amount of code in them. The kernel headers have a lot of cruft that
is kernel-only, and that means that if you compile user space using them,
your compiles will be slower than they should be.

The basic issue is that the kernel will _refuse_ to follow the "namespace
of the day" rules of C89, C99, POSIX, BSD, SuS, GNU .. the list goes on. The
kernel headers are not meant to be used in user space, and will not have the
strict namespace rules that a lot of standards spend so much time playing
with.

There aren't that many things that are actually useful in the kernel
headers anyway. Most of them (like the IPv6 stuff) are really specified in
other places in the first place.

Peter J. Braam put together some ext3 rpms and made them available at ftp://ftp.clusterfilesystem.com/pub/ext3.
P.A.M. van Dam asked where he could find the ext3 patch against the 2.4
kernel, and Andrew Morton replied:

Dawson Engler wrote a utility to check source code for bugs. Specifically,
he said,
"You can look at this checker as
essentially tracking whether the information from an untrusted source
(e.g., from copy_from_user) can reach a trusting sink (e.g., an array index).
The main limiting factor on its effectiveness is that we have a very incomplete
notion of trusting sinks. If anyone has suggestions for other general places
where it's dangerous to consume bad data, please send me an email."
He listed fifteen bugs he'd discovered with this tool. Alan Cox quickly
patched many of them, and Oliver Xymoron added:

I wrote something similar to this for userspace (via ld preload). Most
of my checks followed strings around and made sure they were length checked
so as to avoid stack overflows, but I also checked args to open, etc..

In your case, basically all destinations are trusting sinks at some
level: userspace gives you data to put it somewhere. You might want to
instead check that data is passing through functions that 'detaint', by
checking capabilities, etc. I bet that you'll find that something like 90%
of code paths are covered by a few common security checks. And that most of
the remainder could be rewritten to be.

Maksim Krasnyanskiy claimed the CONFIG_BLUEZ items, saying they belonged to
the Bluetooth subsystem, adding that Linux was the first OS to have official
Bluetooth support. He described those options:

CONFIG_BLUEZ

Bluetooth is low-cost, low-power, short-range wireless technology.
It was designed as a replacement for cables and other short-range
technologies like IrDA. Bluetooth operates in personal area range
that typically extends up to 10 meters.
More information about Bluetooth can be found at http://www.bluetooth.com

To use Linux Bluetooth subsystem, you will need several user-space utilities
like hciconfig and hcid. These utilities and updates to Bluetooth kernel
modules are provided in the BlueZ package.
For more information, see http://bluez.sf.net.

If you want to compile HCI Core as module (hci.o) say M here.

Not unsure ? say N.

CONFIG_BLUEZ_L2CAP

L2CAP (Logical Link Control and Adaptation Protocol) provides connection
oriented and connection-less data transport. L2CAP support is required for
most Bluetooth applications.

Say Y here to compile L2CAP support into the kernel or say M to compile it
as module (l2cap.o).

Not unsure ? say M.

CONFIG_BLUEZ_HCIUART

Bluetooth HCI UART driver.
This driver is required if you want to use Bluetooth devices with serial
port interface.

Say Y here to compile support for Bluetooth UART devices into the kernel
or say M to compile it as module (hci_uart.o).

Not unsure ? say M.

CONFIG_BLUEZ_HCIUSB

Bluetooth HCI USB driver.
This driver is required if you want to use Bluetooth devices with USB
interface.

Say Y here to compile support for Bluetooth USB devices into the kernel
or say M to compile it as module (hci_usb.o).

Not unsure ? say M.

CONFIG_BLUEZ_HCIEMU

Bluetooth Virtual HCI device driver.
This driver is required if you want to use HCI Emulation software.

Say Y here to compile support for Virtual HCI devices into the kernel or
say M to compile it as module (hci_usb.o).

Matt Nelson had a project that would require about 6G of RAM under Intel,
and he wanted to know if there were any caveats he should be aware of. Doug
McNaught suggested getting an Alpha machine, and explained,
"Pointers on IA32 are still 32 bits, so (as I understand it) each
process can address a maximum of 4GB. You would have to allocate multiple
chunks (in shared memory or mmap()ed files, so they're persistent) and map
them in and out as needed (which could get hairy)."
But he added,
"if you can split your task up into multiple
processe such that no process needs to address more than 4GB, an IA32 machine
will work fine."
Later Matt said,
"Thanks
to all that replied. I've never needed to use so much memory before, and
was ignorant to how much magic was implemented in the 64G support on IA32.
Unfortunately, there's not quite enough magic in there for my needs... now
to find an affordable SMP Alpha system...."

Craig Lyons of Promise Technology sent a patch to Andre Hedrick, explaining,
"In the 2.4 kernel there is support for some of our
products (Ultra 66, Ultra 100, etc.). As you may or may not know, our Ultra
family of controllers (which are just standard IDE controllers and do not have
RAID) use the same ASIC on them as our FastTrak RAID controllers do. The 2.4
kernel will recognize our Ultra family of controllers, but there is a problem
in that a FastTrak will not be recognized as a FastTrak, but as an Ultra.
Consequently, the array on the FastTrak is not recognized as an array,
but instead each disk is seen individually, and the users data cannot be
properly accessed. We have a patch that fixes this and are wondering if it
is possible to get this patch into the kernel."
Andre replied,
"I do not want or need your company's patches, period.
I will not take or accept or approve of any dirty code that allows the a
poorly written binary driver that can not control its ISR and it interferes
irresponsiblily with the native ATA driver."

Bert Hubert chastised,
"Craig contacted you to
find out what was wrong and you should explain to him what the problems are,
and how he could solve them. Linus would accept patches written by Bill
Gates if they were licensed right and coded properly, so I don't see why
Promise should be an exception."
He also explained to Craig,
"The procedure is to publish the patch publically and have
people comment and try it. They will often find that your code is not up
to par or does things in ways that do not please the kernel people. No evil
is intended, it is just that the kernel developers are a choosy bunch. But
given the right prodding they will tell you how you could change your code
so that it is acceptable. Alternatively, people here might see what needs
to be done from your patch, and do it themselves."
But Andre said,
"No I would not take their code and apply it.
I might not even want to look at it. All I want is the API rules to the
signatures and we have them now. We do not need their driver."
Rob
Landley felt that, as a maintainer, Andre should be more "open" about their
patches. Andre replied,
"I have seen one version
and I got physically sick."

I would like to publicly thank you for coming to the table of GNU/GPL with
an open perspective. After 90 minutes on the phone, of which 45 minutes
were me pointing out issues promblems and complaints w/ 20 minutes on ways
to work on solutions in the near and distant future and the listening to
your concerns and questions between my moments of interruption.

The next conversion will not have the burst-in moments because it will
be in person or my cell battery will be fully charged.

Since you have stated "I will not make promise, I can not keep" this is
a good thing and it will go a fair way to clean up messes from the past on
both sides.

I look forward to Promise working with Linux in meaningful and productive
ways.

Please reply and correct anything that is mistated by me or verify
the correctness. This will show an action of good-faith before all those
watching here.

Craig replied:

Andre and I did indeed have a nice conversation on the phone. Thank you
again for taking the time to talk with me and offering your assistance. As
I stated on the phone, we are making a large commitment of resources
to supporting Linux by releasing drivers and utilities for our products,
including the FastTrak. I know we have plans to release source for our Ultra
and SuperTrak series cards, but at this point I'm not sure that the way we
are going to be supporting FastTrak is what you would like to see. As I said,
while I cannot guarantee anything that I don't have the authority to deliver,
I will pass on your requests. I will try to be an advocate for Promise in
the Linux community, and an advocate for the Linux community to Promise. If
the company has concerns, I will let you know what they are, and then maybe
you can tell us if we are off-base with those concerns or not.

I would invite anybody to contact me if you have any suggestions,
any requests, whatever. As I told Andre, I won't promise something I can't
personally deliver, but I will do whatever I can to help out. I'm also trying
to get a technical point of contact so that you don't have to deal with a
marketing weenie who doesn't understand half of what you're saying ;).