Making Hardware Just Work

An interesting library someone pointed out after I wrote this is discover from Progeny.
Other related projects include kudzu and of course Linux hotplug and D-BUS.

Problem Statement

This document is an attempt to think through the various elements that go
in to presenting a nice, user-friendly interface to the hardware of a
typical desktop system. No attempt is made to consider issues that
apply to "big iron" servers, because I don't understand those issues.

Let's take a simple example of what should happen from 1,000 feet.
In the following, "system" means an appropriate conglomerate of
kernel and userspace features, it does not imply kernelspace.

I plug in my new digital camera.

If the camera's interface type (e.g. USB) supports it, the presence
of a new device is noticed without user intervention.

For devices that can't be detected on plugin, there's some GUI
application allowing me to explicitly probe or specify the
device information.

The system gets whatever information is available from the device
about the hardware model. Vendor, ProdID, whatever it can get.

The system consults a mapping from hardware model information to
driver information. This mapping is created by merging three sources
of information: user-provided information, OEM-provided information,
and operating-system-vendor-provided information.

If no driver is found, the system delivers a notification
to the desktop environment, and the desktop environment
informs the user that the device is not supported.
Alternatively, the desktop environment asks the user
which driver to try and passes the answer back to the
system; the system a) tries that driver and
b) saves the driver information in the above-mentioned
mapping so it won't have to ask in the future.

The system loads a driver once one has been found in the mapping, or
added to the mapping.

One or more kernel devices are created by the driver
(implying major/minor pair, and /dev file if required).

A notification is sent out from the system to all interested
listeners that a new kernel device has been created.

The user's desktop environment asks the system what sort of device
is represented by the new kernel device: camera, scanner, printer, etc.

If the system does not know, it says "device type is unknown."

If the device type is unknown, the desktop environment asks the
user to provide it. "What did you just plug in?"

After the user provides this information, the desktop environment
passes it back to the system. The system records a mapping from the
hardware model identifying information to the device type. Next time
a kernel device is created for a physical device with the same model
information, the system will report its type accordingly.

The desktop environment allows the user to interact with the device in
device-appropriate ways. For example, view images from a camera,
monitor errors from a printer, or whatever.

Abstraction

Starting from the top down, what should the implementation look like? On
the desktop or application level, one reality is that we will have many
separate applications that care about hardware. Some of these applications
will be trivial applets, others will be standalone apps such as an office
suite, others will be desktop components such as the file manager.

Given these many applications, it's critical to be able to add support for
new hardware models, and even new kinds of bus (USB, PCI, etc.)
without modifying each indidividual application.
However, obviously applications will need to understand the
type of device - where "type" means something like
"camera," "CD-ROM drive," and so on - the user-visible user-interesting
type of the device.

Another requirement on the application level is to be cross-platform.
Thus at some point in the stack, there will need to be a virtualization
with "backends" for various operating systems.

For some time I've been advocating a "hardware abstraction layer." This
layer is simply an interface that makes it possible to add support for new
devices and new ways of connecting devices to the computer, without
modifying every application that uses the device.

Devices

My proposal is that the hardware abstraction layer be a shared library
API. The API would work in terms of Device objects. On Linux a Device
object often maps to a kernel device, but may also map to some entirely
userspace devices (printers are typically all userspace, and may even be
network devices). "Device" in this API is closer to what a user would
consider a device than to any specific implementation concept. The
hardware library would maintain a list of devices that currently exist,
and could provide those upon request.

A Device object would maintain a set of properties in a key-value mapping.
It would be device-dependent which properties existed.
Examples of properties:

Types=Camera,Storage; Types=CD-R,Storage; Types=DVD,Storage

Bus=USB, Bus=PCI

Vendor=Logitech

Not all properties come from the same place.

Some properties would be defined as direct mappings from some hardware
feature. For example, the USB Product field would always be passed
straight through to a property called something like "USB.Product"

Some properties would be "portable" across hardware. For example,
there might be a "Description" property, which would be the best
readable name for the device available. For USB, this might be a
combination of Manufacturer and Product information from the
hardware. PCI provides similar information.

Some properties would not be derived from the hardware itself,
but rather from a mapping maintained by the system.
For example, the "Types" property might be in this category;
the system would maintain a mapping from hardware model
to hardware type.

Some properties might combine the above two; for example,
we might have a mapping that gave internationalized
names for some devices; if the mapping covered a device,
the internationalized name would be used for the above-mentioned
"Description" field, and otherwise the library would fall back
to the information from the hardware. So the "Description"
property would be a function of both the hardware and the
system-maintained mapping.

Some properties might be metadata stored on the device by
the desktop environment or applications. For example
an X-Nautilus-WindowPosition property storing the position
of the window that's used to view storage device contents.

Some properties might come from translations and localizations;
for example overriding the name of a device.

Some properties might come from blacklists and whitelists.

In short, we create a coherent view of the devices on a system and their
properties by merging a set of data sources, including the physical
hardware, drivers, subsystems such as CUPS, a bunch of data that comes
with the operating system, any data that hardware vendors ship with their
hardware, and finally data that users provide manually.

The exact set of information available to be merged depends on
what operating system we're using and what kind of device
we're talking about.

However, the next problem is how to provide functionality specific
to particular types of device. For any camera, the application
needs to be able to get the images from the camera.
For any printer, the application needs to be able to check
what print jobs it has active, and whether there are errors.
For any CD-R, the application needs to be able to burn an
ISO image.

It seems foolish to pile all this functionality into one giant shared
library; instead, we want to build on projects such as CUPS, gphoto,
cdrecord, and so forth. The missing piece, though, is a mapping from the
Device object to those projects. That is, given a Device that I know is a
printer, how do I locate the CUPS print queue that corresponds to that
printer? Given a Device that I know is a camera that can appear as a file
system, how do I mount it?

Say nautilus wants to get images off a camera; you need an API that has a
function like "camera->get_images()." This API probably wouldn't live in
the main hardware library. However, you need a way to get that "camera"
object given a Device*. So the camera API need not live in hardware lib,
but it does need to see the same set of devices seen by the hardware lib
and you need a way to map from Device* to whatever a camera is in the
camera API. Same holds for CUPS, sound server, whatever. Given that
I can list Device* and find two sound cards, how do I specify that
one of them is the output device for the sound server.

So what's needed is a way to identify a Device (remember a device often
means a kernel device, with major/minor pair) that can then be passed to
arbitrary subsystems such as CUPS.

The most obvious solution is to simply pass the Device* from the base
hardware library around between more specialized libraries.
However, this requires buy-in, e.g. the CUPS libs would need to
link to the hardware library.

The other approach is to bail and break the abstraction for this purpose;
i.e. allow getting at the underlying kernel device or CUPS data structure,
and pass that around. This is probably the most practical thing at
first.

An outstanding question is whether applications would use subsystem APIs
directly, or whether the single hardware abstraction library would provide
APIs for each type of device. For example, is there a Printer object in
the base hardware abstraction library that uses CUPS on the backend; or do
applications just use CUPS directly, after asking CUPS for the printer
that corresponds to a Device*. The answer to this may vary according to
the properties of the particular subsystem - is it already a portable
abstraction layer, or is it Linux specific, for example.

A complication: Not all uses of a device are device-type-specific. For
example, the Device object might have a generic operation to disable the
device (presumably this would normally map to unloading its driver).
Thus, the base hardware abstraction library might support these
operations.

Implementation Goals

Here are the major areas of implementation that I see.

1. Loading the right driver

Given some raw hardware information about a device, such as
its PCI or USB ID, load the right driver for the device.
This involves maintaining a database mapping hardware ID
information to kernel driver names and parameters.

It must be possible for an OEM to ship an addition to the database
along with a product. It must also be possible for local sites
to add their own overrides or provide missing mappings.

There should be a hook from this layer to the desktop, where the desktop
is provided with a) whatever description of the device can be scrounged
up, if only "the device you just plugged in," and b) a list of possible
drivers. The desktop then asks the user to pick one of the drivers, and
invokes a routine to record the choice for future reference and then load
the driver. This hook would probably be in the form of a few simple
functions in the hardware abstraction library.

Once a driver is loaded, we should have zero or more new kernel
devices. This means that the list of Device* provided from the hardware
abstraction library would be updated. So the hardware abstraction library
needs some way, from inside an application, to monitor additions and
deletions from the list of loaded devices. This should not be done via
polling, but rather via notification.

2. Creating and maintaining /dev files

In an ideal world, /dev files are completely hidden from applications.
Using them always involves knowledge of specific kinds of hardware, and
thus breaks the abstraction barrier that allows us to add new device
support without changing all the apps. /dev file naming may also vary by
platform, and even by local site.

Currently, /dev naming is sometimes used to specify preferences such as
"the default audio device" or "the default CD-ROM" (/dev/audio,
/dev/mouse, etc.). Using /dev as an implementation detail for this is
fine, but applications should have a way to get the default audio Device*,
then play sound to the default audio Device*, without ever hardcoding the
/dev/audio path in the app. Hardcoding /dev/audio means that the
application must make platform-specific and devicetype/bustype-specific
assumptions. These assumptions are ideally buried in a library, for all
but the most specialized applications.

Anyhow, from an application standpoint, /dev naming is something that should
magically go on behind the scenes of the hardware abstraction library.
But it still has to be implemented somehow.

3. Tracking the list of Device* (and each device's state)

This is the primary task of the hardware abstraction library. It has
several components:

Monitoring kernel, CUPS, etc. devices as they come and go
and merging those into the list of Device*.

Maintaining a database that fills in the properties
of a Device* that don't come from the hardware itself,
such as the type of each device.

Understanding how to read hardware information from the
various kinds of device - USB, PCI, etc.

Synthesizing the properties of each Device* from the hardware
information, the database, user-provided information, and
other available sources.

Exporting generic Device* operations, such as
"disable this device"

4. Providing a means to use each device

This is the secondary task of the hardware abstraction library. Given a
Device known to be a printer or a camera or whatever, the library must
chain to CUPS, gphoto, cdrecord, or whatever is appropriate, allowing the
application to use the device.

In some cases, the hardware abstraction library may offer its own APIs to
use a type of device. This would be especially useful when the existing
subsystem is low-level/crufty, hard to use properly, or unportable.

In other cases, the subsystem may be quite complex, something like
GStreamer or CUPS; wrapping these APIs probably doesn't make sense. But
there still has to be a bridge between the hardware abstraction layer and
the subsystem, so they work together in a coherent way, and above all
agree on the list of available devices.

Implementation Details

1. Loading the right driver

Suggestions:

Use the same database system here that's
used for maintaining information for
Device* properties. That is, use the same
file formats and so forth.

There really must be a way for third party OEMs to add hardware ID to
driver mappings, and this is really the same problem as allowing users
and local sites to add them.

To ask the user which driver to load, D-BUS seems appropriate. It is
a channel from system-space to user-login-session-space.

I don't have much opinion on this otherwise.

2. Creating and maintaining /dev files

As long as this is suitably buried beneath the hardware abstraction
library, it's fairly irrelevant to desktop application developers how it
works. Anything that works is great.

3. Tracking the list of Device* (and each device's state)

The hardware abstraction library should use a "model-view" architecture,
where the Device object and list of Device objects make up the model, and
the application implements some view of it.

It seems appropriate to use D-BUS to notify the hardware abstraction
library of changes to the list of kernel devices, changes to the list
of CUPS printers, etc.

Each Device object would have a backend that was either a kernel device
(major/minor pair), a CUPS printer, or some other kind of device
representation. Information from this backend would then be
integrated with override or supplementary information from a system
database, and the result would be exposed as the public API of the
Device object.

4. Providing a means to use each device

The implementation here is very specific to each type of
device.

Blacklists and whitelists

The core implementation idea presented in this document is to maintain a
list of device objects, where each device object has a set of properties
(key-value pairs).

Blacklists/whitelists of any kind simply become additional properties
to be merged, or perhaps simple rewrite rules that modify a property
in some way. Something like:

if DeviceModel matches blahblah then set BrokenFeatureFoo to true

This is extensible to support any kind of device, device feature,
blacklist/whitelist, user configuration, translations, or whatever; we just
store arbitrary data associated with any device, more or less.