Abstract: A Monitoring system for analysing the traffic being
sent to peripherals on a Universal Serial Bus. The system is composed of a
patch to the Linux kernel and a user space monitoring application.

Acknowlegments:
This project would have been much more difficult without the support and advice
of my supervisor, the helpful and relaxedatmosphere of the hardware bunker, and all the musings of the Linux USB
developers on the linux-usb-devel mailing list.

The Universal Serial Bus is a
system for connecting peripherals to a host computer. It was created to give a
simpler, higher-bandwidth, and more flexible replacement to RS-232 serial and
parallel interfaces. The first Specification for the Universal Serial Bus was
released in 1996 by a consortium of Intel, Compaq, Microsoft, and NEC. Hardware
was soon available and the system has grown to becoming a standard feature on
100% of new personal computers [30]. By the end of 1998 over 10 million USB devices had
been shipped[31].

One of
the key features of USB is that it allows the chaining together of devices in a
tree so that many devices can be connected together via hubs to one port on a
host system. This capability can bring with it a difficulty in analysing what
is going on with the bus in general or with a particular peripheral. USB’s
greater complexities also make it difficult for developers of both devices and
drivers to accurately and effectively debug them. This has lead to a number of
expensive hardware systems that can physically monitor the signals over a USB
cable and interpret the signals in a way that can be analysed by a separate
monitoring computer[1]. This is an
expensive but wholly unobtrusive method of USB monitoring.

The aims
of this project were to create a system that allowed a Host computer with USB
attached to operate normally while selectively monitoring the traffic to and
from devices on that USB. The resulting system was intended to produce minimal
interference in terms of performance and functionality of the USB hardware, OS
drivers, and user mode applications.

There are different possible
solutions to this requirement for monitoring. The most obvious is to intercept
the signals in the wires connecting USB devices to a host system. For many
reasons this is both impractical and only provides a limited set of monitored
information (See Section 3.1 on page 25 for details).

The only realistic alternative
to hardware interception is to monitor information inside the operation system.
This makes any system produced operating system dependent which is unfortunate
but unavoidable given the difficulties of hardware interception. There is no
option of an operating system independent Java implementation as a standardised
java-USB interface is still at an early stage of definition.[8]

Linux is
an open source operating system under a very high degree of active development.
Originally created by one man (Linus Torvalds) Linux is now composed of code
written by thousands of developers around the world. This means that all the
source and documentation is available in the public domain, even down to the
discussions and debates of the principal developers. Linux has come late to USB
support compared to more widely used proprietary operating systems but Linux
use is growing, especially in some niche areas. International Data Corporation
estimates that Linux commercial shipments will increase at a growth rate of 25
percent from 1999 to 2003, compared with a 10 percent growth rate for all other
client operating environments combined. Many devices drivers are being ported
to Linux[25] and new devices are emerging with their first drivers
available in Linux. This combination makes Linux a desirable platform to
develop this project on and a platform with a demonstrable need for a system
like this.

The
Linux USB sub-system almost entirely resides in the kernel of the operating
system (The structure of the Linux USB subsystem is shown in Figure 2 on page 20). Although user-mode device drivers are possible most
device drivers are written to exist as kernel modules. These modules are
written to interface with the core of the Linux USB sub-system; this then
interfaces via a driver to the host controller hardware.

The principal intention of the
monitoring system is to provide information as to what data is being transferred
when and to or from which part of which device. In order to implement this the
kernel has to be changed. This is because the monitoring system must have
access to all the data transfers that occur on the USB. These transfers are
split up in the kernel, as many of the device drivers exist in the kernel.

It is not desirable to have the
whole monitoring system in the kernel. The kernel is a dangerous place to
program due to the damage that can be done. It also has much less library
support for user-mode style applications. So the desired system must consist of
two parts: a modification to the Linux kernel and a user space monitoring
application.

The two parts of the system need
to communicate. The user-space monitoring application must tell the kernel
which transfers to monitor and the kernel based monitoring system must pass the
monitored information back to user-space so that it can be displayed to the
user. Moving data from the kernel space to the user space and vice versa is a
complex issue. After a lot of deliberation the best method was determined to be
using a filesystem interface so the user-space application could use standard
file access functions to query the kernel space. An added advantage was that
this could be implemented as an extension to an existing feature of the Linux
USB sub system.

Using
this system allows maximum flexibility in the implementation of monitoring
applications, as they require no new interface capabilities other simple file
access. Simply opening a particular file and writing a set of characters will
set the monitoring level for a particular connection on a particular USB
device. Reading from the same file will dump the monitored data down to the
user space monitoring application. This turns the system from a symbiotic pair
of programs into a prototype monitoring architecture with an example monitoring
application that supports the additional functionality provided by the kernel
modification. This is a theme that exemplifies the difference between kernel
programming and application programming. The kernel provides specific services,
which can be used in different ways by different applications.

The
sample monitoring application can be implemented in any programming language.
This is because of the standard file access nature of the interface between the
modified kernel and the monitoring application. The monitoring application uses
normal file access to set the level of monitoring and read the monitored data
buffers from the kernel.

The existing Linux USB subsystem
provides a ‘file’ from which contains the topology and configuration of the
buses. The content of this ‘devices’ file enables the monitoring application to
build up a model of the bus. With this model the Monitoring Application has
enough information to set monitoring levels in the modified kernel for
particular types of transfer on particular devices. The monitoring application
can then interpret the monitoring output of the modified kernel so as to display
this information usefully to the user.

The
system that has been produced consists of a 22kb patch to the 2.4 Linux kernel
tree and a Java monitoring application. This combination is capable of
selectively setting the monitoring level for different types of transfers on
different devices, intercepting those transfers, and retrieving details as well
as (optionally) the full data buffer. This information can then be retrieved
from the kernel to user space and displayed in a graphical interface. This
interface allows the comparison of different transfers visually and the
analysis of the data contents using different encoding methods.

This
system has been released to the public domain under the title USBMon via the
World Wide Web. All information relating to USBMon has been released at http://www.dcs.ed.ac.uk/~dxh/public/USB/.
It is the only system available for Linux that can obtain real-time USB
monitoring information from USB devices in Linux with no additional hardware.
It also provides a combination of features unavailable on other proprietary
operating systems.

The
universal serial bus was created as a replacement for the venerable RS-232
serial and parallel communication standards that have featured on almost every
personal computer ever made. USB was intended to remove the need for costly
proprietary interfaces to ISA or PCI busses or the use of costly SCSI
interfaces for peripherals such as scanners and removable hard drives. The
primary deficiencies in the serial and parallel interfaces were seen as being:

·Limited Bandwidth – UART technology and types of cable
specified by RS-232 limits the potential bandwidth of a serial connection to
little over 100 kbits/s

·Lack of Chaining – No matter how many serial ports a
machine had it might never have enough and so the ability to connect more than
one device to a single port was seen as necessary.

·Lack of power distribution – With the exception of mice
in most cases a serial or parallel connection cannot power the peripheral. A
greater degree of power distribution was desired.

USB as defined by the 1.1 Specification[2]
made the following characteristics widely available:

·Up to 127 devices connected together on one bus
utilizing up to five layers of hubs.

USB defines various entities
that have specific meaning in a USB context: Bus, Device, Configuration,
Interface, and Endpoint. With the exception of a Bus each of
these entities has a descriptor that holds various characteristics. The
descriptors are available to the host system as soon as a device attaches to
the bus even if there is no device driver for the device. They are part of the
topological and configuration information.

A USB Device has the
following characteristics:

·A Device has a Class, Subclass, and Protocol.
These correspond to USB defined types of device[4]. For example Classes are defined for mass-storage,
audio, modems, scanners, etc. These classes have different sub-classes and can
conform to various USB defined protocols. It is possible for a device to be
vendor-specific in which case it’s subclass and protocol values are not
relevant. It is also possible for Devices to maintain different interfaces of
different classes in this case the Device is not of a specific class but some
of its interfaces have their own class.

·A Device also carries values for it’s
manufacturer serial number and version numbers as part of the device
descriptor.

·A Device has one or more possible configurations,
one of which is active at any one time.

Each Configuration has the following
characteristics:

·A Configuration is either bus-powered or
self-powered, and either capable of remote-wake up or not.

·A Configuration has a maximum power usage
specified in its descriptor

·A Configuration has one or more Interfaces.

An Interface has the following characteristics:

·An Interface has Class, Subclass, and Protocol.
These are similar to the characteristics of Devices but relate only to a
particular interface.

·An Interface has one or more endpoints.

An Endpoint has the following characteristics:

·An Endpoint is either an Input or an Output

·An Endpoint is of a particular type relating to
the type of data transfer that is possible:

oControl

oBulk

oIsochronous

oInterrupt

·An Endpoint has a Maximum Packet Length that it
can send or receive.

·For Interrupt Endpoints only there is an
Interval value that signals the time between interrupts.

·Control – This is a data transfer that attempts
to change the configuration of the device. This can include a bulk-style data
transfer.

·Bulk – This is the lossless one-time bulk
movement of data to or from the device.

·Isochronous – This relates to a steady stream of
real-time data where the data carries implicit timing in the rate it arrives.
Such transfers require a set allocation of bandwidth. This type of transfer is
normally reserved for real-world streamed data such as an audio channel or
video stream from a camera.

·Interrupt – This transfer type can transfer a
set size of data at regular intervals, timed in milliseconds.

The USB
specification is general enough to allow different implementations of the Host
Controller hardware. There are two common implementations in wide usage, these
are known as UHCI[6] and OHCI[5]. UHCI was created by Intel and is considered a more
simplistic hardware implementation that requires slightly more software
management. OHCI was created by Compaq and is commonly found in all Apple USB
implementations. The two different host controllers both completely support the
USB standard and will work with all USB devices; the only thing that must change
is the Host Controller Driver on the Host Computer.[4]

As part
of the USB 2.0 improvements (as discussed below) a new host controller has
emerged (the only one to support 2.0). This is called EHCI. Having an EHCI host
controller will imply the ability to cope with USB 2.0 devices.

At about
the same time as USB appeared another standard was announced called IEE 1394,
sometimes branded as FireWire[5]
an i.link[6].
IEE 1394 does not have all the chaining capabilities of USB but it does have
vastly greater bandwidth (400MBPS) than USB. IEE 1394 also has the ability to
carry out peer-to-peer connections instead of one acting as a host and the
other as a much simpler device.

The
emerging consumer digital multimedia market has adopted IEE 1394 widely[7].
It is the most common interface on new digital camcorders and is now present on
high-end games platforms. This has demonstrated the demand for
higher-than-existing-USB bandwidth when connecting some peripherals, especially
in the consumer electronics sector. This has spurred the extension of the
existing 1.1 USB specification. In December 2000 the USB 2.0 Specification[3] was released. The principal advantage of USB 2.0 is a
new higher bandwidth capability of 480MBPS.

Currently
the adoption of USB 2.0 and IEE 1394 is of particular note. There is
uncertainty about the adoption of the two standards as many hardware producers
wish to standardise on one system. It is currently uncertain as to whether USB
2.0 will be able to usurp IEE 1394’s dominant position in the consumer
electronics market but it is unlikely that IEE 1394 will replace USB as a
standardised method for connection of peripherals to Personal Computers. An IEE
1394 mouse is a little far fetched. It is unclear whether IEE 1394’s better
peer-to-peer networking capability will make it more popular for the
interconnection of broadband digital consumer devices.[29]

Due to
its open source distribution the Linux Kernel is extremely configurable and it
is very common for experienced users to recompile their own kernel with
slightly altered configuration options. The principal area of configuration is
what parts are included in the kernel as statically linked parts of a
macrokernel and what as dynamically loadable modules, and what is not included
at all. The Linux USB sub-system can be implemented in whole or in part as
dynamically loadable modules. It is common to keep the Linux-USB core
statically linked while dynamically loading the different device drivers. Other
features that can be configured in this way is which Host Controller Driver is
used by the system and the level of debug information sent to the kernel system
log.

The
latest stable version of the Linux kernel (at the time of writing) is 2.4.5.
Version 2.4.0 was released in January 2001; this was a major update from the
previous stable kernel 2.2.17. Minor updates (changes to the third number)
happen quite often and do not usually signify major change. Kernel versions
with odd middle numbers (e.g. 2.3.x) are unstable kernels that are used for
development and experimentation.

The
Virtual File System (VFS) is a core part of the Linux Operating System and key
to much of its functionality. Linux can mount new filesystems within one
unified file-structure. Not all files that appear in the unified filesystem are
sitting on the same disk; in fact not all files are actually files on any disk.
The Virtual File System allows storage devices that use very different formats
to be used together in one filesystem. The VFS has a set interface and
filesystems are written to support it. This results in the details of a
particular filesystem being completely transparent from the user. The user
cannot tell (from how it is accessed) if a file is held on a DOS or EXT2
partition[8] of
an IDE or SCSI hard drive or even a Zip drive on a parallel port interface, it
is completely transparent.

This
flexibility and transparency combined with a very strictly defined interface
that has not changed much over time has lead to the use of the Virtual File
System for different purposes. Files in the VFS can represent devices and
services. Direct interfaces to devices are often found in the /dev file-space.
Services such as CPU usage reporting are reported through files in the /proc
filesystem.

In
Linux a filesystem has an entity called a superblock. The superblock is used to
register and mount the filesystem. The superblock contains all details of how a
filesystem is accessed and points to functions that define inode structures for
particular files. An inode is the kernel reference to a file in a
filesystem. The principal function of the Virtual Filesystem is to cache inodes
and manage their access. User-programs never actually get to find out the inode
of a file. The user program is given a file-descriptor, which acts as a pointer
to a reference of the actual inode of the file. This indirection is a feature
of the Virtual File System cache.

An
inode structure contains details about the file such as modification time,
ownership (user and group), as well as a pointer to a file_operations
structure. The file_operations structure contains pointers to functions
that implement the possible actions on a file such as open, read, write,
release, ioctl, etc.

Linux adopted USB later than its
proprietary rivals. USB support has only really become widespread since the
release of the 2.4 kernel[9],
which was first released in January 2001. Major Distributions have adopted this
kernel in subsequent months and now USB support is a standard feature of most
new Linux installations.

Support
for USB devices in Linux is now growing dramatically, Major device classes are
well supported such as the Human Interface Device (HID) driver (this supports
mice, keyboards etc.) and the mass-storage driver (this is used by floppy
drives, Zip drives, CDRs, even some MP3 players and Digital Cameras).
Proprietary interfaces have also begun to be supported; high profile examples include
Alcatel’s ADSL USB modem and Phillips popular digital cameras[25].

The
Linux USB Subsystem has been developed by a group of developers spread over the
world communicating via the public mailing lists.The basic design is shown in Figure
2. The Linux USB subsystem is a part of the kernel. It
is composed of three layers: The Device Drivers, the USB Core, and the Host
Controller Driver. Due to the nature of the distributed and disjointed development
of open source systems the interfaces between the layers have become tightly
defined. This means that there is a tightly defined interface between drivers
and the USB core, and the USB core and the Host Controller driver.

Linux
USB device drivers are normally kernel based drivers that interface to user
applications via a number of methods such as the /dev device filesystem and the
SCSI filesystem where USB storage devices are represented as if they are SCSI
storage devices. It is possible however, to have user-space based device
drivers. These user space drivers interface to the USB core via the Linux USB
filesystem.

Linux
has a stable driver for the OHCI host controller, it is commonly known as
usb-ohci, however Linux has had several problems with its implementation of the
most common UHCI host controller. Two different implementations now exist,
known by the filenames of their object code: uhci and usb-uhci. Both drivers
are under active development by different people but use significantly
different internal methods. The history is complex but in brief the uhci driver
is older and for a long time had problems with isochronous transfers; while the
newer usb-uhci driver was developed to use similar internal structure and
methods as the OHCI driver. There is active discussion about how best to move
to a simpler situation for UHCI users. This, along with early development of
the EHCI driver has lead to a strong discussion of the possibility and
desirability of merging common functions in the different host controller
drivers in what is referred to as a Host Controller layer. The exact future of
this is currently uncertain.

In theory it should not matter
to users or device driver writers which host controller driver a system is
using, as the interface to the USB core is common between all the Host
Controller drivers.

The
Linux USB core handles all four types of data transfer (control, bulk,
interrupt, and isochronous) via one structure. This is called a URB (Universal
Request Block). A URB contains all important information about a data transfer
including a pointer to the actual data buffer. URBs are queued asynchronously
and can trigger completion routines via a callback function specified in the
URB (this is also extremely important to how monitoring can be achieved). The
Host Controller determines the exact scheduling of the transfers. The URB
structure contains parameters that are required to be set before a URB is
submitted and results that will be filled in by the core and returned when the
URB has completed.

The Linux USB filesystem has its own
filesystem. This can be mounted in the /proc filesystem at /proc/bus/usb. The
/proc filesystem is a memory-only filesystem that allows the kernel mode to
send and receive data as if it were the contents of a file being read from or
written to respectively. When a file in the /proc filesystem is read or written
the filesystem securely calls kernel mode functions that reply to the request
with dynamically generated contents of the requested ‘file’.

The Linux USB filesystem is intended as both
a method for relaying from the kernel to user mode applications the
configuration details of the USB peripherals and as a method for user-mode
device drivers – allowing the sending and receiving of URBs from user-mode
processes.

The configuration information is found in
two files in the top level of the usb filesystem (/proc/bus/usb). These files
are called ‘devices’ and ‘drivers’. The ‘drivers’ file consists of a list of
the currently loaded drivers. The ‘devices’ file contains the details of the
bus topology and the contents of all the device descriptors of attached devices.
This includes all characteristics of devices, configurations, interfaces, and
endpoints. A detailed explanation of the rather complicated formatting of this
file is in [15].

The device interface part of the Linux USB filesystem
is provided in bus directories. There is one directory per bus, the name being
the three-digit bus number allocated by the USB core. For single bus systems
this will almost always be 001. Inside these directories there is a file for
every device, the name being the three-digit device number allocated on
connection. So the device interface for device 1 on bus 1 is at
/proc/bus/usb/001/001. These device interface files can be used to submit data
transfers and carry out other actions on the devices.

There
has been much discussion among the Linux USB developers about the future
direction of development effort. One of the prime efforts is USB 2.0 support.
As well as this there is active discussion of standardisation within Linux of
the way peripheral interfaces such as PCI, USB, IEE 1394, IRDA, and possibly
Bluetooth represent devices and their configuration. This has already led to a
standardisation of the arrangements for Hot-plugging devices. The Hotplug system
is more general than just the USB subsystem and is used by other parts of the
operating system.[22]

The
important aspects of this are really what is not about to change rather than
what is. There is little prospect of the URB system and the Host Controller
interface changing even with the introduction of USB 2.0. The standardisation
between peripheral interfaces in Linux should not affect the internals of the
USB core or device driver design to any great extent (to break existing drivers
on a large scale would be very bad and is extremely unlikely).

USB
has been extremely widely adopted. In 1998 there were 138 million USB enabled
PCs in the world and that is projected to increase to over 500 million in 2001
with 100 per cent of new PCs supporting USB[30]. This level of support from the hardware providers is
matched by almost all widely used operating systems. Different versions of
Windows[27] and Mac-OS[28] both have extensive USB support. Because these
different operating systems are closed source and tend to have more single-user
orientated security models the position of the USB drivers is different. There
are often things called ‘device layers’ which make all devices look similar to
drivers whatever interface they have. This means that monitoring the goings on
of a USB is very different in other operating systems.

There is an application called
USB Snoopy[19] available for Microsoft Windows 98 that allows the
analysis of data sent between drivers and devices. This is often very useful to
software engineers who are attempting to write ‘port’ a driver to a new
operating system such as Linux. USB Snoopy is still at a delicate stage of
development and can cause problems with other Windows applications. A Linux USB
device driver called usb-robot[20] has been developed that can use the output of USB
Snoopy to replay USB transactions on devices where there is no existing Linux
device driver, this is used to reverse engineer drivers.

On Linux there is no equivalent
to the USB Snoopy Windows application in Linux; that is partially the aim of
this project. There is an application available for Linux called USBview[21] that displays the topology and configuration of the
USB attached to the host. Figure
3 shows a screenshot from USBview. USBview only shows
information available in either the devices or drivers files in the Linux USB
filesystem. This data is presented in a split frame with a tree format showing
each device as a node on the left and the details of a selected device in the
right-hand.

The functions that USBview
provides are often inadequate for device developers and driver writers. These
users need not only to be able to tell what configuration and topology devices
and busses have, they need to know exactly what is being communicated and when
between the device and the driver. This has resulted in a significant market
for expensive hardware monitoring systems that physically monitor the wires in
a USB bus. Providers of these include Intel, Cypress, and CATC. These devices
are normally extremely expensive and require an additional monitoring computer
to analyse the data that they produce.

The most
logical solution to USB monitoring on any platform is to unobtrusively monitor
the electrical signals on the cable just outside the host system, analyse these
signals into data transfers, and pass these data transfers to a monitoring
application.

The
principal advantage of this method is that there is no impact on the operation
of the system. No component of the USB system does anything it would not do in
a normal USB set-up. This means that problems that appear when not monitoring
are not likely to be obscured by the presence of monitoring and new problems
are unlikely to be triggered by the presence of monitoring. This makes this
solution very desirable to device developers and device driver writers
attempting to debug exactly what is happening on a system. This type of system
is also very desirable for reverse-engineers attempting to work out how closed
source device/driver combinations are talking to each other.

The main
problem with this method is the technology required to perform it. The
Monitoring hardware must be fast enough to analyse the electrical signals on
the bus, and interface with a communication pipe fast enough to communicate all
the data appearing on the USB. This means both making something almost as
complicated as a host-controller and building a communication pipe with a
bandwidth greater than that of USB (as it must take the maximum USB traffic
plus an overhead of Monitoring configuration information). Since one of the
reasons that USB was created was to increase the available bandwidth of
peripheral connections this is difficult. IEE 1394, and a direct PCI interface
are possibilities but both carry heavy implementation costs in both price of
components and complexity.

Given
that this style of solution is offered by several large companies at high cost
and the feasibility of completing this style of monitoring within the scope of
this project it was decided not to attempt this option.

The
alternative is to intercept the data transfers when already inside the host
system. As is shown in Figure
2 on page 20 the Linux USB Subsystem is composed of three layers,
Host Controller Driver, USB core, and Device Drivers. As different device
drivers interfaced with the USB core only access information relating to the
specific device they are interested in this top level is too far into the host
system for effective monitoring of the USB as a whole. Individual device
drivers are capable of providing their own debugging features if they want.
Added to this device drivers are often distributed in binary only closed source
form (device drivers are an exception in the open-source nature of the GNU
public licence under which Linux is released). Binary only device drivers could
not easily be modified to include added debugging features. It is therefore
desirable to implement monitoring after the signals are decomposed by the
hardware Host Controller and before the communication is split up and passed to
different device drivers.

This
leaves two possible locations for effective USB monitoring: the USB core and
the Host Controller Driver. Monitoring inside the host controller driver would
give access to much more information. The host controller is responsible for
the scheduling of actual transactions and this can be directed and determined
by the driver and as such the timing over individual parts of data transfers
could be monitored. On this evidence the host controller driver seems a very
sensible place to monitor a USB; however there are several problems.

Figure 5: The Kernel
Monitoring Method

Practically,
the number of different drivers and host controllers complicates making major
additions to host controller drivers. The two most common host controllers
(UHCI and OHCI) are significantly different in the amount of scheduling work
that is carried out in the hardware. As such the variety of events and values
that can be measured would be different on different host controllers and in
the case of UHCI host controllers the measurable values would differ depending
upon which driver was loaded. These differences would lead to radically
different monitoring systems for the different combinations. Creating a unified
interface for this style of monitoring would pose several challenges.

As well as these practical
issues there is a strong argument that the additional information that can be
garnered from within the host controller driver is only really relevant to host
controller driver design and implementation which is a very narrow market
compared with device developers and device driver writers. Why do device driver
writers care about scheduling they have no control over? If Host Controller
Driver writers want more information about what Host Controller Drivers are
doing they can insert debugging calls themselves.

So by a
process of elimination the USB core is the only place to implement a kernel
based monitoring system. This is the most sensible location because it is the
only level that operates on a granularity of data that is the same as the
device drivers while having easy access to all data transactions occurring on
the bus. The information is the same across all host controllers and as such is
relevant to the device driver developer.

Having
decided to monitor in the USB core the unit of monitoring becomes clear. This
is the URB (Universal Resource Block). This is the structure used by the device
drivers to submit transactions and understand their results. Events indexed by
URBs are a logical frame of reference for device driver writers and a simple
one for device developers to understand. The URB structure is defined in Appendix A.

The system design is clearly
split into two parts: the kernel patch and the monitoring application. The
kernel patch is a list of additions (and deletions) to the kernel source. This
can be applied to a kernel source, the kernel then configured as to the
system’s requirements in the usual way and compiled.

The monitoring should be
flexible. Monitoring all the data that is passed to or from an endpoint may be
excessive. So three levels of monitoring are defined – No Monitoring, URB
Headers only, and Full Data Monitoring.

The
Monitoring Application is a user-space application that can set the monitoring
level of the kernel element and read its output and display this information in
a useful way.

1.URBs
passing to and from the host computer must be selectively intercepted. This
will involve:

a.Trapping
all URBs after they have completed but before the driver is notified of
completion.

b.Analysing
the trapped URB and determining what (if any) details to monitor of it, by
referring to a maintained list of monitored endpoints.

c.Additionally
the monitored URB should be time-stamped at the time of monitoring.

2.The
relevant information obtained from a URB must be stored in a buffer. Execution
must then return to the normal kernel operations.

3.The
information in the buffer must be transferred to user space when stimulated by
the monitoring application.

4.The
kernel patch should be able to interpret a correctly formatted command sent
from the monitoring application and set the monitoring level appropriately

Attributes that should be present in the kernel patch

1.The
patch should have minimal performance impact on USB transactions that are not
being monitored. This should mean less than a few hundred extra processor
cycles for a non-monitored URB to complete.

2.The
patch should not in any way affect the operation of any other kernel service or
user space application. All existing kernel services should be completely
usable with the patch in place.

3.Changes
to existing functions in the kernel source should be kept to a minimum so as to
reduce the likelihood of developmental changes to the kernel tree breaking the
patch.

The URB structure is defined in
usb.h (in the kernel include files) and is included by all the core source
files (see Appendix A for URB definition). The URB contains a pointer to a
completion function that is key to the monitoring.The completion function is a device driver function that is
referenced when the device driver submits the URB to the USB core. This
function is then used as a callback as soon as the URB has completed. This
callback is in fact called from within the hardware interrupt handler in the
Host Controller Driver that is caused by the completion signal from the Host
Controller[10].

Initially the idea was to
replace this completion call with a new function that performed the monitoring
required and then called the original device driver completion function. This
was simplified slightly to adding in a new function that is called just before
the completion call, with the Host Controller driver’s call to the completion
function remaining in the code. So when modified the Host Controller driver
will call a ‘pre-completion function’ that will perform the monitoring
then the Host Controller driver will call the original device driver defined
completion call.

The advantage of this method is
that it means less change to the URB structure and causes less intermediate
processing to all URBs, thus slightly reducing the performance impact of the
kernel patch. The principal disadvantage is that this requires a modification
to several places in each of the host controllers. This is an annoyance from a
design point of view as it extends slightly the interface between the USB core
and the Host Controller Driver. This decision is finely balanced between
marginal performance impact versus source code ugliness and reduced future
flexibility (in that any new or radically changed host controller driver will
also require minor modifications). This is a deviation from the original plan
where it was aimed to only modify the USB core while leaving the Host
Controller Drivers unaffected by the kernel patch.

It is
all very well calling the pre-completion function for every URB but not all
URBs will need to be monitored. For example it may be that a system includes a
USB keyboard, which will cause a URB to occur every time a key is pressed.

The URB
structure stores the information about the device, endpoint, direction, and
type of the URB in a structure called a pipe. This is in fact an 18-bit
structure encoded in an unsigned integer (see Appendix
B for details). Assuming that the pre-completion
function has access to a list of monitored device-endpoint combinations (see
section 4.2.6 on page 34)
it can use values that represent masks of the pipe structure to check if the
monitored URB is on the list of monitored pipes. So the pre-completion function
will take a URB as a parameter and compare its pipewith each of the
pipe masks it has in a list of monitored pipes.

Once the
pre-completion function has determined that the URB is one that should be
monitored it can copy all the appropriate details into a new URB structure
(including the data buffer depending on monitoring level). The pre-completion
function must also timestamp this new URB. There is no timing information
available within the Linux USB subsystem so this is an addition to the URB
structure in the kernel patch.

Once a URB has been copied the
copy must be stored somewhere. It cannot immediately be sent to user space as
it is being copied in the middle on an interrupt handler so time is of the
essence. The monitored URBs must be stored in a buffer. All the monitored URBs
could be held together in one big list. The disadvantage of this is that
user-space applications would have to obtain the whole list when they may only
want to see if one device has any URBs in its buffer. Separating the monitored
URBs at the device level allows multiple different monitoring applications to
operate concurrently and monitor different devices. The reason for this is
explained below in 4.2.4Passing
the Monitored Information to User Space.

The list
of monitored URBs could be separated at the endpoint level, however this would
add a small amount of extra processing on every monitored URB (calculating the
endpoint, and then finding the correct endpoint structure) as well as
increasing the complexity of transporting URBs to user-space. There are no
obvious advantages in separating the monitored URBs at this level, as it is
very unlikely that two concurrent monitoring applications would want to monitor
different endpoints on the same device. The most logical answer then is to
separate the URBs at the device level maintaining a separate buffer for URBs on
each device.

As well as the pipe
information the URB carries a pointer to a usb_device structure. One
usb_device structure is maintained for each USB device that is currently on the
Bus. This structure contains details about the device and references to
structures describing the configurations. This structure provides the perfect
location to store the monitored URBs. The functions that pass the URBs to
user-space can easily access the correct usb_device structure; this is
explained in section 4.3.1.

Moving data from the kernel mode
to the user mode and vice versa in a relatively secure operating system such as
Linux is surprisingly difficult. The most classical method that was looked at
for some time is the idea of new system calls that could be called from user
mode to both set the monitoring level and request the dumping of a buffer of
monitored URBs down to user mode. This method is largely workable but has
various side effects that make it undesirable from an operating system design
point of view. Firstly this method has some significant security issues in
allowing the dumping of kernel space memory on request; there is ample room for
the introduction of security holes. Also the new system calls would have to be
statically compiled into the Linux kernel. This would change the nature of the
USB sub-system, as at the moment it is possible to compile it all as
dynamically loadable modules. Another problem would be the enumeration of new
system calls. It is entirely probable that new system calls will be introduced
to Linux in the future and these would then introduce incompatibilities with
this kernel patch.

A
simpler method to move data from the kernel space to the user space is to use
the /proc filesystem. Since each ‘file read’ on a file in the /proc filesystem
is serviced by a definable kernel function this function can be set to
dynamically generate the correctly formatted URBs. This system can therefore
dump a monitored URB buffer from the kernel mode to a user application.

The
creation of an individual /proc file in kernel code is quite easy, however it
is much simpler in terms of creating the correct files at the same time to use
the Linux USB sub-system’s own filesystem that is mounted within the /proc
structure. This filesystem contains a directory for every maintained bus with
each bus directory containing a file for every device on that bus, with the
three-digit device number (allocated by the USB core on connection) as the
filename.

So as not to break any currently
working system all these files and their behaviour should not be changed. Under
the modified kernel a new file could be added for every device. As well as a
“***” file (where *** is the device number) a second “***M” file would also be
created. These monitoring files would be set up in the same way as the existing
device files but with different ‘read’ and ‘write’ calls. When a URB for a
monitored endpoint completes it’s relevant details are copied to a monitoring
buffer. When this file is next read then the buffer is outputted to the file.
This action empties the buffer, thereby limiting this system to one monitoring
application in operation at any one time per USB device. This is a limitation
of this design and there is no simple method around it.

The
content of the monitoring file is the information that has been monitored by
the kernel and is being passed to the user-space application. This information
can contain two main parts: the header information and the actual data buffer.
Only on endpoints set to ‘Full Data Monitoring’ is the data buffer included.
The URB header contains values from the URB structure that could be useful to
monitor as well as a time stamp generated at the time that particular URB was
monitored. The values from the URB structure that are passed over are: device
number, endpoint number, pipe, status, size, transfer flags, actual length, and
error count. The meaning of these values is shown in Figure 6.

Device number

The number (as given by the USB core at connection) of
the device this URB was sent to or from

Endpoint number

The number of the endpoint this URB was sent to or from

Pipe

A structure containing device and endpoint numbers as
well as transfer type and direction.

Status

The status of the URB when monitored. This indicates
whether it completed or failed in some way.[11]

Size

The intended size of the data transfer

Transfer Flags

Flags that can affect transfer scheduling

Actual Length

The actual size of the associated data buffer.

Error Count

The number of physical
transmission errors that occurred during the transfer of this URB

The information must be
structured in such a way that the monitoring application can parse it easily
and unambiguously. It is also useful for debugging and advanced use of the
system that the content is easily readable by a user.

A
decision was taken to use plain ASCII text to convey information about a URB.
This fits with the two attributes above and also continues the style of most
/proc files. Linus Torvalds (the original author of Linux) has stated that
binary data should only be present in the /proc filesystem when absolutely
necessary [13].

For the URB header information
plain ASCII text is fine, but the data buffer of the URB must be sent in full
binary format. The header information contains details of the lengths of the
data buffer. The structure of a monitored URB being reported in a monitoring
file is shown in Figure 7.

In order for the Monitoring
Application to easily parse the content of the monitoring file it can be
important that the length of the ASCII header is fixed. This can be achieved by
placing leading zeros at the beginning of possibly multi-digit numbers. In C
notation this is represented by a 0 before the number of digits in a number
modifier, i.e. %02d instead of %2d for a 2 digit number. Note that the status
number in Figure 7 uses the C notation %[SPACE]02d and opposed to %02d.
This means that if the status is negative (which is possible) a minus sign will
be inserted and if positive a space inserted so that the length of the header
will not change. Keeping the header length constant is important for some
parsing systems as some systems deal very differently with ASCII and binary
information.

Given
that the ***M monitoring files are being created anyway to be read from, a
logical extension of this idea is to use the writing case to set the monitoring
level. A simple command structure can easily be defined where a few bytes
represent a monitoring level setting command. Because there are monitoring
files for every device the command need only state the particular endpoint and
the desired monitoring level. The formatting of the command need not be very
complex at all. One option would be simply to have a two-byte command where the
first byte is the endpoint number and the second byte is the monitoring level
(in fact given the maximum number of endpoints this could all be encoded in one
byte), however this is too inflexible for future development. It was decided
that a four-byte command structure where the first and third byte is fixed
would provide greater redundancy for future features. The first and third byte
can be verified by the kernel as a check against accidental input from another
source.

Byte No.

1

2

3

4

Value

‘E’

Endpoint Number

‘L’

Monitoring Level

Figure 8:
Table containing the byte coding of commands to set the monitoring level. All
bytes are plain-text encoded ASCII.

Once the
endpoint value and monitoring level (and the device and bus numbers derived
implicitly from the path of the file) are known by the ‘write’ function in the
filesystem (remember a ‘write’ means information going in to the kernel) it
must store them somewhere. The two considerations on deciding where to store
this small amount of data (the endpoint to be monitored and the level at which
to monitor) are that the location must a) be easily known by the filesystem
call that must write to it and b) be easily known by the pre-completion
function during the monitoring of URBs. Of these two factors the second is much
more important as the pre-completion function is inherently the most
performance-sensitive part of the system.

An option would be to have a
globally visible list of monitored device-endpoint combinations, however this
is seen as bad in terms of kernel design due to it’s lack of scalability and
reliance on one unified global structure. In fact, as it turns out, both
functions can have very easy access to the usb_device structure of the relevant
device. This is possible for the filesystem function because of a field in the file
structure, which is used in the kernel to represent files. There is a space
called ‘private_data’ where the files system can hide a pointer to a structure
of its choice for later reference. For the data outputting functions a pointer
to the appropriate usb_device structure was already being placed there so that
the reading function (a user ‘read’ means a kernel output) had access to the
monitored URBs. This can be re-used in the writing functions to allow the
setting of a monitoring level variable in the usb_device structure.

At first it seems odd for the
pre-completion function to have easy access to the relevant usb_device
structure as it is dealing with URBs appearing on all devices not just one at a
time, but in the URB structure that it gets passed as a parameter there is a
pointer to the relevant usb_device structure so the pre-completion function
will always be able to reference straight to the correct usb_device structure.

Maintaining a list of monitored endpoints on a per device basis
has significant implications on the performance of all URB transfers during any
monitoring compared with keeping a unified list of all monitored device-endpoint
combinations. The speed with which the pre-completion function can determine
whether the URB it is analysing is to be monitored or not is proportional to
the number of comparisons it must make with device-endpoint combinations. Using
this method of storing the monitored endpoints of a device in the usb_device
structure reduces the number of comparisons to the number of monitored
endpoints in that device rather than on the whole system. This means that performance is reduced only for a device on which
there is monitoring being performed, others are largely unaffected.

Within
the Linux USB subsystem source code the implementation of the Linux usb
filesystem to which additions are made is split between two main source files
known as inode.c and devio.c. The devio.c file contains
all the filesystem functions that provide the functionality for the device
interface files in the bus directories. The inode.c file contains the
functions to define these files and build them up in a filesystem and register
that filesystem. So for example when a new device is attached a function in inode.c
is called that defines a new inode structure, adds references to the functions
in devio.c, and adds this inode to the superblock. Then when a directory
in the filesystem is read a function in inode.c will return the list of
inodes in that directory. If the user space wants the filesystem to open a file
it will use the function pointers in the relevant inode structure to call the
device file opening functions defined in devio.c. The same method will
cause the reading functions in devio.c to be called if the file is read.

So in
order to add the monitoring files to this system a number of things must be
done. Changes must be made to inode.c to create inodes for the
additional monitoring files. These inodes must be added to the superblock in
the same way as the device interface files but must have pointers to different
file operation functions replacing the existing functions in devio.c.

These additional
file operations functions must handle the opening, closing, reading and writing
of the files. In order to have access to the appropriate data there is a field
in the inode structure that allows a pointer to the relevant usb_device
structure. This is what allows access to the URB data that has been monitored.
It is important to remember here that file reads translates to the kernel
‘writing’ information to user space and vice versa in reading.

The
added opening function for the monitoring files moves the monitored URB buffer
to a new buffer, so that monitoring can continue to the old buffer while the
user reads this temporary one. This temporary buffer is destroyed by the
closing function. This means that when a monitoring file is opened and closed
the monitored URBs up until the open are lost from kernel space. This is the
attribute of the system that prevents multiple user programs monitoring the
same device at one time.

Implementing
timing routines proved more difficult than expected. This was primarily due to
incompatibilities between the different versions of timing structures in Linux.
Kernel functions do not have access to the standard C library at all but have
most of the functions normally provided by the C library provided by very
similar Linux calls. However there are sometimes conflicts between these
definitions.

There
were several options of what structure the timing data could be stored in. The
classical C time_t type could be used and set by the time() function.
The disadvantage of this is that the granularity is only in seconds, which is
not small enough for many of the possible applications that this timing
information might be wanted for.

Another
option is the timeval structure that includes a time_t-like value
and a value for the number of microseconds within that second. This is
perfectly adequate for the USB monitoring purposes. There is another option
that is a result of the POSIX 4 specification and that is called the timespec
structure, which is doe to supersede the timeval structure. This is
similar to timeval but stores nanoseconds instead of microseconds.
Unfortunately timespec support is incomplete in Linux and although some
of the support functions are available it is not considered ready to be used.

So the
logical decision to use timeval structures was made and implementation
attempted. Obtaining and storing the timeval structure proved no
problem. However, it proved impossible to use the normal functions available in
C to decompose the time_t part of the timeval structure due to
the fact that in the Linux kernel include files there are two possible time.h
files that can be included. One file that provides the timeval
structures and the time-obtaining functions and one that provide the original time_t
structures and the functions to decompose this time_t functions. These
two files cannot be included together due to subtle incompatibilities. This is
only the case for kernel functions.

After
attempting many possible solutions the only workable possibility was to
implicitly code the decomposition of the time_t type rather than rely on the
normally available functions. This is not too complex as date information is
not really relevant so can be disregarded. The time_t type is merely a value
giving the number of seconds since January 1st 1970.

For a period of time during the
development and testing of the kernel patch the development system exhibited
occasional but very fatal kernel crashes (the whole system suddenly froze).
This was very intermittent but usually during Usb activity and almost certainly
connected to the kernel patch. Searching for a bug that can cause this kind of
error is not easy and after some attempts there appeared no easy way of
determining the cause. At a later stage of development when full data
monitoring was finally implemented in full the system exhibited exactly the
same crash symptoms but this time in an entirely repeatable way. The system
crashed when attempting to allocate some memory.

After some research the cause
was discovered. When in the kernel, memory is allocated using kmalloc() instead
of the normal malloc() function. kmalloc() takes an additional parameter which
is a flag specifying the priority. GFP_KERNEL had always been used as the
second value, which specified the correct priority for normal kernel service.
However, the URB intercepting pre_completion function is run from within an
interrupt handler and as such is not allowed to sleep under any circumstances.
In order to stop kmalloc from sleeping under any circumstances it must be
called with GFP_ATOMIC. This means that kmalloc will occasionally fail to
allocate memory but will not sleep and therefore will not cause a systemic
crash. Changing all the kmallocs in the interrupt handler fixed the
intermittent crashes. Only very occasionally does a kmalloc fail in which case
it is reported to the system log.

The only
alterations to the host controller drivers that is required is the addition of
calls to the pre-completion function just prior to all calls to URB completion
functions. The different Host Controller Drivers have different internal
structure so the number and location of these calls varies widely. The ohci host
controller driver centralises its completion call in one function but the
usb-uhci driver makes USB completion calls from 11 different locations in the
source code. This means that there is literally a one line addition required to
the latest ohci host controller driver, whereas it about 15 for the usb-uhci
driver. The smaller the number of changes the more likely the patch is to work
on future versions of these drivers.

For a
lot of the changes to the kernel code there is no option on where to put the
code as it is made up of small modifications to existing functions, however
when introducing new functions into the kernel there is a choice between adding
to an existing file or adding in a new source file to package up all the
functions to do with the patch. This second method would make it easier to make
the patch a fully-fledged kernel configuration option, however this would mean
changing various Makefiles as well as creating the new files.

Primarily
due to the required changes to Makefiles and configuration files it was chosen
not to create a new file for the new monitoring files. Instead the new
functions were placed at the bottom of the devio.c file as almost all
are related to the extensions to the Linux USB filesystem functionality, which
is implemented in the devio.c file.

The
kernel modifications were developed using a stock Linux kernel version 2.4.1[12]. The vast majority of changes that are introduced in
successive versions of the kernel are fixes to old device drivers, new drivers,
or fixes to the Host Controller Drivers.This means that as most of the modifications for this application are to
the USB core and in particular the usbdevfs filesystem there should be little
conflict with later versions of the Linux kernel. The modifications made to the
kernel for this system are mostly additions and specifically adding in new
functions, this should reduce further the likelihood of conflict.

When releasing a working example
of this system the patch was applied to a 2.4.4 kernel and found to fail in two
areas. These two areas were very quickly fixed and were a result of design
changes within one of the Host Controller Drivers (usb-ohci). This highlights
the disadvantages of the method chosen for intercepting URBs (as discussed in
section 4.2.1 on page 30). Whenever the code in a host controller is changed
around the calling of the URB completion callback the kernel patch may no
longer apply. The Linux Patch utility can cope with some degree of small change
but if the actual design changes then this requires manual intervention.

Monitoring
applications can have a multitude of functionality that helps the user deal
with the data that is monitored. This is ancillary to the core purpose of this
application. The main aim of this monitoring application is to fully
demonstrate the functionality of the kernel patch and provide a useful and
usable interface to it.

A simple command-line interface
is too simplistic and inadequate at displaying effectively the monitored data.
A graphical user interface provides a much greater ability to show the complexity
and flexible nature of the monitoring abilities of the kernel patch. The
production of advanced applications using complex GUIs is much simpler now than
it has been in the past and there are many libraries that make the
implementation of ‘standard’ looking GUIs a minimal extra effort on top of the
main application functionality.

The Monitoring Application
must maintain a model of the USB connected to the Host system that the
Application is running on. This model must be flexible enough to hold any
possible state that a USB can be in. It must also be able to hold all available
configuration information about all devices attached to the bus including
details about the configurations, interfaces, and endpoints.

Using Object Orientated
Methodology this model contains various classes of object such as Bus, Device,
Configuration, Endpoint, and Interface. These classes are interconnected by a
form of ownership. For Example a Configuration has one or more Interfaces and
each interface has one or more endpoints associated with it. This leads easily
to a simple Object Model of the whole bus. In Figure
9 a simple object model is shown for a bus, obviously
more than one of these can be held for a system. The details of the implemented
USB model objects are contained in Appendix
D.

This
Object Model provides the perfect location to store the monitored data that is
received from the kernel via the monitoring files in the /proc filesystem. This
is best kept in the Endpoint objects as this is the level at which monitoring
can be switched on or off. Note that this is different from the kernel patch
where URBs are buffered in device-level buffers as opposed to endpoint-level
buffers. This is not a problem but simply highlights different priorities in
different parts of the system. The monitored data can be kept in a new object
called URB. This will contain all the attributes that can be monitored about a
URB as well as a time stamp that is attached by the kernel patch at the time of
monitoring. Optionally the URB object can also contain the data that the URB
was transferring.

All the topological and
configuration information is obtained from the ‘devices’ file in the Linux usb
filesystem, as referred to in section 2.11. Its structure is exhaustively defined in [15]. The file gives seven different types of line each
describing different sets of characteristics that devices, endpoints,
configurations, and interfaces can have. This contains all the characteristics
from the device descriptors available to the Linux USB subsystem. This file can
be parsed to give the structure and contents of the object model. As the object
is parsed the object model can be built up and the fields filled in.

In order
to keep receiving monitored URBs as they are monitored in the kernel the
monitoring application must poll the /proc monitoring files at regular
intervals. This is best achieved in a multi-threaded architecture where a
separate thread is started for each monitored device. These threads can simply
read the contents of the monitoring file (thus emptying the kernel buffer),
construct a URB object for every URB in the monitoring file and place that URB
in the list in the relevant endpoint structure.

This
design does call for a multithreaded architecture accessing a unified set of
data structures. This would imply that there are thread safety contention
issues, however because the ‘reader’ threads are only updating a particular set
of structures that no other reader thread can be updating and the display
threads will only ever read values from these structures for display there is
no danger of thread contention problems.

In
deciding how to best present the data it is important to remember the different
information that people may be interested in finding out. One of the primary
concerns is making it simple to navigate around a bus and select devices of
interest. It can be assumed that target users would want to focus on the
details of one device at a time although retaining the ability to monitor
multiple devices simultaneously.

It is
clear that USBview (see page 22)
has an efficient way of navigating around a USB bus. A tree is a logical
mechanism to navigate around a bus and select a device of interest and USBview
shows this. In USBview another pane contains all information about a selected
device, however here USBview’s solution of having one pane for all information
is inadequate for this system. The potential amount of information that will be
stored in the USB model about a device could be huge and far too big to display
all in one pane. The solution is to break this information down into different
areas and make the second pane selectable as to which information is shown.

The
different types of data that a user of the monitoring application will be able
to access includes:

·Basic configuration information from the device
descriptors (as is given in USBview).

·The Monitoring Level settings of the different
endpoints in each device.

·The monitored URBs for each of the endpoints showing
the characteristics of each of the URBs and the ability to show their data (if
present).

This division lends itself to a
second pane having three possible states each serving the three possible uses
described above. The three states should be easily switchable between but need
not be displayed simultaneously.

Given the way USB devices work,
using different endpoints for different parts of one task, it is important to
be able to see easily the characteristics of monitored URBs on different
endpoints. To this end the URB characteristics can be placed in tables with
scrollbars that allow the viewing of a small number of URBs for one endpoint at
the same time as a small number of URBs on other endpoints.

On monitored URBs with data
attached the data need not be shown all the time. Often there will be a lot of
it and most of the time may not be essential to the detail of monitoring. USB
monitored data can represent many different things. And as such it would be
useful to be able to display the data using different

The
implementation language is significant primarily because of the library
available. The Gtk library is a C based library that is widely used in Linux
GUIs. The Swing Library is an integral part of the Java language that allows
extremely flexible and intelligent presentation in a standardised
cross-platform style.

The
decision as to which language to implement the monitoring application in is
largely irrelevant. Both language/library combinations are more than adequate
for the proposed functionality. In the end the decision was made to use Java
and Swing primarily to use the stronger object-orientated features of Java.
This proved very useful in implementing the USB object model in the Swing GUI.

As
described on page 42 the model of the USB can easily be based on a
collection of five different classes of object: Bus, Device, Configuration,
Interface, and Endpoint. The exact details of these objects are shown in Appendix D.

Many of
the objects need to keep variable sized lists of objects. For example a device
needs to keep a list of child devices attached to it and a list of possible
configuration objects. In these cases variable sized lists have been
implemented as Vectors.

Interfacing
with the kernel via the monitoring files in the /proc filesystem the Monitoring
application must update its usb model as and when it receives new monitoring
information. In order to achieve this the Monitoring Application must regularly
poll all the relevant monitoring files in the /proc filesystem. Java provides
an easy method to set up a thread that will repeat an action repeatedly after a
specified time interval. This is via the Timer and TimerTask
Classes. The Timer class sets up a timer that will run a method in a TimerTask
Object at a set frequency. In setting the frequency of polling it is desirable
to have it often enough so that the monitoring appears almost instantaneous
while not so fast as to impose too high a workload on the kernel functions that
dump the buffers down to the monitoring application.

The actions inside this TimerTask method are simply
to:

1.Open
the relevant monitoring file.

2.Read all the contents into a byte array.

3.Parse the contents of the byte array and construct URB objects
to represent the contents.

4.Place those URB objects in the relevant Endpoint object.

5.Close the file.

The parsing element of this
process is slightly complicated by the way java treats text. Since its
inception Java has treated all characters as being 16-bit Unicode entities. It
has different objects for dealing with character streams (which are converted into
Unicode) and byte streams (which are left as unsigned bytes). A problem arises
with the use in this system of one monitoring file to deliver a stream
containing plain ASCII text descriptions of the URB headers and binary data
representing the data content of the URB in the same file. Here the fixed
length nature of the URB header in the monitoring file becomes important.
Different methods can be used for the header part and the data part. Once the
header part has been read, the presence and size of the data part can be
determined. Then the data part can be read using an alternative data method
that reads the data as bytes instead of Unicode characters.

One
of the most useful features of the Java/Swing combination is the ability to
create standardised user features like tables and trees from different object
types as long as they implement an interface by including a few specific
methods. This means that the USB model objects can be used as the models for
the display structures.

In order
to achieve a tree of devices that can be navigated to select devices of
interest the Device class can be made to implement the TreeModel
interface. This requires the addition of eight methods that allow the
implementation of the tree such as get_child(), get_child_count(), is_root(),
etc…

Using
this method the collection of Device objects in the USB model implements the
tree model. A navigable tree can easily then be created using the JTree class.
The navigable tree is visible in Figure
10.

The
information displayed about a device selected on the tree has to be selected
from three options. The best solution for this is to use a feature of Swing
called Tabbed Panes. This allows the display to be quickly changed between
‘screens’.

The same method that was used to
implement the trees can be used to implement tables. A class can be made to be
an extension of the AbstractTableModel class. This involves adding six new
methods such as getRowCount(),getColumnName(), getValue(), and setValue(). This turns the class into a
table model and a simple table can easily be displayed using the JTable class.

There
are two locations where tables are useful. In the Configurations pane different
interfaces need to show all the details of their endpoints and the set
monitoring level.

The first case can be achieved by making changes to the Interface class. Making
a table from the interface class gives a table of endpoints with their
characteristics. The monitoring level of the endpoint needs to be editable.
There are only three possible values. To avoid erroneous input an editor called
a ComboBox can be attached to this column that allows the selection from a
drop-down menu. Figure 10 shows the configuration pane showing the endpoints of
a USB storage device (in fact a floppy drive).

Figure 10: The Configuration pane showing the
endpoints of a USB storage device

The second case is where the
details of URBs monitored on a URB need to be displayed. In this case each
table model can be represented by an Endpoint class. Similar extensions to
those made to the Interface class can be made to the Endpoint class to allow
the display of the URBs’ details.

The data element of the URB need not
be shown in the table. It is much better if this is shown in an independent
window as then it can be compared side to side with other URB data buffers. To
achieve this another ComboBox can be used to provide a drop-down menu on URB
rows that have an associated data buffer. The data can then be displayed in a
new window. The data that is passed in URBs can have very different meanings
and so different types of encoding can be used to view the data. A tabbed pane
is used to allow access to Hexadecimal, Binary, and Textual views of the data.

On
purely functional and performance grounds a system that unobtrusively
intercepts the USB traffic on the cable outside the host subsystem and passed
the monitored information to a separate monitoring computer provides the most
unobtrusive and accurate monitoring system. This is why large companies succeed
in selling such systems at very high cost. However this system provides a
solution to many problems as effectively as an expensive hardware solution.

Types of
problem that can be detected and analysed using this system that it was not
possible to do before without hardware monitoring:

1.Analyse
the data sent to and from a closed source driver.

2.Debug
the data sent to and from an open source driver without changing the code.

3.Clearly
show the relative activity of different devices on a bus.

4.Determine
the time taken between a driver receiving a URB and sending a reply URB.

5.Determine
whether errors in the data transmissions are occurring (as drivers may not
report them).

Areas that this system cannot
observe and is not able to aid:

1.Analysing
bugs inside the Host Controller hardware.

2.Analysing
bugs inside the Host Controller Driver software.

3.Determining
the accuracy of power distribution across the bus.

4.Obtaining
the timings of sub-parts of data-transfers such as the time difference between
a control request and the acknowledgment.

5.Debug
erroneous transfers from devices that are not of the correct format.

Prior to this project Linux
users were unable to carry out any real monitoring of USBs without specialist
hardware. USBview (as described on page 22) only provides configuration information and only
really displays the information that is already available in the devices file
in the Linux USB filesystem. This project does provide a new capability to USB
on Linux but it does not replace the need for hardware monitoring systems
completely. Experts who are designing complex devices and fine-tuning first
time device drivers will doubtless want to be certain about the exact signals
that are being transmitted. However there is a group of part-time developers
who are writing device drivers for niche devices that have been implemented on
other operating systems. For this class of device driver developer this system
is a very cost-effective solution to debugging the operation of devices and
device drivers.

Writing
code for the kernel is often portrayed as a kind of black art where the
traditional rules of software do not wholly apply, a bit like the quantum
mechanics of programming. This is less true of Linux due to the number of
people involved in its development and the amount of documentation available,
however there are still some major complexities that must be taken into
account.

The
kernel patch has proved largely stable. Although not thoroughly or rigorously
tested for memory leaks and other possible low-visibility defects, the patch
has proved stable being run as the primary kernel on the development system
throughout the progression of the project. This has shown both that the
(patched) kernel is stable for prolonged periods of time (over a week between
re-boots) in that it shows no outward sign of deterioration in either normal
use or USB performance.

After
the kmalloc() priority changes (see section 4.3.3) there were no more unexplained kernel failures.

There
are some areas where the monitoring system is less stable than might be
desired. It only has a partial solution to the issue of hotplugging, and can
give unwanted error messages when devices are disconnected.

The way the monitoring
application handles alternate interfaces is not ideal. The concept of alternate
interfaces was largely forgotten about during the design as none of the devices
that were experimented on had them. The problem is that each alternate device’s
endpoints are given their own endpoint object and assumed to be active (which
is not always true). This does not prevent monitoring but is certainly not
ideal as the display becomes very cluttered by endpoints that are not relevant
to the user.

The
principal addition in the kernel patch that affects all URB transfers
regardless of what the monitoring level is the pre-completion function that is
run on every URB prior to the driver being informed of completion. The amount
of processing that is carried out in this function is a measure of the cost of
monitoring.

Analysis
of the code in this function can give factors that affect the amount of
processing that will occur. The source code to the pre-completion functions is
in Appendix C. The factors that can be deduced from the structure of the code
are:

·For devices on which none of the endpoints is monitored
in any way there is very little processing that will occur. This is because no
comparisons are necessary to check whether the URB needs to be monitored.

·The amount of processing for all transfers on a device
with monitored endpoints will increase with the number of endpoints that are
monitored. This is because even for transfers on endpoints that are not
monitored the function will have to check the endpoint value against more
‘monitored endpoint’ values to determine whether or not this endpoint is on the
list to be monitored.

·Transfers on monitored endpoints will require more
processing than transfers on endpoints that are not monitored. This is because
allocating the memory and copying the details of the URB takes processing time.

·Transfers on endpoints with ‘full data monitoring’ will
require more processing than transfers on endpoints set to ‘URB headers only’.
This is because of the extra memory that must be allocated and the extra
copying.

To
obtain the timing information, a modification can be made to the pre-compilation
function to get the length of time that the system takes to carry out the
processing in that function. In a normal user-space C program this could easily
be done using the clock() call and clock_t variables, however the
problem that was discovered when implementing timing in the monitoring system
(see section 4.3.2 on page 37) resurfaces. The clock() function gives the
number of clock cycles that have occurred. This value looks back to zero every
few minutes on a fast 32-bit machine but is useful for very accurate timing.
Unfortunately the timing functions that are available within the Linux kernel
do not include the clock() function. It seems odd that user space can
access a service from the kernel that code in the kernel cannot easily access
itself, but it was not possible to find a way to do this. There is no evidence
in the kernel itself or in any kernel documentation of a similar function.

An
alternative to the clock() is to use the timing functions used to give a
monitoring time accurate to 1 microsecond. On a fast PC several hundred machine
instructions will occur every microsecond but this is just enough to determine
the delay caused by the processing in the pre-completion function. The absolute
time (in microseconds) can be got at the start of the function and again at the
end. The difference can then be sent to the system log. This does not include
the additional cost of calling the pre-completion function and returning, this
is assumed to be irrelevantly small. The timing results give the approximate
amount of time taken to perform the contents of the pre-completion function. It
is not completely accurate because the start and finish times are not
necessarily on microsecond boundaries so there could be an error of up to 2
microseconds. Taking that into account and after experimenting with this timing
mechanism it was decided to average the results over several transfers.

The
testing that was timed consisted of file transfers to and from a USB storage
device. USB storage devices use the Primary Control pie as well astwo bulk pipes (one in and one out), and an
incoming Interrupt pipe. File transfers will cause activity on all these pipes.
The device was re-mounted between every test in order to clear any file caches.
The first run was also repeated to counter the possible effect of memory caches
in the system.

Two sets
of tests were performed on a relatively high specification machine (650 MHz
Pentium III with 128Mb RAM):

Set 1: URB Only Monitoring – All
endpoints were initially set to ‘No Monitoring’ with the number of endpoints
set to ‘URB only monitoring’ incremented after each timed run. After each timed
run the pre-compilation time for 10 transfers on monitored endpoints and 10 on
unmonitored endpoints were averaged to a figure.

Set 2:
Full Data Monitoring – All endpoints were initially set to ‘No Monitoring’ with
the number of endpoints set to ‘Full Data Monitoring’ incremented after each
timed run. After each timed run the pre-compilation time for 10 transfers on
monitored endpoints and 10 on unmonitored endpoints were averaged to a figure.
Here the timing of monitored data points refers only to 512 byte bulk
transfers. The usb-storage driver will only shift data in blocks of 512 bytes
so this is a good standardisation to eliminate variance depending on the size
of the data that is copied.

Set 1: URB Headers only:

Number of Monitored Endpoints

Average time taken on unmonitored endpoints /
microseconds

Average time taken on monitored endpoints /
microseconds

0

0.5

1

0.6

1.9

2

0.4

1.8

3

0.9

1.4

4

1.5

Figure 13: Table and Graph showing the results of
timing analysis of the pre-completion function.

The
results of this first set of tests are largely as expected with one slight
anomaly. The fact that time taken by monitored transfers is greater than
unmonitored transfers is as expected, as is the fact that the time taken by
unmonitored endpoints appears to marginally increase with the number of
endpoints. The anomaly is that the results indicate that as the number of
monitored endpoints is increased the length of time taken inside monitored
endpoints appears to decrease. There is no obvious explanation for this. The
important conclusions that can be made from these results are that the number
of monitored endpoints (up to at least four of a possible sixteen) does not
have significant impact on the USB performance of a device and that even
monitored transfers are only impacted by about two microseconds per transfer.

Set 2: Full Data Monitoring:

Number of Monitored
Endpoints

Average time taken on
unmonitored endpoints / microseconds

Average time taken on
monitored endpoints / microseconds

0

0.5

1

0.8

5.1

2

0.7

5.3

3

0.8

4.9

4

5.2

The
results of the second set of tests shows that Full Data monitoring is much more
expensive in processing time that simple URB header only monitoring. Full Data
monitoring on a bulk transfer of 512 bytes adds about 5 microseconds to the
amount of time taken to process a URB transfer.

The
significance of this timing data on potential users can only really be guessed
at. USB allows interrupt transfers to happen every millisecond and it is
conceivable that when monitoring such an endpoint the difference between a five
microsecond penalty as opposed to a two microsecond penalty might be important.
Unfortunately it is very difficult (and beyond the scope of this project) to
put these figures in context by benchmarking the other elements of the
Linux-USB subsystem that are essential to its operation.

The monitoring Application’s
performance is much less important than the performance of key parts of the
kernel patch as the monitoring application runs at a much lower priority in
user space and can be interrupted at any time. It is still however relevant as
the monitoring application must be run I order for the monitoring that the
kernel performs is retrieved and displayed. The performance of the Monitoring
Application is heavily dependent on the java virtual machine that is being run
on the host system. On a sample system (650MHz Pentium III with 128Mb SDRAM
running IBM JVM for Linux 1.3) the Monitoring Application never used more than
10 per cent of available CPU utilisation and mostly much less. This is
perfectly acceptable.

At two
points during this project code was released for public consumption and a
notice was made on the linux-usb-developers and linux-usb-users
mailing lists. The code was released as under the name USBMon. These releases
were named versions 0.1 and 0.2 respectively. Version 0.1 only contained a
kernel patch that did not support full data monitoring. Simple command line
tools were supplied with version 0.1 to control and view the monitoring.

Possible future development of
this system could take many possible routes depending on the usage of the
system and implementation decisions made in the main Linux USB tree. The most
obvious next step is to develop the kernel patch into a configuration option in
the Kernel. This could even prepare the patch for possible submission into the
Linux kernel. Before any submission to the Linux kernel the absolute stability
under a much wider range of systems would be necessary. A lot more reviewing of
kernel code by experienced kernel developers would also be desirable.

The amount of monitored data
values monitored by the kernel could also be increased. The interface to
user-space could be made more flexibly defined so as to allow greater
selectivity of monitored events by the monitoring application, for example only
monitoring URBs that returned with error codes.

A
possible addition to the kernel patch is the provision of generalised
statistics for usage and activity over the whole USB. This could take the form
of a file in the /proc filesystem that gave a few global statistics much in the
same way as CPU usage is reported. This could then be used by simple desktop applets
to signal activity on the USB bus.

On
the monitoring application side there are many possible data analysis
mechanisms that could be implemented such as graphs and charts as well as the
calculation of generalised statistics on a device basis.

Class used to store details of a URB that occurred in a Linux USB subsytem.
The exact configuration of this URB object relates to the attributes that this
monitoring application needs to maintain. This is not a definition of what a
URB contains.

[1]
Manufacturers of these systems include Intel, Compaq, CATC, and Catalyst
Enterprises

[2] A minor
update to the 1.0 specification released primarily to clarify issues relating
to hardware device design.

[3] This is
derived from a maximum of 500mA current from a supply that can vary 4.75V –
5.25V as specified in [2] p.134-139

[4] This
situation was not necessarily intended. It is primarily a result of Intel
refusing to licence the UHCI design very widely prompting the development of
OHCI by competitors. Intel has now agreed to licence the USB 2.0 EHCI design
freely [33].

[7] According to
Apple Corp. over 125 peripherals are available at this time, in the year 2000
over 12 million IEE 1394 devices were shipped.

[8] DOS is the
Disk Operating System format common on floppy disks. EXT2 is the most common
native Linux hard drive filesystem.

[9] USB support
was started in the previous 2.2 kernel (starting at 2.2.7). However development
soon moved to the 2.3 experimental kernel tree. USB usage on 2.2 kernels was
possible via the use of a kernel back-port patch that some distributions used
to allow basic USB mouse and Keyboard use. USB usage beyond this was possible
but not easy.

[10] This is the
validation for the accuracy of the timestamp taken at time of monitoring.