Chapter 9 STREAMS Drivers

STREAMS Device Drivers

STREAMS drivers can be considered a subset of device drivers in general and
character device drivers in particular. While there are some differences between STREAMS
drivers and non-STREAMS drivers, much of the information contained in Writing Device Drivers also applies to STREAMS drivers.

Note –

The word module is used differently when talking
about drivers. A device driver is a kernel-loadable module that provides the interface
between a device and the Device Driver Interface, and is linked to the kernel when
it is first invoked.

Basic Driver

A device driver is a loadable kernel module that
translates between an I/O device and the kernel to operate the device.

Device drivers can also be software-only, implementing a pseudo-device such
as RAM disk or a pseudo-terminal that only exists in software.

In the
Solaris operating environment, the interface between the kernel and device drivers
is called the Device Driver Interface (DDI/DKI). This interface is specified in the
Section 9E manual pages that specify the driver entry points. Section 9 also details
the kernel data structures (9S) and utility functions (9F) available to drivers.

The DDI protects the kernel from device specifics. Application programs and
the rest of the kernel need little (if any) device-specific code to use the device.
The DDI makes the system more portable and easier to maintain.

There are three basic types of device drivers corresponding to the three
basic types of devices. Character devices handle data serially
and transfer data to and from the processor one character at a time, the same as keyboards
and low performance printers. Serial block devices and drivers
also handle data serially, but transfer data to and from memory without processor
intervention, the same as tape drives. Direct access block devices
and drivers also transfer data without processor intervention and blocks of storage
on the device can be addressed directly, the same as disk drives.

There are two types of character device drivers: standard character device
drivers and STREAMS device drivers. STREAMS is a separate programming model for writing
a character driver. Devices that receive data asynchronously (such as terminal and
network devices) are suited to a STREAMS implementation.

STREAMS drivers share some kinds of processing with STREAMS
modules. Important differences between drivers and modules include how the application
manipulates drivers and modules and how interrupts are handled. In STREAMS, drivers
are opened and modules are pushed. A device
driver has an interrupt routine to process hardware interrupts.

STREAMS Driver Entry Points

STREAMS drivers have five different points of contact with the kernel:

Table 9–1 Kernel Contact Points

Kernel Contact Point

Description

Configuration entry points

These kernel (dynamically loading) routines
enable the kernel to find the driver binary in the file system and load it into, or
unload it from, the running kernel. The entry points include _init(9E), _info(9E),
and _fini(9E).

Contained in the streamtab, read and process the STREAMS messages that travel through the queue structures.
Examples of STREAMS queue processing entry points are put(9E) and srv(9E).

Interrupt routines

Handle the interrupts from the device (or software interrupts).
It is added to the kernel by ddi_add_intr(9F) when
the kernel configuration software calls attach(9E).

STREAMS Configuration Entry Points

As with other SunOS
5 drivers, STREAMS drivers are dynamically linked and loaded when referred to for
the first time. For example, when the system is initially booted, the STREAMS pseudo-tty
slave pseudo-driver (pts(7D))
is loaded automatically into the kernel when it is first opened.

In STREAMS, the header declarations differ between drivers and modules. The
word “module” is used in two different ways when talking about drivers.
There are STREAMS modules, which are pushable nondriver entities, and there are kernel-loadable
modules, which are components of the kernel. See the appropriate chapters in Writing Device Drivers.

The kernel configuration mechanism must distinguish between STREAMS devices
and traditional character devices because system calls to STREAMS drivers are processed
by STREAMS routines, not by the system driver routines. The streamtab pointer
in the cb_ops(9S) structure provides
this distinction. If it is NULL, there are no STREAMS routines to execute; otherwise,
STREAMS drivers initialize streamtab with a pointer to a streamtab(9S) structure containing
the driver's STREAMS queue processing entry points.

STREAMS Initialization Entry Points

The initialization entry points of
STREAMS drivers must perform the same tasks as those of non-STREAMS drivers. See Writing Device Drivers for
more information.

STREAMS Table-Driven Entry Points

In non-STREAMS drivers, most of the driver's work is accomplished through the
entry points in the cb_ops(9S) structure.
For STREAMS drivers, most of the work is accomplished through the message-based STREAMS
queue processing entry points.

Figure 9–1 shows multiple streams (corresponding to minor devices) connecting to a common
driver. There are two distinct streams opened from the same major device. Consequently,
they have the same streamtab and the same driver procedures.

Figure 9–1 Device Driver Streams

Multiple instances (minor devices) of the same driver are handled during the
initial open for each device. Typically, a driver stores the queue
address in a driver-private structure that is uniquely identified by the minor device
number. (The DDI/DKI provides a mechanism for uniform handling of driver-private structures;
see ddi_soft_state(9F)). The q_ptr of the queue points to the private data structure entry. When the
messages are received by the queue, the calls to the driver put and service procedures pass the address of the queue, enabling the procedures
to determine the associated device through the q_ptr field.

STREAMS guarantees that only one open or close can
be active at a time per major/minor device pair.

STREAMS Queue Processing Entry Points

STREAMS device drivers have processing
routines that are registered with the framework through the streamtab structure.
The put procedure is a driver's entry point, but it is a message
(not system) interface. STREAMS drivers and STREAMS modules implement these entry
points similarly, as described in Entry Points.

The stream head translates write(2) and ioctl(2) calls into messages and sends them
downstream to be processed by the driver's write queue put(9E) procedure. read is seen directly only by
the stream head, which contains the functions required to process system calls. A
STREAMS driver does not check system interfaces other than open and close, but it can detect the absence of a read indirectly
if flow control propagates from the stream head to the driver and affects the driver's
ability to send messages upstream.

For read-side processing, when the driver is ready to send data or other information
to a user process, it prepares a message and sends it upstream to the read queue of
the appropriate (minor device) stream. The driver's open routine generally stores
the queue address corresponding to this stream.

For write-side (or output) processing, the driver receives messages in place
of a write call. If the message cannot be sent immediately to the hardware, it may
be stored on the driver's write message queue. Subsequent output interrupts can remove
messages from this queue.

A driver is at the end of a stream. As a result, drivers must include standard
processing for certain message types that a module might be able to pass to the next
component. For example, a driver must process all M_IOCTL messages;
otherwise, the stream head blocks for an M_IOCNAK, M_IOCACK, or until the timeout (potentially infinite) expires. If a driver does
not understand an ioctl(2), an M_IOCNAK message is sent upstream.

Messages that are not understood by the drivers should be freed.

The stream head locks up the stream when it receives an M_ERROR message,
so driver developers should be careful when using the M_ERROR message.

STREAMS Interrupt Handlers

Most hardware drivers have an interrupt handler routine. You
must supply an interrupt routine for the device's driver. The interrupt handling for
STREAMS drivers is not fundamentally different from that for other device drivers.
Drivers usually register interrupt handlers in their attach(9E)entry
point, using ddi_add_intr(9F).
Drivers unregister the interrupt handler at detach time using ddi_remove_intr(9F).

The system also supports software interrupts. The routines ddi_add_softintr(9F) and ddi_remove_softintr(9F) register and unregister (respectively) soft-interrupt handlers. A software
interrupt is generated by calling ddi_trigger_softintr(9F).

Driver Unloading

STREAMS Driver Code Samples

The following discussion describes characteristics of a STREAMS driver:

Basic hardware/pseudo drivers

This type of driver communicates
with a specific piece of hardware (or simulated hardware). The lp example
simulates a simple printer driver.

Clonable drivers

The STREAMS framework supports a CLONEOPEN facility, which allows multiple streams to be opened from a single
special file. If a STREAMS device driver chooses to support CLONEOPEN,
it can be referred to as a clonable device. The attach(9E) routines
from two Solaris drivers, ptm(7D) and log(7D), illustrate two approaches to cloning.

Multiple instances in drivers

A multiplexer driver is
a regular STREAMS driver that can handle multiple streams connected to it instead
of just one stream. Multiple connections occur when more than one minor device of
the same driver is in use. See Cloning STREAMS Drivers for more information.

Printer Driver Example

Example 9–1 is a sample print
driver for an interrupt-per-character line printer. The driver is unidirectional—it
has no read-side processing. It demonstrates some differences between module and driver
programming, including the following:

Declarations for driver configuration

Open handling

A driver is passed a device number

Flush handling

A driver must loop M_FLUSH messages back upstream

Interrupt routine

A driver registers interrupt handler and processes interrupts

Most of the STREAMS processing in the driver is independent of the actual printer
hardware; in this example, actual interaction with the printer is limited to the lpoutchar function, which prints one character at a time. For purposes of
demonstration, the “printer hardware” is actually the system console,
accessed through cmn_err(9F).
Since there's no actual hardware to generate a genuine hardware interrupt, lpoutchar simulates interrupts using ddi_trigger_softintr(9F).
For a real printer, the lpoutchar function is rewritten to send
a character to the printer, which should generate a hardware interrupt.

The driver declarations follow. After specifying header files
(include <sys/ddi.h> and <sys/sunddi.h>
as the last two header files), the driver declares a per-printer structure, struct lp. This structure contains members that enable the driver to keep
track of each instance of the driver, such as flags (what the driver
is doing), msg (the current STREAMS print message), qptr (pointer to the stream's write queue), dip (the instance's
device information handle), iblock_cookie (for registering an interrupt
handler), siid (the handle of the soft interrupt), and lp_lock (a mutex to protect the data structure from multithreaded race conditions).
The driver next defines the bits for the flags member of struct lp; the driver defines only one flag, BUSY.

Following function prototypes, the driver provides some standard STREAMS declarations:
a module_info(9S) structure
(minfo), a qinit(9S) structure
for the read side (rinit) that is initialized by the driver's open and close entry points, a qinit(9S) structure for the write side (winit) that is initialized by the write put procedure, and
a streamtab(9S) that points
to rinit and winit. The values in the module
name and ID fields in the module_info(9S) structure
must be unique in the system. Because the driver is unidirectional, there is no read
side put or service procedure. The flow control
limits for use on the write side are 50 bytes for the low-watermark and 150 bytes
for the high-watermark.

The driver next declares lp_state. This is an anchor on which
the various “soft-state” functions provided by the DDK operate.
The ddi_soft_state(9F) manual
page describes how to maintain multiple instances of a driver.

The driver next declares acb_ops(9S)
structure, which is required in all device drivers. In non-STREAMS device drivers, cb_ops(9S) contains vectors to the table-driven
entry points. For STREAMS drivers, however, cb_ops(9S) contains
mostly nodev entries. The cb_stream field, however,
is initialized with a pointer to the driver's streamtab(9S) structure.
This indicates to the kernel that this driver is a STREAMS driver.

Next, the driver declares a dev_ops(9S) structure,
which points to the various initialization entry points as well as to the cb_ops(9S) structure. Finally, the driver declares a struct moldrv and a struct modlinkage for use by the
kernel linker when the driver is dynamically loaded. struct moldrv contains
a pointer to mod_driverops (a significant difference between a
STREAMS driver and a STREAMS module—a STREAMS module would contain a pointer
to mod_strops instead).

Example 9–3 shows the lp driver's implementation of the initialization entry points. In lpidentify, the driver ensures that the name of the device being attached
is “lp”.

lpattach first uses ddi_soft_state_zalloc(9F) to
allocate a per-instance structure for the printer being attached. Next it creates
a node in the device tree for the printer using ddi_create_minor_node(9F);
user programs use the node to access the device. lpattach then
registers the driver interrupt handler because the sample is driver pseudo-hardware,
the driver uses soft interrupts. A driver for a real printer would use ddi_add_intr(9F) instead of ddi_add_softintr(9F). A driver for a real printer would
also need to perform any other required hardware initialization in lpattach. Finally, lpattach initializes the per-instance mutex.

In lpdetach, the driver undoes everything it did in lpattach.

lpgetinfo uses the soft-state structures to obtain the required
information.

The STREAMS mechanism allows only one stream per minor device. The driver open routine is called whenever a STREAMS device is opened. open matches the correct private data structure with the stream using ddi_get_soft_state(9F).
The driver open, lpopen in Example 9–4, has the same interface as the module open.

The stream flag, sflag, must have the value 0, indicating
a normal driver open. devp pointers to the major/minor device number
for the port. After checking sflag, lpopen uses devp to find the correct soft-state structure.

The next check, if (q->q_ptr)..., determines if the printer
is already open. q_ptr is a driver or module private data pointer.
It can be used by the driver for any purpose and is initialized to zero by STREAMS
before the first open. In this example, the driver sets the value
of q_ptr, in both the read and write queue structures, to point to the device's per-instance data structure.
If the pointer is non-NULL, it means the printer is already open, so lpopen returns EBUSY to avoid merging printouts from multiple
users.

The driver close routine is called by the stream head. Any
messages left in the queue are automatically removed by STREAMS. The stream is dismantled
and data structures are released.

There are no physical pointers between the read and write queue of a pair. WR(9F) is a queue pointer function. WR(9F) generates the write pointer from
the read pointer. RD(9F) and otherq(9F) are additional queue pointer functions. RD(9F) generates the read pointer from the
write pointer, and otherq(9F) generates
the mate pointer from either.

Driver Flush Handling

If FLUSHW is set, the write message queue is flushed, and
(in this example) the leading message (lp->msg) is also flushed. lp_lock protects the driver's per-instance data structure.

In most drivers, if FLUSHR is set, the read queue is flushed.
However, in this example, no messages are ever placed on the read queue, so flushing
it is not necessary. The FLUSHW bit is cleared and the message
is sent upstream using qreply(9F).
If FLUSHR is not set, the message is discarded.

The stream head always performs the following actions on flush requests received
on the read side from downstream. If FLUSHR is set, messages waiting
to be sent to user space are flushed. If FLUSHW is set, the stream
head clears the FLUSHR bit and sends the M_FLUSH message
downstream. In this manner, a single M_FLUSH message sent from
the driver can reach all queues in a stream. A module must send two M_FLUSH messages to have the same effect.

lpwput queues M_DATA and M_IOCTL messages and if the device is not busy, starts output by calling lpout. Message types that are not recognized are discarded (in the default
case of the switch).

Print Driver Interrupt

lpintr is the driver-interrupt handler registered by the attach routine.

lpout takes a single character from the queue and sends it
to the printer. For convenience, the message currently being output is stored in lp->msg in the per-instance structure. This assumes that the message is
called with the mutex held.

lpoutchar sends a single character to the printer (in this
case the system console using cmn_err(9F))
and interrupts when complete. Of course, hardware would generate a hard interrupt,
so the call to ddi_trigger_softintr(9F) would
be unnecessary.

Driver Flow Control

When the message is queued, putq(9F) increments
the value of q_count by the size of the message and compares the
result to the driver's write high-watermark (q_hiwat) value. If
the count reaches q_hiwat, putq(9F) sets
the internal FULL indicator for the driver write queue. This causes
messages from upstream to be halted (canputnext(9F) returns
FALSE) until the write queue count drops below q_lowat. The driver
messages waiting to be output through lpout are dequeued by the
driver output interrupt routine with getq(9F),
which decrements the count. If the resulting count is below q_lowat, getq(9F) back-enables any upstream queue that
had been blocked.

For priority band data, qb_count, qb_hiwat,
and qb_lowat are used.

STREAMS with flow control can be used on the driver read side to handle temporary
upstream blocks.

To some extent, a driver or a module can control when its upstream transmission
becomes blocked. Control is available through the M_SETOPTS message
(see Appendix A, Message Types) to
modify the stream head read-side flow control limits.

Cloning STREAMS Drivers

To eliminate polling, STREAMS drivers can be made clonable. If a STREAMS
driver is implemented as a clonable device, a single node in the file system can be
opened to access any unused device that the driver controls. This special node guarantees
that each user is allocated a separate stream to the driver for each open call. Each stream is associated with an unused minor device, so the total
number of streams that may be connected to a particular clonable driver is limited
only by the number of minor devices configured for that driver.

In previous examples, each user process connected a stream to a driver by explicitly
opening a particular minor device of the driver. Each minor device had its own node
in the device tree file system. Often, there is a need for a user process to connect
a new stream to a driver regardless of which minor device is used to access the driver.
In the past, this forced the user process to poll the various minor device nodes of
the driver for an available minor device.

The clone model is useful, for example, in a networking environment where a
protocol pseudo-device driver requires each user to open a separate stream over which
it establishes communication. (The decision to implement a STREAMS driver as a clonable
device is made by the designers of the device driver. Knowledge of the clone driver
implementation is not required to use it.)

There are two ways to open as a clone device. The first is to use the STREAMS
framework-provided clone device, which arranges to open the device with the CLONEOPEN flag passed in. This method is demonstrated in Example 9–7, which shows the attach and open routines
for the pseudo-terminal master ptm(7D) driver.
The second way is to have the driver open itself as a clone device, without intervention
from the system clone device. This method is demonstrated in the attach and open routines for the log(7D) device
in Example 9–8.

The ptm(7D) device, which
uses the system-provided clone device, sets up two nodes in the device file system.
One has a major number of 23 (ptm's assigned major number) and
a minor number of 0. The other node has a major number of 11 (the clone device's assigned
major number) and a minor number of 23 (ptm's assigned major number).
The driver's attach routine (see Example 9–7) calls to ddi_create_minor_node(9F) twice.
First, to set up the “normal” node (major number 23); second, to specify CLONE_DEV as the last parameter, making the system create the node with
major 11.

When the special file /devices/pseudo/clone@0:ptmx is opened,
the clone driver code in the kernel (accessed by major 11) passes the CLONEOPEN flag in the sflag parameter to the ptm(7D)open routine. ptm's open routine checks sflag to make sure it is being called
by the clone driver. The open routine next attempts to find an
unused minor device for the open by searching its table of minor
devices. (PT_ENTER_WRITE and PT_EXIT_WRITE are
driver-defined macros for entering and exiting the driver's mutex.) If it succeeds
(and following other open processing), the open routine
constructs a new dev_t with the new minor number, which it passes
back to its caller in the devp parameter. (The new minor number
is available to the user program that opened the clonable device through an fstat(2) call.)

The log(7D) driver uses the
second method; it clones itself without intervention from the system clone device.
The log(7D) driver's attach routine (in Example 9–8)
is similar to the one in ptm(7D).
It creates two nodes using ddi_create_minor_node(9F),
but neither specifies CLONE_DEV as the last parameter. Instead,
one of the devices has minor 0, the other minor CLONEMIN. These
two devices provide log(7D) two interfaces:
the first write-only, the second read-write (see the man page log(7D) for more information). Users open one node or the other. If
they open the CONSWMIN (clonable, read-write) node, the open routine checks its table of minor devices for an unused device. If
it is successful, it (like the ptm(7D)open routine) returns the new dev_t to its caller in devp.

Loop-Around Driver

The loop-around driver is a pseudo-driver that
loops data from one open stream to another open stream. The associated
files are almost like a full-duplex pipe to user processes. The streams are not physically
linked. The driver is a simple multiplexer that passes messages from one stream's
write queue to the other stream's read queue.

To create a connection, a process opens two streams, obtains the minor device
number associated with one of the returned file descriptors, and sends the device
number in an ioctl(2) to the other
stream. For each open, the driver open places the passed queue
pointer in a driver interconnection table, indexed by the device number. When the
driver later receives an M_IOCTL message, it uses the device number
to locate the other stream's interconnection table entry, and stores the appropriate
queue pointers in both of the streams' interconnection table entries.

Subsequently, when messages other than M_IOCTL or M_FLUSH are received by the driver on either stream's write side, the messages
are switched to the read queue following the driver on the other stream's read side.
The resultant logical connection is shown in Figure 9–2. Flow control between the two streams must be handled explicitly,
since STREAMS do not automatically propagate flow control information between two
streams that are not physically connected.

Figure 9–2 Loop-Around Streams

Example 9–9 shows the loop-around
driver code. The loop structure contains the interconnection information
for a pair of streams. loop_loop is indexed by the minor device
number. When a stream is opened to the driver, the driver places the address of the
corresponding loop_loop element in the q_ptr (private
data structure pointer) of the read-side and write-side queues. Since STREAMS clears q_ptr when the queue is allocated, a NULL value of q_ptr indicates
an initial open. loop_loop verifies that this stream is connected
to another open stream.

The open procedure (in Example 9–11) includes canonical clone processing that enables a single
file system node to yield a new minor device/vnode each time the driver is opened.
In loopopen, sflag can be CLONEOPEN, indicating that the driver picks an unused minor device. In this case,
the driver scans its private loop_loop data structure to find an
unused minor device number. If sflag is not set to CLONEOPEN, the passed-in minor device specified by getminor(*devp) is
used.

Because the messages are switched to the read queue following the other stream's
read side, the driver needs a put procedure only on its write side. loopwput (in Example 9–12) shows another use of an ioctl(2).
The driver supports the ioc_cmd value LOOP_SET in
the iocblk(9S) of the M_IOCTL message. LOOP_SET makes the driver connect the
current open stream to the stream indicated in the message. The
second block of the M_IOCTL message holds an integer that specifies
the minor device number of the stream to which to connect.

If these checks pass, the read queue pointers for the two streams are stored
in the respective oqptr fields. This cross-connects the two streams
indirectly, through loop_loop.

The put procedure incorporates canonical flush handling.

loopwput queues
all other messages (for example, M_DATA or M_PROTO)
for processing by its service procedure. A check is made that the
stream is connected. If not, M_ERROR is sent to the stream head.
Certain message types can be sent upstream by drivers and modules to the stream head
where they are translated into actions detectable by user processes. These messages
may also modify the state of the stream head:

M_ERROR

Causes the stream head to lock up. Message transmission between stream
and user processes is terminated. All subsequent system calls except close(2) and poll(2) fail.
Also causes M_FLUSH, clearing all message queues, to be sent downstream
by the stream head.

M_HANGUP

Terminates input from a user process to the stream. All subsequent
system calls that would send messages downstream fail. Once the stream head read message
queue is empty, EOF is returned on reads. This can also result
in SIGHUP being sent to the process group's session leader.

M_SIG/M_PCSIG

Causes a specified signal to be sent to the process group associated
with the stream.

Service procedures are required in this example on both the
write side and read side for flow control (see Example 9–13). The write service procedure, loopwsrv, takes on the canonical form. The queue being written to is not downstream,
but upstream (found through oqptr) on the other stream.

In this case, there is no read-side put procedure so the
read service procedure, looprsrv, is not scheduled
by an associated put procedure, as has been done previously. looprsrv is scheduled only by being back-enabled when its upstream flow
control blockage is released. The purpose of the procedure is to re-enable the writer
(loopwsrv) by using oqptr to find the related
queue. loopwsrv cannot be directly back-enabled by STREAMS because
there is no direct queue linkage between the two streams. Note that no message is
queued to the read service procedure. Messages are kept on the
write side so that flow control can propagate up to the stream head. qenable(9F) schedules the write-side service procedure
of the other stream.

An application using this driver would first open the clone device node created
in the attach routine (/devices/pseudo/clone@0:loopx) two times
to obtain two streams. The application can determine the minor numbers of the devices
by using fstat(2). Next, it joins the two streams by using the
STREAMS I_STRioctl(2) (see streamio(7I)) to pass the LOOP_SETioctl(2) with one of the
stream's minor numbers as an argument to the other stream. Once this is completed,
the data sent to one stream using write(2) or putmsg(2) can be retrieved from the other stream
with read(2) or getmsg(2). The application also can interpose STREAMS modules
between the stream heads and the driver using the I_PUSHioctl(2).

Summarizing STREAMS Device Drivers

STREAMS device drivers are in many ways similar to non-STREAMS device drivers.
The following points summarize the differences between STREAMS drivers and other drivers:

Drivers must have attach(9E) and probe(9E) entry points to initialize the driver.
The attach routine initializes the driver. Software drivers usually have little to
initialize, because there is no hardware involved.

Most drivers have an interrupt handler routine. The driver developer
is responsible for supplying an interrupt routine for the device's driver. In addition
to hardware interrupts, the system also supports software interrupts. A software interrupt
is generated by calling ddi_trigger_softintr(9F).