Oracle Blog

Random thoughts of a disorganized mind...

Sunday Nov 11, 2012

While Solaris 11.1 was under development, we started seeing some errors in
the builds of the upstream X.Org git master sources, such as:

"Display.c", line 65: Function has no return statement : x_io_error_handler
"hostx.c", line 341: Function has no return statement : x_io_error_handler

from functions that were defined to match a specific callback definition
that declared them as returning an int if they did return, but these were
calling exit() instead of returning so hadn't listed a return value.

These had been generating warnings for years which we'd been ignoring, but
X.Org has made enough progress in cleaning up code for compiler warnings
and static analysis issues lately, that the community turned up the default
error levels, including the gcc flag -Werror=return-type and the
equivalent Solaris Studio cc flags -v -errwarn=E_FUNC_HAS_NO_RETURN_STMT,
so now these became errors that stopped the build.
Yet on Solaris, gcc built this code fine, while Studio errored out.
Investigation showed this was due to the Solaris headers, which during
Solaris 10 development added a number of annotations to the headers when
gcc was being used for the amd64 kernel bringup before the Studio amd64
port was ready. Since Studio did not support the inline form of these
annotations at the time, but instead used #pragma for them,
the definitions were only present for gcc.

To resolve this, I fixed both sides of the problem, so that it would work
for building new X.Org sources on older Solaris releases or with older
Studio compilers, as well as fixing the general problem before it broke
more software building on Solaris.

To the X.Org sources, I added the traditional Studio
#pragma does_not_return to recognize that functions like exit()
don't ever return, in patches such as this Xserver patch. Adding a dummy return statement was
ruled out as that introduced unreachable code errors from compilers and
analyzers that correctly realized you couldn't reach that code after a
return statement.

And on the Solaris 11.1 side, I updated the annotation definitions in
<sys/ccompile.h> to enable for Studio 12.0 and later compilers
the annotations already existing in a number of system headers for functions
like exit() and abort(). If you look in that file you'll
see the annotations we currently use, though the forms there haven't gone
through review to become a Committed interface, so may change in the future.

Actually getting this integrated into Solaris though took a bit more work
than just editing one header file. Our ELF binary build comparison tool,
wsdiff,
actually showed a large number of differences in the resulting binaries due
to the compiler using this information for branch prediction, code path
analysis, and other possible optimizations, so after comparing enough of the
disassembly output to be comfortable with the changes, we also made sure to
get this in early enough in the release cycle so that it would get plenty of
test exposure before the release.

It also required updating quite a bit of code to avoid introducing new lint
or compiler warnings or errors, and people building applications on top of
Solaris 11.1 and later may need to make similar changes if they want to keep
their build logs similarly clean.

Previously, if you had a function that was declared with a non-void return
type, lint and cc would warn if you didn't return a value, even if you called
a function like exit() or panic() that ended execution.
For instance:

would previously require a never executed return 0; after the
exit() to avoid lint warning "function falls off bottom without
returning value".

Now the compiler & lint will both issue "statement not reached"
warnings for a return 0; after the final exit(),
allowing (or in some cases, requiring) it to be removed. However, if
there is no return statement anywhere in the function, lint will warn
that you've declared a function returning a value that never does so,
suggesting you can declare it as void. Unfortunately, if your
function signature is required to match a certain form, such as in a
callback, you not be able to do so, and will need to add a
/* LINTED */ to the end of the function.

If you need your code to build on both a newer and an older release, then you
will either need to #ifdef these unreachable statements, or, to keep
your sources common across releases, add to your sources the corresponding
#pragma recognized by both current and older compiler versions, such as:

#pragma does_not_return(exit)
#pragma does_not_return(panic)

Hopefully this little extra work is paid for by the compilers & code
analyzers being able to better understand your code paths, giving you
better optimizations and more accurate errors & warning messages.

When you’re ready to upgrade to the packages from either this repo,
or the support repository, you’ll want to first read
How to Update to Oracle Solaris 11.1 Using the Image Packaging System by Pete Dennis,
as there are a couple issues you will need to be aware of to do that upgrade,
several of which are due to changes in the Free and Open Source Software (FOSS)
packages included with Solaris, as I’ll explain in a bit.

Solaris 11 can update more readily than Solaris 10

In the Solaris 10 and older update models, the way the updates were built
constrained what changes we could make in those releases. To change an
existing SVR4 package in those releases, we created a Solaris Patch, which
applied to a given version of the SVR4 package and replaced, added or deleted
files in it. These patches were released via the support websites (originally
SunSolve, now My Oracle Support)
for applying to existing Solaris 10 installations, and were also merged into
the install images for the next Solaris 10 update release.
(This Solaris
Patches blog post from Gerry Haskins dives deeper into that subject.)

Some of the restrictions of this model were that package refactoring, changes
to package dependencies, and even just changing the package version number,
were difficult to do in this hybrid patch/OS update model. For instance,
when Solaris 10 first shipped, it had the Xorg server from X11R6.8. Over the
first couple years of update releases we were able to keep it up to date by
replacing, adding, & removing files as necessary, taking it all the way
up to Xorg server release 1.3 (new version numbering begun after
the X11R7 split of the X11 tree into separate modules gave each module its
own version).
But if you run pkginfo on the SUNWxorg-server package, you’ll
see it still displayed a version number of 6.8, confusing users as to which
version was actually included.

We stopped upgrading the Xorg server releases in Solaris 10 after 1.3, as
later versions added new dependencies, such as
HAL,
D-Bus,
and libpciaccess,
which were very difficult to manage in this patching model. (We later got
libpciaccess to work, but HAL & D-Bus would have been much harder due to
the greater dependency tree underneath those.) Similarly, every time the
GNOME team looked into upgrading Solaris 10 past GNOME 2.6, they found these
constraints made it so difficult it wasn’t worthwhile, and eventually
GNOME’s dependencies had changed enough it was completely infeasible.
Fortunately, this worked out for both the X11 & GNOME teams, with our
management making the business decision to concentrate on the
“Nevada” branch for desktop users - first as Solaris Express
Desktop Edition, and later as OpenSolaris, so we didn’t have to fight
to try to make the package updates fit into these tight constraints.

Meanwhile, the team designing the new packaging system for Solaris 11 was
seeing us struggle with these problems, and making this much easier to manage
for both the development teams and our users was one of their big goals for
the IPS design they were working on.
Now that we’ve reached the first update release to Solaris 11, we can
start to see the fruits of their labors, with more FOSS updates in 11.1 than
we had in many Solaris 10 update releases, keeping software more up to date
with the upstream communities.

Of course, just because we can more easily update now, doesn’t always
mean we should or will do so, it just removes the package system limitations
from forcing the decision for us. So while we’ve upgraded the X Window
System in the 11.1 release from X11R7.6 to 7.7, the Solaris GNOME team decided
it was not the right time to try to make the jump from GNOME 2 to GNOME 3,
though they did update some individual components of the desktop, especially
those with security fixes like Firefox. In other parts of the system,
decisions as to what to update were prioritized based on how they affected
other projects, or what customer requests we’d gotten for them.

So with all that background in place, what packages did we actually update
or add between Solaris 11.0 and 11.1?

Core OS Functionality

One of the FOSS changes with the biggest impact in this release is the
upgrade from Grub Legacy (0.97) to Grub 2 (1.99) for the x64 platform boot
loader. This is the cause of one of the upgrade quirks, since to go from
Solaris 11.0 to 11.1 on x64 systems, you first need to update the Boot
Environment tools (such as beadm) to a new version that can handle boot
environments that use the Grub2 boot loader. System administrators can
find the details they need to know about the new Grub in the
Administering the GRand Unified Bootloader chapter of the
Booting and Shutting Down Oracle Solaris 11.1 Systems
guide. This change was necessary to be able to support new hardware coming
into the x64 marketplace, including systems using UEFI firmware or booting off
disk drives larger than 2 terabytes.

For both platforms, Solaris 11.1 adds
rsyslog as an optional alternative to
the traditional syslogd, and OpenSCAP
for checking security configuration settings are compliant with site policies.

Note that the support repo actually has newer versions of BIND & fetchmail
than the 11.1 release, as some late breaking critical fixes came through from
the community upstream releases after the Solaris 11.1 release was frozen, and
made their way to the support repository. These are responsible for the other
big upgrade quirk in this release, in which to upgrade a system which already
installed those versions from the support repo, you need to either wait for
those packages to make their way to the 11.1 branch of the support repo, or
follow the steps in the aforementioned upgrade walkthrough
to let the package system know it's okay to temporarily downgrade those.

Developer Stack

While Solaris 11.0 included Python 2.7, many of the bundled python modules
weren’t packaged for it yet, limiting its usability. For 11.1, many more
of the python modules include 2.7 versions (enough that I filtered them out of
the below table, but you can always search on
the package repository server for them.

For other language runtimes and development tools, 11.1 expands the use of
IPS mediated links
to choose which version of a package is the default when the packages are
designed to allow multiple versions to install side by side.

For instance, in Solaris 11.0, GNU automake 1.9 and 1.10 were provided, and
developers had to run them as either automake-1.9 or
automake-1.10. In Solaris 11.1, when automake 1.11 was added, also
added was a /usr/bin/automake mediated link, which points to the
automake-1.11 program by default, but can be changed to another
version by running the pkg set-mediator command.

Mediated links were also used for the Java runtime & development kits in
11.1, changing the default versions to the Java 7 releases (the 1.7.0.x
package versions), while allowing admins to
switch
links such as /usr/bin/javac back to Java 6
if they need to for their site, to deal with
Java 7 compatibility
or other issues, without having to update each usage to use the full versioned
/usr/jdk/jdk1.6.0_35/bin/javac paths for every invocation.

Desktop Stack

As I mentioned before, we upgraded from X11R7.6 to X11R7.7, since a pleasant
coincidence made the X.Org release dates line up nicely with our feature &
code freeze dates for this release. (Or perhaps it wasn’t so
coincidental, after all, one of the benefits of being
the person making the release
is being able to decide what schedule is most convenient for you, and this one
worked well for me.) For the table below, I’ve skipped listing the
packages in which we use the X11 “katamari” version for the Solaris
package version (mainly packages combining elements of multiple upstream
modules with independent version numbers), since they just all changed from
7.6 to 7.7.

In the graphics drivers, we worked with Intel to update the Intel Integrated
Graphics Processor support to support 3D graphics and kernel mode setting on
the Ivy Bridge chipsets, and updated Nvidia’s non-FOSS graphics driver
from 280.13 to 295.20.

Higher up in the desktop stack,
PulseAudio was added for audio support,
and liblouis for Braille support,
and the GNOME applications were built to use them.

The Mozilla applications, Firefox & Thunderbird moved to the current
Extended Support
Release (ESR) versions, 10.x for each, to bring up-to-date security fixes
without having to be on Mozilla’s agressive 6 week feature cycle release
train.

Detailed list of changes

This table shows most of the changes to the FOSS packages between Solaris 11.0
and 11.1. As noted above, some were excluded for clarity, or to reduce noise
and duplication. All the FOSS packages which didn't change the version number
in their packaging info are not included, even if they had updates to fix bugs,
security holes, or add support for new hardware or new features of Solaris.

Package

11.0

11.1

archiver/unrar

3.8.5

4.1.4

audio/sox

14.3.0

14.3.2

backup/rdiff-backup

1.2.1

1.3.3

communication/im/pidgin

2.10.0

2.10.5

compress/gzip

1.3.5

1.4

compress/xz

not included

5.0.1

database/sqlite-3

3.7.6.3

3.7.11

desktop/remote-desktop/tigervnc

1.0.90

1.1.0

desktop/window-manager/xcompmgr

1.1.5

1.1.6

desktop/xscreensaver

5.12

5.15

developer/build/autoconf

2.63

2.68

developer/build/autoconf/xorg-macros

1.15.0

1.17

developer/build/automake-111

not included

1.11.2

developer/build/cmake

2.6.2

2.8.6

developer/build/gnu-make

3.81

3.82

developer/build/imake

1.0.4

1.0.5

developer/build/libtool

1.5.22

2.4.2

developer/build/makedepend

1.0.3

1.0.4

developer/documentation-tool/doxygen

1.5.7.1

1.7.6.1

developer/gnu-binutils

2.19

2.21.1

developer/java/jdepend

not included

2.9

developer/java/jdk-6

1.6.0.26

1.6.0.35

developer/java/jdk-7

1.7.0.0

1.7.0.7

developer/java/jpackage-utils

not included

1.7.5

developer/java/junit

4.5

4.10

developer/lexer/jflex

not included

1.4.1

developer/parser/byaccj

not included

1.14

developer/parser/java_cup

not included

0.10

developer/quilt

0.47

0.60

developer/versioning/git

1.7.3.2

1.7.9.2

developer/versioning/mercurial

1.8.4

2.2.1

developer/versioning/subversion

1.6.16

1.7.5

diagnostic/constype

1.0.3

1.0.4

diagnostic/nmap

5.21

5.51

diagnostic/scanpci

0.12.1

0.13.1

diagnostic/wireshark

1.4.8

1.8.2

diagnostic/xload

1.1.0

1.1.1

editor/gnu-emacs

23.1

23.4

editor/vim

7.3.254

7.3.600

file/lndir

1.0.2

1.0.3

image/editor/bitmap

1.0.5

1.0.6

image/gnuplot

4.4.0

4.6.0

image/library/libexif

0.6.19

0.6.21

image/library/libpng

1.4.8

1.4.11

image/library/librsvg

2.26.3

2.34.1

image/xcursorgen

1.0.4

1.0.5

library/audio/pulseaudio

not included

1.1

library/cacao

2.3.0.0

2.3.1.0

library/expat

2.0.1

2.1.0

library/gc

7.1

7.2

library/graphics/pixman

0.22.0

0.24.4

library/guile

1.8.4

1.8.6

library/java/javadb

10.5.3.0

10.6.2.1

library/java/subversion

1.6.16

1.7.5

library/json-c

not included

0.9

library/libedit

not included

3.0

library/libee

not included

0.3.2

library/libestr

not included

0.1.2

library/libevent

1.3.5

1.4.14.2

library/liblouis

not included

2.1.1

library/liblouisxml

not included

2.1.0

library/libtecla

1.6.0

1.6.1

library/libtool/libltdl

1.5.22

2.4.2

library/nspr

4.8.8

4.8.9

library/openldap

2.4.25

2.4.30

library/pcre

7.8

8.21

library/perl-5/subversion

1.6.16

1.7.5

library/python-2/jsonrpclib

not included

0.1.3

library/python-2/lxml

2.1.2

2.3.3

library/python-2/nose

not included

1.1.2

library/python-2/pyopenssl

not included

0.11

library/python-2/subversion

1.6.16

1.7.5

library/python-2/tkinter-26

2.6.4

2.6.8

library/python-2/tkinter-27

2.7.1

2.7.3

library/security/nss

4.12.10

4.13.1

library/security/openssl

1.0.0.5 (1.0.0e)

1.0.0.10 (1.0.0j)

mail/thunderbird

6.0

10.0.6

network/dns/bind

9.6.3.4.3

9.6.3.7.2

package/pkgbuild

not included

1.3.104

print/filter/enscript

not included

1.6.4

print/filter/gutenprint

5.2.4

5.2.7

print/lp/filter/foomatic-rip

3.0.2

4.0.15

runtime/java/jre-6

1.6.0.26

1.6.0.35

runtime/java/jre-7

1.7.0.0

1.7.0.7

runtime/perl-512

5.12.3

5.12.4

runtime/python-26

2.6.4

2.6.8

runtime/python-27

2.7.1

2.7.3

runtime/ruby-18

1.8.7.334

1.8.7.357

runtime/tcl-8/tcl-sqlite-3

3.7.6.3

3.7.11

security/compliance/openscap

not included

0.8.1

security/nss-utilities

4.12.10

4.13.1

security/sudo

1.8.1.2

1.8.4.5

service/network/dhcp/isc-dhcp

4.1

4.1.0.6

service/network/dns/bind

9.6.3.4.3

9.6.3.7.2

service/network/ftp (ProFTPD)

1.3.3.0.5

1.3.3.0.7

service/network/samba

3.5.10

3.6.6

shell/conflict

0.2004.9.1

0.2010.6.27

shell/pipe-viewer

1.1.4

1.2.0

shell/zsh

4.3.12

4.3.17

system/boot/grub

0.97

1.99

system/font/truetype/liberation

1.4

1.7.2

system/library/freetype-2

2.4.6

2.4.9

system/library/libnet

1.1.2.1

1.1.5

system/management/cim/pegasus

2.9.1

2.11.0

system/management/ipmitool

1.8.10

1.8.11

system/management/wbem/wbemcli

1.3.7

1.3.9.1

system/network/routing/quagga

0.99.8

0.99.19

system/rsyslog

not included

6.2.0

terminal/luit

1.1.0

1.1.1

text/convmv

1.14

1.15

text/gawk

3.1.5

3.1.8

text/gnu-grep

2.5.4

2.10

web/browser/firefox

6.0.2

10.0.6

web/browser/links

1.0

1.0.3

web/java-servlet/tomcat

6.0.33

6.0.35

web/php-53

not included

5.3.14

web/php-53/extension/php-apc

not included

3.1.9

web/php-53/extension/php-idn

not included

0.2.0

web/php-53/extension/php-memcache

not included

3.0.6

web/php-53/extension/php-mysql

not included

5.3.14

web/php-53/extension/php-pear

not included

5.3.14

web/php-53/extension/php-suhosin

not included

0.9.33

web/php-53/extension/php-tcpwrap

not included

1.1.3

web/php-53/extension/php-xdebug

not included

2.2.0

web/php-common

not included

11.1

web/proxy/squid

3.1.8

3.1.18

web/server/apache-22

2.2.20

2.2.22

web/server/apache-22/module/apache-sed

2.2.20

2.2.22

web/server/apache-22/module/apache-wsgi

not included

3.3

x11/diagnostic/xev

1.1.0

1.2.0

x11/diagnostic/xscope

1.3

1.3.1

x11/documentation/xorg-docs

1.6

1.7

x11/keyboard/xkbcomp

1.2.3

1.2.4

x11/library/libdmx

1.1.1

1.1.2

x11/library/libdrm

2.4.25

2.4.32

x11/library/libfontenc

1.1.0

1.1.1

x11/library/libfs

1.0.3

1.0.4

x11/library/libice

1.0.7

1.0.8

x11/library/libsm

1.2.0

1.2.1

x11/library/libx11

1.4.4

1.5.0

x11/library/libxau

1.0.6

1.0.7

x11/library/libxcb

1.7

1.8.1

x11/library/libxcursor

1.1.12

1.1.13

x11/library/libxdmcp

1.1.0

1.1.1

x11/library/libxext

1.3.0

1.3.1

x11/library/libxfixes

4.0.5

5.0

x11/library/libxfont

1.4.4

1.4.5

x11/library/libxft

2.2.0

2.3.1

x11/library/libxi

1.4.3

1.6.1

x11/library/libxinerama

1.1.1

1.1.2

x11/library/libxkbfile

1.0.7

1.0.8

x11/library/libxmu

1.1.0

1.1.1

x11/library/libxmuu

1.1.0

1.1.1

x11/library/libxpm

3.5.9

3.5.10

x11/library/libxrender

0.9.6

0.9.7

x11/library/libxres

1.0.5

1.0.6

x11/library/libxscrnsaver

1.2.1

1.2.2

x11/library/libxtst

1.2.0

1.2.1

x11/library/libxv

1.0.6

1.0.7

x11/library/libxvmc

1.0.6

1.0.7

x11/library/libxxf86vm

1.1.1

1.1.2

x11/library/mesa

7.10.2

7.11.2

x11/library/toolkit/libxaw7

1.0.9

1.0.11

x11/library/toolkit/libxt

1.0.9

1.1.3

x11/library/xtrans

1.2.6

1.2.7

x11/oclock

1.0.2

1.0.3

x11/server/xdmx

1.10.3

1.12.2

x11/server/xephyr

1.10.3

1.12.2

x11/server/xorg

1.10.3

1.12.2

x11/server/xorg/driver/xorg-input-keyboard

1.6.0

1.6.1

x11/server/xorg/driver/xorg-input-mouse

1.7.1

1.7.2

x11/server/xorg/driver/xorg-input-synaptics

1.4.1

1.6.2

x11/server/xorg/driver/xorg-input-vmmouse

12.7.0

12.8.0

x11/server/xorg/driver/xorg-video-ast

0.91.10

0.93.10

x11/server/xorg/driver/xorg-video-ati

6.14.1

6.14.4

x11/server/xorg/driver/xorg-video-cirrus

1.3.2

1.4.0

x11/server/xorg/driver/xorg-video-dummy

0.3.4

0.3.5

x11/server/xorg/driver/xorg-video-intel

2.10.0

2.18.0

x11/server/xorg/driver/xorg-video-mach64

6.9.0

6.9.1

x11/server/xorg/driver/xorg-video-mga

1.4.13

1.5.0

x11/server/xorg/driver/xorg-video-openchrome

0.2.904

0.2.905

x11/server/xorg/driver/xorg-video-r128

6.8.1

6.8.2

x11/server/xorg/driver/xorg-video-trident

1.3.4

1.3.5

x11/server/xorg/driver/xorg-video-vesa

2.3.0

2.3.1

x11/server/xorg/driver/xorg-video-vmware

11.0.3

12.0.2

x11/server/xserver-common

1.10.3

1.12.2

x11/server/xvfb

1.10.3

1.12.2

x11/server/xvnc

1.0.90

1.1.0

x11/session/sessreg

1.0.6

1.0.7

x11/session/xauth

1.0.6

1.0.7

x11/session/xinit

1.3.1

1.3.2

x11/transset

0.9.1

1.0.0

x11/trusted/trusted-xorg

1.10.3

1.12.2

x11/x11-window-dump

1.0.4

1.0.5

x11/xclipboard

1.1.1

1.1.2

x11/xclock

1.0.5

1.0.6

x11/xfd

1.1.0

1.1.1

x11/xfontsel

1.0.3

1.0.4

x11/xfs

1.1.1

1.1.2

P.S. To get the version numbers for this table, I ran a quick perl script over
the output from:

Thursday Oct 25, 2012

One of the first places you can see Solaris 11.1 changes are in the docs,
which have now been posted in
the Solaris 11.1 Library
on docs.oracle.com.
I spent a good deal of time reviewing documentation for this release,
and thought some would be interesting to blog about, but didn't review
all the changes (not by a long shot), and am not going to cover all
the changes here, so there's plenty left for you to discover on your own.

Just comparing the
Solaris 11.1 Library list of
docs against the
Solaris 11 list
will show a lot of reorganization and refactoring of the doc set,
especially in the system administration guides. Hopefully the new break down
will make it easier to get straight to the sections you need when a task is
at hand.

Packaging System

Unfortunately, the excellent in-depth guide for how to build packages for the
new Image Packaging System (IPS) in Solaris 11 wasn't done in time
to make the initial Solaris 11 doc set. An interim version was
published shortly after release, in PDF form on the
OTN IPS page. For Solaris 11.1 it was included
in the doc set, as Packaging and Delivering Software With the Image Packaging System in Oracle Solaris 11.1, so should be easier to find, and
easier to share links to specific pages the HTML version.

Also added in this release is a document containing the
lists of
all the packages in each of the major package groups in Solaris 11.1
(solaris-desktop, solaris-large-server, and solaris-small-server).
While you can simply get the contents of those groups from the package
repository, either via the web interface or the pkg command line, the
documentation puts them in handy tables for easier side-by-side comparison,
or viewing the lists before you've installed the system to pick which one
you want to initially install.

Security

One of the things Oracle likes to do for its products is to publish security
guides for administrators & developers to know how to build systems that
meet their security needs. For Solaris, we started this with Solaris 11,
providing a guide for sysadmins to find where the security relevant
configuration options were documented. The
Solaris 11.1 Security Guidelines
extend this to cover new security features, such as
Address Space Layout Randomization (ASLR) and
Read-Only Zones, as well as adding additional guidelines for existing
features, such as
how to limit the size of tmpfs filesystems, to avoid users
driving the system into swap thrashing situations.

In parallel, we updated the
Solaris C Libary Functions security considerations list with
details of Solaris 11 enhancements such as FD_CLOEXEC flags, additional *at()
functions, and new stdio functions such as
asprintf() and
getline().
A number of code examples throughout the Solaris 11.1 doc set were updated to
follow these recommendations, changing unbounded strcpy() calls to strlcpy(),
sprintf() to snprintf(), etc. so that developers following our examples start
out with safer code. The Writing Device Drivers guide even had the
appendix updated to list which of these utility functions,
like snprintf() and strlcpy(), are now available via the Kernel DDI.

Little Things

Of course all the big new features got documented, and some major efforts
were put into refactoring and renovation, but there were also a lot of smaller
things that got fixed as well in the nearly a year between the Solaris 11
and 11.1 doc releases - again too many to list here, but a random sampling
of the ones I know about & found interesting or useful:

The sample dcmd sources in /usr/demo/mdb were updated to
include ::help output, so that developers like myself who follow the
examples don't forget to include it (until a helpful code reviewer pointed it
out while reviewing the mdb module changes for Xorg 1.12).
The README file in that directory was updated to show the correct paths for
installing both kernel & userspace modules, including the 64-bit variants.