Port2Serialconsole

Some notes on using the second serial port as a node console.

You might need to do this if you have a machine which has some sort of
System Management interface that uses the first serial port. In theory,
we should be able to co-exist with it, but we have had no luck thus far.

Anyway, there are several levels you need to change:

1. pxeboot:
This is our all-in-one PXE-load bootstrap program. It is built from the
FreeBSD boot code sources in /share/freebsd/4.7/src/sys/boot. Make a local
copy of the /share/freebsd/4.7/src tree as /usr/src on some node, and then
edit i386/Makefile.inc and add (or uncomment) the line:
CFLAGS += -DCOMPORT=0x2f8
This assumes that your second serial port is in the "usual" place. If
not, you may have to go into the BIOS (or just look at FreeBSD/Linux
boot time messages) and find what address it is at.
Now do "make" in the boot directory. Go to the i386/emuboot
(or /usr/obj/usr/src/sys/boot/i386/emuboot if using a separate build tree)
and copy "pxeboot" to your boss node. If all your machines are using
serial port2, then you can just install it as /tftpboot/pxeboot.emu.
If only some nodes have it, well...we have no infrastructure for that
right now. Install the file as /tftpboot/pxeboot-sio2.emu and hand edit
/usr/local/etc/dhcpd.conf, adding a "filename" option for all nodes
which require the different pxeboot.
2. Linux:
A. Change /etc/lilo.conf and run /etc/lilo. To change lilo.conf, at the
beginning change:
serial=0,115200n8
to:
serial=1,115200n8
and for every kernel entry change the command line additions from:
append="console=ttyS0,115200"
to:
append="console=ttyS1,115200"
B. Change /etc/inittab. Change the line:
S0:2345:respawn:/sbin/agetty -L 115200 ttyS0 vt100
to:
S1:2345:respawn:/sbin/agetty -L 115200 ttyS1 vt100
C. Change /etc/securetty. Add ttyS1 at the end.
Otherwise you cannot login as root on the console.
3. FreeBSD:
Oh man, was this a pain in the ass! Who wrote that BSD console support
code anyway? Hmm...it has a "University of Utah" copyright...doh!
Never mind! Anyway, you need to build a new boot load and new kernels
for this:
A. For the boot loader, go back to step 1 where you built pxeboot. If you
did a make for the whole boot tree, you should also have recreated a boot
loader that will work for you. Go to i386/loader (or /usr/obj/..blahblah..)
and copy loader over to /boot.
B. To build a kernel, go to your local source tree in /usr/src/sys/i386/conf
and modify the TESTBED, TESTBED-DELAY, TESTBED-JAIL, and TESTBED-LINKDELAY
(yeah, yeah, we need to reduce this to a single kernel...). Change:
device sio0 at isa? port IO_COM1 flags 0x10 irq 4
device sio1 at isa? port IO_COM2 irq 3
to:
device sio0 at isa? port IO_COM1 irq 4
device sio1 at isa? port IO_COM2 flags 0x10 irq 3
i.e., move the "flags 0x10" option from sio0 to sio1. Now rebuild your
kernels either the old fashioned way:
config TESTBED
cd ../../compile/TESTBED
make depend all
or the new fangled way:
cd /usr/src
make buildkernel KERNCONF=TESTBED
Build all four kernel configs. They are installed as follows:
TESTBED/kernel -> /kernel.100HZ (hard link to /kernel)
TESTBED-LINKDELAY/kernel-> /kernel.1000HZ (link to /kernel.linkdelay)
TESTBED-DELAY/kernel -> /kernel.10000HZ (link to /kernel.delay)
TESTBED-JAIL/kernel -> /kernel.jail
getty should work just fine as it uses "/dev/console" which is magically
redirected to the correct serial device (lets hear it for the guys who
wrote the BSD console code!)
4. Now remake your BSD and Linux images.
5. Think you are done? Guess again. Now you need to modify the BSD "newnode",
"frisbee" and "freebsd" MFS kernels as well. Go back to your local kernel
source tree and modify the configs: GENERIC, TESTBED-PXEBOOT, and
TESTBED-PXEBOOT-EXT2FS (yeah, yeah, just shut yer yap and go do it!) as
you did before. Then build the kernels and install them on the boss node:
GENERIC/kernel -> /tftpboot/freebsd.newnode/boot/kernel
TESTBED-PXEBOOT/kernel -> /tftpboot/frisbee/boot/kernel
TESTBED-PXEBOOT-EXT2FS/kernel -> /tftpboot/freebsd/boot/kernel
Then go into each of those target directories (/tftpboot/*/boot) and
run "./prepare". That script just gzips the kernel so it loads faster.
Now, be thankful that you at least didn't have to mount the MFS itself!
Again, you are big time screwed if only some of your nodes should be
using serial port 2 as the console. If so, then you are going to have
install the kernels under a different name and modify the emulab
infrastructure (bootinfo) to tell pxeboot which MFS to load for the
different nodes.