Loom Communications

Emulating The AT&T 3B2 Computer

On November 29, 2014, I posted the following message to the Classic
Computer Mailing List.

Folks,
For some reason I got it in my head that writing an AT&T 3B2
emulator might be a good idea. That idea has pretty much
been derailed by lack of documentation.
I have been unable to find any detailed technical description
of any 3B2 systems. Visual inspection of a 3B2 300 main board
reveals the following major components:
- WE32100 CPU
- WE32101 MMU
- 4 x D2764A EPROM
- TMS2797NL floppy disk controller
- PD7261A hard disk controller
- SCN2681A dual UART
- AM9517A Multimode DMA Controller
How these are addressed is anybody's guess.
To even dream of doing an emulator, I at least need to know
the system memory map -- what physical addresses these devices
map to. Without that it's pretty pointless to get started.
If anyone has access to this kind of information, please drop
me a line. Otherwise, I'll just put this one on the far-back
burner!
-Seth

When I sent that message, I had no idea that it would lead to almost
four years of effort, and perhaps if I had known I would have given
up. But thankfully I didn't, and today I'm happy to say that the
effort was, at long last, successful. The 3B2/400 emulator works well
enough that I have released it to the world and rolled it back into
the parent SIMH source tree.

Because this project required so much reverse engineering, and because
documentation about the 3B2 is still so scarce and hard to come by, I
wanted to take the time to document how the emulator came about.

Background

In the mid 1980s, when big companies were entering the computer market
left and right, AT&T made a bold and (in retrospect) unwise move by
introducing not just one computer, but three.

AT&T entered the PC compatible market with the PC 6300, which was
really just a re-branded Olivetti M24. Simultaneously, they entered
the UNIX market with the AT&T UNIX PC (a personal, single-user UNIX
system based on the 68010 microprocessor), and finally with the 3B2,
which was a small computer designed as a workgroup server. The 3B2
could accommodate up to 50 simultaneous users connected by dumb
terminal or modem, and could even support AT&T's "Blit" bitmapped
terminal. It probably helped that AT&T Bell Labs invented UNIX, so the
company had some history behind it when it brought the 3B2 to market.

Marketing wise, none of these computers did blockbuster sales. The
3B2, though, had decent success due to its small size and modest cost
 just $9,950 entry level.

The 3B2 was special largely because it was so perfectly designed for
running UNIX. Its CPU, the WE 32000 series, was designed by Bell Labs
and Western Electric based on earlier work building a CPU meant to map
to the C programming language. The CPU was ideally suited to UNIX
because it was ideally suited to C. In fact, the 3B2 became the base
porting platform for AT&T UNIX System V Release 3 (SVR3).

When I learned about the 3B2, I quickly became interested in playing
with one. Unfortunately, they're pretty rare to come by
nowadays. Maybe ten or twenty years ago they were fairly common on the
surplus market, but not any more. I went looking for an emulator
instead, but no emulators were to be found, so I did something
regrettable: I decided to write my own.

Emulating the 3B2

The first step in any emulator is gathering up all the source material
and documentation you can find about the system you want to
recreate. This usually means schematics, maintenance documents,
offline diagnostics, design memos, source code, firmware, anything
that shows the internals of the system and how it was
implemented. This is arguably the most important step, because without
this you can't even get started.

Well, bad news: For the 3B2, none of this was available. No
schematics. No internals docs. No offline diagnostics. No
firmware. Nothing. Despite my best effort, Google yielded very little
but high-level user manuals and some floppy diskette images. The only
goldmine I found was a WE32100 microprocessor manual

I immediately sent out a call online to see if anyone had anything
more. Eventually, I was able to gather three critical sources: First,
a copy of the 3B2/400 boot ROM, dumped by a friend with access to a
real machine. Second, a high resolution picture of a 3B2/400
motherboard. And third, a copy of the AT&T UNIX SVR3 kernel source
code.

At last, I had something to go on.

Figuring Out the Basics

Visual inspection of the 3B2 motherboard was a first critical
step. I've annotated the photograph above. It revealed every major
component in the system, and allowed me to look up data sheets for
each part. Still, I had no idea how the whole system was actually
connected. If you're going to emulate a computer, you need to know how
all the pieces fit together, and that can't be gleaned from a
picture. That's when I turned to the SVR3 kernel source code. These
sources are pretty revealing, but of course, they would have to
be. The kernel needs to know the layout of the system in order to
operate.

The first thing I wanted to know was, what does the memory map look
like? What physical addresses does the system use to communicate with
all of these devices? Thankfully a little poking around led me to the
kernel linker table, showing the memory location for every component
in the system. This block told me pretty much everything I needed to
know to get started. Although there are no comments, the big clue here
came from digging through the source code to find these symbol names
and making some educated guesses about what exactly they corresponded
to.

Decoding the Firmware

At around the same time that I was trying to understand the memory map
and system components, I started to decode the ROM image so I could
figure out how the 3B2 boots. I didn't (and still don't) have the
firmware source code, so I had to disassemble the boot ROM, and
unfortunately there was no disassembler. I had to write one.

Writing a basic disassembler is pretty straight-forward for the most
part. You just read in some bytes, figure out what instructions and
operands those bytes correspond to, and spit out the mnemonic
instructions as text. I'd written disassemblers for the 6502, but of
course the WE32100 is much more complex, so it took a little longer
than I was planning.

Once decoded, reading the firmware was pretty useful. For one thing,
it has a lot of CPU exercise tests, tests that would eventually verify
that the emulated CPU was doing the right things. But more importantly
it provides a slew of functions that can be called by the SVR3 boot
code. These added to my understanding of the system architecture.

Armed with the knowledge gleaned from the SVR3 source code and the
boot ROM, I had enough to get started writing something.

The Emulation Framework

I've written emulators for small computers before, but this time, for
a system so complex, I knew I wanted to use a framework that already
existed and had a lot of support. I turned to the SIMH framework
(available on GitHub), which I consider to be more or less the gold
standard for emulating minicomputers. SIMH gave me the ability to
bootstrap things very, very quickly, because most of the hooks for
emulating a minicomputer are already there. Console and serial IO,
disk image attaching and detaching, debugging support, it's all baked
in. So I rolled up my sleeves, forked SIMH on GitHub, and started
work.

The CPU

The first thing I worked on was a CPU emulation. I assumed
(incorrectly) that this was going to be the most challenging part, and
of course an emulator can't do anything useful without it, so I got a
head start as soon as I could.

The WE32100 is a complex microprocessor, and I'm grateful that I had
an excellent document to follow, "Maxicomputing in Microspace", the
official WE 32100 Processor Information Manual. This manual covers in
excruciating detail the inner workings of the WE32100 and each
instruction. It also describes each interrupt and exception
microsequence in such glorious detail that emulating them
became… well, not easy. But certainly manageable.

Within a month or so, I had enough of the CPU emulated that it could
pass the ROM diagnostics and try to print something to the console.

The Console

Console emulation was tricky. The 3B2 has two serial ports on board,
CONSOLE and CONTTY. CONSOLE is, naturally, where the console terminal
plugs in, and it is the port that receives all the boot messages.

CONSOLE and CONTTY are both controlled by a single SCN2681A dual-UART
chip. The SCN2681A datasheet is pretty good, but not quite as detailed
as I'd like, so a lot of educated guesses had to be made about how the
SCN2681A works internally. The first pass was actually pretty
brain-dead and didn't behave properly at all, but did print boot
messages well enough to verify that the CPU was trying to come up.

As the emulator progressed and I learned more about the SCN2681A, the
CONSOLE and CONTTY emulation got better. SIMH provided almost all of
the boilerplate necessary to handle keyboard input and terminal
output. I will leave out the gory details, but eventually a full
3-character receive ring buffer, transmit buffer, all control
registers, and modem line control were accurately emulated.

By this time, I was able to interact fully with the on-board ROM
monitor and its small set of built-in commands. The next challenge was
to try to boot UNIX from a floppy image.

The Floppy Drive

When the CPU and console at least mostly working, I turned my
attention to the floppy disk controller, a Western Digital 2797
compatible chip. Again, a datasheet was available, and described the
chip well enough to get started emulating it.

The 3B2 floppy drive is an off the shelf CDC model that supports
quad-density diskettes (96tpi, 80 track DSDD). SIMH once again
provided all the boilerplate necessary to read and write a diskette
image, so all I really needed to do was figure out how the 3B2 system
talked to the controller, emulate enough of the controller to respond
correctly to commands, and then read and write sectors on a mounted
image.

With a skeleton controller written, I was finally able to try to boot
the first UNIX installation diskette. Naturally, it didn't
work. (Nothing works the first time in an emulator)

The Hard Drive

I didn't know it at the time, but it turns out that having a working
hard drive controller is essential to getting UNIX to boot off of a
floppy. The 3B2 has an NEC ?PD7261A Winchester controller, and
supports one or two MFM hard disks in 31MB, 43MB, 72MB, or 161MB
capacities.

SIMH provides excellent hard disk image access routines. You really
just need to use the SIMH disk library, and a few routines allow
reading and writing sectors to the disk image. But, that was only part
of the challenge. The larger problem was figuring out how the
Winchester controller interfaced with the host and responded to
commands. In some ways it was very similar to the floppy
controller. In other ways, much different.

I soon discovered that a key problem was going to be interrupts (more
on that later). Computers do not operate infinitely quickly: Disks are
slow, it may take 250 milliseconds to do a disk operation, an
extraordinarily long time when your CPU is operating at an instruction
every microsecond. Normally the system will issue a command to the
disk controller, and then go off to do other things while it waits for
I/O to complete. When the disk controller has completed its work, it
has to interrupt the system board and say "my work is done".

Part of the job of emulating a disk controller, then, is to
artificially delay certain actions and only interrupt after a certain
number of microseconds or milliseconds has gone by. SIMH supports this
through several routines that put jobs on an internal execution queue
with a specified delay in either simulated machine steps, or
microseconds through a calibrated clock.

It took ages, but eventually the hard disk controller emulation was
working well enough to accurately read and write a virtual hard disk
image.

System Timers

The 3B2 has a timer chip, the NEC ?PD8253C, that provides three
interval timers. This is a very complex chip, in that each
independent timer can be configured a broad number of ways and perform
a number of different functions. It was, in fact, one of the harder
parts of the system to get working accurately.

The most important timer is Timer #1, configured by the boot ROM to
deliver a 100 Hz square wave. Once every 10ms, the timer generates a
Level 15 interrupt. This interrupt, it turns out, drives much of
UNIX. Without it, UNIX can't schedule or switch tasks. So it was
essential to get right.

This is another area where SIMH shines. SIMH has support for fully
calibrated system timers, meaning that you can tell SIMH to trigger an
action once every 10 ms of wall clock time, and it will do so. This is
how the 3B2 emulator's 100Hz system timer works.

The MMU

By this time, I was actually getting very close to booting UNIX. In
fact, I was able to get much of the way through the boot process,
until the computer turned on virtual memory. Then everything came
crashing down, because I needed something very important: A Memory
Management Unit.

The 3B2 uses a WE32101 MMU, a custom part manufactured by Western
Electric and part of the WE32000 chipset. The MMU provides translation
between physical and virtual memory addresses, and it's a complicated
piece of engineering. In fact, it was tougher to get the MMU working
correctly than it was to get the CPU working correctly.

At its heart, the MMU has a set of configuration registers that tell
it where to find translation tables in main memory. Without going too
much into detail here, the translation is controlled by a very complex
state machine, and the MMU must cause an exception in the CPU whenever
translation cannot happen. These exceptions are essential to get
right, because the operating system relies on them to keep its process
virtual memory tables up to date.

Frustratingly, the only document I had at the time that described the
MMU was the WE32101 data sheet. The data sheet was good, but not
great, and although it described the translation algorithm at a high
level, it didn't go into great detail.

The MMU took what felt like ages to get done, and little edge cases
kept popping up as I went along. It wasn't until very recently indeed
that I felt like the MMU emulation was completely polished and
(hopefully) bug-free.

… And The Rest

Finally, there were only a few bits and pieces of glue logic left to
emulate. Among them were:

The DMA controller for moving blocks of memory around between the
system board and peripherals.

The battery-backed Time-of-Day clock that provides the current date
and time on power-up.

A set of latches called the CSR that hold run-time and interrupt
status.

The Non-Volatile RAM (NVRAM), which holds system configuration
between boots.

All of these were important to a working system, some more than
others. The DMA, CSR, and NVRAM were particularly important, but were
also thankfully trivial to get working.

Interrupts

But still, the 3B2 would not boot.

After all this work, it was still incredibly buggy. System timing
felt impossible to get right. It all came down to interrupts.

The floppy controller, hard disk controller, timer, and UART all
generate interrupts in the system. I was so sure I had all of the
interrupts correct, but no matter how much I played with interrupt
timing, the system would crash before getting to the OS installation
screen. At several points I gave up, frustrated, and shelved the
emulator for months at a time, convinced that I was missing some
important architectural detail that lay hidden in internals
documentation I would never see.

It wasn't until late last year that I had the breakthrough necessary
to lead to success. You see, I had been working under the assumption
that interrupts are fired and handled (or not handled) immediately
after firing. If the CPU is ready for an interrupt, it would handle
it. If not, it would ignore it. But that's not actually how interrupts
in the 3B2 work.

What really happens is that interrupts are latched. When a peripheral
generates an interrupt, the system latches the interrupt signal in the
CSR. If the CPU isn't ready to handle the interrupt yet, that's fine,
it'll keep processing until it is. Once the CPU is ready, only then
does it acknowledge and handle the latched interrupt, and clear the
CSR latch.

This was the missing piece, the silver bullet. I spent an afternoon
refactoring my interrupt code, and booted the installation floppy
image again.

This time, it worked. I was greeted by this screen, and a did a little
dance of joy:

At long last, I installed UNIX, and it was good.

Ongoing Work

An emulator like this never really feels done. Since I got the base
system working, there have been several areas I continue to work
on. Of course, every now and then I find a bug, and fixing bugs is
priority number one. There have been several critical bugs with the
MMU, with system timing, and with the UART. At the moment, I don't
know of any outstanding bugs, but of course that doesn't mean there
aren't any yet to be discovered.

Outside of bugs, most of the work has been around adding support for
expansion cards. The 3B2/400 has a backplane with 12 slots that can
accept I/O expansion cards, many of which are based around a common
architecture called "Common I/O", or CIO for short. So far, I have
implemented two expansion cards:

The PORTS card, which supports four dial-up or remote terminal
serial ports.

The CTC card, which adds a removable 23MB Cipher QIC tape
cartridge.

There are more cards I'd like to do, but reverse engineering these
cards takes a lot of time and effort. PORTS and CTC were low-hanging
fruit, and other cards will be much more challenging.

Conclusion

Four years is a long time. Of course I didn't work on the emulator
non-stop for four years. I have a day job, and a life that needs
attending to. There were long periods when I didn't work on the
emulator at all.

But now that it's in a state that I can share it with the rest of the
world, it feels good. I'm glad I tackled this project, saw it through,
and made something that works and can help preserve this piece of
computing history.

Now it's time to look for my next project.

Copyright 2018 by Seth Morabito.
Proudly published with Emacs and Org Mode