IBM z196 Mainframe Architecture

Introduction

IBM’s zArchitecture mainframes have a long and storied history and occupy a unique place in the computing landscape. The starting point, the System/360 was envisioned as a family of computers that could serve any potential customer – from business to scientific computing. The first systems were released in 1964, after years of development and a budget that reportedly grew to $5B. The S/360 introduced the concept of compatibility by defining an instruction set architecture that was separate from the initial implementation. In addition to the first ISA, it was also the first system to use microcode and subsequent models were the first computers to use virtualization in production. The technical achievements yielded tremendous commercial success for IBM well into the next century. As recently as 2006, mainframes and related products and services were responsible for a quarter of IBM’s revenue and half of the profit.

While wildly successful, mainframes were intertwined with legal troubles for IBM. They were a focal point of early antitrust cases against the company, most notably by rival Control Data and the Department of Justice. IBM’s legal experiences shaped the industry in many ways. In 1969, they unbundled software and services from the underlying hardware – which essentially created the market for computer software. It also opened the door for compatible mainframes from other companies, such as Amdahl, Hitachi, Fujitsu, Siemens, NEC and others. Some of these mainframe clones continue to exist today, particularly in Europe and Japan.

The mainframe significantly predates the concept of RISC, which was pioneered in the 1980’s. zArchitecture, along with x86 are classic complex instruction sets and ironically, amongst the most commercially successful. This is most likely due to the unwavering commitment to software compatibility from IBM, which was subsequently inherited by Intel and the PC. Mainframes evolved over time from 24-bit addressing, to 31-bit addressing in 1982 and eventually the 64-bit zArchitecture in 2000. The latter signaled the end for most mainframe clones, which were predominantly 31-bit.

One of the more unique elements of the mainframe is the I/O architecture. I/O is managed by a dedicated channel processor, to offload work from the main CPU. Channel programs are invoked with a single instruction on the main CPU and executed by the I/O processor. In some ways, this is the ultimate form of CISC – a single zArchitecture instruction might be the equivalent of millions of simple operations. The emphasis on I/O and system architecture is the hallmark of IBM’s mainframes.

IBM’s mainframes are expensive computers, priced well beyond proprietary RISC/UNIX systems, but they can often be an economically attractive platform due to the overall efficiency and stability. The high cost of a mainframe has given the processors a reputation as engineering projects with infinite budgets. This reputation is not entirely true, but it is understandable how this myth developed.

The name ‘mainframe’ stems from packaging all the components of the computer into a single frame. Early mainframes were designed with bipolar logic and spanned multiple chips. The first mainframe based on a CMOS microprocessor (i.e. single chip processor) was the 9672-G1 in 1994. The microprocessors have subsequently evolved to integrate greater system level functionality, mirroring the path of Moore’s Law. Given the cost of the overall mainframe and the relatively smaller role of the microprocessor, it is eminently understandable that IBM would consistently choose to design large CPUs with exotic packaging and high power consumption, in order to maximize overall performance. This gives IBM’s engineers a unique opportunity, since most other microprocessor design teams are constrained by the economics of a component, rather than system, business model.

The z196 is the latest mainframe microprocessor from IBM. It is a 5.2GHz, quad-core, 3 issue, out-of-order execution design that was released in late 2010. The z196 is manufactured on IBM’s 45nm SOI process with 13 layers of metal and deep trench capacitor eDRAM. IBM’s process technology was slightly customized for the z196, with two extra layers of metal and a 4th Vt option for critical paths. The z196 has roughly the same 15 FO4 pipeline depth as its predecessor, the quad-core z10. The new cores are 25% more power efficient and IBM’s engineers relied on process technology improvements to increase the frequency from 4.4GHz up to 5.2GHz.

Physically, the z196 is implemented in a 512mm2 die with 1.3B transistors and over 9000 pins for I/O and power. The processor has extensive circuit and architectural RAS features such as hardened latches, a recovery unit, residue checking and a redundant memory channel for parity. In aggregate, IBM estimates that the overhead for error checking is around 20-25% for digital logic. The z196 maintains the same ~250W power envelope as the previous generation while adding out-of-order execution and a shared L3 cache. Six microprocessors and two storage controller chips are packaged in a 103 layer ceramic MCM that can be water-cooled for reliability and leakage reduction, and dissipates 1.8KW.