Linux on POWER

Anyone familiar with GNU/Linux will not be surprised by the fact that this operating system runs on almost all known processors. However, very few people are aware that mere support just might not be enough. You’ll also need to keep an up-to-date repository of code. This is especially true when it comes to serious hardware such as POWER.

I guess that almost all desktop systems in the office or home are derived from one prototype, the IBM PC (personal computer), which was built around the concept of an open architecture that IBM introduced 30 years ago. It’s true that the company itself didn’t quite understand how revolutionary open hardware architecture would be a few years later, in the mid-80s. Whatever the case, the fact remains that the PC became popular because many independent manufacturers could produce a wide range of compatible peripherals. This, in turn, allowed a competitive market to develop, eventually making desktop PCs available to everyone.

The open development model of Linux first embraced the desktop market of x86 computers, and then proceeded to “capture” other architectures as well (now more than 20), ranging from x86, ARM and MIPS, to Alpha, SPARC, PowerPC, IA64 and S390.

While “desktop” platforms (x86 and ARM) have many different Linux distributions and other operating systems that can run on them, the “pure” server architectures like IA64, S390 and Power cannot boast of this — the number of distributions is much, much smaller. This fact is understandable. Quality and the cost of server hardware for industrial applications is immeasurably higher than for the usual PC. Moreover, not every Linux distribution vendor can find the resources to build and provide long-term support for major server hardware architecture — even if they are willing to.

Some time ago, I got my hands on an IBM IntelliStation POWER 285 workstation, considered the closest model to large industrial servers. The main brain of the station is a dual-core POWER 5+ processor.

Although announced back in 2005, the system is serious stuff even now. Judge for yourself: 4 GB RAM (max capacity 32 GB), with a disk subsystem that has 2×146 GB SCSI drives (max disk storage 1.2 TB). It has a self-diagnostic system and memory “self-healing” technology known as ChipKill, a remote HMC (Hardware Management Console) and a pair of NICs working at 1 Gbit. This system is unbeatable for an entry-level enterprise server and simply forces you to take another respectful look at IBM — Big Blue creates innovative and striking products.

Well, that’s enough of an introduction. Let’s learn Linux installation and operation on IBM hardware. And later decide how it is possible to effectively exploit Linux on Power.

Remote access

IBM produces equipment that has OpenFirmware (OF) inside. This is analogous to the traditional PC BIOS used on x86 systems. With OF’s help, you can assign which source should be used to start an OS, perform a number of diagnostic procedures, etc. OF itself uses a ServiceProcessor — a special unit embedded into the big tower. When you plug in the power cord, first the ServiceProcessor starts. That initially sets up a remote access option, and then awaits the user’s response.

The user has to press the hardware Start button on the front panel. So, when plugged in, IntelliStation is never completely turned off — even with the main CPUs off, the ServiceProcessor doesn’t sleep. The scheme is reminiscent of a similar product — iLO, developed by Compaq/HP. iLO also lets one start and control a server remotely.

It’s worth noting that IBM also has additional functions run by HMC (Hardware Management Console), among which are planning and allocation of physical resources to logical partitions (LPAR/DLPAR), splitting main CPU performance onto logical CPUs (Capacity On Demand) and related actions that provide virtualisation management on POWER-based processors. Unfortunately, I can only mention these capabilities, and not go any deeper into the subject, because that would be beyond the scope of this review.

So, in order to establish an access link to the ServiceProcessor and later install Linux, you will need an RS232(F)<->RS232(F) cable and a dedicated workstation, from which you can directly control your IntelliStation 285. As a console, any PC with an installed command-line tool like minicom or surecom (a GUI alternative) will suit. Set the communication speed to 19,200 bit/s, the standard default rate for IBM hardware, plug in the power cord and wait for the login prompt after running minicom -D /dev/ttyS0 -s 19200 -o. See Figure 1.

Figure 1: Control Power hardware via RS232 cable

To start the whole system and gain access to OF, from where you can select an OS source, choose: 1. Power/Restart Control -> 1. Power On/Off System -> 8. Power on. After that, in approximately two minutes, you will hear the noise of the fans — the big system has started initialisation and a minute later the hardware will be completely initialised, signalled via two speaker beeps.

At the same time, on the console, you should see a message from SMS (System Management Services) as in Figure 2, where you choose 1 to enter the system OF menu and then select 1. Select Install/Boot Device to choose the boot device (which you will run the installer from).

Figure 2: SMS greeting before entering OpenFirmware menu

You should see something like Figure 3, where I chose 4. IDE, because the built-in CD-ROM is connected to the IDE connector. In your version, it might be 3. CD/DVD.

The installation procedure reminds me of what happens on the x86, with just one difference — in our case, control is via a serial line in text VT102 mode, so some characters may not be displayed properly as you can see in Figure 5.

Figure 5: SUSE Linux installation could be fun

Several menu items, such as the network address, may require you to return to re-edit these fields. We can use Tab/Alt+Tab to move between text interface fields. These keyboard shortcuts are functional even in this very minimal mode.

Note: Installing SLES 10 involved no difficulty in navigation or in selecting the text buttons. However, with SLES 11, there was the problem of having to check the Internet connection (Test Connection to the Internet Via Ethernet Controller). The problem is that there is no systems setting to exclude or interrupt this operation when you are in text mode via a serial line. In this case, simply restart the system and assign the necessary IP address after a reboot (via serial connection and minicom). After that, you are encouraged to connect via SSH over the Ethernet connection.

Embedded Linux on ServiceProcessor

In addition to control via a serial line, IntelliStation (and in general, all IBM products) offers management via an ordinary Web browser. This Web control mechanism (Advanced System Management — see Figure 6) is provided by a tiny computer within the IntelliStation, with Linux and BusyBox utilities.

Figure 6: Management via Web interface

This ServiceProcessor also uses Power technology — a PowerPC 405 CPU from IBM [Datasheet PDF] but with a much smaller form factor. This uses a low-power processor FSP (Flexible Service Processor) running at about 200 MHz (see Figure 7).

Figure 7: Embedded Linux runs on PowerPC 405 from IBM

ASM provides the same controls as the serial-line SMS but in a more usable manner (Figure 8). It lets you manage the system remotely — start/stop it, view logs, allocate resources (like Capacity on Demand parameters), etc.

Figure 8: ASM's user-friendly controls presentation

Let’s briefly see how this subsystem is launched. When you connect the mains power to the box, the ServiceProcessor automatically starts the embedded Linux. After initially testing some components of a larger block, ServiceProcessor displays on a front panel the readiness status for full system start. When you hit the “Power On” button, OpenFirmware loads and ultimately passes control to your Linux boot loader, yaboot.

Like with the large system, this small subsystem has two independent Ethernet controllers, referred to as HMC 1 and HMC 2. It makes sense to connect them to separate LANs/VLANs for a special management network. First, it helps access restriction and security. Second, you can avoid a network traffic overload on a single LAN.

IP addresses for these ports can be supplied by a DHCP server on a management network. If one isn’t found, default addresses are automatically assigned: 192.168.2.147 (port C7-T1, on HMC1) and 192.168.3.147 (port C7-T2, HMC2). You can log in via telnet, with the default credentials admin/admin.

As we see, Embedded Linux uses JFFS/CRAMFS filesystems, which are common in tiny products. An interesting feature is NVRAM memory mounted as storage (/dev/mtdblock/nvram/nvram). Reading data from its files gives the status of different subsystems. You can also change some values using commands from /opt/fips/bin — particularly, you can even try to launch several subsystems (or maybe the full system box) with /opt/fips/bin/powr_fsp_spcn, which checks the current system status, and accordingly launches /bin/powr_L4 or /bin/powr_sqh. The ASM Web server (named webs), when you click the button Save settings and power on does the same.

Available distros for POWER

Originally, I planned to use Red Hat Enterprise Linux as the main OS on this system. However, neither RHEL 5.7 nor 6.2 Beta, nor even 6.2 started correctly. RHEL 6.2 Beta and a more recent version 6.2 spews a kernel trap error during initial startup. Yes, the IntelliStation 285 wasn’t in the supported hardware list, but neither was it for SLES 10/11. Still, SLES 10 and 11 support it unofficially. Let us see how the installation process goes, and whether we can bring the system to production mode.

In addition to RHEL and SLES, numerous systems based on the POWER processor even supported Ubuntu 10.04 (LTS edition). However, my experience proved that this model wasn’t suitable for Ubuntu — it cannot start properly. Moreover, newer versions of Ubuntu come without any POWER/PowerPC architecture support at all, unfortunately.

I checked if the most recent release (6.0.3) of Ubuntu’s parent distribution, Debian, could boot. It wasn’t able to start — it drops directly into OpenFirmware. Another failure! Well, let’s not despair. Quite possibly, the Debian community will fix it.

Oddly enough, a completely amateurish project called Crux PPC has a fully working distribution — Crux PPC 2.7a, which not only loads on Power hardware, but also provides video output via correctly-set framebuffer mode. None of the other distributions mentioned can boast of that! All provided only a serial-line text mode. In general, Crux PPC runs as a LiveCD and has no installer, so I advise professionals to use it. Again, this topic is out of the scope of this article, so I will leave it for later.

Performance

As I mentioned a bit earlier, IBM products are one level, or a “head above” their competitors. Comparing the very different Power and x86 architectures is hard, because there are no objective criteria for such a comparison. However, from an ordinary user’s perspective, we can try the 7Zip archiver in its benchmark mode. This program is completely open source, so anyone can compile it.

Let us compare IntelliStation based on the POWER5+ processor with 2 cores and 4 GB RAM, manufactured in 2005, with an Intel Core 2 Duo (2 cores with hyper-threading) and 4 GB RAM, but released three years later (2008). First, the IntelliStation:

As you can see, the performance doesn’t differ much. Even a slightly outdated IBM station can compete with a relatively new x86 product.

Despite the fact that equipment based on the Power processor is marketed by IBM as a corporate power-horse for commercial operating systems like AIX and IBM i (produced by IBM itself), we have seen that it can also be used with Linux. Thus, virtually the entire software stack that successfully works on x86 can also be run on Power — Apache, Java, WebSphere, MySQL, etc.

Taking into account the fact that IBM hardware usually comes with unique technologies like the hardware management console and the hardware virtualisation hypervisor, you can easily create an effective solution by combining the best features from both the hardware and software sides — the corporate and the public world.

The author has specialised in Linux and FOSS technologies for more than a decade. His professional spheres of interest include, but are not limited to, robotics, embedded systems, statistics and algorithmic methods.

ah, just a note. CRUX PPC (like x86 main project CRUX) isn’t amateurish but it’s a DIY community distro like Gentoo or Debian. It also, obviously, has an installer.

Moonferret

With a bit of work, both Ubuntu & Debian will run on this machine. I have mine running Ubuntu 10.04. It involved editing the yaboot.conf after install and swapping the graphics board (GXT6500p) for a Mac Radeon 7000 (no BIOS flash needed – Just works! )

Moonferret

With a bit of work, both Ubuntu & Debian will run on this machine. I have mine running Ubuntu 10.04. It involved editing the yaboot.conf after install and swapping the graphics board (GXT6500p) for a Mac Radeon 7000 (no BIOS flash needed – Just works! )