The Background – Linux as a Fast Follower and the Need for Hot Patching

No doubt about it, Linux has made impressive strides in the last 15 years, gaining many features previously associated with high-end proprietary Unix as it made the transition from small system plaything to core enterprise processing resource and the engine of the extended web as we know it. Along the way it gained reliable and highly scalable schedulers, a multiplicity of efficient and scalable file systems, advanced RAS features, its own embedded virtualization and efficient thread support.

As Linux grew, so did supporting hardware, particularly the capabilities of the ubiquitous x86 CPU upon which the vast majority of Linux runs today. But the debate has always been about how close Linux could get to "the real OS", the core proprietary Unix variants that for two decades defined the limits of non-mainframe scalability and reliability. But "the times they are a changing", and the new narrative may be "when will Unix catch up to Linux on critical RAS features like hot patching".

Hot patching, the ability to apply updates to the OS kernel while it is running, is a long sought-after but elusive feature of a production OS. Long sought after because both developers and operations teams recognize that bringing down an OS instance that is doing critical high-volume work is at best disruptive and worst a logistical nightmare, and elusive because it is incredibly difficult. There have been several failed attempts, and several implementations that "almost worked" but were so fraught with exceptions that they were not really useful in production.[i]

Red Hat held its 2015 summit last week in Boston. One of the most important announcements was the general availability of version 3 of OpenShift. After my discussion with Jim Whitehurst, president and CEO of Red Hat, as well as other executives, partners and, clients, I believe that Red Hat has made a strategic move and is taking the lead in enterprise-class container solutions for hybrid cloud enablement. This is because:

Red Hat has an early-mover advantage in platform refactoring.OpenShift and Cloud Foundry, two major open source PaaS platforms, both started refactoring with container technology last year. The developers of Cloud Foundry are still working hard to complete the platform’s framework after implementing Diego, the rewrite of its runtime. But OpenShift has already completed its commercial release, with two major replacements around containers: It replaced Gears, its original homegrown container model, with Docker and replaced Broker, its old orchestration engine, with Kubernetes.

In a world where OS and low-level platform software is considered unfashionable, it was refreshing to see the Linux glitterati and cognoscenti descended on Boston for the last three days, 5000 strong and genuinely passionate about Linux. I spent a day there mingling with the crowds in the eshibit halls, attending some sessions and meeting with Red Hat management. Overall, the breadth of Red Hat’s offerings are overwhelming and way too much to comprehend ina single day or a handful of days, but I focused my attention on two big issues for the emerging software-defined data center – containers and the inexorable march of OpenStack.

Containers are all the rage, and Red Hat is firmly behind them, with its currently shipping RHEL Atomic release optimized to support them. The news at the Summit was the release of RHEL Atomic Enterprise, which extends the ability to execute and manage containers over a cluster as opposed to a single system. In conjunction with a tool stack such as Docker and Kubernates, this paves the way for very powerful distributed deployments that take advantage of the failure isolation and performance potential of clusters in the enterprise. While all the IP in RHEL Atomic, Docker and Kubernates are available to the community and competitors, it appears that RH has stolen at least a temporary early lead in bolstering the usability of this increasingly central virtualization abstraction for the next generation data center.

The rise of the DevOps role in the enterprise and the increasing requirements of agility beyond infrastructure and applications make the platform-as-a-service (PaaS) market one to watch for both CIOs and enterprise architecture professionals. On December 9, the membership of Cloud Foundry, a major PaaS open source project, announced the formation of the Cloud Foundry Foundation.

In my view, this is as important as the establishment of OpenStack foundation in 2012, which was a game-changing move for the cloud industry. Here’s why:

PaaS is becoming an important alternative to middleware stacks. Forrester defines PaaS as a complete application platform for multitenant cloud environments that includes development tools, runtime, and administration and management tools and services. (See our Forrester Wave evaluation for more detail on the space and its vendors.) In the cloud era, it’s a transformational alternative to established middleware stacks for the development, deployment, and administration of custom applications in a modern application platform, serving as a strategic layer between infrastructure-as-a-service (IaaS) and software-as-a-service (SaaS) with innovative tools.

Cloud Foundry is one major open source PaaS software. Cloud Foundry as a technology was designed and architected by Derek Collison and built in the Ruby and Go programming languages by Derek and Vadim Spivak (wiki is wrong!). VMware released it as open source in 2011 after Derek joined the company. Early adopters of Cloud Foundry include large multinationals like Verizon, SAP, NTT, and SAS, as well as Chinese Internet giants like Baidu.

[Apologies to all who have just read this post with a sense of deja-vue. I saw a typo, corrected it and then republished the blog, and it reset the publication date. This post was originally published several months ago.]

Having been away from the Linux scene for a while, I recently took a look at a newer version of Linux, SUSE Enterprise Linux Version 11.3, which is representative of the latest feature sets from the Linux 3.0 et seq kernel available to the entre Linux community, including SUSE, Red Hat, Canonical and others. It is apparent, both from the details on SUSE 11.3 and from perusing the documentation on other distribution providers, that Linux has continued to mature nicely as both a foundation for large scale-out clouds as well as a strong contender for the kind of enterprise workloads that previously were only comfortable on either RISC/UNIX systems or large Microsoft Server systems. In effect, Linux has continued its maturation to the point where its feature set and scalability begin to look like a top-tier UNIX from only a couple of years ago.

Among the enterprise technology that caught my eye:

Scalability – The Linux kernel now scales to 4096 x86 CPUs and up to 16 TB of memory, well into high-end UNIX server territory, and will support the largest x86 servers currently shipping.

I/O – The Linux kernel now includes btrfs (a geeky contraction of “Better File System), an open source file system that promises much of the scalability and feature set of Oracle’s popular ZFS file system including checksums, CoW, snapshotting, advanced logical volume management including thin provisioning and others. The latest releases also include advanced features like geoclustering and remote data replication to support advanced HA topologies.

On April 23, IBM rolled out the long-awaited POWER8 CPU, the successor to POWER7+, and given the extensive pre-announcement speculation, the hardware itself was no big surprise (the details are fascinating, but not suitable for this venue), offering an estimated 30 - 50% improvement in application performance over the latest POWER7+, with potential for order of magnitude improvements with selected big data and analytics workloads. While the technology is interesting, we are pretty numb to the “bigger, better, faster” messaging that inevitably accompanies new hardware announcements, and the real impact of this announcement lies in its utility for current AIX users and IBM’s increased focus on Linux and its support of the OpenPOWER initiative.

Technology

OK, so we’re numb, but it’s still interesting. POWER8 is an entirely new processor generation implemented in 22 nm CMOS (the same geometry as Intel’s high-end CPUs). The processor features up to 12 cores, each with up to 8 threads, and a focus on not only throughput but high performance per thread and per core for low-thread-count applications. Added to the mix is up to 1 TB of memory per socket, massive PCIe 3 I/O connectivity and Coherent Accelerator Processor Interface (CAPI), IBM’s technology to deliver memory-controller-based access for accelerators and flash memory in POWER systems. CAPI figures prominently in IBM’s positioning of POWER as the ultimate analytics engine, with the announcement profiling the performance of a configuration using 40 TB of CAPI-attached flash for huge in-memory analytics at a fraction of the cost of a non-CAPI configuration.[i]

Every culture has its coming of age rituals — Confirmation, Bar Mitzvah, being hunted by tribal elders, surviving in the wilderness, driving at high speed while texting — all of which mark the progress from childhood to adulthood. In the high-tech world, one of the rituals marking the maturation of a company is the user group. When a company has a strategy it wants to communicate, a critical mass of customers, and prospects bright enough that it wants to highlight them rather than obscure them, it is time for a user group meeting.

This year, having passed a year since the acquisition of Novell by AttachMate and its subsequent instantiation as a standalone division, as well as being its 20th anniversary, SUSE had its first user group meeting. All in all, the portents were good, and SUSE got its core messages across to an audience of about 500 of its users as well as a cadre of the more sophisticated (IMHO) industry analysts.

Among My Key Takeaways:

SUSE is a stable company with rational management — With profitable revenues of over $200M and a publicly stated plan to hit $234 for the next fiscal year, SUSE is a reasonably sized company (technically a division of $1.3B Attachmate, but it looks and acts like an independent company), with growth rates that look to be a couple of points higher than its segment.

SUSE’s management has done an excellent job of focusing the company — SUSE, acknowledging its size disadvantage over competitor Red Hat, has chosen to focus heavily on enterprise Linux, publicly disavowing desktop and mobile device directions. SUSE’s claim is that their market share in the core enterprise segment is larger than their overall market share compared to Red Hat. This is a hard number to even begin to tweeze out, but it feels like a reasonable claim.

In the latest evolution of its Linux push, IBM has added to its non-x86 Linux server line with the introduction of new dedicated Power 7 rack and blade servers that only run Linux. “Hah!” you say. “Power already runs Linux, and quite well according to IBM.” This is indeed true, but when you look at the price/performance of Linux on standard Power, the picture is not quite as advantageous, with the higher cost of Power servers compared to x86 servers offsetting much if not all of the performance advantage.

Enter the new Flex System p24L (Linux) Compute Node blade for the new PureFlex system and the IBM PowerLinuxTM 7R2 rack server. Both are dedicated Linux-only systems with 2 Power 7 6/8 core, 4 threads/core processors, and are shipped with unlimited licenses for IBM’s PowerVM hypervisor. Most importantly, these systems, in exchange for the limitation that they will run only Linux, are priced competitively with similarly configured x86 systems from major competitors, and IBM is betting on the improvement in performance, shown by IBM-supplied benchmarks, to overcome any resistance to running Linux on a non-x86 system. Note that this is a different proposition than Linux running on an IFL in a zSeries, since the mainframe is usually not the entry for the customer — IBM typically sells to customers with existing mainframe, whereas with Power Linux they will also be attempting to sell to net new customers as well as established accounts.

I recently had an opportunity to spend some time with SUSE management, including President and General Manager Nils Brauckmann, and came away with what I think is a reasonably clear picture of The Attachmate Group’s (TAG) intentions and of SUSE’s overall condition these days. Overall, impressions were positive, with some key takeaways:

TAG has clarified its intentions regarding SUSE. TAG has organized its computer holdings as four independent business units, Novell, NetIQ, Attachmate and SUSE, each one with its own independent sales, development, marketing, etc. resources. The advantages and disadvantages of this approach are pretty straightforward, with the lack of opportunity to share resources aiming the business units for R&D and marketing/sales being balanced off by crystal clear accountability and the attendant focus it brings. SUSE management agrees that it has undercommunicated in the past, and says that now that the corporate structure has been nailed down it will be very aggressive in communicating its new structure and goals.

SUSE’s market presence has shifted to a more balanced posture. Over the last several years SUSE has shifted to a somewhat less European-centric focus, with 50% of revenues coming from North America, less than 50% from EMEA, and claims to be the No. 1 Linux vendor in China, where it has expanded its development staffing. SUSE claims to have gained market share overall, laying claim to approximately 30% of WW Linux market share by revenue.

Focus on enterprise and cloud. Given its modest revenues of under $200 million, SUSE realizes that it cannot be all things to all people, and states that it will be focusing heavily on enterprise business servers and cloud technology, with less emphasis on desktops and projects that do not have strong financial returns, such as its investment in Mono, which it has partnered with Xamarin to continue development,.

Most Java applications don’t need a fancy container that has umpteen features. Do you want to pay for a car that has windshield wipers on the headlights? (I wish I could afford it.) Most Java applications do not need these luxuriant features or can be designed not to need them. Many firms do, in fact, deploy enterprise-class Java web applications on Apache Tomcat. It works. It is cheap. It can save tons of dough.

Expensive Java Application Servers Sometimes Add Value

There is a need for luxury. But, you probably don’t need it to provide reliable, performant, and scalable Java web applications. Application server vendors will argue that:

You need an application container that supports EJBs. EJB3 fixed the original EJB debacle, but why bother? Use Spring, and you don’t need an EJB-compliant container. Many applications don’t even need Spring. EJBs are not needed to create scalable or reliable applications.