The Server Time Protocol (STP) feature is designed to provide the capability for multiple servers
and Coupling Facilities to maintain time synchronization with each other, without requiring a
Sysplex Timer®. The servers and Coupling Facilities that are capable of supporting Server Time
Protocol are:

IBM System z9 Enterprise Class (z9 EC)

IBM System z9 Business Class (z9 BC)

IBM eServer® zSeries® 990

IBM eServer zSeries 890

STP is designed to support a multisite sysplex configuration up to 100 km (62 miles) and to allow
use of Coupling Links to transport timekeeping information.

The Server Time Protocol feature was previewed July 27, 2005. Refer to Hardware
Announcement
105-241
, dated July 27, 2005.

Non-raised-floor support for z9 BC

A new feature, designed to allow the IBM System z9 Business Class (z9 BC) to operate in a
non-raised-floor environment, is being offered.

The Sysplex Timer's Licensed Internal Code (LIC) has been upgraded to support using STP in a Mixed
Coordinated Timing Network. The required Sysplex Timer LIC is shipped along with the STP feature,
and must be installed by the IBM Service Support Representative prior to migrating from a Sysplex
Timer-based External Time Reference network to any STP Coordinated Timing Network.

Executive summary

The Server Time Protocol (STP) feature is designed to allow events occurring in different z9 EC, z9
BC, z990, and z890 servers, and Coupling Facilities (CFs), to be properly sequenced in time. STP is
designed to reduce or eliminate the requirement for Sysplex Timers. STP supports a multisite
sysplex up to 100 km (62 miles) without requiring an intermediate site. Previously, an intermediate
site was recommended to locate one of the Sysplex Timers, when the multisite sysplex distance
exceeded 40 km (25 miles).

Server Time Protocol

Server Time Protocol is a server-wide facility that is implemented in the Licensed Internal Code
(LIC) of z9 EC, z9 BC, z990, z890, and CFs and presents a single view of time to Processor
Resource/Systems Manager (PR/SM). STP uses a message-based protocol in which timekeeping
information is passed over externally defined Coupling Links  InterSystem Channel-3 (ISC-3) links
configured in peer mode, Integrated Cluster Bus-3 (ICB-3) links, and Integrated Cluster Bus-4
(ICB-4) links. These can be the same links that already are being used in a Parallel Sysplex® for
Coupling Facility (CF) message communication.

By using the same links to exchange timekeeping information and Coupling Facility messages in a
Parallel Sysplex, STP can scale with distance. Servers exchanging messages over short-distance
links such as ICB-3 and ICB-4 are designed to meet more stringent synchronization requirements than
servers exchanging messages over long-distance links such as ISC-3 (distances up to 100 km), where
the synchronization requirements are less stringent. This is an enhancement over the current
Sysplex Timer implementation, which does not scale with distance.

The STP design has introduced a new concept called Coordinated Timing Network (CTN). A CTN is a
collection of servers and Coupling Facilities that are time synchronized to a time value called
Coordinated Server Time. STP supports the following functions:

Initialize the time manually, including Time Zone offset, Daylight Saving Time (DST) offset, and
Leap seconds offset.

Initialize the time by dialing out to a time service so that Coordinated Server Time can be set to
within 100 milliseconds of an international time standard such as Coordinated Universal Time (UTC).

Schedule periodic dial-outs to a time service so that Coordinated Server Time may be gradually
steered to an international time standard.

Adjust Coordinated Server Time by up to +/- 60 seconds. This improves upon the Sysplex Timer's
capability of adjusting time by up to +/- 4.999 seconds.

Schedule changes to offsets listed above. STP can automatically schedule Daylight Saving Time,
based on the selected Time Zone, a capability that did not exist on the Sysplex Timer.

STP enhancements

Unlike the Sysplex Timer, which distributes time to multiple servers in a star pattern, STP
distributes time messages in layers, or stratums. In a timing network based on STP, the stratum
level is used as a means to define the hierarchy of a server in the timing network. A Stratum 1
server or CF is the highest level in the hierarchy in the STP network. A Stratum 2 server or CF
uses STP messages to synchronize to a Stratum 1 server. A Stratum 3 server or CF uses STP messages
to synchronize to a Stratum 2 server. STP supports up to three layers (Stratum 3).

Since STP can support a network of up to three stratums, it is possible to configure a timing
network that spans three or more sites. The maximum fiber distance from a Stratum 1 server to a
Stratum 2 server is 100 km, and the maximum distance from a Stratum 2 server to a Stratum 3 server
is also 100 km. This capability allows Parallel Sysplex configurations which may result in reducing
the Recovery Time Objective (RTO).

The Server Timer Protocol feature is designed to simplify hardware configurations by:

Allowing clock synchronization for z9 EC, z9 BC, z990, and z890 servers and Coupling Facilities
without requiring the Sysplex Timer and dedicated timer links. This helps reduce the need for
separate hardware that needs to be ordered and maintained.

Supporting a multisite timing network of up to 100 km (62 miles) over fiber optic cabling, thus
allowing a sysplex to span these distances. This overcomes the limitations of the Sysplex Timer to
Sysplex Timer links being supported only up to 40 km. STP is designed to eliminate the need for a
third site to locate the second Sysplex Timer for installations where the multisite sysplex spans a
distance greater than 40 km but less than 100 km (the current limit for Sysplex Timer to server
connection).

Potentially reducing the cross-site connectivity required for a multisite Parallel Sysplex.
Dedicated links are no longer required to transport timing information since STP and Coupling
Facility messages may be transmitted over the same links.

Additional value from Server Time Protocol may be obtained by:

Eliminating infrastructure requirements (space, power, and so forth) needed to support Sysplex
Timers

Eliminating Sysplex Timer maintenance costs

Eliminating fiber optic infrastructure requirements for:

Patch/trunk cables

Dense Wavelength Division Multiplexer (DWDM) ports

Dark fiber between sites for External Time Reference (ETR) and Control Link Oscillator (CLO) links
(links between Sysplex Timers)

Concurrent migration and coexistence

STP introduces the concept of a Coordinated Timing Network (CTN), to meet two key goals of System
z customers:

Concurrent migration from an existing External Time Reference (ETR) network to a timing network
using STP.

Capability of servers that cannot support STP to be synchronized in the same network as servers that
support STP (z9 EC, z9 BC, z990, z890).

A CTN can be configured in two ways:

Mixed CTN (ETR and STP), which requires a Sysplex Timer. The Sysplex Timer provides the timekeeping
information in a Mixed CTN.

A Mixed CTN allows you to meet the above requirements of concurrent migration and coexistence. The
coexistence allows for a sysplex configuration where members of the sysplex may reside on a z800 or
z900 server.

STP-only CTN, which does not require a Sysplex Timer.

Concurrent migration:
Continuous availability is a key business requirement for most enterprises. Systems that are
designed to deliver continuous availability combine the characteristics of high availability and
near continuous operations to deliver high levels of service  targeted at 24x7. High
availability is an attribute of a system that provides service at agreed upon levels and can mask
unplanned outages from end users. Near continuous operations, on the other hand, is the attribute
of a system designed to continuously operate and mask planned outages from end users.

With proper hardware, software, and connectivity planning, STP allows customers who want to migrate
from an existing ETR network to an STP CTN to do so concurrently, so that they can maintain
continuous operations during the migration process. Concurrent migration may be possible, where
supported, if you wish to go back from an STP CTN to an ETR network. This capability could be
useful if you wish to use a change window to do STP testing and then go back to a previous
configuration concurrently before the start of production time.

The following concurrent migrations are supported:

Concurrent migration from an ETR network to an STP-only CTN

Concurrent migration from an ETR network to a Mixed CTN

Concurrent migration from a Mixed CTN to an STP-only CTN

Concurrent migration from an STP-only CTN to a Mixed CTN

Concurrent migration from a Mixed CTN to an ETR network

Coexistence:
It is crucial that STP allows servers and Coupling Facilities (CFs) synchronized with the
Sysplex Timer and servers and CFs synchronized with Coordinated Server Time (CST) to coexist and be
synchronized to each other. A Mixed CTN allows this coexistence in a Parallel Sysplex only for z900
and z800 servers and CFs, which must be attached to the Sysplex Timer. The Sysplex Timer provides
the timekeeping information for both the ETR network and the Mixed CTN.

Non-raised-floor support for z9 BC

An IBM System z9 Business Class (z9 BC) feature may be ordered to allow use of the z9 BC in a
non-raised-floor environment. This capability may help ease the cost of entry into the z9 BC; a
raised floor may not be necessary for some infrastructures.

The non-raised-floor z9 BC implementation is designed to meet all electromagnetic compatibility
standards. Feature #7998 must be ordered if the z9 BC is to be used in a non-raised-floor
environment.

There are new ICB cables which must be used to support this configuration when connecting from z890
or z800 servers to a non-raised-floor z9 BC.

When connecting from a z890 to a non-raised-floor z9 BC using ICB-3 or ICB-4, the following ICB
cables are required:

ICB-3 Cable for NRF, feature #0231 (replaces feature #0227)

ICB-4 Cable for NRF, feature #0232 (replaces feature #0228)

When connecting from a z890 to a non-raised-floor z9 BC, the additional ICB-3 or ICB-4 cables must
be ordered on the z9 BC server.

When connecting from a z800 to a non-raised-floor z9 BC using ICB-3, the following cable is
required:

ICB-3 Cable for NRF, feature #0231 (replaces feature #0227)

When connecting from a z800, this ICB-3 cable must be ordered on the z9 BC server.

Accessibility by people with disabilities

A U.S. Section 508 Voluntary Product Accessibility Template (VPAT) containing details on the
product's accessibility compliance can be requested via IBM's Web site

Section 508 of the U.S. Rehabilitation Act

IBM System z9 is capable on delivery, when used in accordance with IBM's associated documentation,
of satisfying the applicable requirements of Section 508 of the Rehabilitation Act of 1973, 29
U.S.C. Section 794d, as implemented by 36 C.F.R. Part 1194, provided that any Assistive Technology
used with the product properly interoperates with it.

Server Time Protocol

The mainframe's use of z/Architecture provides a means by which programs can establish
time-of-day (TOD) and determine the ordering of serialized events such as updates to a database, a
log file, or other data structures. The TOD clock resolution must be sufficient so that every
consecutive TOD request, whether executed on different central processors (CPs) in the same server,
or on different servers, will produce increasing values. These unique time stamps can be used to
reconstruct, recover, and allow the ordering of serialized updates to shared data.

When operating in a clustered environment, an External Time Reference (ETR) is used to accommodate
the ordering of the time stamps across servers in a sysplex. Prior to the availability of STP, a
common timer called the IBM Sysplex Timer was required to keep the TOD clocks of all participating
systems in synchronism with each other to within a small number of microseconds, dictated by the
fastest possible passing of data from one server to another via a Coupling Facility (CF) structure.
The Sysplex Timer also provides for automatic setting of the time and date in the system without the
need for an operator to enter the time and date manually through the system console of each system.

The next generation of time synchronization, Server Time Protocol (STP), is being made available as
the follow-on to the Sysplex Timer, providing the same capabilities as the Sysplex Timer to meet the
z/Architecture requirements described above. STP is designed to provide the capability for multiple
System z9 EC, z9 BC, z990, and z890 servers and CFs to maintain time synchronization with each
other. STP is designed to provide time synchronization for servers that have been configured to be
in a Parallel Sysplex or a sysplex (without a Coupling Facility) as well as servers that are not in
a sysplex, but need to be time synchronized. STP is a message-based protocol in which timekeeping
information is passed over data links between servers. The timekeeping information is transmitted
over externally defined Coupling Links  InterSystem Channel-3 (ISC-3) links configured in peer
mode, Integrated Cluster Bus-3 (ICB-3) links, or Integrated Cluster Bus-4 (ICB-4) links.

STP can be ordered as a feature of the z9 EC, z9 BC, z990, or z890 server or Coupling Facility and
may eliminate the requirement for a Sysplex Timer.

STP will support a multisite timing network of up to 100 km (62 miles) over fiber optic cabling,
thus allowing a Parallel Sysplex to span these distances. This overcomes the limitation of Sysplex
Timer to Sysplex Timer links being supported only up to 40 km (25 miles).

Non-raised-floor for z9 BC

The System z9 Business Class (z9 BC) was specifically designed for small and mid-sized mainframe
environments. It is designed to offer a variety of growth options and excellent price/performance
for those environments requiring a lower capacity entry point and more granular growth than may be
available with the z9 EC. By introducing the non-raised-floor features of the z9 BC, entry into the
z9 BC may be eased for those infrastructures that can take advantage of the capability.

Enhanced Coordinated Server Time accuracy

IBM intends to enhance the accuracy of initializing and maintaining Coordinated Server Time to an
international time standard such as Coordinated Universal Time (UTC). The then current server is
planned to have the capability of attaching to an external time source, such as a Global Positioning
System (GPS) receiver.

Network Time Protocol (NTP) client support

IBM intends to enhance the STP design to provide Network Time Protocol (NTP) client capability, so
that Coordinated Server Time may be initialized and maintained to time provided by an NTP server.
The purpose of this function is to allow the same time across an enterprise comprised of
heterogeneous platforms.

All statements regarding IBM's plans, directions, and intent are subject to change or withdrawal
without notice. Any reliance on these Statements of General Direction is at the relying party's
sole risk and will not create liability or obligation for IBM.

Business Partner information

If you are a Direct Reseller - System Reseller acquiring products from IBM, you may link directly to
Business Partner information for this announcement. A PartnerWorld ID and password are required
(use IBM ID).

IBM Integrated Technology Services

IBM services include business consulting, outsourcing, hosting services, applications, and other
technology management.

These services help you learn about, plan, install, manage, or optimize your IT infrastructure to be
an On Demand Business. They can help you integrate your high-speed networks, storage systems,
application servers, wireless protocols, and an array of platforms, middleware, and communications
software for IBM and many non-IBM offerings. IBM is your one-stop shop for IT support needs.

For details on available services, contact your IBM representative or visit

Specified operating environment

Hardware requirements

Sysplex Timer® Licensed Internal Code upgrade:
The Sysplex Timer's Licensed Internal Code (LIC) has been upgraded to support using STP in
a Mixed Coordinated Timing Network (CTN). The required Sysplex Timer LIC is shipped along with the
STP feature, and must be installed by the IBM Service Support Representative prior to migrating from
a Sysplex Timer based External Time Reference (ETR) network to any STP CTN.

Hardware Management Console:
Management of STP-enabled servers requires V2.9.1 Licensed Internal Code (LIC) of the
Hardware Management Console (HMC) application. Previously installed HMCs (feature numbers 0075 and
later) can be upgraded to V2.9.1 LIC by requesting your IBM Service representative to order ECA 241
for each server that has one or more HMC features to be upgraded.

If an existing HMC is to be loaded with the new HMC LIC, the attached servers will first need to be
at specific driver levels and may need specific MCLs to be installed. These MCLs may be loaded
without an outage, as long as the server is at the driver level listed. If you need to upgrade the
server to the required driver level, an outage will be necessary.

z990 and z890 (Driver 55)

MCL 132 for EC J13486

z900 and z800 (Driver 3Gf)

MCL 194 for EC J11213

9672 G5, G6, and Multiprise® 3000 (Model H55 or H75 only) (Driver 26)

MCL 174 for EC F99918

MCL 009 for EC J11172

MCL 009 for EC J10392

MCL 006 for EC J11920

EC Levels and MCLs for STP support:
z9 EC, z9 BC, z990, and z890 servers and Coupling Facilities can be made STP-capable by
installing required STP Licensed Internal Code (LIC). The LIC may be installed concurrently. The
following are the STP LIC requirements:

z990 and z890

The z990 and z890 must be at EC Driver level 55. Prior to installing the STP feature number, have
your SSR download and install all of the latest MCLs that are available for this driver. Several
STP prerequisite MCLs will be installed as a result.

IBM System z9

The z9 EC and z9 BC must be at EC Driver level 63. Prior to installing the STP feature number, have
your SSR download and install all of the latest MCLs that are available for this driver. Several
STP prerequisite MCLs will be installed as a result.

After the MCLs are installed, the z9 EC, z9 BC, z990, and z890 become STP-capable servers. STP
functions cannot be used until the STP enablement feature is installed.

Enabling hardware feature for STP:
Feature number 1021 is required for your server or Coupling Facility to enable STP on your
hardware. This is a chargeable feature for each server and Coupling Facility.

Feature number 1021 provides the following:

Installation instructions are provided. The IBM Service Support Representative (SSR) will perform
this part of the installation.

A required Sysplex Timer EC LIC disk is provided. This LIC should be installed on any Sysplex Timer
where there is a migration planned from ETR network to any CTN configuration. This should be
installed prior to installing the server enablement LIC.

A required server or Coupling Facility MCL diskette or CD is provided. This MCL enables STP.

Feature number 1021 should be installed on any STP-capable server or Coupling Facility that will be
used in a Mixed CTN or STP-only CTN. STP enablement LIC can be loaded without disruption to current
operations. The loading of the enablement LIC is done via the server's Support Element.

The hardware requirements for the features and functions contained in this announcement are
identified.

Machine Change Levels (MCLs) are required.
Descriptions of the MCLs are available now through Resource Link.

Software requirements

Note:
z/OS®.e is exclusive to z9 BC, z890, and z800.

STP is available as a feature on z9 EC, z9
BC, z990, and z890 servers, and is supported by z/OS and z/OS.e V1.7 and above.

Even though the planned availability version of z/OS and z/OS.e V1.7 and V1.8 have function to
support STP, additional software maintenance is required. Maintenance being installed on the z/OS
and z/OS.e system image can be done using the "rolling IPL" process, which allows you to avoid a
sysplex-wide outage. The "rolling IPL" process allows coexistence of the following system images:

z/OS and z/OS.e V1.7 or above with the maintenance

z/OS and z/OS.e V1.7 or above without the maintenance

z/OS and z/OS.e V1.4 through z/OS and z/OS.e V1.6

Software support for timing-only links:
Timing-only links are Coupling Links that allow two servers to be synchronized using STP
messages, when a CF does not exist at either end of the Coupling Link. If timing-only links are
required for the configuration, then PTFs are also needed.

z/OS and z/OS.e V1.4 through z/OS and z/OS.e V1.6 considerations:
z/OS provides compatibility and flexibility as images migrate by allowing several releases
of z/OS and z/OS.e to coexist in a Sysplex or a Parallel Sysplex® multisystem configuration. z/OS
and z/OS.e V1.7 are compatibility supported with releases back to z/OS and z/OS.e V1.4. This means
that z/OS and z/OS.e V1.4 can coexist in the same sysplex as z/OS and z/OS.e V1.7. Although an
STP-only CTN must consist of only z/OS and z/OS.e V1.7 or later systems, a mixed-CTN can include any
coexistence-supported releases, as long as the toleration PTFs are installed. For example, z/OS and
z/OS.e V1.7 can coexist with z/OS V1.4 and z/OS.e in the same timing network in the same sysplex.

Software levels that support z9 BC will also support the non-raised-floor z9 BC.

You should consult the Preventive Service Planning (PSP) buckets for STP related maintenance. They
are listed in the 2084DEVICE, 2086DEVICE, 2094DEVICE, and 2096DEVICE PSP buckets for the z990, z890,
z9 EC, and z9 BC, respectively.

To simplify the identification of PTFs needed to support Server Timer Protocol, a functional PSP
bucket has been created. You can use the Enhanced Preventive Service Planning Tool (EPSPT), which
is available on the "Technical Help Database for Mainframe Preventive Service Planning Buckets" Web
site

and the Enhanced PSP Tool to reconcile the recommended service for
the STP function and the hardware device against your systems target
zones. Detailed information on the EPSPT is available on the IBM
Education Assistant Web site

Hardware Management Console (HMC):
The HMC is for the exclusive use of the HMC application.
Customer applications cannot reside on the HMC. The ESCON® Director
and Sysplex Timer applications cannot reside on the HMC. TCP/IP is the
only supported communication protocol. The HMC supports z9 EC and z9 BC.
It may also be used to support z990, z890, z900, z800, G5, G6, and
Multiprise 3000 servers provided it is upgraded to a new AROM level.

Planning information

Customer responsibilities

Multisite extended distance considerations:
At STP planned availability, Dense Wavelength Division Multiplexer (DWDM) products are
planned to be qualified to support transporting STP messages over ISC-3 links in a multisite sysplex
configuration. The list of qualified DWDM products that support STP may be found on Resource Link

ICB-3 cables (#0227) or ICB-4 cables (#0228) may be used to carry STP
traffic.

ICB-3 cables for non-raised-floor NRF (#0231) or ICB-4 cables for NRF
(#0232) may be used to carry STP traffic.

ISC-3 cables, customer-supplied 9 micron single mode fiber optic cables
terminated with an LC Duplex connectors, may also be used to carry STP
traffic.

Security, auditability, and control

The z9 EC, z9 BC, z990, and z890 use the security and auditability features and functions of host
hardware, host software, and application software.

The customer is responsible for evaluation, selection, and implementation of security features,
administrative procedures, and appropriate controls in application systems and communications
facilities.

IBM Global Services has transformed its delivery of hardware and software support services to put
you on the road to higher systems availability. IBM Electronic Services is a Web-enabled solution
that provides you with an exclusive, no-additional-charge enhancement to the service and support
available on the IBM eServer® platform. These services provide the opportunity for greater system
availability due to faster problem resolution and preemptive monitoring. IBM Electronic Services is
comprised of two separate, but complementary, elements: IBM Electronic Services news page and IBM
Electronic Service Agent.

IBM Electronic Services news page provides you with a single Internet entry point that replaces the
multiple entry points traditionally used by customers to access IBM Internet services and support.
The news page enables you to gain easier access to IBM resources for assistance in resolving
technical problems.

The IBM Electronic Service Agent is no-additional-charge software that resides on your IBM eServer
system. It is designed to proactively monitor events and transmit system inventory information to
IBM on a periodic, customer-defined timetable. The IBM Electronic Service Agent tracks system
inventory, hardware error logs, and performance information. If the server is under a current IBM
maintenance service agreement or within the IBM warranty period, the Service Agent automatically
reports hardware problems to IBM. Early knowledge about potential problems enables IBM to provide
proactive service that may result in higher system availability and performance. In addition,
information collected through the Service Agent will be made available to IBM service support
representatives when they are helping answer your questions or diagnosing problems.