Bullseye! Tokyo Stock Exchange hits the mark with "arrowhead"

Throughout much of the past year Automated Trader readers have been in frequent contact with the editorial team with questions about the Tokyo Stock Exchange’s forthcoming “arrowhead” trading platform. The platform went live at the beginning of 2010, so we recently seized the opportunity to catch up with Yuichiro Yamamoto, Manager of the exchange’s IT Development Department, shortly after “arrowhead” completed its first quarter of live trading. Automated Trader’s Founder, Andy Webb, spoke to Mr Yamamoto about “arrowhead” as well as the exchange’s philosophy regarding automated trading and its future technology plans.

Yuichiro Yamamoto

Andy Webb: How did the "arrowhead" project come
about?

Yuichiro Yamamoto: By 2003 it was apparent that
our legacy exchange matching system for equities was experiencing
capacity constraints. The TSE embarked on a series of upgrades to
enhance capacity, but it was nevertheless apparent that a new
trading platform would ultimately be required.

The formal planning and request for proposal (RFP) process for
this new platform began in 2006, but prior to that we visited
numerous exchanges around the world to assess the available
exchange technology already deployed in the market place. Our
objective was to identify the most cutting edge technology
currently in use, but also to extrapolate that information to
gain an insight into probable future developments and
requirements.

We announced the RFP and vendor selection process in August 2006,
before selecting Fujitsu Corporation as the developer for our
next generation trading system in December 2006. Exchange staff
(with input from market participants) then worked closely with
Fujitsu over the next three years in developing "arrowhead",
which subsequently went live on January 4th 2010.

Andy Webb: Did your selection of Fujitsu as
developer imply that the TSE had already made a conscious policy
decision to build rather than buy the new trading platform?

Yuichiro Yamamoto: No, not at all. We opened the
door to all categories of vendor for competitive tender, but we
simply took the view that Fujitsu's proposed solution offered the
best overall package.

Andy Webb: What does the "arrowhead" platform
consist of?

Yuichiro Yamamoto: The platform was designed as
an integrated whole, with Fujitsu providing both the hardware as
well as the software. (See Figure 1 for a breakdown of the
hardware and middleware used in "arrowhead".) The matching engine
runs on Fujitsu PRIMEQUEST mission-critical servers, which use
Intel Itanium 64-bit processors.

"arrowhead" runs on Red Hat Enterprise Linux (RHEL), which we
were already using for our derivatives trading system. As RHEL
had delivered reliable and rapid performance in that role, we
opted to use it again for the stock trading platform. A further
consideration was RHEL's easy scalability, which was an important
part of the design of "arrowhead".

Hardware

PRIMEQUEST (mission-critical IA server)

PRIMERGY (industry standard x86 server)

ETERNUS (disk storage system for SAN environment)

IPCOM (network server)

SR-S (secure switch)

Si-R (IP access router)

Middleware

(PRIMESOFT)

Ultra-high-speed data management software

Interstage (business application platform software)

Symfoware (high-reliability database management software)

Systemwalker (enterprise system management software)

PRIMECLUSTER (clustering software)

ETERNUS SF (storage management software)

Figure 1: Overview of hardware and middleware in "arrowhead"

The platform's primary and secondary operating sites are
connected to the exchange's "arrownet" optical ring network,
which also links to the primary and secondary exchange offices
and of course market participants. (See Figure 2 - TSE Systems
Schematic Diagram.)

Andy Webb: How are system maintenance and
operational roles divided between Fujitsu and TSE personnel?

Yuichiro Yamamoto: As I mentioned earlier, the
system was built by the TSE in close collaboration with Fujitsu
personnel. Now "arrowhead" is live, maintenance and any system
enhancements (such as server cluster expansion) are the
responsibility of Fujitsu, while TSE personnel handle day-to-day
system operation.

Andy Webb: In view of the rate of order volume
growth in modern electronic markets, scalability is presumably
crucial. How flexible is the technology in this respect?

Yuichiro Yamamoto: "arrowhead" consists of four
primary business functions:

Order gateways through which market participants submit
orders

The order matching engine

The market data distribution system

Market surveillance

Each of these functions has its own dedicated server cluster
(though the functions are linked at the software level by the
PRIMESOFT middleware). These separate clusters mean that upgrades
to an individual business function can be undertaken without
disturbing the other functions. We maintain "arrowhead" with
double the capacity required to handle predicted peak volumes.
Nevertheless, if a hardware upgrade to any cluster is required,
this takes at most one week - so we can respond very rapidly to
any increases in activity.

Figure 2: TSE Systems Schematic Diagram

Andy Webb: How does the platform perform in
terms of latency?

Yuichiro Yamamoto: It typically takes less than
two milliseconds for an order to complete the round trip between
a gateway server and the matching engine. (In other words, for an
order message from a gateway server to hit the matching engine
and for the gateway server to receive an order acknowledgement
back from the matching engine.) In the case of participants using
the TSE's co-location service, the typical round trip between the
participant's co-location server and the gateway server is around
150 microseconds.

Andy Webb: What about redundancy of critical
systems?

Yuichiro Yamamoto: "arrowhead" keeps transaction
data resident in memory, which allows data access in the very low
microsecond range. These memory-resident transaction data are
processed in parallel on triply redundant servers, so if one
server fails the switchover to another server will occur in
seconds without any data loss.

Andy Webb: Apart from improving speed and
resilience in the trading process, what other implications has
the introduction of the new platform had on the TSE trading
process?

Yuichiro Yamamoto: In addition to accelerating
the distribution of market information, it has also allowed us to
significantly expand the depth of market data provided to market
participants. For example, the range of depth of book quotes
above/below the currently traded price provided by the FLEX
Standard data feed has been increased from five to eight, while
the FLEX Full feed distributes all order information on all
issues in real time.

We also took the opportunity with the launch of "arrowhead" to
introduce a number of other innovations to the quoting/trading
process. These include sequential trade quotes, implemented
revisions to bid/offer price limits and special quote renewal
price intervals, as well as order-matching conditions in itayose
call auctions, which are all intended to promote smoother
transaction flow and enhanced liquidity.

Taken as a whole, we believe that these changes - in conjunction
with the implementation of "arrowhead" - also facilitate the
efficient use of execution algorithms on the TSE.

Andy Webb: Would you consider selling
"arrowhead" technology to other exchanges?

Yuichiro Yamamoto: Yes, we would be very happy
to provide our technology to other exchanges. To date it has
proven robust, with no outages or disruption since the launch
date. Apart from the technology itself, the development process
has been a valuable learning process for us that we could share
with other exchanges. For example, as part of the project we had
to develop an extremely detailed technical requirements document.
While this was extremely challenging to accomplish, it ultimately
resulted in robust technology that was implemented in accordance
with the go live deadline.

Andy Webb: What is the exchange's attitude to
high frequency trading and what controls do you have in place to
counter the "message flooding" that some other exchanges have
experienced?

Yuichiro Yamamoto: We welcome high frequency
traders and believe that the capacity and speed of "arrowhead"
combined with the introduction of our co-location services will
facilitate their business. However, we do not offer sponsored
access and at present have no plans to do so.

As regards control, we can monitor and manage the traffic on each
participant's gateway server; this forms part of the surveillance
business function of "arrowhead" referred to earlier. We also
apply a fixed maximum limit to each gateway in terms of orders
per second so as to prevent message flooding.

Andy Webb: Have the recent technology changes
driven a significant increase in the number of foreign
participants in the Tokyo market?

Yuichiro Yamamoto: It is difficult to say for
certain, as it is the securities firms that have a direct
connection to the exchange rather than end participants. However,
the discernible change in trade and message volumes since the
change of "arrowhead" might suggest that an increasing number of
overseas participants are now trading on the exchange.

Andy Webb: How so?

Yuichiro Yamamoto: The number of completed
trades has remained broadly speaking stable, but there has been
an increase (though not dramatic) in the number of order messages
being transmitted. This would be consistent with a growing number
of order modifications or order slices that might suggest the
presence of automated and algorithmic trading.

Andy Webb: What has been the market response to
the TSE's co-location service?

Yuichiro Yamamoto: The co-location service
opened to market participants in May 2009 for options and
futures, with connectivity for equities, ETFs and REITs being
made available as from January 4th of this year. Demand has been
strong, so while we still have sufficient capacity to accommodate
further participants, we are currently expanding the available
co-location space to allow for future demand.

Andy Webb: Does "arrowhead" cover all the
instruments traded on the TSE?

Yuichiro Yamamoto: No it only trades cash
stocks, convertible bonds and exchange traded funds. For stock
options we use a version of Euronext LIFFE's options trading
platform that went live last October. For futures we currently
use a system that was developed in house, but our intention is to
make a strategy decision regarding the development of a next
generation futures trading system this year. Once this is
complete, it will complete the replacement of all the exchange's
core trading infrastructure.