Our goal with Trapeze and Slice is to push the performance bounds for network
storage systems using Myrinet and similar networks.
Although our work with Slice is preliminary, our initial prototype
shows the performance that can be achieved with network storage
systems using today's technology and the right network support.

Figure 4 shows read and write bandwidths from disk for
high-volume sequential file access through the FreeBSD read and write
system call interface using the current Slice prototype. For these tests,
the client was a DEC Miata (Personal Workstation 500au) with a
500 MHz Alpha 21164 CPU and a 32-bit 33 MHz PCI bus using the Digital 21174 ``Pyxis''
chipset. The block I/O servers have a 450 MHz Pentium-III on an Asus P2B
motherboard with an Intel 440BX chipset, and either of two
disk configurations: (1) four Seagate Medalist
disks on two separate Ultra-Wide SCSI channels, or (2)
four IBM DeskStar 22GXP drives on separate Promise Ultra/33
IDE channels. All machines are
equipped with Myricom LANai 4.1 SAN adapters, and run kernels built from
the same FreeBSD 4.0 source pool. The Slice client uses a simple
round-robin striping policy with a stripe grain of 32KB. The test program
(dd) reads or writes 1.25 GB in 64K chunks, but it does not touch
the data. Client and server I/O caches are flushed before each run.

Each line in Figure 4 shows the measured I/O bandwidth
delivered to a single client using a fixed number of block I/O servers.
The four points on each line represent the number
of disks on each server; the x-axis gives the total number of disks used
for each point.
The peak write bandwidth is 66 MB/s; at this point the client CPU
saturates due to copying at the system call layer.
The peak read bandwidth is 97 MB/s. While reading at 97 MB/s the client CPU
utilization is only 28%, since FreeBSD 4.0
avoids most copying for large reads by page remapping at the system call layer.
In this configuration, an unpredicted 8KB page fault from I/O server memory
completes in under 150 s.

We have also experimented with TCP/IP communication using Trapeze.
Recent point-to-point TCP bandwidth tests (netperf) yielded a peak bandwidth
of 956 Mb/s through the socket interface on a pair of Compaq XP 1000
workstations using Myricom's LANai-5 NICs. These machines have a 500 MHz
Alpha 21264 CPU and a 64-bit 33 MHz PCI bus with a Digital 21272
``Tsunami'' chipset, and were running FreeBSD 4.0 augmented with page
remapping for sockets [7].