Data Acquisition and Quality Control

Stations from most networks operated by the BSL transmit data
continuously to the BSL facilities on the UC Berkeley campus
for analysis and archive. In this chapter, we describe activities
and facilities which cross-cut the individual networks described
in Chapters 38, 40 and 41, including
procedures for data acquisition and
quality control, and sensor testing capabilities and procedures.
This year the computer and
networking
facilities used for data acquisition moved from McCone Hall
to the University's seismically safe building at 2195 Hearst Ave.

While some of these activities are continuous from year to year,
we have identified changes or activities which are specific to
2005-2006.

Until 2005-2006, both the BSL staff monitoring routine data acquisition,
and the computers and facilities to
acquire, process, and archive the data were situated
in McCone Hall. There the BSL has facilities
designed to provide air conditioning, 100-bit switched network, and
reliable power with UPS and generator. During this year,
the computers and telemetry equipment associated with data collection
and archival were moved to the new campus computer facility
in 2195 Hearst Avenue.

After several years of actively working with the campus, the BSL
has finally relocated the infrastructure supporting the critical
operations of data acquisition, processing, archiving, and data distribution
to a more robust facility than McCone Hall. With assistance from the Office
of the Vice Chancellor for Research, the BSL has been granted space in 2195
Hearst, a recently completed building on the Oxford Tract. 2195 Hearst was
constructed to current seismic codes, and the hardened campus computer
facility within was designed with special attention for post-earthquake
operations. The computer center contains state-of-the art seismic bracing,
UPS power and air conditioning with generator backup, and extensive security
and equipment monitoring.

During 2005-2006, the BSL completed the relocation of equipment
to the new facilities in
2195 Hearst. This includes all of its data acquisition and
real-time earthquake processing computers, as well as
the data archive and distribution
computers. Following the computer move, all telemetry
equipment (5 T1s lines, dedicated leased phone circuit to our paging service,
dialin/dialout modems, as well as various radio and VSAT communication
equipment) were also transferred to the new location over the course of
several months. The final elements were moved in February, 2006.
During the transition, the private network used for
seismic data acquisition and earthquake processing was temporarily bridged
between McCone Hall and 2195 Hearst using an encrypted tunnel across the
campus backbone network.

In the past, mission-critical earthquake monitoring and
review processes ran on several computers in McCone
Hall. Thus, these computer systems run on circuits
with both UPS and generator power. Air conditioning is provided
through both "building air" and two additional AC units.
Over the years, the BSL has experienced problems with both
the McCone generator system and the air conditioning.

With the move of many BSL and NCEDC operations servers to
the campus computer center at 2195 Hearst (SRB1), our generator
power and air conditioning resources in the BSL server
room in 237 McCone have better matched our needs
over the past year. The BSL generator and UPS battery
system supported servers during one brief power outage
this year. The air conditioning systems for room 237
required maintenance and some parts replacement, but no
serious problems resulted during these events. The BSL generator
is maintained by Physical Plant Capital Services and was run
without load twice monthly.

BSL is developing a long range plan with UCB Communications
Network Services (CNS), a division of Information Services and Technology,
to replace the generator with a larger, 100 kW unit, and to upgrade
the UPS battery backup systems. This joint project
is designed to provide generator/UPS power for the two CNS-operated
network equipment closets serving all of McCone Hall, in addition to
providing emergency power to the BSL suite and the BSL engineering
lab in room 298 McCone. BSL and CNS will present this plan
to the Vice Chancelor Academic Council for approval and
assistance with funding for this proposal.

Central-site data acquisition for the BDSN/NHFN/MPBO is performed by two
computer systems located at the BSL (Figure 44.1).
These acquisition systems are also
used for the Parkfield-Hollister electromagnetic array and for the BARD
network. A third system is used primarily as data exchange system with the
USNSN and transmits data to the USNSN from HOPS,
CMB, SAO, WDC, HUMO, MOD, MCCM, and YBH. Data acquisition for the HRSN follows a more
complicated path, as described in Chapter 41.

Figure 44.2:
Dataflow in the REDI processing environment,
showing waveform data
coming in from the Quanterra data loggers (Q) into comserv. From
comserv, data are logged to disk (via datalog), distributed to other
computers (mserv), fed into the CDA for REDI processing, and spooled
into a trace ring for export.

The BSL uses the comserv program for central data acquisition,
which was developed by Quanterra. The comserv program receives
data from a remote Quanterra data logger, and redistributes the data to one
or more comserv client programs. The comserv clients used by REDI
include datalog, which writes the data to disk files for archival
purposes, cdafill, which writes the data to the shared memory
region for REDI analysis, and other programs such as the seismic
alarm process, the DAC480 system, and the feed for the
Memento Mori Web page (Figure 44.2).

The two computers that perform data acquisition also serve
as REDI processing systems. In order to facilitate REDI processing,
each system maintains a shared memory region that contains the most
recent 30 minutes of data for each channel used by the REDI analysis
system. All REDI analysis routines first attempt to use data in the
shared memory region, and will only revert to retrieving data from
disk files if the requested data is unavailable in the shared memory region.

Each BDSN datalogger that uses frame relay telemetry is configured to enable
data transmittion simultaneously to two different computers over two different frame
relay T1 circuits to UCB. However, the BSL normally actively enables and uses only one
of these data stream from each station at any given time.
The comserv client program cs2m receives
data from a comserv and multicasts the data over a private ethernet. The
program mcast, a modified version of Quanterra's comserv
program, receives the multicast data from cs2m, and provides
a comserv-like interface to local comserv clients. This allows each
REDI system to have a comserv server for every station, and each of the two
systems have a complete copy of all waveform data.

We have extended the multicasting approach to handle data received from other
networks such as the NCSN and UNR. These data are received by Earthworm
data exchange programs, and are then converted to MiniSEED and multicast
in the same manner as the BSL data. We use mserv on both REDI
computers to receive the multicast data, and handle it in an identical
fashion to the BSL MiniSEED data.

In 2006, the BSL established a real-time data feed of all BSL waveform between
the BSL acquisition systems and the NCEDC computers using the open source
Freeorb software. This allows the NCEDC to provide near-real-time access to
all BSL waveform data through the NCEDC DART (Data Availabile in Real Time)
system.

BSL seismic data are routinely monitored for state-of-health. An
automated analysis is computed weekly
to characterize the seismic noise level recorded by each broadband
seismometer. The estimation of the Power Spectral Density (PSD)
of the ground motion recorded at a seismic station provides an objective
measure of background seismic noise characteristics over a
wide range of frequencies. When used routinely, the PSD
algorithm also provides an objective measure of seasonal
and secular variation in the noise characteristics and
aids in the early diagnoses of instrumental problems.
A PSD estimation algorithm was developed in the early
1990's at the BSL for characterizing
the background seismic noise and as a tool for quality control.
As presently implemented, the algorithm sends the results via email
to the engineering and some research staff members and generates
a bargraph output which compares all the BDSN
broadband stations by components. A summary of the
results for 2005-2006 is displayed in Figure 38.2. Other
PSD plots for the NHFN, HRSN, and MPBO are shown in
Figures 40.2, 41.3,
respectively.

Four years ago, we expanded our use of the weekly
PSD results to monitor trends in the noise level at each station.
In addition to the weekly bar graph, additional figures showing the
analysis for the current year are produced. These cumulative PSD
plots are generated for each station and show the noise level in
5 frequency bands for the broadband channels. These cumulative
plots make it easier to spot certain problems, such as failure of
a sensor. In addition to the station-based plots, a summary plot for
each channel is produced, comparing all stations. These figures
are presented as part of a noise analysis of the BDSN on the WWW
at http://www.seismo.berkeley.edu/seismo/bdsn/psd/.

In addition to the PSD analysis developed by Bob Uhrhammer, the BSL has
implemented the Ambient Noise Probability Density Function (PDF) analysis
system developed by McNamara and Buland (2004). This
system does its noise analysis over all the data of a given time period (week
or year), including earthquakes, calibration pulses, and cultural noise. This
is in contrast to Bob Uhrhammer's PSD analysis, which looks at only the
quietest portion of data within a day or week. Pete Lombard of the BSL
extended the McNamara code to cover a larger frequency range and support
the many different types of sensors employed by the BSL. Besides the
originally supported broadband sensors, our PDF analysis now includes surface
and bore-hole accelerometers, strain meters, and electric and magnetic field
sensors. These enhancements to the PDF code, plus a number of bug fixes, were
provided back to the McNamara team for incorporation in their work.
The results of the PDF analysis are presented on the web at
http://moho/seismo/PDF/. Figure 44.3 shows noise analysis
results for a typical week. We review these plots as part of our assessment
of station health.

Figure 44.3:
Noise analysis results for the week of 07/02/06 at
the newest BDSN
station MCCM, on the BHZ
component. The prominent feature at short periods are produced by
waves from the nearby earthquake off-shore
of Fort Ross, California,
(2006/07/06, 20:43 UTC; 3.7). At long periods, the
surface waves of a 6.6 earthquake in the Aleutian Islands
(2006/07/08, 20:40 UTC) dominate the spectrum.

The BSL has set up an instrumentation test facility in the Byerly
Seismographic Vault in order to systematically determine and to compare
the characteristics of up to eight sensors at a time. The test
equipment consists of an eight-channel Quanterra Q4120 high-resolution
data logger and a custom interconnect panel that provides isolated
power and preamplification, when required, to facilitate the connection
and routing of signals from the sensors to the data logger with
shielded signal lines. This year
a GPS rebroadcaster was installed, so that
all data loggers in the Byerly vault will operate on
the same time base. Upon acquisition of the 100 samples-per-second
(sps) data from the instruments under test, PSD analysis and spectral
phase coherency analysis are used to characterize and compare the
performance of each sensor. Tilt tests and seismic signals with a
sufficient signal level above the background seismic noise are also
used to verify the absolute calibration of the sensors. A simple
vertical shake table is used to assess the linearity of a seismic
sensor.

The sensor testing facility of the BSL is described in detail in the
2001-2002 Annual Report.

In February 2006, we embarked on a project to develop
new electronics for the STS-1 very broadband seismometer. This is a
collaborative project with
Tom VanZandt of Metrozet, LLC (Redondo Beach, CA) and is funded by a
grant from NSF through the
IRIS/GSN program.

The STS-1 VBB (Wielandt and Streckeisen, 1982; Wielandt and
Steim, 1986), widely viewed as the finest VBB sensor in the world,
is currently the principal broad-band seismometer used by the
Incorporated research Institutions for Seismology (IRIS) Global
Seismographic Network (GSN), GEOSCOPE, and several other global
or regional seismic networks operated by members of the
Federation of Digital Broad-Band Seismograph Networks (FDSN).
The installed base (approximately 750 sensor axes) represents a
very significant international investment for low frequency
seismology. The BDSN includes 10 STS-1's in its network.
Unfortunately, many of the STS-1 seismometers, which were manufactured
and installed 10-20 years ago, are encountering both operational
failures and age-related errors (Ekström and Nettles, 2005).
This problem
is exacerbated by the fact that sensors are no longer being
produced or supported by the original manufacturer, G.
Streckeisen AG (Pfungen, Switzerland). The nature and severity
of this problem has been discussed widely. For example, a report
from a recent broadband seismic sensor workshop (Ingate et al, 2004)
highlights the unique value of the installed base of STS-1
sensors, as well as the current lack of replacements with
equivalent long period performance. In the absence of focused
action by the seismological community, the state-of-health of
the existing STS-1 instruments will continue to decline.
Numerous efforts, both commercial and government-funded, are
underway to develop future replacements (IRIS Workshop, 2004).
Regardless of how one views the potential of these new approaches
to delivering a manufacturable, STS-1-equivalent product, given the
present funding environment, it is clear that they all would mandate
outright replacements of the existing STS-1 sensors.

In collaboration with its commercial partner, Metrozet, LLC
(Redondo Beach, CA), the BSL is developing and testing new
electronic hardware, and methods for mechanical repair, for
the STS-1. The intent of this effort is to develop simple and
economical long-term solutions to current and anticipated
problems with the existing STS-1 sensors. A primary aim is
to develop a fully-tested, modern electronics module that will
be a drop-in replacement for the original electronics. This
will provide users with a legitimate option for replacing old
modules that are no longer functioning. This new electronics
design will address environmental packaging problems that have
led to operational errors and failures in the existing instruments.
This effort will also provide the opportunity to implement a set of
electronic improvements that will make the installation and
operation of the sensors more efficient.

In the first half of 2006, Metrozet developed the first
prototype and reverse engineered electronics for the STS-1,
while the BSL engineering staff
constructed a test-bed at the Byerly Vault (BKS) and developed the
capability to simultaneously test 6-8 STS-1 components. Much time was
spent locating spare STS-1's and the associated environmental shields and
bringing them back to Berkeley.

Doug Neuhauser, Bob Uhrhammer, Peggy Hellweg, Pete Lombard, and Rick
McKenzie are involved in the data acquisition and quality control of
BDSN/NHFN/MBPO data. Development of the sensor test facility and
analysis system
was a
collaborative effort of Bob Uhrhammer, Tom McEvilly, John Friday, and
Bill Karavas. IRIS and DTRA provided, in part, funding for and/or
incentive to set up and operate the facility and we thank them for
their support. Bob Uhrhammer, Peggy Hellweg, Pete Lombard
Doug Neuhauser and Barbara Romanowicz contributed to the
preparation of this chapter. The STS-1 project is funded by NSF through the
IRIS/GSN program.