Project Summary

View and print project details including project summary, purpose, associations to Biological Opinions,
and area. To learn more about any of the project properties, hold your mouse cursor over the field label.

Note 1: The StreamNet contract is being revised mid year to incorporate changes in data type and data service priorities as determined through a regional data management workshop and from guidance by CBFWA. This revision will result in shifts in effort among the various types of data served by the project and services provided to data sources and data users. There is no change in scope of the project or in the total budget.

Note 2: Project costs have, in most cases, been allocated equally across all individual titles within each Work Element, since it is not possible to accurately separate them for the large number of individual titles and the fact that in some cases given work effort (cost) can address several titles at the same time.

StreamNet is a cooperative, multi-agency data compilation and data management project authorized by the Northwest Power Planning Council's Fish and Wildlife Program (FWP). It is funded primarily by the Bonneville Power Administration (BPA) through the FWP as part of its program to protect, mitigate, and enhance fish and wildlife resources affected by the development and operation of hydroelectric facilities on the Columbia River and tributaries. Other funding has also been obtained in the past from agencies including the Environmental Protection Agency and the National Marine Fisheries Service.

The Pacific States Marine Fisheries Commission (PSMFC) administers the project. PSMFC staff are also responsible for the regional components of the project, including maintaining the regional database, assuring regional data standardization, making data available in Geographic Information System (GIS) formats, building and operating Internet based data delivery systems, and posting the data for public access.

Three fourths of the project consists of sub-projects within the state fish and wildlife agencies, Columbia River Intertribal Fish Commission and US Fish and Wildlife Service to develop data and databases within the respective agencies and facilitate data transfer in regionally consistent format. In addition to administratively housing the StreamNet sub-projects, these cooperating agencies also contribute in kind support. The kind and amount of support varies between agencies. All agencies provide at least some salary support for their respective StreamNet Project Leader (from one or two months to full time). Several agencies contribute use of servers or other computer equipment and services that are not covered by charges for indirect costs. All contribute time by biologists and in some cases data entry staff to provide data to the project.

StreamNet functions to obtain, standardize, georeference and disseminate fish related data. It focuses on the kinds of data primarily collected by the state, tribal and federal fisheries management agencies for use in management and research programs. Work is organized around eight BPA defined Work Elements:

Council recommendation:
Fund as proposed with the following supplemental recommendations through FY 2013:

- Data access under this work should continue to evolve towards a more accessible platform for various users and optimize dynamic web-services to facilitate coordinated data-sharing and data depiction.

- As feasible, this work should expand to include additional managers (and data collecting entities3) that currently cannot easily provide access to their data, whether raw or synthesized, to improve accessibility to their data.

- StreamNet should strive to be a comprehensive data portal (e.g. linking to and depicting data from other sources etc.) for locating fish data needed to inform Program implementation and broad Program evaluation, emphasizing on using web-services. With respect to salmonid fish data, data collectors could provide their data directly to StreamNet while non-salmonid fish data could be made accessible to StreamNet through web-services from resident fish databases or a resident fish data portal.

- Data stored and accessed through StreamNet should include synthesized information, e.g. population estimates, needed for informing Program implementation and broad Program evaluation.

- Data made accessible through StreamNet should focus on data funded by Bonneville and priority data for the program. Identification of Bonneville funded projects that collect fish data should be based on project information available at cbfish.org.

- As necessary, prioritization of Bonneville funded data should be informed by Bonneville and Council’s evaluation and reporting needs for the program (e.g., ISRP retrospective reports, Report to Congress, and HLI reports), and Bonneville FCRPS BiOp reports. Furthermore, if the PERC moves forward, it would be expected that the council recommendations based on the guidance from this committee would be incorporated in this work.

- Sponsor to participate on the PERC as requested by the Council to assist in developing recommendations of the PERC.

Resolve issues concerning Deliverable #2 (update existing StreamNet datasets), as follows: (1) StreamNet proposes to stop updating or to provide only opportunistic updating of some of its primary datasets for an unspecified number of years until data collection activities for the Coordinated Assessment (CA) project are completed. The sponsors need to clarify how this will this impact the Council's Fish and Wildlife Program and other projects and programs that require updated StreamNet datasets to complete their work; (2) A regional discussion on which (if any) data types should be permanently dropped from StreamNet needs to be held; and (3) The sponsors need to clarify whether derived value data being collected for the Coordinated Assessments project meet the needs for reporting High Level Indicators (HLIs) for viable salmonid population (VSP) parameters.

Design and implement a plan for internal and external effectiveness monitoring. Previous ISRP reviews cited "Lack of clarity of who uses StreamNet, site use, and user satisfaction." The sponsors responded that "Site usage and use by agency is reported annually in our annual reports" and that it is difficult to assess satisfaction because it is used over the internet. A very strong rationale for any project is that it is achieving its objectives, and it is important to assess how well StreamNet is meeting the needs of agencies, tribes, and other users. The ISRP suggests that the sponsors provide two letters of reference from each agency working with StreamNet, one from the administrative level and the other from the staff level, outlining progress, improvements, limitations and shortcomings of the approach, and whether alternative forums or approaches might better meet agency needs.

Provide a report describing in detail the data quality assurance and quality control (QA/QC) procedures used by StreamNet. In the FY 2007-09 review, the ISRP encouraged the sponsors "to complete the draft document describing QA/QC procedures soon." In this proposal, the sponsors state, "We hope to develop a report describing the entire QA/QC process more fully in the future." The lack of well-documented QA/QC procedures reduces confidence in the quality of StreamNet datasets and data management systems.

The StreamNet project has clearly played an important role in providing information to regional programs and meeting regional objectives. However, the proposal would be improved by inclusion of a description and evaluation of project significance with respect to regional coordination of data management under the Council's Draft Monitoring Evaluation Research and Reporting Plan, Subbasin Plan objectives, Council's 2006 Research Plan, and other regional plans.

The problem statement focuses on the difficulty that StreamNet has in acquiring data from the Basin's management agencies. If regional data networks develop, however, the need for a central facility like StreamNet may decrease. A long-term goal is to move regional data dissemination toward a distributed “Exchange Network” model. The proposal would be improved by a more detailed description of this model. As those capabilities are developed, StreamNet is working with the Coordinated Assessment project to provide critical metrics, and the ISRP supports this effort.

The primary objective of StreamNet, to provide easily assessable regional data for agencies and others, is highly important. StreamNet appears to be succeeding in this objective, but it could have provided statistics on numbers of users of the database from various agencies (see Qualification #2). Although these data are reportedly located in annual reports, these data should have been summarized in the proposal.

The StreamNet website encourages submission of datasets that may be of interest to others. Although the proposal mentioned a number of other dataset projects, it was not clear to what extent StreamNet datasets might overlap with other datasets that are made available online. The proposal would be improved by inclusion of a dataflow chart showing how all of the datasets and database organizations integrate among themselves and avoid duplication.

The sponsors note that Pisces has some errors in their accounting of project reports, and that they have had difficulty in meeting deadlines for the annual reports and quarterly status reports because "input has to be obtained and consolidated from six subcontracting agencies."

The proposal lists a number of accomplishments, and it describes how it has responded to previous deficiencies. They collaborate with agencies to help standardize data collection so that the data can be stored more readily. StreamNet has responded to most past ISRP comments (see Qualification #3), and they are attempting to fix problem areas as described in detail in the proposal.

It would be useful to obtain information from active agency participants on StreamNet about the percentages of data from their agencies that find their way into the program each year, as well as their perceptions of how successful and useful the project is and what can be done to increase the quantity and quality of data entered (see Qualification #2).

StreamNet has focused on maintaining and updating a set of fish related data over many years and seems to have done a good job with those data and the information technology available. Better results could have been obtained if data storage in StreamNet had been viewed as higher priority among the states and tribes.

One important negative result relevant to the ISRP’s ongoing resident fish review is that provision of resident fish data to StreamNet is still not an agency priority. The sponsors state, "We are unable to change this situation, given that even when the NPCC Chair requested that we include more resident fish data, he also stated outright that there would be no additional funding to support the effort."

StreamNet devised an internet-based approach to disseminating data that are standardized and georeferenced across agency lines. StreamNet is, however, labor intensive, and with current staffing they have to focus on updating existing data sets, and have little time available to work to locate and standardize additional types of data.

StreamNet is evolving in response to input from agencies and user groups, demonstrating adaptive management for example by working with the Coordinated Assessments Project. However, this will result in delays by three agencies (WDFW, IDFG, and ODFW) in the updating of primary StreamNet databases (see Qualification #1). The ISRP concurs with the sponsors’ statements that a regional discussion is needed on which datasets, if any, to completely eliminate.

About three fourths of the project consists of sub-projects with states and tribes to develop data and databases and make these data available via StreamNet. StreamNet is proposing to expand the project to include the Colville, Shoshone-Bannock and CRITFC member tribes to the project. However, CRITFC’s StreamNet budget was moved to Columbia Basin Fish Accords. The current relationship with CRITFC is unclear. Has CRITFC data not been stored on StreamNet in the past? The proposal would be improved by addressing these issues.

Retrospective Evaluation of Results

StreamNet’s primary past contribution to the Fish and Wildlife Program has been to provide access to summarized, interoperable fisheries datasets collected by the Basin’s fisheries agencies. Data are provide via the project's website (www.streamnet.org) through an online data query system and interactive map applications. The primary data sets include:

Anadromous fish distribution (generalized)

Resident fish distribution (generalized)

Adult abundance in the wild, redd counts

Adult abundance in the wild, spawner counts

Adult abundance in the wild, dam/weir counts

Adult abundance in the wild, estimates of spawner population

Hatchery returns (anadromous).

StreamNet’s goal of providing updated data within a year of data collection in the field has not always been met because of delays from internal reviews and in release of data by the agencies that collected the data.

StreamNet also develops and disseminates a variety of other data types including stream network hydrography, fish barriers, protected areas, hatcheries, dams, and other structural facilities, and fish age data. StreamNet also disseminates independent data sets that do not fit the StreamNet data exchange format and are archived in the Data Store (www.streamnet.org/datastore_search.cfm), where they are searchable and downloadable, along with metadata and functions as a data archive, as suggested by the ISRP (ISRP 2000-3). They also provide source documents for all data contained in the StreamNet database to the StreamNet Library.

Additional past contributions include:

Initiating development of internal database systems in some partner agencies;

Responding to data-related requests from participants in the Fish and Wildlife Program, for example their lead role in developing an initial draft Data Exchange Template for the Coordinated Assessments project and hiring specialists to assist agencies in describing data management gaps and needs, which was used as a template to help the state and tribal agencies determine their capacity to locate and provide the specified indicators and metrics (www.pnamp.org/sites/default/files/ca-lessons_learned_report-2011-05-17.pdf);

Redesign and ongoing implementation of an online data query application for the StreamNet database (http://test.streamnet.org/); and

A guide on data sharing, Considerations for Regional Data Collection, Sharing and Exchange (Schmidt and the StreamNet Steering Committee (2009), and a condensed ‘top ten list’ format (StreamNet, 2010). Many of the concepts discussed in the data sharing guide were adopted by the Coordinated Assessments project and in new data-related requirements for BPA contracting.

StreamNet results can be evaluated in part by review of their responses to issues raised in the past at workshops and by groups like NED, PNAMP, CBFWA and the ISRP, as follows:

Timeliness of data updates: Addressed in part by encouraging and supporting agencies to develop internal database systems and initiate agency-wide approaches to data management.

Lack of data from some tribes: Addressed in part by initiating work with the CRITFIC member and other tribes.

Need for additional types of data: Addressed in part by assisting agencies with developing internal database capacity to allow more efficient data management and sharing.

Lack of derived data: Addressed in part by assisting agencies in development of a data exchange network approach, whereby agencies other than StreamNet provide derived estimates and supporting data via web services.

Lack of standardized field sampling: Addressed in part by collaborating with management agencies and regional scale entities to coordinate what is monitored and how, e.g., the Coordinated Assessments project.

Not enough resident fish data: Not addressed due to a lack of funding to support the effort.

Lack of standardization in data collected, collection methods, and data standards: Addressed in part because data collection issues were not addressed, but data standards were addressed through the Data Exchange Format (www.streamnet.org/reports_pubs.cfm.

Unclear priorities for the types of data provided through StreamNet: Addressed in part by organizing a workshop with CBFWA in fall 2006 and prioritizing abundance data to support the Status of the Resource report, but regional consensus recommendations were not developed.

Lack of clarity of who uses StreamNet, site use, and user satisfaction: Addressed in part by reporting site usage in annual reports, but user satisfaction could not be determined via an online user survey.

Lack of description of QA/QC procedures: Addressed only by a brief description in the 2012 proposal.

Lack of adequate metadata: Addressed in part by working through the PNAMP Metadata Work Group, but limited primarily to general descriptions due to lack of original metadata with the data submitted by data-collection agencies.

Justification of the amount of staff and infrastructure: Addressed in the 2012 proposal.

The StreamNet strategic plan (Schmidt 2009) emphasizes developing internal database capabilities within the data source agencies and a distributed network for dissemination of data. As the distributed network develops, the need for a central location for data management should be evaluated by the agencies and entities collecting, disseminating, and using the data, as centralized coordination and management of such a distributed network will require considerable resources.

The primary relationships described include those with PNAMP, CBFWA, and CRITFC, and there appears to be good coordination and collaboration with these entities. This section of the proposal would be improved by more information on BPA-funded projects and project numbers, including data collection projects, that are necessary for successful completion of proposed StreamNet objectives and deliverables. The sponsors note that agencies are the major limiting factor for StreamNet, that is, the efficiency with which data can be located and accessed within agencies and converted to regional standards. Clearly, provision of data to StreamNet is not always a high priority for management agencies.

The ISRP supports the StreamNet strategic plan (Schmidt, 2009) that emphasizes providing more support for developing internal database capabilities within the data source agencies. However, progress has been slow in some agencies, because it is not viewed as high priority. Although not discussed in this proposal, the Tribal Data Network proposal (#200850700) appears to be the companion proposal to facilitate this move.

The ISRP supports the increased focus on providing derived estimates and assisting data source agencies with development of internal data systems for storage and dissemination of data.

The sponsors describe important emerging limiting factors with respect to regional-scale data coordination in the proposal. Concerns are being addressed to at least some extent by the Coordinated Assessments project. Nevertheless, the ISRP concludes that there is a need for improved coordination of data management at the regional scale that will necessarily involve discussions with the Council and BPA, as well as agreement and support from the states, tribes, and other agencies and entities involved in providing data to StreamNet.

4. Deliverables, Work Elements, Metrics, and Methods

The ISRP supports StreamNet’s proposed new work in several areas: (1) to help develop indicator and metric data for the Coordinated Assessment project; (2) to collaboratively establish data needs and priorities, agree on standardized formats and definitions for sharing, and initiate sharing of the selected data as routine operations; and (3) to revise the data query system to improve user friendliness, increasing speed, and linking tabular and GIS data.

The ISRP is concerned that some data currently collected (Deliverable #2) will be put on hold until a distributed network can be established (see Qualification #1). Because the primary focus of regional data coordination is the Coordinated Assessment project, it is not likely that Deliverable #2 can be met within the period covered by this proposal. Although this is reflected in a reduction in funding for this deliverable, as discussed earlier in the proposal there is a need for regional consensus on this issue and whether some Streamnet datasets should be completely eliminated.

Among the three work elements listed for this project, none has metrics. The guidance given on the proposal submission site emphasizes an “emphasis on outcomes,” discussion of hypotheses, quantitative (and qualitative) measures and metrics, summary tables and graphs, and trends. Data management activities are amenable to scientific analysis. Key questions, hypotheses, relationships, data gathering and analysis, reporting of results, and revisions based on what is learned are expected. Greater emphasis on trying to measure outcomes and include in the proposal an adaptive management framework for designing, implementing, evaluating, and revising data management activities is recommended.

A log-in system to StreamNet might allow the program to more effectively evaluate public usage.

4a. Specific comments on protocols and methods described in MonitoringMethods.org

The project is 100% RM&E and Data Management with three work elements:

None of these work elements is associated with metrics in MonitoringMethods.org; however, it would be useful for retrospective evaluation of project results to develop quantifiable metrics or these work elements that could be used to track trends in data management project results.

Qualifications

In summary, StreamNet’s objectives are clearly stated, and most deliverables appear to be on track to meet the objectives (see Qualification #1). The project is of benefit to the Fish and Wildlife Program. However, an effectiveness monitoring plan needs to be developed and implemented (Qualification #2), and QA/QC methods are not documented in sufficient detail (Qualification #3).

The ISRP supports the project’s shift in focus to increased emphasis on derived estimates, such as indicators and metrics to support regional scale reporting under the ESA, as per the Coordinated Assessment (CA) project (www.pnamp.org/project/3129). Acquiring data from the tribes is a major step forward. The ISRP supports the strategic plan (Schmidt, 2009) that emphasizes developing internal database capabilities within the data source agencies and a distributed network for dissemination of data. However, the need of a central location for data should be revaluated as a distributed network system is developed. Coordination and management of such a distributed network will require considerable resources. The ISRP concludes from information in the proposal that there is substantial room for improvement in regional coordination of data management in the Columbia River Basin. This will necessarily involve discussions with the Council and BPA, as well as agreement and support from the states, tribes, and other management agencies and entities involved in collecting and providing data to StreamNet.

The need to standardize data protocols continues. The central role of BPA in funding data collection in the basin should provide a mechanism to require standardization of data reporting, protocols, and methods. However, the sponsor's response provided reasonable explanation of the position of StreamNet relative to the data standards issue and the difficulties of moving the issue forward based on voluntary agency agreement. The proposal describes past, present, and future features of StreamNet in a subdued manner. Nevertheless, the importance of having data development and dissemination activities in the basin is clear.

StreamNet is complex, and the staff is attempting to meet the needs of a diverse audience. As we learned in the 2000 review, this is not easy. Workshops to establish priority needs, better mechanisms to track use and effectiveness, documented QA/QC procedures, moves toward standardization without offending clients (both users and suppliers of data), specific data development on hatchery releases, and other topics that the ISRP questioned appear to be underway and in the right direction. For example, the ISRP encourages the sponsors to complete the draft document describing QA/QC procedures soon.

The base program is fundable and serves an important role in the Basin. The ISRP strongly supports expanding the tasks and objectives of StreamNet to provide the most utility to the basin. The "Fundable" recommendation is qualified, however, because the program needs to develop measures of effectiveness and assess its impact in terms of those measures. The project should have in place a system for monitoring and evaluating its performance. The program still needs to develop more in-depth measures of monitoring effectiveness and assess its impact in terms of user satisfaction. Use of the services should be documented, and more focus should be placed on outputs rather than inputs. A systematic way of evaluating effectiveness is needed. Who are the users? Were these users satisfied? Is tracking software used (e.g., Web Trends)? The sponsors should provide some evaluative performance information to address these questions.

The ISRP recommends that the project receive an independent project review on the quality of its service delivery soon. The sponsors are receptive to the idea of an independent performance review.