Atmospheric Sciences & Global Change Research Highlights

Clarifying the Uncertain

The fractional-factorial analysis method employs a fraction of decisions, decision variables, and uncertainties and applies it to GCAM-USA to identify and assess key sensitivities of the model output regarding a specific energy and climate policy question. Enlarge Image.

Results: In a direct evaluation of the uncertainty characterization process being applied as part of the Platform for Regional Integrated Modeling and Analysis (PRIMA) framework, Pacific Northwest National Laboratory scientists examined building energy policy under multiple energy-efficiency settings and several distinct uncertainty scenarios using a sub-regional version of the Global Change Assessment Model known as GCAM-USA. Their goal was to develop a decision-focused (i.e., relevant to a particular stakeholder) sensitivity analysis that identified factors that most influence stakeholder decisions. In an effort to generate an option that falls between ad hoc one-variable-at-a-time sensitivity assessments (for large models) or Monte Carlo analysis of all potential uncertainties (for small models), PNNL scientists used a middle-course, fractional-factorial analysis method to test the building energy policies. Their work showed that the number of factors required for modeling a given GCAM outcome actually was small, greatly reducing the modeling and computational burden in the presence of uncertainty.

Why It Matters: By using a fractional-factorial method, the researchers were able to determine that sensitivity patterns exist to assist decision makers in making impactful mitigation and adaptation decisions. Moreover, the method reduced the number of model runs needed to identify and compare key sensitivities that determine which decisions are important to investigate with formal Monte Carlo simulations. In most cases, it would be computationally impossible to fully analyze every potential source of uncertainty in a large, data-driven model such as GCAM-USA. This option provides a viable computational model for uncertainty characterization with significant time and cost benefits while still revealing important drivers for decisions and more detailed uncertainty analysis.

Methods: The fractional-factorial analysis method employs a specifically selected subset (or fraction) of decisions, decision variables, and uncertainties and applies it to an integrated assessment model (GCAM-USA) to identify and assess key sensitivities of the model output to socioeconomic and Earth system uncertainties involving a specific energy and climate policy question. In this case, they examined sub-national energy and climate policy related to the energy efficiency of buildings. Drawing from information culled from stakeholder groups concerned about the regional effects of building energy codes and equipment energy efficiency, they used three residential and commercial building energy efficiency scenarios: no change (fixed at 2005 levels), energy-efficiency improvements to reduce energy use per unit of floor space by 50% (by 2050), and aggressive energy-efficiency improvements to cut energy use per unit of floor space by 80% (by 2050). Then, they created plausible future scenarios for uncertainties relevant in almost any region: population, wealth, carbon policy, and climate change.

The full-factorial analysis of all possible combinations required 162 GCAM runs, so they used an incremental fractional-factorial strategy that initially reduced the overall number of GCAM model runs from 162 to 54. Based on their comparison of the full (162) and fractional-factorial (54) model outcomes, the statistically sensitive variables remained the same. It actually took only seven combinations out of the 54 to explain more than 98 percent of variability in energy use, carbon dioxide emissions, and cost. The specificity of the decisions, variables, and uncertainties in this experiment demonstrated how sensitivity analysis can allow analysts to determine the uncertainties that matter in a given decision context versus those that do not. This allows decision makers to concentrate on resolving the uncertainties that truly matter with confidence in the outcomes.

What's Next? As this work succeeded in demonstrating a viable empirical sensitivity analysis, the next step will involve engaging this method to assist regional decision makers in making energy and climate policy decisions under uncertainty.

Acknowledgments: This research was supported by PRIMA, conducted under PNNL's Laboratory Directed Research and Development Program. The research used Evergreen computing resources at PNNL's Joint Global Change Research Institute at the University of Maryland, which is supported by the U.S. Department of Energy, Office of Science's Integrated Assessment Research Program. Long-term support for GCAM also is provided by the Integrated Assessment Research Program.

User Facilities

Additional Information

Share

Fractional-factorial Analysis

In this case, fractional-factorial analysis involves strategically selecting a small fraction of the possible combinations of experimental conditions (design points) versus all possible combinations in the full-factorial method. The selections are carefully designed to concentrate on specific decisions, decision variables, and uncertainties. The principle is that this sensitivity analysis points to the actual few variables that significantly affect decision criteria to better identify relevant uncertainties.