Quantative uncertainty scoringQuantative uncertainty scoring

"Uncertainty is not a number"

The quantification of yield gaps is subject to various sources of uncertainty, such as uncertainties in data, models, and concepts. Identification and estimation of these uncertainties provides valuable information to users of the atlas. The reported yield gaps in the atlas therefore are accompanied by qualitative estimations of uncertainties regarding the presented results, including remarks by experts on the justification of these estimations. The protocol given here describes the used approach how the uncertainty estimates are obtained. The assumption is that the GYGA framework is ‘true', i.e. it is accepted as given and there are no relevant conceptual uncertainties. Other sources of uncertainties have been framed within this conceptual view.

Terminology and explanation

Protocol parameter: phase in the GYGA protocol as described in the methods section. The relevant parameters are weather data, soil data, crop area, cropping system, modelling, and scaling.

A qualitative estimation of the ‘size' of uncertainties (from 1 to 3 to indicate an increasing level of uncertainty) in the data used for estimating a given parameter.

For example, how much confidence is there in the weather data? Are there "suspicious" data?

The remarks section provides a concise justification of the assessment.

The guideline on the classification criteria for level of uncertainty given below can be used, but experts may use their own approach for estimation uncertainties if there are good reasons for this.

Sensitivity:

A qualitative estimation of the sensitivity of the protocol parameter (from 1 to 3 to indicate an increasing level of sensitivity), i.e. in how far is it expected that the protocol parameter relatively contributes to the end result (Yx).

Example: weather plays a more important role for Yw than for Yp; therefore the sensitivity to weather is lower for Yp than for Yw.

Example: sensitivity to rooting depth will be greater for countries with predominantly shallow soils versus countries with predominantly deeper soils.

Example: In countries with low actual yields (Ya), the yield gap (Yg) is less sensitive to Ya.

Effect on yield:

A qualitative estimation of the propagated effect of the estimated uncertainty on the end result (Yx) given the sensitivity. The effect on yield is determined by using the estimated level of uncertainty (1, 2, or 3) and sensitivity (1, 2, or 3) at the hand of the look-up table given below. In this table, the levels of uncertainty (U) and sensitivity (S) have been extended to five levels to accommodate intermediate scores arising from averaging scores from two or more individual assessments. For instance, when expert A scores 1 and expert B scores 2, the average score is 1.5. In case of a divergent score (e.g. one expert scores a 1, and another a 3) this is explicitly indicated in the remarks.

Overall Uncertainty Score:

The "overall uncertainty score" is calculated from the values in the "effect on yield" column.

This score is determined by using an arithmetic mean. However, users can use the raw scores to calculate different scores if desired.

The overall uncertainty score is represented using the five-color-scale in the atlas, which is similar to the one given above. It is the primary information presented to the Atlas users.

Classification criteria for level of uncertainty

Potential yield

Protocol parameter

Criteria

Low uncertainty (1)

Medium uncertainty (2)

High uncertainty (3)

Weather

# years of measured quality data (Tmax, Tmin)

>10

3-10

0-2

If "suspicious data" : level -1 (please indicate why you think that the data are suspicious)

soil

NA

Crop area

observed deviations between SPAM-2005 and current land use; corrected or not; expectation there will be many more deviations (please justify this expectation)

Source and disaggregation by crop-water regime; if suspicious level -1 (please indicate why you think that the data are suspicious)

Crop-specific national data

Crop-aggregated national data

expert data

Number of (recent) years

# recent years for Rainfed / Irrigated

> 10 / > 5

5-10 / 3-5

<5 / <3

Scaling

Spatial resolution

District -

NUTS3 "equivalents"

Province -

NUTS2 "equivalents"

Region -

NUTS1 "equivalents"

Yield Gap

The uncertainty of Yp, Yw and Ya has been assessed separately

We take the average values of "uncertainty" from the Yp-, Yw- and Ya-sheets to fill in the score

(Yp or Yw) and Ya are given a weight to calculate the overall "effect on Yg". Currently, both components have an identical weight. The weights are given as 1, but may be adjusted by the user if there is proper justification for doing this. We are currently considering to make the weight dependent on the level of the actual yield, as uncertainty of low actual yields will have a minor effect on the uncertainty of the yield gap.

Score sheet

The Template is named GYGA_Uncertainty_COUNTRY_CROP.xls

please rename the file for each country-crop combination (eg GYGA_Uncertainty_GHANA_MILLET.xls)

Grey fields to be filled (other fields are locked):

Four sheets for Yp, Yw, Ya and Yg

Country and Crop in first sheet only (Yp)

"Uncertainty" and "Remarks" per country-crop combination ;

"Sensitivity" per crop;

For the assessment follow the criteria given for each protocol step. Please supply concise justifications for each protocol parameter (one to three lines) in the remarks section when doing the assessment.

The guideline is intended not to be compulsory, but as a first estimate. If you feel that a score should be different that is perfectly acceptable as long as it is justified by proper arguments. Please note that the remarks will be explicitly included in the atlas.