Chapter 7. Evaluating Risks

This chapter continues the discussion of Phase II of the specification development
process. This chapter is intended to provide "how to use" best practices
in evaluating the risks associated with the initial acceptance procedures that
have been developed up to this point. The steps that are involved in this part
of the process are identified in the flowchart in figure 25. The numbers in
boxes before the titles of the following sections refer to the corresponding
box in the flowchart.

Establishing the limits to be used for acceptance is an important step. Making the limits too restrictive deprives the contractor of a reasonable opportunity to meet the specification. Making them not sufficiently restrictive makes them ineffective in controlling quality. Selection of the limits relates to the determination of risks. The concept of risks for acceptance is similar to that discussed in chapter 5 for verification testing to evaluate whether test results came from the same population. The two types of risk discussed in chapter 5 are the seller's (or contractor's) risk, a, and the buyer's (or agency's) risk, b. The a risk is also called a Type I risk, and the b risk is also called a Type II risk. A well-written QA acceptance plan takes these risks into consideration in a manner that is fair to both the contractor and the agency. Too large a risk for either party undermines credibility.

39.1. Risks: Definitions and Concepts

39.1.1. Risks. Before proceeding further, some terms need to be formally defined. The TRB glossary (2) includes the following definitions:

Seller's risk (a)-also called risk of a type I error. The probability that an acceptance plan will erroneously reject acceptable quality level (AQL) material or construction with respect to a single acceptance quality characteristic. It is the risk the contractor or producer takes in having AQL material or construction rejected.

Buyer's risk (b)-also called risk of a type II error. The probability that an acceptance plan will erroneously fully accept (100 percent or greater) rejectable quality level (RQL) material or construction with respect to a single acceptance quality characteristic. It is the risk the highway agency takes in having RQL material or construction fully accepted. [The probability of having RQL material or construction accepted (at any pay) may be considerably greater than the buyer's risk.]

The a and b risk levels that might be appropriate vary depending upon the material or construction process that is involved. The appropriate risk level is a subjective decision that can vary from agency-to-agency. In reality, it is likely that few agencies have developed and evaluated the risk levels associated with their acceptance plans. While risk levels are an agency decision, AASHTO R-9, "Acceptance Sampling Plans for Highway Construction," suggests the risk levels indicated in table 21. (22) It should be noted that large sample sizes, on the order of 10 to 20 or more, may be required to achieve some of the risk levels stipulated in this table.

1Critical: when the requirement is essential
to preservation of life.
Major: when the requirement is necessary for the prevention of substantial
financial loss.

Minor: when the requirement does not materially affect performance.
Contractual: when the requirement is established only to provide
uniform standards for bidding.

As noted in the section on verification testing in chapter 5, the
concept of a and b risks
derives from statistical hypothesis testing where there is either a right or
wrong decision. As such, when a and b
risks are applied to materials or construction they are only
truly appropriate for the case of a pass/fail or accept/reject decision
and, in fact, may lead to considerable confusion if an attempt is made to apply
them to the payment adjustment case. When materials not only can be accepted
or rejected, but can also be accepted at an adjusted payment, then additional
interpretations or clarifications must be applied to the definitions for these
risks in an effort to manipulate them to apply to the payment adjustment situation.

For example, in the definition for buyer's risk above, it states that b is the probability that RQL material will be accepted at 100 percent payment or greater. The definition must then go on to point out that there is a much greater probability that the RQL material will receive some reduced payment. While it is not stated as directly, the same reasoning is true for the seller's risk. The definition indicates that a is the probability that AQL material will be rejected. Although not stated in the definition, it is also true that there is a much greater probability that the AQL material will be accepted at a reduced payment.

39.1.2. OC Curves. The buyer's and seller's risks are very narrowly defined to occur at only two specific quality levels. The buyer's risk is the probability of accepting material that is exactly at the RQL level of quality, while the seller's risk is the probability of rejecting material that is exactly at the AQL level of quality. These definitions do not therefore provide a very good indication of the risks over a wide range of possible quality levels. To evaluate how the acceptance plan will actually perform in practice, it is necessary to construct an OC curve. The TRB glossary (2) includes the following definition:

OC curve — A graphic representation of an acceptance plan that shows the relationship between the actual quality of a lot and either (1) the probability of its acceptance (for accept/reject acceptance plans) or (2) the probability of its acceptance at various payment levels (for acceptance plans that include pay adjustment provisions).

An example of an OC curve for a pass/fail or accept/reject acceptance plan, case (a) in the above definition, is shown in figure 26. Probability of acceptance is shown on the vertical axis for the range of quality levels indicated on the horizontal axis. An example of an OC curve for an acceptance plan with payment adjustment provisions, case (b) in the above definition, is shown in figure 27. The axes are the same as for figure 26, but there are multiple curves, one for each of several selected payment levels, plotted.

Each curve plotted in figure 27 represents the probability of receiving a payment factor equal to or greater than the one indicated for the line. For example, for the OC curves in figure 27, material that is of exactly AQL quality has approximately a 45 percent chance of receiving a payment factor of 1.04 (104 percent) or greater. This same AQL material has approximately a 55 percent chance of receiving full payment (100 percent) or greater, which also means that it has approximately a 45 percent chance of receiving less than 100 percent payment. This AQL material has essentially a 100 percent chance of receiving a payment factor of 0.80 (80 percent) or greater.

On the other hand, for the OC curves in figure 27, material that is of exactly RQL quality has approximately a 30 percent chance of receiving a payment factor of 0.80 (80 percent) or greater, and nearly an 80 percent chance of receiving a payment factor of 0.70 (70 percent) or greater. Similar payment probabilities can be determined for any level of actual quality, and additional curves could be developed for any specific value of payment factor.

39.1.3. Expected Payment Curves. Figure 27 clearly shows that consideration of onlya and brisks is clearly not sufficient when payment adjustments are used. From figure 27 it can also be seen that using multiple OC curves is not an easy way to evaluate an acceptance plan. It would be convenient to have a single curve that can represent the operation of the plan as opposed to many different curves for each plan. Another way to present the payment performance for an acceptance plan is with what is call an expected payment (EP) curve. The TRB glossary (2) includes the following definition:

EP curve — A graphic representation
of an acceptance plan that shows the relation between the actual quality of
a lot and its EP (i.e., mathematical pay expectation, or the average pay the
contractor can expect to receive over the long run for submitted lots of a
given quality). [Both OC and EP curves should be used to evaluate how well
an acceptance plan is theoretically expected to work.]

An example of an EP curve is shown in figure 28. Quality levels are indicated on the horizontal axis in the usual manner, but instead of probability of acceptance, the vertical axis gives the expected (long-term average) payment factor as a percent of the contract price.

Although the risks have a different interpretation when associated with EP curves than with OC curves, the same type of information is provided. For the example in figure 28, AQL work receives an expected payment of 100 percent, as desired, while truly superior work that is better than the AQL receives an expected payment of 102 percent. At the other extreme, RQL work corresponds to an expected payment of 70 percent. For still lower levels of quality, the curve levels off at a minimum expected payment of 50 percent.

Simplified Example: a and b
Risks and an OC Curve

A simplified example of how risks are related to specification limits can be illustrated by considering primitive acceptance plans that were based on measuring and accepting a property based on only one test. Suppose that an accept/reject acceptance plan for asphalt content has been developed based on the definitions for AQL material and RQL material that follow.

Define AQL Material. It is assumed that asphalt content follows a normal distribution. It has been determined that for asphalt content, acceptable material has a standard deviation of about 0.20 percent when the mean is close to the target JMF value. If the JMF has established the target as 6.0 percent asphalt content, the AQL is therefore a lot (population) with a mean of 6.0 percent and a standard deviation of 0.20 percent. Figure 29 shows an AQL population.

Define RQL Material. Additionally, unacceptable material might be defined as that for which the mean differs from the target value by 0.4 percent or more, as long as the standard deviation does not exceed 0.20 percent. (Other definitions would be equally valid.) The RQL is therefore a lot (population) with a standard deviation of 0.20 percent and a mean of 5.6 percent or lower, or 6.4 percent or higher. Examples of RQL populations are shown in figure 30.

Determine a Risk. Suppose the agency
has established the specification limits, i.e., the limits within which individual
asphalt content results must fall, as the JMF ± 0.40. For a JMF target
value of 6.0 percent, this establishes the specification limits as 5.60 percent
and 6.40 percent. An AQL population is shown along with the specification limits
in figure 31. The a risk is the probability that a single test result from this
AQL lot would be outside of the allowable specification range of 5.60 percent
to 6.40 percent. This is the a risk to the contractor because if a test falls
outside these limits the agency will erroneously reject the material. The Z-statistics
can be calculated and used in conjunction with the standard normal distribution
table (table 7) to determine this probability to be 0.0456 or 4.56 percent.

Determine b Risk. Figure 32 shows
an RQL population with its mean at 5.60 percent and standard deviation of 0.20
percent. The RQL population could also have its mean at 6.40 percent. The RQL
population can either be too low or too high, but not both at the same time.
The b risk is the probability that a single test result from this RQL lot would
be within of the allowable specification range of 5.60 percent to 6.40 percent.
This is the risk to the agency because if a test result falls in this range,
the agency will erroneously accept the material. From figure 32 this probability
can be seen to be 0.50 or 50 percent.

Develop the OC Curve. Similarly, the probabilities of acceptance for lots with means of any value, e.g., 5.20 percent, 5.40 percent, 5.60 percent, etc., can be calculated and plotted to form the OC curve shown in figure 33. The AQL and RQL are also noted on the figure. It should be noted that it is purely coincidental that the OC curve in figure 33 has the appearance of a normal curve.

39.2. OC Curves for PWL or PD Acceptance Plans

As with any acceptance plan that bases the acceptance decision on a sample, there are risks associated with PWL or PD acceptance plans. The above example demonstrated the calculation of risks for a simple acceptance plan based on an assumed known standard deviation, and with the acceptance decision based on only a single test result. The risks associated with PWL or PD acceptance plans cannot be calculated so easily.

For PWL or PD acceptance plans the risks are almost always determined by means of computer simulation. It is, however, possible to illustrate the risks associated with using a sample to estimate PWL by means of a simplified attributes example.

Simplified Example

Assume that we have a bag that has 100 marbles. Further assume that the bag
has 70 white marbles and 30 blue marbles. Also assume that we wish to take
a sample of 10 marbles to estimate the percentage of the marbles in the bag
that are blue.

It is easy to estimate the percentage of blue marbles from a sample of 10
marbles. However, each sample of 10 marbles will not yield the same percentage
of blue marbles. The first sample of 10 marbles might contain 3 blue marbles,
thereby yielding an estimate of 30 percent blue marbles. However, it could
also have only one blue marble, or five blue marbles. In each of these cases
the estimate from the sample will be fairly far from the true value of 30
percent.

The histogram in figure 34 shows the results of 100 samples, each with 10
marbles. While the individual sample results could be quite far from the actual
percentage in the population, the average of the 100 samples is quite close
to the true population value. Also, most of the sample values are close to
the actual population percentage, with fewer values as the estimate becomes
farther from the actual population percentage. Although simplified, this example
clearly shows how the PWL values estimated from samples can vary. The long-run
average of the sample averages will tend to equal the true population PWL
value, but there is a risk that any individual estimate may either over-estimate
or underestimate the true population PWL value.

39.2.1. Computer Simulation. As noted above, calculating the risks for actual PWL acceptance plans is much more involved than the simplified example from figure 34. In fact, computer simulation is almost always used to developa and brisks, as well as OC and EP curves. OCPLOT, a user-friendly program that develops OC and EP curves by computer simulation, was developed as part of FHWA Demonstration Project No. 89. This program is explained in detail in the report for that project, (18) and is also presented in appendix M along with some examples.

OCPLOT can be used to develop OC curves for accept/reject acceptance plans. It can also be used with a stipulated payment equation to determine the probability of receiving a lot payment factor greater than or equal to any specific value. In this way, it can be used to plot multiple OC curves similar to those in figure 27. The program can also develop EP curves for a given payment equation. The program is also capable of simulating acceptance plans containing retest provisions.

39.3. Evaluating the Risks

39.3.1. Accept/Reject Acceptance Plans. How potential risks are evaluated depends upon the type of acceptance plan that is used. The evaluation of risks is rather straightforward for accept/reject (pass/fail) acceptance plans. As noted above,a and brisks and OC curves were developed specifically for this type of situation. Therefore, they can be directly used to assess the risks to both parties.

To reiterate, the a risk is the probability that AQL material will be rejected;
while the b risk is the probability that RQL material will be accepted. However,
since contractors will not operate at only these two quality levels, to fully
consider risks the OC curve, which illustrates probability of acceptance for
any quality level, must be developed for the acceptance plan under consideration.
An example will help to illustrate how this can be done.

Example: Accept/Reject Acceptance Plans-OC Curves

The previously discussed OCPLOT program can be used to determine thea and brisks and to plot the OC curve for a sample acceptance plan. Suppose that
an agency decides to use asphalt content as an accept/reject property for
an HMAC pavement (note, this is not
recommended, but is used here solely for the purpose of illustrating the use
of an OC curve for an accept/reject situation). Further suppose that the agency
has established for asphalt content a lower specification limit of 5.60 percent
and an upper specification limit of 6.40 percent. The agency has decided to
use the PWL, based on a sample of size 4, as the quality measure. The agency
has selected 90 PWL for the AQL and 50 PWL for the RQL. The lot will be accepted
if the estimated PWL is greater than or equal to 70. Table 22 and figure 35
show the results of the OCPLOT analysis of this proposed acceptance plan.

From table 22 it can be seen that the seller's risk is a = 1.000 - 0.905
= 0.095 (or 9.5 percent) and the buyer's risk is b = 0.144 (or 14.4 percent).
Further, both table 22 and figure 35 show the probability of acceptance over
the total range of possible lot quality levels, as defined by the actual PWL
for the lot. The agency would need to decide whether or not it considers these
levels of risk to be appropriate.

39.3.2. Payment Adjustment Acceptance Plans. The evaluation
of risks becomes much more complicated when the acceptance plan includes payment
adjustment provisions. The concepts of a and b
risks, which were developed from hypothesis testing where there is a yes or
no decision, i.e., reject or fail to reject the null hypothesis, are not sufficient
when the decision involves not only accept or reject, but also accept at an
adjusted payment level.

The TRB glossary (2) definitions for seller's and buyer's risks that are presented above do not attempt to incorporate the concept of payment adjustments. The seller's risk is defined as the probability that an acceptance plan will erroneously reject AQL material or construction. This disregards the fact that the material or construction can be accepted at full payment, increased payment, or decreased payment. In other words, whether or not a lot received 105 percent, 100 percent, 75 percent, or 50 percent payment would have no impact with regard to the seller's risk based on this definition. Obviously, however, these different payment levels would have quite an impact on how the contractor perceived its risks.

Similarly, the buyer's risk is defined as the probability that an acceptance plan will erroneously fully accept (100 percent or greater) RQL material or construction. Once again, this definition disregards the impact of partial payments when determining the buyer's risk. However, when considering its risks the agency will certainly be interested in the probability of accepting RQL material at reduced payment levels as well as at 100 percent payment or greater.

The use ofa and brisks to evaluate payment adjustment acceptance plans is
simply not sufficient. Some additional method or methods is/are necessary to
properly evaluate the risks when payment adjustments are added to the acceptance
decision options. The expected payment, or EP, (see figure 28) is another method
for considering the payment adjustment aspects of the acceptance plan. However,
EP alone is also not sufficient to fully evaluate the risks that are involved.
Multiple OC curves for various payment levels (see figure 27) should also be
developed when evaluating acceptance plans with payment adjustment provisions.
An example will help to illustrate the evaluation of risks for payment adjustment
acceptance plans.

Example: Payment Adjustment Acceptance Plans-EP Curves

Consider the previous asphalt content example where the sample size was 4,
the allowable specification range was 5.60 percent to 6.40 percent, and the
AQL and RQL were defined as 90 PWL and 50 PWL, respectively. However, instead
of a simple accept/reject acceptance plan, the agency chooses to use equation
28 to establish the payment factor for a lot:

where: PF = payment factor for the lot, as a percent of contract
price.

PWL = estimated PWL value for the lot.

From the above equation, it can be seen that the maximum payment factor is
105 percent at 100 PWL, while the payment factor at the AQL will be 100 percent
and the payment factor at the RQL will be 80 percent. It is generally accepted
that the average payment for AQL material should be 100 percent. In this example,
the payment factor at the AQL is 100 percent, exactly as intended. However,
if the payment equation is not developed properly, the average payment factor
may turn out to be above or below 100 percent at the AQL. If this is the case,
the agency should determine if an expected payment other than 100 percent
for AQL material is acceptable.

With the above information, the OCPLOT program can be used to develop the
EP curve shown in figure 36. It can be seen in this figure that, as desired,
the EP for AQL material is 100 percent. This means that a contractor that
consistently produces material that just meets the minimum requirements, i.e.,
AQL material, will receive an average
payment factor of 100 percent in the long-run. Similarly, the EP for RQL material
is 80 percent as desired from the payment equation.

The EP curve has the advantage of combining all of the possible payment levels
into a single expected, or long-term average, payment for each given level of
quality. While it is a major improvement over only considering a and b risks, the use of the EP alone still has some
serious deficiencies. The primary deficiency in the use of EP alone is that,
while it considers the average long-term payment factor, it fails to consider
for a given quality level the variability of the individual lot payment factors
that comprise this long-term average. This variability is directly related to
the sample size. That is, the variability about the average payment factor decreases
as the size of the individual samples increases. To
fully evaluate the risks it is necessary to also consider this variability about
the expected payment values.

The OCPLOT program output can be used to demonstrate this variability of the individual lot payment factors. Figure 37 shows for an AQL population a histogram that displays the individual lot PWL estimates along with their corresponding payment values for 1,000 simulated lots using a sample of size of 4 for each individual lot. Figure 38 shows similar information for an RQL population. The high degree of variability of the individual lot payment factors is obvious from this histogram. However, over a large number of lots, the high and low estimates for lot PWL will tend to balance out to give the correct average payment factor.

If, however, there are only a small number of lots on a project, then it will be possible that a significantly low estimated PWL value could negatively impact the payment that the contractor should have received. Similarly, larger PWL estimates could be obtained that would provide a larger payment than is deserved. A contractor would be wise to target a quality level above the AQL, particularly on smaller projects, to ensure that this variability of individual lot PWL estimates does not create a problem. However, as discussed elsewhere in this manual, it is often the practice of contractors to bid projects with the anticipation of receiving the maximum incentive payments. If this is the case, it is unlikely that contractors will target their processes at the AQL. It is more likely that they will target their processes for greater than AQL quality to try to maximize their incentive payment. In this event, the variability of the individual lot payment factors will not likely pose a serious problem to either the contractor or the agency.

The variability associated with the estimate of the lot PWL can be reduced by increasing the size of the sample obtained from each lot. Figure 39 shows a histogram that displays for an AQL population the individual lot PWL estimates along with their corresponding payment values for 1,000 simulated lots using a sample of size of 20 for each individual lot. Figure 40 shows similar information for an RQL population. When these figures are compared with figures 37 and 38, for samples of size 4, the smaller spread of the individual PWL and payment factor estimates is apparent.

Even if it reduces the variability of the PWL estimates, and hence the risk to both the contractor and the agency, it may not be practical or economical to use large sample sizes unless correspondingly large lot sizes are also used. The use of a very large lot, possibly even the total project, will allow larger sample sizes, but also introduces problems of its own. As noted in chapter 6, a major assumption that is required is that all of the material and/or construction processes remain consistent throughout the total lot.

Over the course of a long project, changes in weather, materials, rolling patterns, mix designs, etc., are likely to lead to variations throughout the project. Combining all of these together may result in a normal distribution, albeit one with a larger variability than the individual production lots, but this may not be the best method to evaluate a project. If there are "bad" segments on a project, it might be better to see them penalized on a lot-by-lot basis than to have them lumped together with the "good" material from all of the other lots.

While figures 37 through 40 clearly illustrate the relative variabilities of the individual PWL and payment factor estimates associated with different sample sizes, they do not provide any quantitative measure for the variabilities. One way to quantify these variabilities would be to calculate the standard deviation of the individual PWL or payment estimates. This is not discussed in this manual, but is presented and discussed in the technical report for this project.(17)

Example: Payment Adjustment Acceptance Plans-Multiple OC Curves

Another step that is necessary to evaluate fully the risks for a payment
adjustment acceptance plan is to plot OC curves, such as those shown in figure
27, associated with receiving various payment factors. As shown in appendix
M, the OCPLOT program can be used to develop these curves, although each curve
must be developed individually and then manually combined onto a single set
of axes.

Suppose that the OCPLOT program is used to develop multiple OC curves for
the asphalt content acceptance plan from the previous example. Figure 41 shows
OC curves for the probability of receiving greater than or equal to various
levels of payment factor for a sample of size of 4 using the payment relationship
shown in equation 28. These OC curves would be considered along with the EP
curve from the previous example to evaluate the risks associated with the
acceptance plan.

While the EP curve in figure 36 shows that the average long-term payment
is 100 percent for AQL material, the OC curves in figure 41 show that the
probability is less than 60 percent that any individual lot of AQL material
will receive 100 percent payment or greater. This means that there is nearly
a 40 percent chance that a contractor would receive less than full payment
for a lot that was of AQL quality. This risk, which would be considered to
be a (if a is defined as the probability that AQL material will receive less
than full payment), seems high. However, it is somewhat offset by the fact
that the OC curves also indicate that there is over a 40 percent chance of
receiving a payment of 104 percent or greater.

The OC curves and EP curves describe the operation of the acceptance plan such that the risks can be evaluated throughout the entire quality regime. If the risks are considered acceptable, no modifications to the initial acceptance plan are necessary. However, if the risks are considered unacceptable in terms of being too high for both or either party, a reassessment of the acceptance plan is necessary.

As shown in the previous section, there is no easy answer to the question "Are the risks acceptable?" since this is to a great extent a subject of opinion, and opinions may vary from agency-to-agency. Table 21 can provide some guidance regardinga andb risk levels, but these risks are not very useful when price adjustment acceptance plans are used. Even in accept/reject acceptance plans, thea andb risks apply at only two specific levels of quality. An OC curve is still necessary to evaluate the risks over the full range of possible quality levels.

When a price adjustment acceptance plan is used it is essential that
the agency develop both an EP curve and OC curves for the probability
of receiving various payment factors over the total range of quality levels.
The agency may also wish to look at histograms of individual payment factors
to obtain a picture of how much variability is associated with the payment
factor determination. This is shown in figures 37 through 40.

The decision regarding what does or does not constitute an acceptable level of risk will to a great extent be a subjective one. There is, however, one factor that is not subjective. There is generally universal agreement that the expected payment should be 100 percent for quality that is at exactly the AQL. Although it should not be confused with the statistical risk,a, the agency may wish to consider the "average payment" risk to the contractor, if the EP is less than 100 percent at the AQL, or to the agency, if the EP is greater than 100 percent at the AQL. The EP at the RQL quality level is another point that is often specifically considered.

It must be remembered that the EP alone is not a complete measure, particularly of the likelihood that any individual lot will receive a correct payment factor. The variability of the individual payment factors about the EP curve must also be considered. Ultimately, the decision regarding what constitutes acceptable or unacceptable risks rests with the individual agency. While the determination of acceptable risks rests solely with the agency, by way of the joint industry/agency task force discussed in earlier chapters, there should be contractor input into this decision.

Word of Caution

The procedures that have been presented in the previous sections, as
well as the OCPLOT program that is discussed, are primarily for the case
of acceptance based on a single property. When, as will often be the
case, there are multiple acceptance properties it will be necessary for
the agency to develop sophisticated computer simulation methods to complete
a full analysis of the risks. These analyses will be quite involved and
will be dependent upon the quality characteristics chosen for acceptance
and whether or not a performance model for predicting service life has
been adopted by the agency. Another factor that will impact the analysis
is whether a composite quality measure has been developed, or whether
the individual quality measures will in some way, perhaps by adding,
multiplying, or averaging, be combined into a composite payment factor.
All of the possibilities cannot possibly be covered in this manual.

It is very likely that the agency will need to seek outside assistance
to help in developing the simulation routines necessary to fully evaluate
the risks. Universities as well as other agencies and consultants who
have already developed such procedures are potential sources for this
outside assistance. Once the agency has developed appropriate OC and
EP curves for the acceptance plan, it should supply this information
to the contractors that work in the State. Otherwise, each contractor
will individually be required to seek outside help to fully understand
the risks associated with the new acceptance plan, and this is not very
cost effective for the contractors or ultimately for the agency.

If the risks are considered unacceptable they are likely to be too high rather than too low. To reduce the risks it may be possible to change the specification limits, the acceptance limits, and/or increase the sample size. The most straightforward approach would be to increase the sample size per lot.

An increase in the sample size may be accomplished by either increasing the lot size or increasing the sampling frequency. For example, if the lot size were 1800 Mg of HMAC, and the sampling frequency was one test per 450 Mg, then the number of tests per lot could be increased from 4 to 8 by increasing the lot size to 3600 Mg. On the other hand, the number of tests per lot could be increased from 4 to 8 by keeping the lot size as 1800 Mg but increasing the sample frequency to one test per 225 Mg.

Another way to change the risk levels would be to change the specification limits or the acceptance limits. This may be related to the definition of AQL and/or RQL material. For example, for the example presented above for asphalt content, the AQL was defined as a population with a mean of 6.00 percent and a standard deviation of 0.20 percent. Using this definition, the specification (and, in this case, acceptance) limits for an accept/reject decision based on a single test result were set at plus or minus two standard deviations from the target value of 6.00 percent, i.e., 6.00 percent ± (2 ´ 0.20 percent) or 5.60 percent to 6.40 percent. This provided an a risk to the seller of 0.0456, or 4.56 percent. This risk can be reduced to nearly zero by setting the specification (and acceptance) limits at ± 3s rather than ± 2s. However, this will also increase the brisk, unless the definition of RQL is changed.

For accept/reject acceptance plans based on PWL, the acceptance limit could be reduced, say from 90 PWL to 85 PWL, to lower the a risk. It could also be raised, say from 90 PWL to 95 PWL, to increase the a risk. It must be noted that whenever a is changed, b will also change unless the sample size is changed as well. For acceptance plans with price adjustments, the payment equation could be changed to increase or decrease the expected payment values. While these changes would impact the EP values and the "payment" risks at the AQL and RQL, they would not necessarily change the "statistical" risks,a and b.

New OC curves and EP curves must be developed for any changes that are proposed
to the initial acceptance plan provisions. This is the only way to determine
what impact the changes will have on the risks to both the contractor and the
agency. The agency should not proceed with developing the finalized draft specification until an acceptance plan has been developed for which the agency believes the risks are appropriate.

Once all of the preceding steps have been completed, the agency can move forward to finalize the wording for the initial draft specification. At this point the agency is ready to move forward to the implementation phase of the specification development process.

It is obvious from the above discussions that a great deal of thought should be put into the development of an acceptance plan. There are many "pieces" to the puzzle that must fit together for the acceptance plan to be well-written
and to work as intended. However, there are many resources that can be used to help accomplish this goal. QA acceptance plans have been under development and evolution for over three decades. This history can be an invaluable resource for any agency that is in the process of developing QA acceptance plans.