Tuesday, 4 June 2013

Some Thoughts on Rational Decision-making

How should administrative law cope with genuine uncertainty, in which
probabilities cannot be attached to outcomes? I argue that there is an
important category of agency decisions under uncertainty is which it is rational to be arbitrary.
Rational arbitrariness arises when no first-order reason can be given
for the agency’s choice one way or another within a certain domain, yet
the agency has valid second-order reasons to make some choice or other.
When these conditions obtain, even coin-flipping may be a perfectly
rational strategy of decisionmaking for agencies.

Courts should
defer to rationally arbitrary decisions by agencies. There is a proper
role for courts in ensuring that agencies have adequately invested
resources in information-gathering, which may dispel uncertainty. Yet in
some cases the value of further investments in information-gathering
will itself be genuinely uncertain. If so, courts should defer to
agencies’ second-order choices about informational investments on the
same grounds that justify deference to agencies’ first-order choices
under uncertainty.

The idea that coin flipping might be a rational decision-making process is provocative, but Vermeule is referring to a limited (though important) class of decisions. To fall into this class, a decision must exhibit "first-order uncertainty" -- genuinely unquantifiable uncertainty due to absence of information about facts or individuals' future actions -- and also "second-order uncertainty" -- impossible to gather further information to resolve the first-order uncertainty.
One of his examples involves an agency that must decide whether or not to list a species as endangered. The evidence is not conclusive either way. Further information-gathering will not assist, such that the following situation could be said to arise:

[A] hallmark of problems calling for a rationally arbitrary decision is a kind of mirror-image reversibility. If the agency chooses A over B, and the court overturns that decision as arbitrary, it will also be the case that the agency’s choice of B over A could be overturned on exactly the same ground. Indeed, any choice the agency makes could be overturned for lack of first-order reasons. Recognizing this, the reviewing court should realize that the agency may be facing a situation in which no non-arbitrary choice is feasible.

What is the agency to do in such circumstances? Sometimes it can extrapolate from past experience, default to the status quo or use conventional wisdom. But in some circumstances, Vermeule argues, these options will not avail the agency any more than a coin toss or other means of randomization.

This arresting logic is not to be pushed too far. Vermeule notes that courts have a role "in ensuring that agencies have adequately invested resources in information-gathering". This seems quite correct. Agencies should explain the existence of first- and second-order uncertainty. If they do not provide cogent reasons or evidence to support a claim that a decision was reached under uncertainty, the courts should intervene. A 'reasoned decision-making' requirement is as appropriate here as it is elsewhere.

A recent Canadian example comes to mind. Tervita Corporation v. Commissioner of Competition, 2013 FCA 28 involved a situation in which the Competition Tribunal had to put its mind to the economic effects of a proposed merger of landfill operators. Both quantitative and qualitative effects had to be considered. Evidently, not all of these could be measured with precision. In part, the analysis in Tervita depended on the answer to a question about the future actions of individuals and companies: how would others react to a decision to allow or forbid the merger? I think this is a modest example of what Vermeule styles as "strategic uncertainty" (uncertainty "arising from the strategic interdependence of actors’ choices").

[148]An
objective offset analysis means that the quantification of both gains in
efficiency and anti-competitive effects must be carried out whenever it is
reasonably possible to do so. When precise quantification is not reasonably possible
for a given element, a rough estimate is to be preferred to a subjective
judgement call. When neither a precise quantification nor a rough estimate is
reasonably possible for a given element, then of course there will be a certain
degree of discretion in attributing weight to any remaining qualitative gain in
efficiency or effect, but this discretion must be curtailed and limited by the
principles of reasonableness. In other words, any weight given to the remaining
unquantifiable qualitative effects must be reasonable, i.e.,it
must be supported by the evidence, and the reasoning behind the Tribunal’s
weighting must be clearly articulated or otherwise discernable.

A coin toss is very much a last resort! And if a coin toss is chosen, every step up to the choice should be explained in an objective fashion.

A more difficult question arising from Vermeule's analysis is in what circumstances limitations of time and resources can be a basis for refusing to resolve first-order uncertainty. In particular, there are areas of administrative law where the need for speedy and cost-effective decisions is quite pressing and might be thought to outweigh the need for perfect rationality.

An arbitrator found in favour of the union. A categorical approach was adopted. In "dangerous" environments, evidence of an existing alcohol problem is needed before proceeding to random tests which compromise employee privacy. In "ultra-dangerous"environments (nuclear facilities, airplanes, etc), no evidence is needed. Here, the arbitrator found, the workplace was not ultra-dangerous, even though there was a potential for catastrophic accidents. Accordingly, evidence was needed. In the arbitrator's view, the available evidence of alcohol-related incidents did not justify the imposition of random testing. The employer's decision was unreasonable.

On judicial review, both the first-instance judge and the New Brunswick Court of Appeal struck down the arbitrator's decision: once the potential for catastrophe was evident, imposing random alcohol testing had to be reasonable. As Robertson J.A. concluded:

[57]In summary, it is not difficult to support the
contention that Irving’s kraft paper mill qualifies as an inherently dangerous
workplace as would a chemical plant. This is why evidence of an existing
alcohol problem in the workplace was not required to support its policy of
random alcohol testing. This is why the arbitration board’s decision cannot stand
and the application judge was correct in determining that its decision should
be set aside and the grievance dismissed. In the circumstances, it is
unnecessary to delve into issues dealing with the board’s perception that
evidence of near disasters is required to justify the imposition of alcohol or
drug testing policies.

But the reasoning of both the arbitrator and the reviewing courts took place in a significant factual vacuum. The possibility of catastrophic accident was noted, but it was not quantified. Surely it could have been. The possibility that alcohol impairment would lead to catastrophic accidents is also, in principle, a risk that can be quantified. It was not. This is really an example of risk rather than what Vermeule styles as "brute uncertainty" ("epistemically unattainable facts that in some sense lie out there"), but in the absence of analysis, risk seems to shade into uncertainty.

The interesting question is whether resources should have been invested in calculating the risks. My tentative answer is no: as long as the reasoning process of the arbitrator was clear and supported by evidence in the record, there was ample material for the reviewing courts to assess the rationality of the reasoning process and ultimate decision. In turn, the reviewing courts were able to explain where the arbitrator's logic fell short. A much more satisfying decision could have been reached if time and money had been expended on gathering more information. But whether that would have satisfied the interests of the parties is doubtful. (Perhaps those living and working near the paper mill should also have a say, but the structure of the arbitration process did not allow for that.)

At a minimum, however, a reviewing court should require alertness on the part of the arbitrator to the fact that risks had not been adequately calculated and sensitivity to the possibility that some mechanism for calculating the risks (perhaps expert evidence) could be used. If the arbitrator decided not to calculate the risks (because of limitations of time and resources), he should have explained why. The objective reasonableness standard emphasized in Tervita requires, I think, alertness and sensitivity to this problem.