1 Problem data

We consider a dam manager intenting to maximize the intertemporal payoﬀ obtained by selling
power produced by water releases, when the water inﬂows (rain, outﬂows from upper dams) are
random. However, the manager must also respect a minimal volume during the Summer months for
tourism reasons.

1.1 Dam dynamics

We model the dynamics of the water volume in a dam by

with

time t ∈𝕋 := {t0,…,T} is discrete (such as days), and t denotes the beginning of the
period [t,t + 1[,

S(t) volume (stock) of water at the beginning of period [t,t + 1[, belonging to the
discrete set , made of water volumes, where is the maximum
dam volume,

a(t) inﬂow water volume (rain, etc.) during [t,t+1[, belonging to

decision-hazard: a(t) is not available at the beginning of period [t,t + 1[

q(t) turbined outﬂow volume during [t,t + 1[, decided at the beginning of period
[t,t + 1[, supposed to depend on S(t) but not on a(t), belonging to the discrete set
, where q♯ is the maximum which can be turbined by time unit
(and produce electricity),

the spilled volume

The dam manager is supposed to make a decision, here turbining q(t) at time t, before knowing
the water input a(t). Such a case is called decision-hazard. The constraint on the water turbine q(t)
is

(1)

A scenario is a sequence of uncertainties:

(2)

1.2 Criterion: intertemporal payoﬀ

The manager original problem is one of payoﬀ maximization where turbining one unit of water has
unitary price p(t). On the period from t0 to T, the payoﬀs sum up to

(3)

where

the sequence

(4)

of prices is supposed to be known in advance (in other models, it could be progressively
revealed to the manager),

the ﬁnal term gives value to the water volume in the dam at the horizon
T.

1.3 Constraint: minimal volume during the Summer months

For “tourism” reasons, the following constraint is imposed

In what follows, we shall be more speciﬁc about the sense with which this constraint has to be
satisﬁed, namely in probability.

1.4 Water turbined strategy

A strategy Rule : 𝕋 × 𝕏 → 𝕌 assigns a water turbined to any state S of dam stock
volume and to any decision period t ∈ 𝕋. Once given, we obtain uncertain volume trajectories
and turbined trajectories
produced by the “closed-loop” dynamics

(5)

and function of the scenario a(⋅). Thus, in the end, we obtain an uncertain payoﬀ

In the macro trajectories, the output CC is the mean payoﬀ averaged over the scenarios: by
the law of large numbers, CC is an approximation of the expected payoﬀ if the number of scenarios
is large enough (Monte Carlo method).

// We start with a zero value of water at the end of the year zero_final_payoff_vector=zeros(states);// ----------------------------------------------------// SIMULATIONS// ---------------------------------------------------- [FEEDBACK,VALUE]=SDP(zero_final_payoff_vector);// optimal strategyfunction uu=optimal_rule(tt,xx) uu=FEEDBACK(tt,xx-state_min+1);endfunction// Trajectories simulations and visualization SS0=0; [SS,HH,CC]=trajectories(SS0,Scenarios,optimal_rule); xset("window",10);// xbasc(); plot2d(bTT,SS') xtitle('Stock volumes in a dam following an optimal strategywith a zero final value of water', ...'(time)','(volume)')// Payoff histogram xset("window",20);xbasc(); histplot(100,CC) xtitle('Histogram of the optimal payoff with a zero final value of water') disp('The minimum of the optimal payoff is '+string(min(CC))); disp('The mean of the optimal payoff is '+string(mean(CC))); disp('The maximum of the optimal payoff is '+string(max(CC)));

Question 1Picture the trajectories of the stocks corresponding to the optimal strategy.Evaluate the optimal expected payoﬀ, and compare it with the value functionevaluated at the initial time t0and the initial stock. Explain why these two quantitiesshould be close. What do you observe for the ﬁnal stocks? Explain why.

2.4 Optimal strategy when the ﬁnal “value of water” is not zero

Till now, there was no gain in leaving water in the dam at the ultimate decision period. From now
on, we consider that the “value of water” is given by

// ----------------------------------------------------// SIMULATIONS// ---------------------------------------------------- [FEEDBACK,VALUE]=SDP(final_payoff_vector);// optimal strategyfunction uu=optimal_rule(tt,xx) uu=FEEDBACK(tt,xx-state_min+1);endfunction// Trajectories simulations and visualization [SS,HH,CC]=trajectories(SS0,Scenarios,optimal_rule); xset("window",10);// xbasc(); plot2d(bTT,SS') xtitle('Stock volumes in a dam following an optimal strategy with a final value of water', ...'(time)','(volume)')// Payoff histogram xset("window",20);xbasc(); histplot(100,CC) xtitle('Histogram of the optimal payoff with a final value of water') disp('The minimum of the optimal payoff is '+string(min(CC))); disp('The mean of the optimal payoff is '+string(mean(CC))); disp('The maximum of the optimal payoff is '+string(max(CC)));

Figure 1: Stock volumes in a dam following an optimal strategy, with a ﬁnal value of water

Question 2Picture the trajectories of the stocks corresponding to the optimal strategy.Evaluate the optimal expected payoﬀs for diﬀerent values of the initial stock, andcompare them with the value functionevaluated at the initial time t0and theinitial stock. Display the histogram of the optimal payoﬀ. Compare the mean of theoptimal payoﬀ with the upper and lower bounds of the distribution.

Figure 2: Histogram of the optimal payoﬀ, with a ﬁnal value of water

2.5 Evaluation of the probability to satisfy the tourism constraint

Let denote an optimal stock trajectory.

Question 3Evaluate the probability

(16)

that the water volumeremains above F% ofduring the months of July and August,where F% varies between 0% and 100%.

Figure 3: Probability that the summer tourism constraint is satisﬁed under the optimal
strategy, with a ﬁnal value of water

Question 5Launch the above code (maybe you will have to reduce the time step, or thehorizon, and adapt the code in consequence if the computation takes too much time). Visualizethe maximal viability probability starting from an almost full dam. Draw iso-probabilitycurves. Comment on what you observe.