MIDAS: A mixed integer dynamic approximation scheme

Abstract

Mixed integer dynamic approximation scheme (MIDAS) is a new sampling-based algorithm for solving finite-horizon stochastic dynamic programs with monotonic Bellman functions. MIDAS approximates these value functions using step functions, leading to stage problems that are mixed integer programs. We provide a general description of MIDAS, and prove its almost-sure convergence to a \(2T\varepsilon \)-optimal policy for problems with T stages when the Bellman functions are known to be monotonic, and the sampling process satisfies standard assumptions.

Keywords

This research was carried out with financial support from the Laboratoire de Finance des Marchés de l’Energie at INRIA, the Programme Gaspard Monge pour l’Optimisation of the FMHJ Foundation, EDF and Meridian Energy Limited. The first author acknowledges support of the New Zealand Marsden Fund under contract UOA1520.

Proposition 1

For every \(x\in X_{\delta }\),

$$\begin{aligned} \bar{Q}^{k+1}(x)= Q^{k+1}(x). \end{aligned}$$

Proof

For a given point \(x\in X_{\delta }\), consider \(w_{h}\), \(z_{i}^{h}\), \(i=1,2, \ldots ,N\), \(h=1,2,\ldots ,k\) that are feasible for MIP(x). If \(w_{h}=0\), \( h=1,2,\ldots ,k\), then \(\varphi \le \bar{V}\) is the only constraint on \(\varphi \) and so \(\bar{Q}^{k}(x)=\bar{V}\). But \(w_{h}=0,\)\(h=1,2,\ldots ,k\) means that for every such h, \(z_{i}^{h}=1\) for some component i giving

for any such h then choosing \(w_{h}=1\) for any of these would yield a value of \(\varphi \) strictly lower than the value obtained by choosing \( w_{h}=0\) for all of them. So \(w_{h}=0\) is optimal for \(h\notin \mathcal {H}^{k+1}(x)\) . It follows that \(\mathcal {H}^{k+1}(x)= \{h:w_{h}=1\}\). Thus the optimal value of MIP(x) is