► Two failures of the dynamic programming (DP) approach to the stochasticoptimalcontrol problem are investigated. The first failure arises when we wish to solve…
(more)

▼ Two failures of the dynamic programming (DP) approach to the stochasticoptimalcontrol problem are investigated. The first failure arises when we wish to solve a class of certain singular stochasticcontrol problems in continuous time. It has been shown by Lasry and Lions (2000) that this difficulty can be overcome by introducing equivalent standard stochasticcontrol problems. To solve this class of singular stochasticcontrol problems, it remains to solve the equivalent standard stochasticcontrol problems. Since standard stochasticcontrol problems can be solved by applying the DP approach, this then solves the first failure. In the first part of the thesis, we clarify the idea of Lasry and Lions and extend their work to the case of controlled processes with jumps. This is particularly important in financial modelling where such processes are widely applied. For the purpose of application, we applied our result to an optimal trade execution problem studied by Lasry and Lions (2007b). The second failure of the DP approach arises when we wish to solve a multiperiod portfolio selection problem in which a mean-standard-deviation type criterion (a non-separable criterion) is used. We formulate such a problem as a discrete time stochasticcontrol problem. By adapting a pseudo dynamic programming principle, we obtain a closed form optimal strategy for investors whose risk tolerances are larger than a lower bound. As a consequence, we develop a multiperiod portfolio selection scheme. The analysis is performed in the market of risky assets only, however, we allow both market transitions and intermediate cash injections and offtakes. This work provides a good basis for future studies of portfolio selection problems with selection criteria chosen from the class of translation-invariant and positive-homogeneous risk measures.
Advisors/Committee Members: Goldys, Beniamin, The University of Sydney, Penev, Spiridon , Mathematics & Statistics, Faculty of Science, UNSW.

Wu, W. (2016). Limitations of dynamic programming approach: singularity and time inconsistency. (Doctoral Dissertation). University of New South Wales. Retrieved from http://handle.unsw.edu.au/1959.4/56208 ; https://unsworks.unsw.edu.au/fapi/datastream/unsworks:40264/SOURCE02?view=true

Chicago Manual of Style (16th Edition):

Wu, Wei. “Limitations of dynamic programming approach: singularity and time inconsistency.” 2016. Doctoral Dissertation, University of New South Wales. Accessed September 15, 2019.
http://handle.unsw.edu.au/1959.4/56208 ; https://unsworks.unsw.edu.au/fapi/datastream/unsworks:40264/SOURCE02?view=true.

Wu W. Limitations of dynamic programming approach: singularity and time inconsistency. [Doctoral Dissertation]. University of New South Wales; 2016. Available from: http://handle.unsw.edu.au/1959.4/56208 ; https://unsworks.unsw.edu.au/fapi/datastream/unsworks:40264/SOURCE02?view=true

► My PhD thesis concentrates on the field of stochastic analysis, with focus on stochastic optimization and applications in finance. It is composed of two parts:…
(more)

▼ My PhD thesis concentrates on the field of stochastic analysis, with focus on stochastic optimization and applications in finance. It is composed of two parts: the first part studies an optimal stopping problem, and the second part studies an optimalcontrol problem.
The first topic considers a one-dimensional transient and downwards drifting diffusion process X, and detects the optimal times of a random time(denoted as ρ). In particular, we consider two classes of random times: (1) the last time when the process exits a certain level l; (2) the time when the process reaches its maximum. For each random time, we solve the optimization problem
infτ E[λ(τ- ρ)+ +(1-λ)(ρ - τ)+]
overall all stopping times. For the last exit time, the process should stop optimally when it runs below some fixed level k the first time, where k is the solution of an explicit defined equation. For the ultimate maximum time, the process should stop optimally when it runs below a boundary which is the maximal positive solution (if exists) of a first-order ordinary differential equation which lies below the line λs for all s > 0 .
The second topic solves an optimal consumption and investment problem for a risk-averse investor who is sensitive to declines than to increases of standard living (i.e., the investor is loss averse), and the investment opportunities are constant. We use the tools of stochasticcontrol and duality methods to solve the resulting free-boundary problem in an infinite time horizon. Briefly, the investor consumes constantly when holding a moderate amount of wealth. In bliss time, the investor increases the consumption so that the consumption-wealth ratio reaches some fixed minimum level; in gloom time, the investor decreases the consumption gradually. Moreover, high loss aversion tends to raise the consumption-wealth ratio, but cut the investment-wealth ratio overall.

► Motivated by the limitations of current optimalcontrol and reinforcement learning methods in terms of their efficiency and scalability, this thesis proposes an iterative stochastic…
(more)

▼ Motivated by the limitations of current optimalcontrol and reinforcement learning methods in terms of their
efficiency and scalability, this thesis proposes an iterative
stochasticoptimalcontrol approach based on the generalized path
integral formalism. More precisely, we suggest the use of the
framework of stochasticoptimalcontrol with path integrals to
derive a novel approach to RL with parameterized policies. While
solidly grounded in value function estimation and optimalcontrol
based on the stochastic Hamilton Jacobi Bellman (HJB) equation,
policy improvements can be transformed into an approximation
problem of a path integral which has no open algorithmic parameters
other than the exploration noise. The resulting algorithm can be
conceived of as model-based, semi-model-based, or even model free,
depending on how the learning problem is structured. The new
algorithm, Policy Improvement with Path Integrals (PI2),
demonstrates interesting similarities with previous RL research in
the framework of probability matching and provides intuition why
the slightly heuristically motivated probability matching approach
can actually perform well. Applications to high dimensional robotic
systems are presented for a variety of tasks that require optimal
planning and gain scheduling.; In addition to the work on
generalized path integral stochasticoptimalcontrol, in this
thesis we extend model based iterative optimalcontrol algorithms
to the stochastic setting. More precisely we derive the
Differential Dynamic Programming algorithm for stochastic systems
with state and control multiplicative noise. Finally, in the last
part of this thesis, model based iterative optimalcontrol methods
are applied to bio-mechanical models of the index finger with the
goal to find the underlying tendon forces applied for the movements
of, tapping and flexing.
Advisors/Committee Members: Schaal, Stefan (Committee Chair), Valero-Cuevas, Francisco (Committee Member), Sukhatme, Gaurav S. (Committee Member), Todorov, Emo (Committee Member), Schweighofer, Nicolas (Committee Member).

In this thesis, production systems facing abandonments are studied. These problems are modeled as stochastic scheduling problems with due dates. In the literature, few results exist concerning the optimalcontrol of such systems. This thesis aims at providing optimalcontrol policies for systems with impatience. We consider a generic system with a single machine, on which jobs have to be processed. Processing times, due dates (or patience time) and release dates are random variables. A weight is associated to each job and the objective is to minimize the expected weighted number of late jobs. In our study, we use different models, taking into account the specific features of real life problems. For example, we make a difference between impatience, when a customer has been waiting for too long, and abandonment, when a customer leaves the system after getting impatient. In the class of static list scheduling policies, we provide optimal schedules for problems with impatience. In the class of preemptive dynamic policies, we specify conditions under which a strict priority rule is optimal and we give a new heuristic, both extending previous results from the literature. We study variants and extensions of these problems, when several machines are available or when preemption is not authorized.

This thesis introduces a selection of models for optimal execution of financial assets at the tactical level. As opposed to optimal scheduling, which defines a…
(more)

▼

This thesis introduces a selection of models for
optimal execution of financial assets at the tactical level. As
opposed to optimal scheduling, which defines a trading schedule for
the trader, this thesis investigates how the trader should interact
with the order book. If a trader is aggressive he will execute his
order using market orders, which will negatively feedback on his
execution price through market impact. Alternatively, the models we
focus on consider a passive trader who places limit orders into the
limit-order book and waits for these orders to be filled by market
orders from other traders. We assume these models do not exhibit
market impact. However, given we await market orders from other
participants to fill our limit orders a new risk is borne:
execution risk.We begin with an extension of Guéant et al. (2012b)
who through the use of an exponential utility, standard Brownian
motion, and an absolute decay parameter were able to cleverly build
symmetry into their model which significantly reduced the
complexity. Our model consists of geometric Brownian motion (and
mean-reverting processes) for the asset price, a proportional
control parameter (the additional amount we ask for the asset), and
a proportional decay parameter, implying that the symmetry found in
Guéant et al. (2012b) no longer exists. This novel combination
results in asset-dependent trading strategies, which to our
knowledge is a unique concept in this framework of literature.
Detailed asymptotic analyses, coupled with advanced numerical
techniques (informing the asymptotics) are exploited to extract the
relevant dynamics, before looking at further extensions using
similar methods.We examine our above mentioned framework, as well
as that of Guéant et al. (2012), for a trader who has a basket of
correlated assets to liquidate. This leads to a higher-dimensional
model which increases the complexity of both numerically solving
the problem and asymptotically examining it. The solutions we
present are of interest, and comparable with Markowitz portfolio
theory. We return to our framework of a single underlying and
consider four extensions: a stochastic volatility model which
results in an added dimension to the problem, a constrained
optimisation problem in which the control has an explicit lower
bound, changing the exponential intensity to a power intensity
which results in a reformulation as a singular stochasticcontrol
problem, and allowing the trader to trade using both market orders
and limit orders resulting in a free-boundary problem.We complete
the study with an empirical analysis using limit-order book data
which contains multiple levels of the book. This involves a novel
calibration of the intensity functions which represent the
limit-order book, before backtesting and analysing the performance
of the strategies.

Blair, J. (2016). Modelling approaches for optimal liquidation under a
limit-order book structure. (Doctoral Dissertation). University of Manchester. Retrieved from http://www.manchester.ac.uk/escholar/uk-ac-man-scw:300421

Chicago Manual of Style (16th Edition):

Blair, James. “Modelling approaches for optimal liquidation under a
limit-order book structure.” 2016. Doctoral Dissertation, University of Manchester. Accessed September 15, 2019.
http://www.manchester.ac.uk/escholar/uk-ac-man-scw:300421.

Blair J. Modelling approaches for optimal liquidation under a
limit-order book structure. [Doctoral Dissertation]. University of Manchester; 2016. Available from: http://www.manchester.ac.uk/escholar/uk-ac-man-scw:300421

University of Manchester

6.
Blair, James.
Modelling approaches for optimal liquidation under a limit-order book structure.

► This thesis introduces a selection of models for optimal execution of financial assets at the tactical level. As opposed to optimal scheduling, which defines a…
(more)

▼ This thesis introduces a selection of models for optimal execution of financial assets at the tactical level. As opposed to optimal scheduling, which defines a trading schedule for the trader, this thesis investigates how the trader should interact with the order book. If a trader is aggressive he will execute his order using market orders, which will negatively feedback on his execution price through market impact. Alternatively, the models we focus on consider a passive trader who places limit orders into the limit-order book and waits for these orders to be filled by market orders from other traders. We assume these models do not exhibit market impact. However, given we await market orders from other participants to fill our limit orders a new risk is borne: execution risk. We begin with an extension of Guéant et al. (2012b) who through the use of an exponential utility, standard Brownian motion, and an absolute decay parameter were able to cleverly build symmetry into their model which significantly reduced the complexity. Our model consists of geometric Brownian motion (and mean-reverting processes) for the asset price, a proportional control parameter (the additional amount we ask for the asset), and a proportional decay parameter, implying that the symmetry found in Guéant et al. (2012b) no longer exists. This novel combination results in asset-dependent trading strategies, which to our knowledge is a unique concept in this framework of literature. Detailed asymptotic analyses, coupled with advanced numerical techniques (informing the asymptotics) are exploited to extract the relevant dynamics, before looking at further extensions using similar methods. We examine our above mentioned framework, as well as that of Guéant et al. (2012), for a trader who has a basket of correlated assets to liquidate. This leads to a higher-dimensional model which increases the complexity of both numerically solving the problem and asymptotically examining it. The solutions we present are of interest, and comparable with Markowitz portfolio theory. We return to our framework of a single underlying and consider four extensions: a stochastic volatility model which results in an added dimension to the problem, a constrained optimisation problem in which the control has an explicit lower bound, changing the exponential intensity to a power intensity which results in a reformulation as a singular stochasticcontrol problem, and allowing the trader to trade using both market orders and limit orders resulting in a free-boundary problem. We complete the study with an empirical analysis using limit-order book data which contains multiple levels of the book. This involves a novel calibration of the intensity functions which represent the limit-order book, before backtesting and analysing the performance of the strategies.

► This thesis solves estimation and control problems in computational neuroscience, mathematically dealing with the first-passage times of diffusion stochastic processes. We first derive estimation algorithms…
(more)

▼ This thesis solves estimation and control problems in computational
neuroscience, mathematically dealing with the first-passage times of diffusion
stochastic processes. We first derive estimation algorithms for model parameters
from first-passage time observations, and then we derive algorithms for the
control of first-passage times. Finally, we solve an optimal design
problem which combines elements of the first two: we ask how to elicit
first-passage times such as to facilitate model estimation based on said
first-passage observations.
The main mathematical tools used are the Fokker-Planck partial differential
equation for evolution of probability densities, the Hamilton-Jacobi-Bellman
equation of optimalcontrol and the adjoint optimization principle from optimalcontrol theory.
The focus is on developing computational schemes for the
solution of the problems. The schemes are implemented and are tested for a wide
range of parameters.

► This dissertation consists of four parts that revolve around structured stochastic uncertainty and optimalcontrol/estimation theory.In the first part, we consider the continuous-time setting of…
(more)

▼ This dissertation consists of four parts that revolve around structured stochastic uncertainty and optimalcontrol/estimation theory.In the first part, we consider the continuous-time setting of linear time-invariant (LTI) systems in feedback with multiplicative stochastic uncertainties. The objective is to characterize the conditions of Mean-Square Stability (MSS) using a purely input-output approach. This approach leads to uncovering new tools such as stochastic block diagrams. Various stochastic interpretations are considered, such as It\=o and Stratonovich, and block diagram conversion schemes between different interpretations are devised. The MSS conditions are given in terms of the spectral radius of a matrix operator that takes different forms when different stochastic interpretations are considered. The second part applies the developed theory to analyze the mean-square stability and performance of stochastic cochlear models. The analysis is carried out for a generalized class of biomechanical models of the cochlea, that is formulated as a stochastic spatially distributed system, by allowing stochastic spatio-temporal perturbations within the cochlear amplifier. The simulation-free analysis explains the underlying mechanisms that give rise to cochlear instabilities such as spontaneous otoacoustic emissions and/or tinnitus. Furthermore, nonlinear stochastic simulations are carried out to validate the predictions of the theoretical analysis. The third part revisits the development of numerical methods to solve optimalcontrol problems using a function-space approach. This approach has the advantage of unifying the framework upon which the various (existing) numerical methods are based on. In fact, this approach motivates the definition of various system and projection operators that make the derivations conceptually transparent. Furthermore, the function-space approach builds useful geometric intuitions that inspire the development of new projection-based methods.In the last part, we propose a methodology of optimal path design for sensors through a distributed environment. We consider time-limited scenarios where the sensors can only make a small number of measurements, but where some portion of a physics-based model is available for the field of interest (such as temperature). We consider both point-wise and tomographic sensors. The main idea is to recast the sensor path planning problem as a deterministic optimalcontrol problem to minimize metrics related to the optimal estimation error covariance.

Filo, M. G. (2018). Topics in Stochastic Stability, Optimal Control and Estimation Theory. (Thesis). University of California – eScholarship, University of California. Retrieved from http://www.escholarship.org/uc/item/7nh0k0dp

Note: this citation may be lacking information needed for this citation format:Not specified: Masters Thesis or Doctoral Dissertation

Note: this citation may be lacking information needed for this citation format:Not specified: Masters Thesis or Doctoral Dissertation

Council of Science Editors:

Filo MG. Topics in Stochastic Stability, Optimal Control and Estimation Theory. [Thesis]. University of California – eScholarship, University of California; 2018. Available from: http://www.escholarship.org/uc/item/7nh0k0dp

Note: this citation may be lacking information needed for this citation format:Not specified: Masters Thesis or Doctoral Dissertation

►Stochasticoptimalcontrol has seen significant recent development, motivated by its success in a plethora of engineering applications, such as autonomous systems, robotics, neuroscience, and…
(more)

▼Stochasticoptimalcontrol has seen significant recent development, motivated by its success in a plethora of engineering applications, such as autonomous systems, robotics, neuroscience, and financial engineering. Despite the many theoretical and algorithmic advancements that made such a success possible, several obstacles remain; most notable are (i) the mitigation of the curse of dimensionality inherent in optimalcontrol problems, (ii) the design of efficient algorithms that allow for fast, online computation, and (iii) the expansion of the class of optimalcontrol problems that can be addressed by algorithms in engineering practice. The aim of this dissertation is the development of a learning stochasticcontrol framework which capitalizes on the innate relationship between certain nonlinear partial differential equations (PDEs) and forward and backward stochastic differential equations (FBSDEs), demonstrated by a nonlinear version of the Feynman-Kac lemma. By means of this lemma, we are able to obtain a probabilistic representation of the solution to the nonlinear Hamilton-Jacobi-Bellman PDE, expressed in form of a system of decoupled FBSDEs. This system of FBSDEs can then be simulated by employing linear regression techniques. We present a novel discretization scheme for FBSDEs, and enhance the resulting algorithm with importance sampling, thereby constructing an iterative scheme that is capable of learning the optimalcontrol without an initial guess, even in systems with highly nonlinear, underactuated dynamics. The framework we develop within this dissertation addresses several classes of stochasticoptimalcontrol, such as L2, L1, risk sensitive control, as well as some classes of differential games, in both fixed-final-time as well as first-exit settings.
Advisors/Committee Members: Tsiotras, Panagiotis (advisor), Theodorou, Evangelos A. (advisor), Haddad, Wassim M. (committee member), Zhou, Haomin (committee member), Popescu, Ionel (committee member).

► The behavior of physical, chemical, and biological systems can exhibit significant sensitivity to uncertainty or variation in system parameters. This factor arises in practical…
(more)

▼ The behavior of physical, chemical, and biological systems can exhibit significant sensitivity to uncertainty or variation in system parameters. This factor arises in practical control problems in many areas of science and engineering when there is uncertainty in the parameters of a single control system, or when a collection of structurally similar systems with variations in common parameters must be steered using a common control signal. Analysis of these cases has given rise to the subject of ensemble control, which was motivated by practical control design problems in the fields of nuclear magnetic resonance spectroscopy and imaging, neuroscience, and sensorless robotic manipulation. This dissertation focuses on the investigation of fundamental properties and the development of optimal controls for deterministic and stochastic linear ensemble systems. Although the ensemble controllability for deterministic linear ensemble systems has been characterized in previous studies, explicit controllability conditions remain undiscovered. In this dissertation, explicit controllability conditions for a class of time-invariant linear ensemble systems with linear parameter variation are constructed. This class of ensemble control systems arises from practical engineering and physical applications, such as the transport of quantum particles and control of uncertain harmonic systems. The construction is based on the notion of polynomial approximation, and the conditions are related to the rank of the system matrices and are easy to verify. In addition to the study of deterministic ensemble control systems, we extend our work to a stochastic case where the ensemble systems are subject to random dynamic disturbances. Such disturbances can greatly affect the behavior of systems, which can become particularly challenging to control in a desired manner as a result, especially when feedback cannot be used to attenuate disturbances. We study optimal steering problems involving stochastic linear ensemble systems driven by Gaussian noise and Poisson counters. In particular, we seek to minimize the statistical objectives of the mean square error (MSE) and the error in the mean of the terminal state of the ensemble with respect to the desired state.
Advisors/Committee Members: Hiro Mukai, Heinz Schaettler, Istvan Kiss.

Qi, J. (2014). Control of Deterministic and Stochastic Linear Ensemble Systems. (Doctoral Dissertation). Washington University in St. Louis. Retrieved from https://openscholarship.wustl.edu/eng_etds/71

► In stochasticoptimalcontrol theory, the complete specification of the probability density function of the random cost functional might be considered the most a…
(more)

▼ In stochasticoptimalcontrol theory, the
complete specification of the probability density function of the
random cost functional might be considered the most a designer can
do when formulating an optimalcontrol law. The field of cost
cumulant controls has made considerable advances towards this
capability in the past few decades, largely because of the
advantage gained from controlling the cost cumulants instead of the
cost moments. However, current cost cumulant control paradigms have
left the deliberate specification of the probability density
function for the random cost outside the designer’s direct
influence. The ability to design control laws upon the desired
shape and location of the cost density would be highly valuable to
control engineers, since there is evidence that the shape of the
cost density under high-performance controllers directly
corresponds to the resulting closed-loop system behavior.
This dissertation proposes
a Multiple-Cumulant Cost Density-Shaping (MCCDS) optimization
problem for the LQG framework. The control solution to the MCCDS
optimization is derived using dynamic programming techniques; it is
the finite-horizon, linear state-feedback control that minimizes a
smooth, convex, scalar function of arbitrarily-many initial cost
cumulants and target initial cost cumulants. The MCCDS theory is
shown to generalize the Linear Quadratic Gaussian (LQG), “k Cost
Cumulant” (kCC), and Risk-Sensitive (RS) control paradigms for zero
targets and linear performance indices. Additionally, the MCCDS
framework enables the minimization of well-known distance functions
between the cost density and a target cost density, such as the
Kullback-Leibler Divergence, Bhattacharyya Distance, and the
Hellinger Distance. The
finite-horizon MCCDS control is extended to the infinite-horizon in
this dissertation. Other areas of investigation include MCCDS
performance index construction, cost density-shaping minimax and
Nash games, and a Statistical Target Selection (STS) iterative
procedure for control design. Together MCCDS and STS enable the
design of control laws with optimality among a family of target
cost densities, which might be regarded as a new approach to robust
LQG control design. For structures excited by seismic disturbances,
numerical experiments show that the MCCDS controls identified thru
STS can achieve greater vibration suppression than nominal kCC
controllers, without any compromise to robust
stability.
Advisors/Committee Members: Dr. Ron Diersing, Committee Member, Dr. Peter Bauer, Committee Member, Dr. Vijay Gupta, Committee Member, Dr. Yih-Fang Huang, Committee Member, Dr. Panos Antsaklis, Committee Chair.

▼ Local electricity markets can be defined broadly as
"future electricity market designs involving domestic customers,
demand-side response and energy storage". Like current deregulated
electricity markets, these localised derivations present specific
stochastic optimisation problems in which the dynamic and random
nature of the market is intertwined with the physical needs of its
participants. Moreover, the types of contracts and constraints in
this setting are such that ``games'' naturally emerge between the
agents. Advanced modelling techniques beyond classical mathematical
finance are therefore key to their analysis. This thesis aims to
study contracts in these local electricity markets using the
mathematical theories of stochasticoptimalcontrol and
games.Chapter 1 motivates the research, provides an overview of the
electricity market in Great Britain, and summarises the content of
this thesis. It introduces three problems which are studied later
in the thesis: a simple control problem involving demand-side
management for domestic customers, and two examples of games within
local electricity markets, one of them involving energy storage.
Chapter 2 then reviews the literature most relevant to the topics
discussed in this work.Chapter 3 investigates how electric space
heating loads can be made responsive to time varying prices in an
electricity spot market. The problem is formulated mathematically
within the framework of deterministic optimalcontrol, and is
analysed using methods such as Pontryagin's Maximum Principle and
Dynamic Programming. Numerical simulations are provided to
illustrate how the control strategies perform on real market
data.The problem of Chapter 3 is reformulated in Chapter 4 as one
of optimal switching in discrete-time. A martingale approach is
used to establish the existence of an optimal strategy in a very
general setup, and also provides an algorithm for computing the
value function and the optimal strategy. The theory is exemplified
by a numerical example for the motivating problem. Chapter 5 then
continues the study of finite horizon optimal switching problems,
but in continuous time. It also uses martingale methods to prove
the existence of an optimal strategy in a fairly general
model.Chapter 6 introduces a mathematical model for a game
contingent claim between an electricity supplier and generator
described in the introduction. A theory for using optimal switching
to solve such games is developed and subsequently evidenced by a
numerical example. An optimal switching formulation of the
aforementioned game contingent claim is provided for an abstract
Markovian model of the electricity market.The final chapter studies
a balancing services contract between an electricity transmission
system operator (SO) and the owner of an electric energy storage
device (battery operator or BO). The objectives of the SO and BO
are combined in a non-zero sum stochastic differential game where
one player (BO) uses a classic control with continuous effects,
whereas the other player (SO) uses an impulse…
Advisors/Committee Members: PESKIR, GORAN G, Moriarty, John, Peskir, Goran.

► Uruguay is a pioneer in the use of renewable sources of energy and can usually satisfy its total demand from renewable sources. Control and optimization…
(more)

▼ Uruguay is a pioneer in the use of renewable sources of energy and can usually satisfy its total demand from renewable sources. Control and optimization of the system is complicated by half of the installed power - wind and solar sources - be- ing non-controllable with high uncertainty and variability. In this work we present a novel optimization technique for efficient use of the production facilities. The dy- namical system is stochastic, and we deal with its non-Markovian dynamics through a Lagrangian relaxation. Continuous-time optimalcontrol and value function are found from the solution to a sequence of Hamilton-Jacobi-Bellman partial differential equations associated with the system. We introduce a monotone scheme to avoid spurious oscillations in the numerical solution and apply the technique to a number of examples taken from the Uruguayan grid. We use parallelization and change of variables to reduce the computational times. Finally, we study the usefulness of extra system storage capacity offered by batteries.

► Fokker-Planck equations, along with stochastic differential equations, play vital roles in physics, population modeling, game theory and optimization (finite or infinite dimensional). In this thesis,…
(more)

▼ Fokker-Planck equations, along with stochastic differential equations, play vital roles in physics, population modeling, game theory and optimization (finite or infinite dimensional). In this thesis, we study three topics, both theoretically and computationally, centered around them. In part one, we consider the optimal transport for finite discrete states, which are on a finite but arbitrary graph. By defining a discrete 2-Wasserstein metric, we derive Fokker-Planck equations on finite graphs as gradient flows of free energies. By using dynamical viewpoint, we obtain an exponential convergence result to equilibrium. This derivation provides tools for many applications, including numerics for nonlinear partial differential equations and evolutionary game theory. In part two, we introduce a new stochastic differential equation based framework for optimalcontrol with constraints. The framework can efficiently solve several real world problems in differential games and Robotics, including the path-planning problem. In part three, we introduce a new noise model for stochastic oscillators. With this model, we prove global boundedness of trajectories. In addition, we derive a pair of associated Fokker-Planck equations.
Advisors/Committee Members: Dieci, Luca (advisor), Chow, Shui Nee (committee member), Zhou, Haomin (committee member), Egerstedt, Magnus (committee member), Gangbo, Wilfrid (committee member).

▼ This thesis presents novel methods for computing optimal pre-commitment strategies in time-inconsistent optimalstochasticcontrol and optimal stopping problems. We demonstrate how a time-inconsistent problem can often be re-written in terms of a sequential optimization problem involving the value function of a time-consistent optimalcontrol problem in a higher-dimensional state-space. In particular, we obtain optimal pre-commitment strategies in a non-linear optimal stopping problem, in an optimalstochasticcontrol problem involving conditional value-at-risk, and in an optimal stopping problem with a distribution constraint on the admissible stopping times. In each case, we relate the original problem to auxiliary time-consistent problems, the value functions of which may be characterized in terms of viscosity solutions of a Hamilton-Jacobi-Bellman equation.

► We investigate the impact of cash reserves upon the optimal behaviour of a modelled firm that has uncertain future revenues. To achieve this, we build…
(more)

▼ We investigate the impact of cash reserves upon the
optimal behaviour of a modelled firm that has uncertain future
revenues. To achieve this, we build up a corporate financing model
of a firm from a Real Options foundation, with the option to close
as a core business decision maintained throughout. We model the
firm by employing an optimalstochasticcontrol mathematical
approach, which is based upon a partial differential equations
perspective. In so doing, we are able to assess the incremental
impacts upon the optimal operation of the cash constrained firm, by
sequentially including: an optimal dividend distribution; optimal
equity financing; and optimal debt financing (conducted in a novel
equilibrium setting between firm and creditor). We present
efficient numerical schemes to solve these models, which are
generally built from the Projected Successive Over Relaxation
(PSOR) method, and the Semi-Lagrangian approach. Using these
numerical tools, and our gained economic insights, we then allow
the firm the option to also expand the operation, so they may also
take advantage of favourable economic conditions.
Advisors/Committee Members: JOHNSON, PAUL PV, Johnson, Paul, Evatt, Geoffrey.

►Control of autonomous vehicle teams has emerged as a key topic in the control and robotics communities, owing to a growing range of applications that…
(more)

▼Control of autonomous vehicle teams has emerged as a key topic in the control and robotics communities, owing to a growing range of applications that can benefit from the increased functionality provided by multiple vehicles. However, the mathematical analysis of the vehicle control problems is complicated by their nonholonomic and kinodynamic constraints, and, due to environmental uncertainties and information flow constraints, the vehicles operate with heightened uncertainty about the team's future motion. In this dissertation, we are motivated by autonomous vehicle control problems that highlight these uncertainties, with in particular attention paid to the uncertainty in the future motion of a secondary agent. Focusing on the Dubins vehicle and unicycle model, we propose a stochastic modeling and optimal feedback control approach that anticipates the uncertainty inherent to the systems. We first consider the application of a Dubins vehicle that should maintain a nominal distance from a target with an unknown future trajectory, such as a tagged animal or vehicle. Stochasticity is introduced in the problem by assuming that the target's motion can be modeled as a Wiener process, and the possibility for the loss of target observations is modeled using stochastic transitions between discrete states. An optimalcontrol policy that is consistent with the stochastic kinematics is computed and is shown to perform well both in the case of a Brownian target and for natural, smooth target motion. We also characterize the resulting optimal feedback control laws in comparison to their deterministic counterparts for the case of a Dubins vehicle in a stochastically varying wind. Turning to the case of multiple vehicles, we develop a method using a Kalman smoothing algorithm for multiple vehicles to enhance an underlying analytic feedback control. The vehicles achieve a formation optimally and in a manner that is robust to uncertainty. To deal with a key implementation issue of these controllers on autonomous vehicle systems, we propose a self-triggering scheme for stochasticcontrol systems, whereby the time points at which the control loop should be closed are computed from predictions of the process in a way that ensures stability.

► This dissertation presents a series of three essays that examine applications and computational issues associated with the use of stochasticoptimalcontrol modeling in the…
(more)

▼ This dissertation presents a series of three essays that examine applications and computational issues associated with the use of stochasticoptimalcontrol modeling in the field of economics. In the first essay we examine the problem of valuing brownfield remediation and redevelopment projects amid regulatory and market uncertainty. A real options framework is developed to model the dynamic behavior of developers working with environmentally contaminated land in an investment environment with stochastic real estate prices and an uncertain entitlement process. In a case study of an actual brownfield regeneration project we examine the impact of entitlement risk on the value of the site and optimal developer behavior. The second essay presents a numerical method for solving optimal switching models combined with a stochasticcontrol. For this class of hybrid control problems the value function and the optimalcontrol policy are the solution to a Hamilton-Jacobi-Bellman quasi-variational inequality. We present a technique whereby approximating the value function using projection methods the Hamilton-Jacobi-Bellman quasi-variational inequality may be recast as extended vertical non-linear complementarity problem that may be solved using Newton's method. In the third essay we present a new method for estimating the parameters of stochastic differential equations using low observation frequency data. The technique utilizes a quasi-maximum likelihood framework with the assumption of a Gaussian conditional transition density for the process. In order to reduce the error associated with the normality assumption sub-intervals are incorporated and integrated out using the Chapman-Kolmogorov equation and multi-dimensional Gauss Hermite quadrature. Further improvements are made through the use of Richardson extrapolation and higher order approximations for the conditional mean and variance of the process, resulting in an algorithm that may easily produce third and fourth order approximations for the conditional transition density.
Advisors/Committee Members: Roger von Haefen, Committee Member (advisor), Paul Fackler, Committee Chair (advisor), Denis Pelletier, Committee Member (advisor), John Seater, Committee Member (advisor).

► This research work presents a hierarchical nonlinear optimization-based framework for planning and scheduling of supply networks modeled as multi-class stochastic queueing networks. More precisely, the…
(more)

▼ This research work presents a hierarchical nonlinear
optimization-based framework for planning and scheduling of supply
networks modeled as multi-class stochastic queueing networks. More
precisely, the framework has two decision layers. At the top level
a tactical processing plan is designed. This is accomplished by
means of a nonlinear optimalcontrol formulation, along with
suitable solution algorithms, to compute on a rolling-horizon basis
a tactical processing plan which yields the lowest cost
expected-value inventory trajectory. In doing so, raw materials,
inventory and demand mismatching costs are considered. As the main
outcome, a target inventory trajectory is obtained for each
inventory buer in the network at all times within a rolling
planning time window. The bottom layer in turn deals with a
distributed scheduling framework to best track the inventory
targets generated by the tactical processing plan. In this regard,
a processing schedule is generated for each server in the system so
that sequence-dependent and inventory holding costs are minimized
for each server within the current planning time window.
Uncertainty in the inventory accumulation and depletion processes
as well as network interrelations is accounted for by means of a
robust optimization formulation. Several numerical examples are
presented in order to illustrate the framework mechanics as well as
the algorithmic issues inherent to the different optimization steps
performed on a hierarchical basis. The planning framework can also
be adapted to standard Enterprise Resource Systems (ERPs) to best
support the planning and scheduling functions on a regular
basis.

A Work Project, presented as part of the requirements for the Award of a Masters Degree in Finance from the NOVA – School of Business…
(more)

▼

A Work Project, presented as part of the requirements for the Award of a Masters Degree in Finance from the NOVA – School of Business and Economics

The increasing global attention to greenhouse emissions and the recent creation of EU Emission Trading Scheme has clearly suggested the need of consistent methods to value projects aimed to reduce gases. This need particularly concerns companies that have to find a way to both remain profitable and conform to new legal requirements. Multiple ways of cutting emission costs are available nowadays: short term abatement measures, which primary involve switching production machinery from coal to gas; long term abatement measures, which envisage the implementation of new types of projects .e.g Clean Development Mechanism or Joint Implementation Mechanism suggested by Kyoto Protocol -. In this work we study the impact of the introduction of both kinds of policy in a pricing model for CO2 allowances.

► Tuberculosis (TB) is currently one of the major public health challenges in South Africa, and in many countries. Mycobacterium tuberculosis is among the leading causes…
(more)

▼ Tuberculosis (TB) is currently one of the major public health challenges in South Africa, and in many countries. Mycobacterium tuberculosis is among the leading causes of morbidity and mortality. It is known that tuberculosis is a curable infectious disease. In the case of incomplete treatment, however, the remains of Mycobacterium tuberculosis in the human system often results in the bacterium developing resistance to antibiotics. This leads to relapse and treatment against the resistant bacterium is extremely expensive and difficult. The aim of this work is to present and analyse mathematical models of the population dynamics of tuberculosis for the purpose of studying the effects of efficient treatment versus incomplete treatment. We analyse the spread, asymptotic behavior and possible eradication of the disease, versus persistence of tuberculosis. In particular, we
consider inflow of infectives into the population, and we study the effects of screening. A sub-model will be studied to analyse the transmission dynamics of TB in an isolated population. The full model will take care of the inflow of susceptibles as well as inflow of TB infectives into the population. This dissertation enriches the existing literature with contributions in the form of optimalcontrol and stochastic perturbation. We also show how stochastic perturbation can improve the stability of an equilibrium point. Our methods include Lyapunov functions, optimalcontrol and stochastic differential equations. In the stability analysis of the DFE we show how backward bifurcation appears. Various phenomena are illustrated by way of simulations.
Advisors/Committee Members: Witbooi, Peter J (advisor).

Adebiyi AO. Mathematical modeling of the population dynamics of tuberculosis
. [Internet] [Thesis]. University of the Western Cape; 2016. [cited 2019 Sep 15].
Available from: http://hdl.handle.net/11394/4928.

Note: this citation may be lacking information needed for this citation format:Not specified: Masters Thesis or Doctoral Dissertation

Council of Science Editors:

Adebiyi AO. Mathematical modeling of the population dynamics of tuberculosis
. [Thesis]. University of the Western Cape; 2016. Available from: http://hdl.handle.net/11394/4928

Note: this citation may be lacking information needed for this citation format:Not specified: Masters Thesis or Doctoral Dissertation

► This dissertation considers a stochastic dynamic system which is governed by a multidimensional diffusion process with time dependent coefficients. The control acts additively on…
(more)

▼ This dissertation considers a stochastic dynamic system which is governed by a multidimensional diffusion process with time dependent coefficients. The control acts additively on the state of the system. The objective is to minimize the expected cumulative cost associated with the position of the system and the amount of control exerted. It is proved that Hamilton-Jacobi-Bellman’s equation of the problem has a solution, which corresponds to the optimal cost of the problem. We also investigate the smoothness of the free boundary arising from the problem.
In the second part of the dissertation, we study the backward parabolic problem for a nonlinear parabolic equation of the form ut + Au(t) = f (t, u(t)), u(T ) = ϕ, where A is a positive self-adjoint unbounded operator and f is a Lipschitz function. The problem is ill-posed, in the sense that if the solution does exist, it will not depend continuously on the data. To regularize the problem, we use the quasi-reversibility method to establish a modified problem. We present approximated solutions that depend on a small parameter ε > 0 and give error estimates for our regularization. These
results extend some work on the nonlinear backward problem. Some numerical examples are given to justify the theoretical analysis.
Advisors/Committee Members: Jose L. Menaldi.

This dissertation demonstrates that there is high revenue potential in using limit order book imbalance as a state variable in an algorithmic trading strategy. Beginning…
(more)

▼

This dissertation demonstrates that there is high revenue potential in using limit order book imbalance as a state variable in an algorithmic trading strategy. Beginning with the hypothesis that imbalance of bid/ask order volumes is an indicator for future price changes, exploratory data analysis suggests that modelling the joint distribution of imbalance and observed price changes as a continuous-time Markov chain presents a monetizable opportunity. The arbitrage problem is then formalized mathematically as a stochasticoptimalcontrol problem using limit orders and market orders with the aim of maximizing terminal wealth. The problem is solved in both continuous and discrete time using the dynamic programming principle, which produces both conditions for market order execution, as well as limit order posting depths, as functions of time, inventory, and imbalance. The optimal controls are calibrated and backtested on historical NASDAQ ITCH data, which produces consistent and substantial revenue.

An economic agent is faced with the problem of deciding how to allocate her wealth among consumption and investment in an underlying financial market, in order to maximize the expected utility derived from the instantaneous consumption over a given time interval and her wealth at the final horizon. We assume that the financial market under consideration consists of a riskless asset and a risky asset, the latter having the special feature that its temporal evolution is given by a stochastic differential equation with mean reversion coefficients. Dynamic programming techniques will be employed to solve this StochasticOptimalControl problem with the goal of finding the optimal strategies for consumption and investment and compare it with Merton's optimal strategies.

▼ Constraint handling is difficult in model predictive control (MPC) of linear differential inclusions (LDIs) and linear parameter varying (LPV) systems. The designer is faced with a choice of using conservative bounds that may give poor performance, or accurate ones that require heavy online computation. This thesis presents a framework to achieve a more flexible trade-off between these two extremes by using a state tube, a sequence of parametrised polyhedra that is guaranteed to contain the future state. To define controllers using a tube, one must ensure that the polyhedra are a sub-set of the region defined by constraints. Necessary and sufficient conditions for these subset relations follow from duality theory, and it is possible to apply these conditions to constrain predicted system states and inputs with only a little conservatism. This leads to a general method of MPC design for uncertain-parameter systems. The resulting controllers have strong theoretical properties, can be implemented using standard algorithms and outperform existing techniques. Crucially, the online optimisation used in the controller is a convex problem with a number of constraints and variables that increases only linearly with the length of the prediction horizon. This holds true for both LDI and LPV systems. For the latter it is possible to optimise over a class of gain-scheduled control policies to improve performance, with a similar linear increase in problem size. The framework extends to stochastic LDIs with chance constraints, for which there are efficient suboptimal methods using online sampling. Sample approximations of chance constraint-admissible sets are generally not positively invariant, which motivates the novel concept of âsample-admissible' sets with this property to ensure recursive feasibility when using sampling methods. The thesis concludes by introducing a simple, convex alternative to chance-constrained MPC that applies a robust bound to the time average of constraint violations in closed-loop.

► We study a class of stochastic duopoly games inspired by the two time-scale feature of many markets. The firms convert their short-term “local” advantage driven…
(more)

▼ We study a class of stochastic duopoly games inspired by the two time-scale feature of many markets. The firms convert their short-term “local” advantage driven by exogenous infinitesimal shocks into a more durable gain through long-term market dominance. As an extension of existing literature, we consider two asymmetric players each of whom adopts timing strategies to increase her profitability and possibly bring negative externality to the rival. In turn, this leads us to more general settings of nonzero-sum games. Characterizing Nash equilibrium as a fixed-point of each player’s best-response to her rival, we construct threshold-type Feedback Nash Equilibrium via best response iteration. Our main contribution is explicitly constructing equilibria for types of duopoly games that represent a wide range of industries. Motivated by the competition among sectors of power generators, we consider a duopoly of producers with finite options to increase their production capacity. We study nonzero-sum games in which two players compete for market dominance via switching controls. We also study mixed switching and impulses games inspired by the vertical competition among the producers and consumers of a commodity. Our analysis quantifies the dynamic competition effects and brings economic insights.

► This study aims to address the problem of attitude control of spacecraft in presence of thrust uncertainty, which leads to stochastic accelerations. Spacecraft equipped with…
(more)

▼ This study aims to address the problem of attitude control of spacecraft in presence of thrust uncertainty, which leads to stochastic accelerations. Spacecraft equipped with electric propulsion and other low thrust mechanisms, often experience random fluctuations in thrust. These stochastic processes arise from sources such as uncertain power supply output, varying propellant flow rate, faulty thrusters, etc. Mission requirements and mass/fuel limitations demand an optimal and proactive method of control to mitigate the thrust uncertainty and parasitic torque. Stabilizing stochasticoptimalcontrol of the satellite attitude dynamics is derived through formulation of the Hamilton-Jacobi-Bellman equation associated with a stochastic differential equation. The solution to the Hamilton-Jacobi-Bellman partial differential equation is approximated through the method of Al’brekht [1]. Extension of Albrekht method for a stochastic system was first presented in [2]; detailed derivations of linear and nonlinear stochasticcontrol laws along with their analytical and numerical analyses are presented in this thesis. A planning method is then discussed to lower the error due to local nature of the control.
Advisors/Committee Members: Namachchivaya, Navaratnam S. (advisor), Ho, Koki (advisor).

► In this thesis we investigate single and multi-player stochastic dynamic optimization prob-lems. We consider both discrete and continuous time processes. In the multi-player setup we…
(more)

▼ In this thesis we investigate single and multi-player stochastic dynamic optimization prob-lems. We consider both discrete and continuous time processes. In the multi-player setup we investigate zero-sum games with both complete and partial information. We study partially observable stochastic games with average cost criterion and the state process be-ing discrete time controlled Markov chain. The idea involved in studying this problem is to replace the original unobservable state variable with a suitable completely observable state variable. We establish the existence of the value of the game and also obtain optimal strategies for both players. We also study a continuous time zero-sum stochastic game with complete observation. In this case the state is a pure jump Markov process. We investigate the nite horizon total cost criterion. We characterise the value function via appropriate Isaacs equations. This also yields optimal Markov strategies for both players.
In the single player setup we investigate risk-sensitive control of continuous time Markov chains. We consider both nite and in nite horizon problems. For the nite horizon total cost problem and the in nite horizon discounted cost problem we characterise the value function as the unique solution of appropriate Hamilton Jacobi Bellman equations. We also derive optimal Markov controls in both the cases. For the in nite horizon average cost case we shown the existence of an optimal stationary control. we also give a value iteration scheme for computing the optimalcontrol in the case of nite state and action spaces.
Further we introduce a new class of stochastic processes which we call stochastic processes with \age-dependent transition rates". We give a rigorous construction of the process. We prove that under certain assunptions the process is Feller. We also compute the limiting probabilities for our process. We then study the controlled version of the above process. In this case we take the risk-neutral cost criterion. We solve the in nite horizon discounted cost problem and the average cost problem for this process. The crucial step in analysing these problems is to prove that the original control problem is equivalent to an appropriate semi-Markov decision problem. Then the value functions and optimal controls are characterised using this equivalence and the theory of semi-Markov decision processes (SMDP). The analysis of nite horizon problems becomes di erent from that of in nite horizon problems because of the fact that in this case the idea of converting into an equivalent SMDP does not seem to work. So we deal with the nite horizon total cost problem by showing that our problem is equivalent to another appropriately de ned discrete time Markov decision problem. This allows us to characterise the value function and to nd an optimal Markov control.
Advisors/Committee Members: Ghosh, Mrinal Kanti.

► Natural disasters are one of the constant challenges for designing new and strengthening existing infrastructures. Such hazards in the past have incurred significant loss of…
(more)

▼ Natural disasters are one of the constant challenges
for designing new and strengthening existing infrastructures. Such
hazards in the past have incurred significant loss of life and
economic damage; therefore, further research is warranted in this
area to enhance the health and minimize the cost of maintaining and
upgrading infrastructures, improve residents’ comfort, and enable
achieving higher levels of life safety. To this end, the field of
hazard mitigation and control focuses on performance improvement,
safety, and cost effectiveness of structures mostly through
minimizing large deformations of seismic-excited structures and
suppressing the damage and collapse in dynamic systems due to
excessive vibrations. Past developments in active and semi-active
control designs, such as the commonly used state space controllers
(e.g. linear quadratic regulator for fully observed systems and
linear quadratic Gaussian for partially observed systems), consider
linear feedback strategies. Meanwhile, such control strategies
require linearization, and the system is usually linearized based
on linear elastic properties. The control force is proportional to
the state space vector and the dynamics and constraints of control
devices are mainly ignored. The objective functions have
restrictive forms, and are solely dependent on a second order
convex function of the response variables. To overcome the
aforementioned shortcomings, this dissertation develops new
stochasticcontrol algorithms for active and semi-active control
strategies. This research concentrates on the development of
frameworks that incorporate nonlinearity of the system, uncertainty
of the excitation, and constraints and dynamics of the control
device. Control designs are developed based on different objective
functions such as higher order polynomials of response variables,
reliability of the structure, and life cycle cost of the system
considering hazard risks in seismic prone areas. In particular, a
nonlinear sliding mode control algorithm based on stochastic
linearization is developed; this method supports higher order
objective functions and therefore enhances the ability of designers
to achieve design objectives. The proposed control algorithm is
designed, optimized, and tested on a seismically excited multi-span
bridge equipped with semi-active magnetorheological dampers. Next,
a stochasticcontrol algorithm is presented based on a proposed
stochastic averaging method called enhanced stochastic averaging.
This method conserves the nonlinear behavior of the system and the
stochastic nature of the excitation in optimalcontrol design. In
order to directly minimize the probability of failures, the
stochasticcontrol algorithm is extended to a reliability-based
control algorithm. These control algorithms are implemented in a
system with nonlinear soil-structure interactions. Furthermore, a
risk-based control methodology is developed to minimize life cycle
cost of a nonlinear multi-story building subjected to seismic
excitations. The findings of these proposed…
Advisors/Committee Members: Shafieezadeh, Abdollah (Advisor).