In this article, we consider the stochastic optimal control problem of discrete-time linear systems subject to Markov jumps and multiplicative noise under three kinds of performance criterions related to the final value of the expectation and variance of the output. In the first problem it is desired to minimise the final variance of the output subject to a restriction on its final expectation, in the second one it is desired to maximise the final expectation of the output subject to a restriction on its final variance, and in the third one it is considered a performance criterion composed by a linear combination of the final variance and expectation of the output of the system. We present explicit sufficient conditions for the existence of an optimal control strategy for these problems, generalising previous results in the literature. We conclude this article presenting a numerical example of an asset liabilities management model for pension funds with regime switching.; CNPq (Brazilian National Research Council)[304866/03-2]; FAPESP (Research Council of the State of Sao Paulo)[03/06736-7]

In this paper, we consider the stochastic optimal control problem of discrete-time linear systems subject to Markov jumps and multiplicative noises under two criteria. The first one is an unconstrained mean-variance trade-off performance criterion along the time, and the second one is a minimum variance criterion along the time with constraints on the expected output. We present explicit conditions for the existence of an optimal control strategy for the problems, generalizing previous results in the literature. We conclude the paper by presenting a numerical example of a multi-period portfolio selection problem with regime switching in which it is desired to minimize the sum of the variances of the portfolio along the time under the restriction of keeping the expected value of the portfolio greater than some minimum values specified by the investor. (C) 2011 Elsevier Ltd. All rights reserved.; CNPq (Brazilian National Research Council) [301067/09-0]; Brazilian National Research Council (CNPq); USP; USP

Weather forecasting models are computationally intensive applications and traditionally they are executed in parallel machines. However, some issues prevent these models from fully exploiting the available computing power. One of such issues is load imbalance, i.e., the uneven distribution of load across the processors of the parallel machine. Since weather models are typically synchronous applications, that is, all tasks synchronize at every time-step, the execution time is determined by the slowest task. The causes of such imbalance are either static (e.g. topography) or dynamic (e.g. shortwave radiation, moving thunderstorms). Various techniques, often embedded in the application’s source code, have been used to address both sources. However, these techniques are inflexible and hard to use in legacy codes. In this thesis, we explore the concept of processor virtualization for dynamically balancing the load in weather models. This means that the domain is over-decomposed in more tasks than the available processors. Assuming that many tasks can be safely executed in a single processor, each processor is put in charge of a set of tasks. In addition, the system can migrate some of them from overloaded processors to underloaded ones when it detects load imbalance. This approach has the advantage of decoupling the application from the load balancing strategy. Our objective is to show that processor virtualization can be applied to weather models as long as an appropriate strategy for migrations is used. Our proposal takes into account the communication pattern of the application in addition to the load of each processor. In this text...

A mathematical model has been developed for describing the dynamic operation of the N-formylmorpholine extractive distillation column and the corresponding solvent recovery column in the benzene extraction plant. The NRTL equation was used to calculate the equilibrium and thermodynamic properties of the mixtures. The validity of the model in terms of temperature, pressure and split fraction was examined using actual plant data at steady-state conditions. Comparison between model results and plant data shows good consistency. In order to improve the control of the process and selection of the optimal control strategy, the model was used to find the optimum values of the constants of the controllers with Nelder-Mead algorithm during unsteady-state operation by minimizing the deviation from steady-state conditions. The outcome of this study could be used by operators and engineers to increase the productivity of the unit.

A dual-motor coupling-propulsion electric bus (DMCPEB) is modeled, and its optimal control strategy is studied in this paper. The necessary dynamic features of energy loss for subsystems is modeled. Dynamic programming (DP) technique is applied to find the optimal control strategy including upshift threshold, downshift threshold, and power split ratio between the main motor and auxiliary motor. Improved control rules are extracted from the DP-based control solution, forming near-optimal control strategies. Simulation results demonstrate that a significant improvement in reducing energy loss due to the dual-motor coupling-propulsion system (DMCPS) running is realized without increasing the frequency of the mode switch.

We introduce a model to discuss an optimal investment problem of an insurance company using a game theoretic approach. The model is general enough to include economic risk, financial risk, insurance risk, and model risk. The insurance company invests its surplus in a bond and a stock index. The interest rate of the bond is stochastic and depends on the state of an economy described by a continuous-time, finite-state, Markov chain. The stock index dynamics are governed by a Markov, regime-switching, geometric Brownian motion modulated by the chain. The company receives premiums and pays aggregate claims. Here the aggregate insurance claims process is modeled by either a Markov, regime-switching, random measure or a Markov, regime-switching, diffusion process modulated by the chain. We adopt a robust approach to model risk, or uncertainty, and generate a family of probability measures using a general approach for a measure change to incorporate model risk. In particular, we adopt a Girsanov transform for the regime-switching Markov chain to incorporate model risk in modeling economic risk by the Markov chain. The goal of the insurance company is to select an optimal investment strategy so as to maximize either the expected exponential utility of terminal wealth or the survival probability of the company in the ‘worst-case’ scenario. We formulate the optimal investment problems as two-player...

Contrary to static mean-variance analysis, very few papers have dealt with dynamic mean-variance analysis. Here, the mean-variance efficient self-financing portfolio strategy is derived for n risky assets in discrete and continuous time. In the discrete setting, the resulting portfolio is mean-variance efficient in a dynamic sense. It is shown that the optimal strategy for n risky assets may be dominated if the expected terminal wealth is constrained to exactly attain a certain goal instead of exceeding the goal. The optimal strategy for n risky assets can be decomposed into a locally mean-variance efficient strategy and a strategy that ensures optimum diversification across time. In continuous time, a dynamically mean-variance efficient portfolio is infeasible due to the constraint on the expected level of terminal wealth. A modified problem where mean and variance are determined at t=0 was solved by Richardson (1989). The solution is discussed and generalized for a market with n risky assets. Moreover, a dynamically optimal strategy is presented for the objective of minimizing the expected quadratic deviation from a certain target level subject to a given mean. This strategy equals that of the first objective. The strategy can be reinterpreted as a two-fund strategy in the growth optimum portfolio and the risk-free asset.

Simple behavioural rules, or “rules of thumb”, which lead to behaviour that closely approximates an optimal strategy, have generated a lot of recent interest in the field of foraging behaviour. In this paper, we derive rules of thumb from a stochastic simulation model in which the foragers behave optimally. We use a particular biological system: the patch leaving behaviour of a parasitoid. We simulate parasitoids whose patch leaving behaviour is determined by a stochastic dynamic programming (SDP) model, while allowing parasitoids to make mistakes in their estimation of host density when arriving in a patch. We use Cox's proportional hazards models to obtain statistical rules of thumb from the simulated behaviour. This represents the first use of a proportional hazard approximation to generate rules of thumb from a complex optimal strategy.; http://www.elsevier.com/wps/find/journaldescription.cws_home/623/description#description

This paper develops a dynamic stochastic
general equilibrium model to analyze and derive simple
budget rules in the face of volatile public revenue from
natural resources in a low-income country like Niger. The
simulation results suggest three policy lessons or rules of
thumb. When a resource price change is positive and
temporary, the best strategy is to save the revenue windfall
in a sovereign fund, and use the interest income from the
fund to raise citizens' consumption over time. This
strategy is preferred to investing in public capital
domestically, even when private investment benefits from an
enhanced public capital stock. Domestic investment raises
the prices of domestic goods, leaving less money for
government to transfer to households; public investment is
not 100 percent effective in raising output. In the presence
of a negative temporary resource price change, however, the
best strategy is to cut public investment. This strategy
dominates other methods, such as trimming government
transfers to households...

Online auctions are arguably one of the most important and distinctly new applications of the internet. The predominant player in online auctions, eBay, has over 18.9 milllion users, and it was the host of over $5 billion worth of goods sold in the year 2000. Using methods from approximate dynamic programming and integer programming, we design algorithms for optimally bidding for a single item online auction, and simultaneous or overlapping multiple online auctions. We report computational evidence using data from eBay's web site from 1772 completed auctions for personal digital assistants and from 4208 completed auctions for stamp collections that show that (a) the optimal dynamic strategy outperforms simple but widely used static heuristic rules for a single auction, and (b) a new approach combining the value functions of single auctions found by dynamic programming using an integer programming framework produces high quality solutions fast and reliably.; Singapore-MIT Alliance (SMA)

The design of optimal dynamic disturbance accommodation controller with
limited model information is considered. We adapt the family of limited model
information control design strategies, defined earlier by the authors, to
handle dynamic controllers. This family of limited model information design
strategies construct subcontrollers distributively by accessing only local
plant model information. The closed-loop performance of the dynamic controllers
that they can produce are studied using a performance metric called the
competitive ratio which is the worst case ratio of the cost a control design
strategy to the cost of the optimal control design with full model information.; Comment: Fixed Typos, Updated Introduction and References. This manuscript is
an early version of the results presented in arXiv:1112.5032 prepared for the
presentation at the American Control Conference 2012

We consider a remote estimation problem with an energy harvesting sensor and
a remote estimator. The sensor observes the state of a discrete-time source
which may be a finite state Markov chain or a multi-dimensional linear Gaussian
system. It harvests energy from its environment (say, for example, through a
solar cell) and uses this energy for the purpose of communicating with the
estimator. Due to the randomness of energy available for communication, the
sensor may not be able to communicate all the time. The sensor may also want to
save its energy for future communications. The estimator relies on messages
communicated by the sensor to produce real-time estimates of the source state.
We consider the problem of finding a communication scheduling strategy for the
sensor and an estimation strategy for the estimator that jointly minimize an
expected sum of communication and distortion costs over a finite time horizon.
Our goal of joint optimization leads to a decentralized decision-making
problem. By viewing the problem from the estimator's perspective, we obtain a
dynamic programming characterization for the decentralized decision-making
problem that involves optimization over functions. Under some symmetry
assumptions on the source statistics and the distortion metric...

A distinctive feature of a clustered observational study is its multilevel or
nested data structure arising from the assignment of treatment, in a non-random
manner, to groups or clusters of individuals. Examples are ubiquitous in the
health and social sciences including patients in hospitals, employees in firms,
and students in schools. What is the optimal matching strategy in a clustered
observational study? At first thought, one might start by matching clusters of
individuals and then, within matched clusters, continue by matching
individuals. But, as we discuss in this paper, the optimal strategy is the
opposite: first match individuals and, once all possible combinations of
matched individuals are known, then match clusters. In this paper we use
dynamic and integer programming to implement this strategy and extend optimal
matching methods to hierarchical and multilevel settings. In particular, our
method attempts to replicate a paired clustered randomized study by finding the
largest sample of matched pairs of treated and control individuals within
matched pairs of treated and control clusters that is balanced according to
specifications given by the user. We illustrate our method on a case study of
the comparative effectiveness of public versus private voucher schools in
Chile...

Searching for objects amongst clutter is a key ability of visual systems.
Speed and accuracy are often crucial: how can the visual system trade off these
competing quantities for optimal performance in different tasks? How does the
trade-off depend on target appearance and scene complexity? We show that the
optimal tradeoff strategy may be cast as the solution to a partially observable
Markov decision process (POMDP) and computed by a dynamic programming
procedure. However, this procedure is computationally intensive when the visual
scene becomes too cluttered. Therefore, we also conjecture an optimal strategy
that scales to large number of clutters. Our conjecture applies to homogeneous
visual search and for a special case of heterogenous search where the
signal-to-noise ratio differs across location. Using the conjecture we show
that two existing decision mechanisms for analyzing human data, namely
diffusion-to-bound and maximum-of-output, are sub-optimal; the optimal strategy
instead employs two scaled diffusions.; Comment: 19 pages, 6 figures

In the present paper, we study the optimal execution problem under stochastic
price recovery based on limit order book dynamics. We model price recovery
after execution of a large order by accelerating the arrival of the refilling
order, which is defined as a Cox process whose intensity increases by the
degree of the market impact. We include not only the market order but also the
limit order in our strategy in a restricted fashion. We formulate the problem
as a combined stochastic control problem over a finite time horizon. The
corresponding Hamilton-Jacobi-Bellman quasi-variational inequality is solved
numerically. The optimal strategy obtained consists of three components: (i)
the initial large trade; (ii) the unscheduled small trades during the period;
(iii) the terminal large trade. The size and timing of the trade is governed by
the tolerance for market impact depending on the state at each time step, and
hence the strategy behaves dynamically. We also provide competitive results due
to inclusion of the limit order, even though a limit order is allowed under
conservative evaluation of the execution price.; Comment: 24 pages, 14 figures

From the Hamilton-Jacobi-Bellman equation for the value function we derive a
non-linear partial differential equation for the optimal portfolio strategy
(the dynamic control). The equation is general in the sense that it does not
depend on the terminal utility and provides additional analytical insight for
some optimal investment problems with known solutions. Furthermore, when
boundary conditions for the optimal strategy can be established independently,
it is considerably simpler than the HJB to solve numerically. Using this method
we calculate the Kelly growth optimal strategy subject to a periodically reset
stop-loss rule.; Comment: 13 pages, 4 figures. Submitted to Quantitative Finance 29 May 2013

Submitted for publication to 'The Economic Journal'.; In the literature on risk, one generally assume that uncertainty is uniformly distributed over the entire working horizon, when the absolute risk-aversion index is negative and constant. From this perspective, the risk is totally exogenous, and thus independent of endogenous risks. The classic procedure is "myopic" with regard to potential changes in the future behavior of the agent due to inherent random fluctuations of the system. The agent's attitude to risk is rigid. Although often criticized, the most widely used hypothesis for the analysis of economic behavior is risk-neutrality. This borderline case must be envisaged with prudence in a dynamic stochastic context. The traditional measures of risk-aversion are generally too weak for making comparisons between risky situations, given the dynamic complexity of the environment. This can be highlighted in concrete problems in finance and insurance, context for which the Arrow-Pratt measures (in the small) give ambiguous results (see Ross, 1981).; This article is based on research undertaken with support from the European Community's PHARE ACE Programme 1998 under grant P98-2103-S.; Peer reviewed

This dissertation consists of three chapters relating to dynamic demand models of storable goods and their application to taxes that are imposed on soft drinks. Broadly speaking, the first chapter builds the estimation strategy for dynamic demand models of storable goods that allows for unobservable heterogeneous preferences in household's tastes. The second chapter uses the estimation strategy developed in the first chapter to study the policy implications of taxes that are imposed on sugary soft drinks. The last chapter explores and provides an explanation for the level of pass-through for soda taxes.

To be more specific, the first chapter develops techniques for incorporating systematic brand preferences in dynamic demand models of storable goods. Dynamic demand models are important for correctly measuring price elasticities of products that can be stockpiled. However, most of the literature excludes systematic preferences over consumers' brand tastes. This chapter resolves this issue by incorporating random coefficient Logit models into a dynamic demand framework and hence allows for realistic demand substitution patterns. It builds on Hendel and Nevo's 2006 Econometrica paper, where the authors introduce a model of dynamic demand that flexibly incorporates observable heterogeneity and estimates it via a three-step procedure that separates brand and volume choices. While a powerful tool...