Cauchy-Schwarz inequality and optimization 2

Cauchy-Schwarz inequality and optimization 2

Example 8.1. Given the initial state for some and a discount factor we need to maximize the discounted stream of future rewards

(1)

The maximization is over actions (you can think of them as consumptions). The initial state is the initial wealth; at each step, the consumption is chosen between zero and the wealth left from the previous period, This reduces the amount available for consumption in the next period:

(2)

Step 1. Here we reformulate the problem to remove the states from the constraints. From (2) we have Fixing an integer then we obtain

(3)

Since in each state consumption is nonnegative, the states can only decrease over time. On the other hand, they cannot be negative. Hence, the sequence is nonincreasing and bounded from below by zero. Therefore the limit exists and is nonnegative. Letting in (3) we get

The problem becomes: maximize (1) subject to

(4)

Step 2. Getting rid of square roots. Denoting we have to maximize subject to

Step 3. Applying the result on the Cauchy-Schwarz inequality. The result from the previous post does not depend on the number of variables. For our present needs it is formulated as follows: Let be fixed numbers and let be variables satisfying the restriction Then the infinite weighted sum is maximized at the point

and the maximized value is

Letting we get and

with the maximized value

And no clumsy terminology and notation from the Infinite Horizon Dynamic Programming method!