Abstract. We investigate a model of hybrid control system in which both discrete and contin- uous controls are involved. In this general model, discrete controls act on the system at a given set interface. The state of the system is changed discontinuously when the trajectory hits predeﬁned sets, namely, an autonomous jump set A or a controlled jump set C where the controller can choose to jump or not. At each jump, the trajectory can move to a diﬀerent Euclidean space. We prove the continuity of the associated value function V with respect to the initial point. Using the dy- namic programming principle satisﬁed by V , we derive a quasi-variational inequality satisﬁed by V in the viscosity sense. We characterize the value function V as the unique viscosity solution of the quasi-variational inequality by the comparison principle method.

1. Introduction. Many complicated control systems, like ﬂight control and transportation, perform computer coded checks and issue logical as well as contin- uous control commands. The interaction of these diﬀerent types of dynamics and information leads to hybrid control problems. Thus hybrid control systems are those having continuous and discrete dynamics and continuous and discrete controls. Many control systems, which involve both logical decision making and continuous evolution, are of this type. Typical examples of such systems are constrained robotic systems [1] and automated highway systems [8]. See [5], [6], and the references therein for more examples of such systems. In [5], Branicky, Borkar, and Mitter presented a model for the most general hybrid control system in which continuous controls are present and, in addition, discrete controls act at a given set interface, which corresponds to the logical decision making process as in the above examples. The state of the system is changed discontinuously when the trajectory hits these predeﬁned sets, namely, an autonomous jump set A or a controlled jump set C where the controller can choose to jump or not. They prove right continuity of the value function corresponding to this hybrid control problem. Using the dynamic programming principle they arrive at the partial diﬀerential equation satisﬁed by the value function, which turns out to be the quasi-variational inequality, referred hereafter as QVI. In [4], Bensoussan and Menaldi study a similar system and prove that the value function u is close to a certain u εwhich they mention to be continuous indicating the use of the basic ordinary diﬀerential equation estimate for continuous trajectories and the continuity of the ﬁrst hitting time (see [4, Theorem 2.5 and Remark 3.5]). They

∗Received by the editors November 1, 2004; accepted for publication (in revised form) March 24, 2005; published electronically October 7, 2005. This work was partially supported by DRDO 508 and ISRO 050 grants to Nonlinear Studies Group, IISc.

http://www.siam.org/journals/sicon/44-4/61807.html

†Department of Mathematics, Indian Institute of Science, Bangalore 560012, India (sheetal@ math.iisc.ernet.in). This author is a UGC Research Fellow and the ﬁnancial support from UGC is gratefully acknowledged. ‡IISc-TIFR Mathematics Program, TIFR Center, P.O. Box 1234, Bangalore 560012, India (mythily@math.tifrbng.res.in).

1259

1260 SHEETAL DHARMATTI AND MYTHILY RAMASWAMY

prove its uniqueness as a viscosity solution of the QVI in a certain special case where the autonomous jump set is empty and the controlled jump set is the whole space. In our work, we study this problem in a more general case in which the au- tonomous jump set is nonempty and the controlled jump set can be arbitrary. Our model is based on that of [5]. Our main aim is to prove uniqueness in the most gen- eral case when the sets A and C are nonempty and also to obtain precise estimates to improve the earlier continuity results. Our motivation comes from the fact that in all the real-life models mentioned above, logical decision making is always involved as well as the continuous control. This will correspond to a nonempty autonomous jump set A. Here we prove the local H¨older continuity of the value function under a transver- sality condition, the same as the one assumed in [5] and [4] (see (2.36) in [4]). For this we need to follow the trajectories starting from two neighboring points, through their continuous evolution, and through their discrete jumps since the autonomous jump set is nonempty. This involves careful estimation of the distance between the trajectories in various time intervals and summing up these terms to show that the distance remains small for initial points suﬃciently close enough. Although the basic estimates used are similar to those available in the literature (e.g., [3], [4]), the crucial point in our proof is the convergence of the above summation. This also allows us to get the precise H¨older exponent for the continuity of the value function. As in [5] and [4], using the dynamic programming principle, we arrive at the QVI satisﬁed by the value function. Then we show that the value function is the unique viscosity solution of the QVI. Our proof is very diﬀerent from [4]. Their approach using a ﬁxed point method does not seem to be suitable, as it is for the general case of a nonempty autonomous jump set. Our approach is based on the comparison principle in the class of bounded continuous functions. It is inspired by earlier work on impulse and switching control and game theoretic problems in the literature, namely, [2], [7], [9], particularly the idea of deﬁning a sequence of new auxiliary functions. But the presence of the autonomous and controlled jump sets leads to diﬀerent equations on these sets, and hence some new ideas are needed to arrive at the conclusion.

2. Notation and assumptions. In a hybrid control system, as in [5], the state vector during continuous evolution is given by the solution of the following problem:

The trajectory also undergoes discrete jumps when it hits predeﬁned sets A, the autonomous jump set, and C, the controlled jump set. A predeﬁned set D is the destination set for both autonomous jumps as well as controlled jumps:

A

= A i×

{i},

A i⊆ Ω i⊆ R di,

i

C

= C i×

{i},

C i⊆ Ω i⊆ R di,

i

D

= D i× {i},

D i⊆ Ω i⊆ R di.

i

HYBRID CONTROL SYSTEMS AND VISCOSITY SOLUTIONS

1261

The trajectory starting from x ∈ Ω i, on hitting A, that is the respective A i⊆ Ω i, jumps to the destination set D according to the given transition map g. g uses discrete controls from the discrete control set V 1and can move the trajectory from A ito D j⊆ Ω j⊆ R dj. The trajectory then will continue its evolution under f jtill it again hits A or C, in particular A jor C j. On hitting C the controller can choose either to jump or not to jump. If the controller chooses to jump, then the trajectory is moved to a new point in D. In this case the controller can also move from Ω ito any of the Ω j. This gives rise to a sequence of hitting times of A, which we denote by σ i, and a sequence of hitting times of C, where the controller chooses to make a jump which is denoted by ξ i. Thus σ iand ξ iare the times when continuous and discrete dynamics interact. Hence the trajectory of this problem is composed of continuous evolution given by (2.1) between two hitting times and discrete jumps at the hitting times. We

denote (X(σ

by X (ξ i) . In general we take the trajectory to be left continuous so that X x(σ i)

i + ) will be denoted by x iand

X x(ξ

, u(·))

−

i

), u(·)) by x iand g(X(σ

−

i

) and X x(ξ i) means X x(ξ

−

i

), v) by x iand the destination of X(ξ

−

i

), whereas X x(σ

i

+

means X x(σ

i + ) will be denoted by X x(ξ i) .

We give the inductive limit topology on Ω, namely,

(x n, i n) ∈ Ω converges to (x, i) ∈ Ω

if for some N

large and ∀n ≥ N,

i n= i,

x, x n∈ Ω i,

Ω i⊆ R di

for some i,

and

x n− x Rdi< ε.

With the understanding of the above topology we suppress the second variable i from Ω. We follow the same for A, C , and D . We make the following basic assumptions on the sets A, C, D, and on functions f and g. (A1): Each Ω iis the closure of a connected, open subset of R di.

We assume the following conditions on the cost functionals. (C1): K is Lipschitz continuous in the x variable with Lipschitz constant K 1and is uniformly continuous in the u variable. Moreover, K is bounded by K 0. (C2): C aand C care uniformly continuous in both variables and bounded below by C > 0. Moreover, C ais Lipschitz continuous in the x variable with Lipschitz constant C 1and is bounded above by C 0. Also we assume

C c(x, y) < C c(x, z) + C c(z, y)

∀x ∈ C i, z ∈ D ∩ C j, y ∈ D.

We now give two simple examples of hybrid control systems. For more examples, see [5]. Example 2.1 ( collisions ). Consider the ball of mass m which is moving in vertical and horizontal directions in a room under gravity with gravitational constant g. The dynamics can be given as

x˙ =

v x, y˙ = v y,

v˙ x= 0,

v˙ y= −mg.

On hitting the boundaries of the room A 1= {(x, y)|y = 0, or y = R 1} we instantly set v yto −ρv yfor some ρ ∈ [0, 1], the coeﬃcient of restitution. Similarly we reset v xto −ρv xon hitting the boundary A 2{(x, y)|x = 0 or x = R 2}. Thus in this case the sets A 1and A 2are autonomous jump sets. We can generalize the above system by allowing dynamics to occur in diﬀerent R dafter hitting. The next example illustrates the importance of the transversality condition, in the absence of which the optimal trajectory and hence the optimal control may fail to exist. Example 2.2. Consider the dynamical system in R 2given by

x˙ 1(t)=1,

x 1(0) = 0,

x˙ 2(t) = u,

x 2(0) = 0,

where u ∈ [0, 1], and when the trajectory hits the set A given by A = {(x 1, x 2)|(x 1−

1) 2+(x 2+1) 2= 1} it jumps to (10 10, 10 10). The cost is given by

x 2(t)|, 210 10}. Here the vector ﬁeld (u, 1) is not transversal to the boundary at (1, 0) for u = 0. Hence optimal trajectory does not exist and, moreover, the value function is discontinuous at (1, 0).

∞ e −tmin{|x 1(t)+

0

HYBRID CONTROL SYSTEMS AND VISCOSITY SOLUTIONS

1263

In the following sections we are interested in exploring the value function of the hybrid control problem deﬁned in (2.7). In section 2 we show that the value function is bounded and locally H¨older continuous with respect to the initial point. In section 3, we use viscosity solution techniques and the dynamic programming principle to derive a partial diﬀerential equation satisﬁed by V in the viscosity sense, which turns out to be the Hamilton–Jacobi–Bellman QVI. Section 4 deals with uniqueness of the solution of the QVI. We give a comparison principle proof characterizing the value function as unique viscosity solution of the QVI.

3. Continuity of the value function. Let the trajectory given by the solution

of (2.1) and starting from the point x be denoted by X x(t, u(·)). Since x ∈ Ω, it belongs in particular to some Ω i. Then we have from the theory of ordinary diﬀerential equations

(3.1)

|X

x

(3.2)

(t, u(·)) − X (t, u(·))| ≤ e

z

¯

Lt

|x − z|,

¯

|X x(t, u(·)) − X x( t, u(·))| ≤ F |t − t|,

where F and L are as in (A4). Deﬁne the ﬁrst hitting time of the trajectory as

T (x) = inf {t > 0 | X x(t, u) ∈

u

A} .

Notice that this T (x) is in particular with respect to A ias x ∈ Ω i. By assuming a suitable transversality condition on ∂A iand ∂C iwe prove the continuity of T in the topology of R di. This is equivalent to proving the continuity of T on Ω with respect to the inductive limit topology on Ω. Hereafter by convention we assume the topology to be of that Ω i, in which the respective points belong. Theorem 3.1. Assume (A1)–(A7). Let X (t) be the trajectory given by the solution of (2.1). Let the ﬁrst hitting time T (x) be ﬁnite. Then it is locally Lipschitz continuous, i.e., there exists a δ 1> 0 depending on f, ξ 0, and the distance function from ∂A isuch that for all y, y¯ in B(x, δ 1), a δ 1neighborhood of x in Ω

where B(A i, δ) is a δ neighborhood of A iand d(x) is a signed distance of x from ∂A igiven by

d(x) =

⎧

⎪

−dist(x, ∂A i)

◦

if x ∈ A i,

⎨ if x ∈ ∂A i,

0

⎪ ⎩ dist(x, ∂A i)

¯

if x ∈ A

c

i .

For simplicity of notation we drop the suﬃx i from now on, remembering that the distances are in R di. It is possible to choose R > 0 such that in a small neighborhood of ∂A, say B(∂A, R), the above signed distance function d is C 1, thanks to our assumption (A2).

1264 SHEETAL DHARMATTI AND MYTHILY RAMASWAMY

Now for x 0∈ ∂A choose u 0in U such that u 0(t) = u 0for all t and r 0< R such

that

(3.3)

f (x, u

0

) · Dd(x) < −ξ 0

∀x ∈ B(x 0, r 0).

Observe that we can choose r 0independent of x 0by using compactness of ∂A. Now consider the trajectory starting from x, given by

Hence for all x belonging to B(¯x, δ 1), T is bounded. Let this bound be T 0. Then we have

|T(x) − T(¯x)| < C|x − x¯|e LT0.

Hence we conclude that the ﬁrst hitting time of trajectory is locally Lipschitz contin-

uous with respect to the initial point. Now we take up the issue of continuity of the value function. For this proof we need some estimates on hitting times of trajectories starting from two nearby points. We prove these estimates in the following lemmas. We ﬁx the controls u¯ and v¯ and suppress them in the following calculations. Lemma 3.2. Let σ 1and Σ 1be the ﬁrst hitting times of trajectories evolving with ﬁxed controls u¯ and v¯ according to (2.1) starting from x and z, respectively. Let x 1and z 1be points where these trajectories hit A for the ﬁrst time:

x 1= X x(σ 1),

z 1= X z(Σ 1),

x 1, z 1∈ ∂A.

If |x − z| < δ 1, where δ 1is as in Theorem 3.1, then

(3.9)

|x

1

− z

1

| ≤ (1 + F C)e

L(Σ

1

∨σ

1

)

|x − z|.

Proof. Note here that by Theorem 3.1 we have the estimate on |σ 1− Σ 1| given by (3.8),

(3.10)

|σ

1

− Σ

1

| < Ce

L(Σ

1

∨σ

1

)

|x − z|.

Using this we estimate |x 1− z 1|. Without loss of generality we assume that Σ 1> σ 1,

|x 1− z 1| = |X x(σ 1)− X z(Σ 1)|

≤ |X x(σ 1)−

X z(σ 1)| + |X z(σ 1)− X z(Σ 1)|.

Using (3.1) we get

|X x(σ 1)− X z(σ 1)| < e Lσ1|x − z|,

while (3.2) and (3.10) lead to

|X z(σ 1)− X z(Σ 1)| ≤ F|σ 1− Σ 1| ≤ FCe LΣ1|x − z|.

Combining these estimates, we get

|x 1− z 1| ≤ e LΣ1|x − z|(1 + F C)

for

z ∈ B(x, δ 1).

Observe that the destination points of x 1and z 1, which are denoted by x 1= g(x 1, v¯) and z 1= g(z 1, v¯), may belong to Ω j⊆ R dj. Without loss of generality we assume that x 1, z 1∈ Ω 2⊆ R d2, and the evolution of trajectories takes place in Ω 2till the next hitting time. Let σ 2and Σ 2be the next hitting times of the trajectories when they hit A once again. The next lemma deals with the estimate of |σ 2− Σ 2|. Lemma 3.3. Let the ﬁrst hitting time of trajectories starting from x and z, and evolving with ﬁxed control u¯, be σ 1and Σ 1, and the second hitting times are σ 2and Σ 2. Then there exists δ 2such that for |x − z| < δ 2,

Let σ iand Σ ibe the ith hitting times of trajectories starting from x and z, respectively. With the above notation we assume that x i, z i∈ Ω i+1⊆ R di+1. We apply Theorem 3.1 and the above lemmas recursively to ﬁnd estimates on successive hitting times and points where trajectories hit A. We generalize the above estimates for the ith hitting times of trajectories when they hit A. For simplicity of calculations we denote F C + G(F C + 1) by P hereafter. Remark 3.4. Let the control u¯ be ﬁxed. Let σ iand Σ ibe the ith consecutive hitting time of the trajectory starting from x and z, respectively, when they hit A, and let x i, z ibe the points on ∂A where trajectories hit A. Then proceeding along lines similar to those of Lemmas 3.2 and 3.3 we get the estimates for |σ i−Σ i| and |x i−z i| which are given by

|σ i− Σ i| ≤

Ce LΣiP i−1|x − z|,

|x i− z i| ≤

e LΣi(F C + 1)P i−1|x − z|

whenever |x − z| < δ i, where δ i:= min{δ 1,

Theorem 3.5 (continuity of the value function). Under the assumptions of Theorem 3.1, value function V of hybrid control problem deﬁned by (2.7) is bounded and locally H¨older continuous with respect to the initial point. Proof. First we show that the value function is bounded. For any u ∈ U and

v ∈ V 1,

δ 2,

,

δ1 e−LΣ iPi− 1

}.

V

(x) ≤ ∞ K(X x(t), u(t))e −λtdt +

0

∞

i=0

C a(X(σ i), v)e −λσi.

By our assumptions (C1) and (C2),

V (x) ≤ K 0+∞ e −λtdt +

0

+∞

i=1

C 0e λσi≤ K0+ C 0

λ

+∞

i=1

e−λσ i.

From (A5), recalling that β = inf d(A i, D i),

(3.17)

Hence we get

(3.18)

∞

e

σ

i+1

≥

σ

i

−λσ

i

≤ e

−λσ

1

+

β

sup |f (x, u)|

∞

e

−λβ/F

i

i=1 i=1

≥

≤ e

σ

i

+ β/F.

−λσ

1

1

1 − e

−λβ/F

,

HYBRID CONTROL SYSTEMS AND VISCOSITY SOLUTIONS

1269

leading to

V (x) ≤ K

λ

+C0e−λσ 1

1

1−e−λβ/F .

This proves V (x) is bounded. We now show that V deﬁned in (2.7) is locally H¨older continuous with respect to the initial point. Let x, z ∈ Ω. Regarding V (x) as in (2.7), we assume that the controller chooses not to make any controlled jumps. Note that the controller has this choice because in the interior of C he can always choose not to jump. On the boundary of C that is ∂C by the transversality condition, vector ﬁeld is nonzero and hence he can continue the evolution without jumping. Thus in any case he can choose

not to jump. Then given ε > 0, we can choose the controls u, v depending on ε such that