Numerical treatment of nonlinear optimal control problems

abstract: In this paper, we present iterative and non-iterative methods for the solution of nonlinear optimal control problems (NOCPs) and address the sufficient conditions for uniqueness of solution. We also study convergence properties of the given techniques. The approximate solutions are calculated in the form of a convergent series with easily computable components. The efficiency and simplicity of the methods are tested on a numerical example.


Introduction
Optimal control is a subject where it is desired to determine the inputs to a dynamical system that optimize (i.e., minimize or maximize) a specified performance index while satisfying any constraints on the motion of the system.Optimal control theory has received a great deal of attention and has found applications in many fields of science and engineering [20,22,23,15,16,17,18,21,24].Because of the complexity of most applications, optimal control problems are most often solved numerically.Numerical methods for solving optimal control problems date back nearly five decades to the 1950s with the work of Bellman [3,4,5,6,7,8].From that time to the present, the complexity of methods and corresponding complexity and variety of applications has increased tremendously making optimal control a discipline that is relevant to many branches of engineering.
Numerical methods for solving optimal control problems are divided into two major classes: indirect methods and direct methods.It is seen that indirect methods and direct methods emanate from two different approaches.On the one hand, the indirect approach solves the problem indirectly (thus the name, indirect) by converting the optimal control problem to a boundary-value problem.As a result, in an indirect method the optimal solution is found by solving a system of differential equations that satisfies endpoint and/or interior point conditions [11,14,12].On the other hand, in a direct method the optimal solution is found by transcribing an infinite-dimensional optimization problem to a finite-dimensional optimization problem [2,9,10,13].
The two different ideas of indirect and direct methods have led to a dichotomy in the optimal control community.Researchers who focus on indirect methods are interested largely in differential equation theory (e.g., see [1]), while researchers who focus on direct methods are interested more in optimization techniques.These two approaches have much more in common than initially meets the eye.Specifically, in recent years researchers have deeply investigated the connections between the indirect and direct forms.These two classes of methods are merging as time goes by.
This paper is organized as follows.In Section 2, we define a nonlinear quadratic optimal control problem.In Section 3, we establish existence and uniqueness of the solution of the defined control problem.Stability of solution of the problem is presented in Section 4. In Section 5, we present an iterative numerical method and give convergence analysis of the introduced method.Section 6, provides a noniterative method and analyzes its convergence.Finally, in Section 6, we present optimal control results for a test problem.

Nonlinear Quadratic Optimal Control Problem
We consider the nonlinear dynamical system of the form where x(t) ∈ ℜ n and v(t) ∈ ℜ m are the state vector and control function, respectively, and x 0 , x f are the initial and final states at t 0 , t f .Moreover, f (t, x(t)) ∈ ℜ n and g(t, x(t)) ∈ ℜ n×m are two real-valued continuously differentiable functions and have continuous first derivative with respect to x.
The objective is to find the optimal control law v * (t) that minimizes the quadratic objective functional subject to the control system (2.1),where Q is a symmetric positive semi-definite n × n matrix and R is a symmetric positive definite m × m matrix.
Consider Hamiltonian of the control system (2.1) as where k(t) ∈ ℜ n is known as co-state vector.According to the pontryagin's maximum principle [32], the optimality condition for system (2.1) obtained by the following nonlinear equations. where For the sake of simplicity, let us define the right hand sides of (2.3) as follows (2.4) Because the initial value of k(t) is not known, thus system (2.3) leads to where φ 1 ∈ ℜ n and φ 2 ∈ ℜ are two functions and β ∈ ℜ is an unknown parameter.
In order to obtain an approximate solution of system (2.5), let us integrate the system (2.5) with respect to t, using the initial condition we obtain We consider (2.6), which in turn can be written as where

Existence and Uniqueness of The Solution
First, we want to find the sufficient conditions for the uniqueness of solution in the space C([t 0 , t f ] → ℜ n+1 ) of real-valued continuous functions on the interval Here, we will use the infinity-norm .∞ , that is given (3.1) → ℜ be continuous and there exists a positive constant 0 < L, such that then, the problem (2.7) has a unique solution.
Proof.We introduce an operator Φ, acting on C([t 0 , t f ] → ℜ n+1 ), defined by Observe that the right hand side of (3.3) is a continuous function on the interval Next we define the weighted norm .θ , where θ > 0, on C([0, T ] → ℜ n+1 ) as the following where L is constant.Our proof is proceed as follows: Step 1.We know that Q is continuous, then the operator Step 2. We can show that Φ maps bounded sets into bounded sets in ) be bounded; i.e., there exists a positive constant r such that For each Z ∈ B r , we have Since, Q is continuous, we have thus Φ maps B r into itself.
Step 3. We prove that Φ is a contraction map.Let Z, Z ∈ C([0, T ] → ℜ n+1 ), then we obtain Because Then Hence, by choosing θ sufficiently large, we make e +θLs ds}] < 1. (3.10) Hence for the chosen value of θ, the operator Φ is a contraction map.Now, by application of the Contraction Mapping Theorem Φ has a unique fixed point in C([0, T ] → ℜ n+1 ), which is also the unique solution of the problem.

Stability of Solution
In this section, we discuss stability of solution for (2.7) by using the concept of stability.First, we give the definition of the stability of solution for (2.7).Definition 4.1.A solution Z(t) to (2.7) is said to be stable on the interval [t 0 , t f ] if for every ǫ > 0 there exists δ > 0 such that for all Z satisfying Z(t 0 ) − Z ∞ < δ the solution W (t) for following problem satisfies Z(t) − W (t) ∞ < ǫ for all t in [t 0 , t f ], Theorem 4.1.Assume that Q(t, Z(t)) is a continuous vector function on Suppose also, that there exists a positive constant L such that, holds whenever (t, Z(t)) and (t, Z(t)) lie in the rectangle U , and let Q(t, Z(t)) ∞ ≤ M for some real number M on U .Then the solution Z(t) is stable on the interval it follows that Now set Φ(t) = Z(t) − W (t) ∞ and λ = Z 0 − Z ∞ ; then, (4.6) can be written as From (4.7) (4.9), we have Thus, given ǫ > 0 as the above definition, we choose δ = ǫ exp(L(t 0 − t f )) and the proof is completed now.

Iterative Method (IM)
In this section, we apply an iterative method to solve the optimal control system problem.We consider (2.6) which can be written as We shall look for series solution to (5.1) and (5.2) as ) and define the partial sums and the following sequential method ) From (5.6), we have (5.8) Now, we consider (5.9) Therefore, we have ) where (5.13)

Convergence Analysis of IM
In the following, we provide sufficient conditions for the convergence of IM solution series.Theorem 5.1.Assume that Q(t, Z(t)) is a continuous vector function on Assume also that Q(t, Z(t)) satisfies a uniform Lipschitz condition with respect to Z(t) on U with Lipschitz constant γ, i.e., for all t ∈ [t 0 , t f ], and all Z(t) and Z(t), and let Q(t, Z(t)) ∞ ≤ M , for some real number M on U .Then the IM series solution converges to the solution of (2.1) on I = [t 0 , t 0 + χ], where Proof.The assumptions imply the following estimations Numerical treatment of nonlinear optimal control problems 203 From (5.17), we have therefore the sequence ∞ i=0 Z i (t) is absolute convergence.

Non-Iterative Method (NIM)
By considering (2.7), operators L and N can be defined in the following way We choose Z 0 (t) = Z(t 0 ), as an initial approximation guess.By using the above definition, we construct the following homotopy where p ∈ [0, 1] is the so-called homotopy parameter, Z(t, p) , and Z 0 defines the initial approximation of the solution of (2.7).Let the solution of (6.1) be in the form In order to determine the functions Z j , j = 1, 2, ..., we substitute (6.2) into Equation H(Z, p) = 0 and collect terms of the same powers of p, to obtain where Then, the solution of (5.9) has the form If it is difficult to determine the sum of series (6.2) for p = 1, then as an approximate solution of the equation, we approximate the solution Z(t) by the truncated series Z j (t).(6.6)

Convergence analysis of NIM
In the following, we provide sufficient conditions for the convergence of SM solution series.Theorem 6.1.Consider the optimal control system that is rewritten as Suppose that the following conditions are satisfied.
Then the SM solution series is absolutely convergent.

Proof:
Define the sequence of partial sums {Z k (x)} ∞ k=0 .Now, we are going to prove that {Z k (t)} ∞ k=0 is absolute convergence.Let Z k (t) be an arbitrary partial sums, subtract Z(t) from Z k (t), we have Proceeding by induction we obtain

Illustrative Example
In this section, an illustrative example is considered to examine the effectiveness and the accuracy of the proposed methods for solving NOCPs.All of the computations have been performed by using the Maple software package.In this example, we report mean-absolute error which is defined as: where (x(t), k(t)) T is the exact solution and (x n (t), k n (t)) T is the approximate solution.
Example 1. (see [33]) Consider the following optimal control problem minimize J[x(t), v(t)] =  1 shows the approximation values of β by using IM and NIM solution for different values of n.For Examples 1, the mean-absolute errors obtained by IM and NIM have been illustrated in Fig. 1.From Fig. 1 we can see that the approximate solutions obtained by proposed methods are in prefect agreement with the exact solution.The obtained errors also show that the accuracy of the IM is slightly better.

Numerical treatment of nonlinear optimal control problems 205 Figure 1 :
Figure 1: The mean absolute error for IM and NIM solutions with n = 8.