Global Optimization Via Neural Networks and D.C. Programming

Abstract

The ultimate goal of this work is to provide a general global optimization
method. Due to the difficulty of the problem, the complete task is divided
into several sections than can be summarized as a modeling phase followed by a
global optimization phase. Each of the various sections draws from different
engineering fields. What this work suggests is an interface and common grounds
between these.

The modeling phase of the procedure consists of converting a general problem
into a given formulation using a particular type of neural network. The
architecture required for the neural network forms a new class: the pseudo
multilayer neural network. It is introduced and compared to more classical
neural network architectures such as the regular multilayer neural network.
However, a key difference between these is that the behavior of the usual
multilayer neural network has to be programmed, while an extremely efficient
procedure is given here to synthesize the pseudo multilayer neural network.
Thereby any initial problem can be systematically converted into a pseudo
multilayer network without going through the undesired programming steps such
as the backpropagation rule.

The second phase of the work consists of translating the initial global
optimization problem into the global minimization of a target function related
to the neural network model. Systematic procedures are again given here.

The last phase consists of globally minimizing the target function. This is
done via the so-called DC programming technique where DC stands for
"Difference of Convex." The pseudo multilayer was created such that
it can systematically be converted into a DC formulation, and therefore be
compatible with DC programming. A translation procedure to go from the pseudo
multilayer neural network to the DC formulation is given. When a DC program is
applied to this last formulation, the resulting solution can be directly mapped
to the global minimum of the target function previously defined, thereby
producing the global solution of the neural network modeling the initial
problem. Therefore, the optimal solution of the original problem is known as
well.