Abstract Over the recent years, several studies have been carried out by the researchers to describe a general, ?exible and powerful design method based on modern heuristic optimisation algorithms for in?nite impulse response (IIR) digital ?lters since these algorithms have the ability of ?nding global optimal solution in a nonlinear search space. One of the modern heuristic algorithms is the arti?cial immune algorithm which implements a learning technique inspired by human immune system. However, the immune system has not attracted the same kind of interest from researchers as other heuristic algorithms. In this work, an arti?cial immune algorithm is described and applied to the design of IIR ?lters, and its performance is compared to that of genetic and touring ant colony optimisation algorithms. r 2005 Elsevier Ltd. All rights reserved.Keywords: Immune algorithm; Genetic algorithm; Ant colony algorithm; IIR ?lter design

1. Introduction In electronic equipment of any kind, analog functions have been increasingly replaced by digital algorithms. The advantages of digital signal processing (DSP) are based on the fact that the performance of the applied digital algorithms is always predictable. There is no dependence on the tolerances of the electrical components as in analog systems. The adjustment of components which is often necessary in analog circuits becomes absolute. Also, extremely low frequencies can be processed without the problems which occur in analog systems. Almost any DSP algorithm or processor can reasonably be described as a digital ?lter although this term is most commonly used for systems which transmit (or reject) well-de?ned frequency ranges. Digital ?lters can be classi?ed into two groups in terms of the?Corresponding author. Tel.: +90 352 4374901/40706; fax: +90 352 4375267. E-mail addresses: kalinlia@erciyes.edu.tr (A. Kalinli), nurhan_k@erciyes.edu.tr (N. Karaboga).

structure: non-recursive and recursive. A non-recursive ?lter is referred to as ?nite impulse response (FIR) ?lter and recursive ?lter as in?nite impulse response (IIR) ?lter. Although the output of the non-recursive ?lter depends only on present and previous inputs, the output from a recursive one depends on one or more previous output values, as well as on inputs. In other words, it involves feedback. From the DSP point of view, its great advantage is computational economy. However, there is a potential disadvantage: The general IIR ?lter design problem is highly nonlinear in the ?lter coef?cients although the FIR design problem is linear (Stearns, 1981; Shynk, 1989). Popular modern heuristic algorithms include genetic algorithm (GA), simulated annealing (SA), tabu search (TS), ant colony optimisation (ACO) algorithm, neural networks (NNs) and arti?cial immune algorithm (IA). GA is a best-known model of evolutionary computation which mimics the natural selection and the basic operators of genetic science, and it was invented by Holland (1975). The SA algorithm has been modelled on the annealing of solids in nature (Kirkpatrick et al.,

1983). TS is based on the principles of intelligent problem solving and was proposed by Glover (1986). ACO algorithm simulates the behaviour of real ant colonies (Dorigo et al., 1991). NNs have been modelled on the mechanism of the brain (Kohonen, 1996). IA is based on the learning mechanism of natural immune system (Farmer et al., 1986). Over the recent years, several studies have been carried out by the researchers to describe a general, ?exible and powerful design method based on modern heuristic optimisation algorithms for IIR ?lters since these algorithms have the ability of ?nding global optimal solution in nonlinear search spaces (Etter et al., 1982; Nambiar and Mars, 1992; Tang et al., 1996; Ng et al., 1996; Chen et al., 2001; Kalinli and Karaboga, 2003; Karaboga et al., 2004; Kalinli and Karaboga, 2004). Etter et al. have proposed an adaptive GA for recursive adaptive IIR ?lter design. Nambiar and Mars have incorporated SA concepts into GA for designing optimal adaptive IIR ?lters. Tang et al. have introduced GA as an engineering optimisation algorithm for signal processing problems including IIR ?lters. A new hybrid search methodology in which the genetic type search was embedded into gradient-descent algorithms have been developed by Ng et al. and the coef?cients of IIR ?lters were evolved in a random manner. Chen et al. have applied an adaptive simulated annealing to digital IIR ?lter design. The application of IA to the digital IIR ?lter design has been introduced by Kalinli and Karaboga. Karaboga et al. have described a new design method based on ACO for designing digital IIR ?lters. A novel adaptive digital IIR ?lter design approach using TS algorithm has been proposed by Kalinli and Karaboga. It is a known fact that every optimisation algorithm has its own advantages and disadvantages. This fact is also valid for modern heuristic algorithms, too. For instance, the performance of TS and SA usually depends on the starting point of the search since these two algorithms employ a form of neighbourhood search mechanism. The main features of ACO algorithm are distributed computation, positive feedback and constructive greedy search. Of these algorithms, GA, NN and arti?cial IA are the approaches inspired from biological systems. In the literature, there exist several models proposed for NN and evolutionary computation. GA is a stochastic optimisation algorithm employed for combinatorial and continuous optimisation problems. GA can ef?ciently search large solution spaces due to its parallel structure and the probabilistic transition rules employed in the operators. However, a basic GA has two main drawbacks: lack of good local search ability and premature convergence. The natural immune system uses learning, memory, and associative retrieval to solve recognition and classi?cation tasks. Speci?cally, it learns to recognize relevant patterns, remember patterns that have been seen previously, and

use combinatorics to construct pattern detectors ef?ciently. The natural immune system is a great source of inspiration for developing intelligent problem solving techniques and however a few computational models have been developed. So, arti?cial immune systems emerged in the 1990s as a new computational research area. An IA simulates a learning technique carried out by natural immune system. However, the immune system has not attracted the same kind of interest from researchers as the NN or the evolutionary computation. But, over the last years, there has been increasing interest in the area of arti?cial immune system and their applications (Miyamoto et al., 2004; Watkins et al., 2004; Xiaoping et al., 2003; Karaboga et al., 2001; Kalinli and Karaboga, 2003; Hong et al., 2000). In the work described by Kalinli and Karaboga (2003), only the application of an IA to the IIR ?lter design has been introduced. In Xiaoping et al. (2003), the authors employed the IA for FIR ?lter design. In this work, a method based on IA is described for the digital IIR ?lter design and its performance is compared to that of GA, TS and the touring ACO (TACO) algorithm, which is a continuous version of ant colony algorithm (Karaboga et al., 2004). In Section 2, the basic principles of the natural immune system and the IA used are described. Section 3 presents the design problem. In Section 4, ?rstly the performance of IA is tested by using a set of well-known test functions and secondly IA is applied to the design of IIR ?lter, and the results produced are discussed.

2. Natural immune system and immune algorithm used 2.1. Natural immune system The natural immune system is a distributed novelpattern detection system with several functional components positioned in strategic locations throughout the body. The main purpose of the immune system is to recognize all cells (or molecules) within the body and categorize those cells as self or non-self. The non-self cells are further categorized in order to stimulate an appropriate type of defensive mechanism. The immune system learns through evolution to distinguish between foreign antigens (e.g. bacteria, viruses, etc.) and the body’s own cells or molecules. The lymphocyte is the main type of immune cell participating in the immune response that possesses the attributes of speci?city, diversity, memory, and adaptability, there are two subclasses of the lymphocyte: T and B. Each of these has its own function. In particular, B cells secrete antibodies that can bind to speci?c antigens. The immune system has evolved to mechanisms: innate (non-speci?c) and adaptive (speci?c) immunity which are linked to and in?uence each other. Adaptive

immunity consists of two branches: the humoral immunity which is carried out by the B cells and their products, and the cellular immunity by T cells. Both branches follow a similar sequence of steps for defence. When an antigen enters the body, self-regulatory mechanisms determine the branch of the immune system to be activated, the intensity of the response, and its duration. Speci?cally, regulation of both immunity is conducted by a population of T cells. T cells regulate immune responses by release of soluble molecules such as lyphokines, cytokines, interleukins in order to activate B cells (Farmer et al., 1986; Hunt and Cooke, 1996). The B cells originated in the bone marrow collectively form what is known the immune network. This network acts to ensure that once useful B cells are generated, they remain in the immune system until they are no longer required. When a B cell of natural immune system encounters an antigen, an immune response is elicited, which causes the antibody to bind the antigen (if they match) so that the antigen can be neutralized. If the antibody matches the antigen suf?ciently well, its B cell becomes stimulated and can produce related clones which are incorporated into the immune network. The level of stimulation depends on both how well the B cell’s antibody matches the antigen and how it matches other B cells in the immune network. If the stimulation level rises above a given threshold, the B cell becomes enlarged and starts replicating itself several times, producing clones of itself. To allow the immune system to be adaptive, the clones that grow also turn on a mutation mechanism that generates at high frequencies point mutations in the genes coding speci?cally the antibody molecule. This mechanism is called somatic hypermutation (Kepler and Perelson, 1993). If the stimulation level falls below a given threshold, the B cell does not replicate and in time it will die off. Survival of the B cells produced by the bone marrow or by hyper mutation depends on their af?nity to the antigen and to the other B cells in the network. The algorithm simulating the principles of natural immune system is called immune algorithm. The main steps of the IA used in this work are presented in Fig. 1. In the initialisation step, a population of B cells are randomly formed. Firstly, theConstruct the initial population REPEAT Calculate the stimulation level of cells Compute the similarity of cells with the highest stimulated cell Form a sub-population by using the similarity values Apply somatic mutation Calculate the similarity of each cell with others cells Remove some of the cells stimulated at low level from the population Generate new cells and insert some of them into the network UNTIL (generation = maxgeneration)

stimulation levels of all cells in the population are calculated and the highest stimulated cell is determined. Secondly, the similarities of all other cells are computed with respect to this highest stimulated cell and a sub-population of cells is constructed from the population depending on the calculated similarity values. Thirdly, the cells of the sub-population are cloned according to their similarity values and then the somatic mutation is applied to the cells in clones. Lastly, the similarity of each cell with all others in the rest of population is calculated and some of the cells with the low similarity are replaced with the randomly produced cells. 2.2. Immune algorithm used In the model used in this work, each B cell is represented with a binary string which is a possible solution to the problem. At the ?rst step, an initial population of B cells is formed of the cells produced randomly. Then, all cells are evaluated for the problem and a ?tness value is assigned to each cell in the population. Depending on the ?tness values, cells are ordered and the cell with the highest ?tness value is assumed as the antigen. Since the stimulation level of a cell depends on both the similarity with the antigen and the similarity with all other cells in the network, two different sub-populations are constructed based on these similarity values. Therefore, the similarity of all cells with the antigen is computed and the ?rst subpopulation is constructed of the cells with the highest match score with the antigen by using a K-Tournament selection scheme. After this selection operation each cell in the sub-population is mutated by applying a onepoint mutation operator. In the one-point mutation, every bit in the string representing a cell is checked and if a randomly produced number between 0 and 1 is less than the mutation rate, then the value of that bit is reversed. The resulting mutated cells are only kept if they improve their own similarity with the antigen. After the ?rst sub-population is formed, the similarity of each cell in the rest of network with all cells in the network is calculated and then these cells are ordered depending on their match scores. Some of B cells having low similarity values which are also called to be stimulated at low level

are removed from the population. New B cells are randomly produced and those with the similarity above predetermined level with the cells in the population are added into the population. These operations are repeated at each generation. In our model, the similarity between cells is calculated as shown in Fig. 2. In this example, each parameter (a, b) is represented with ?ve bits. Therefore, a cell (a solution to the problem) is a binary string with the length of 10 bits. Each bit in a sub-string used to represent a or b parameters has different weight in the similarity calculation since the signi?cance of each bit in a binary string is different in the conversion from binary to decimal numbers. For example, the least signi?cant bit has the minimum weight ?20 ? 1? as the most signi?cant one has the maximum weight ?24 ? 16?.

representation as follows: y? k ? ?M X i?1

bi y? k ? i ? ?

L X i?0

ai x?k ? i?;

(1)

where x(k) and y(k) are the ?lter’s input and output, respectively, and M (XL) is the ?lter order. The transfer function of this IIR ?lter can be obtained by the ztransformation of Eq. (1) in the following general form: PL ?i A?z? i?0 ai z H ?z? ? ? . (2) P ?i B?z? 1 ? M i?1 bi z Hence, the design of this ?lter can be considered as an optimisation problem of the cost function J(w) stated as the following:w2 W

min J ?w?,

(3)

3. Description of the problem Impulse response with in?nite length can be realised with a limited number of coef?cients by also using delayed values of the output to calculate the current output value. The calculation of the current output value is based on the current input value and the previous input and output values. The use of delayed output values gives the ?lter a feedback or recursive structure. In a general recursive (IIR) ?lter realisation, the current and L delayed values of the input are multiplied by the coef?cients ai, M delayed values of the output are multiplied by coef?cients bi, and all the resulting products are added. Sorting input and output terms to both sides of the equal sign yields the sum

is compared with that of GA and TACO algorithm since these two algorithms carry out the populationbased search and use probabilistic transition rules as IA, and secondly the application of IA for the design of IIR ?lter design is described and the results are discussed.

4.1. Evaluation of IA using test functions In order to evaluate the performance of IA, a set of well-known test functions including the functions de?ned by De Jong (1975), which are given in Table 1, is used. Number of parameters, solutions, parameter bounds and length of solution for the test functions are presented in Table 2. Fl (Sphere): The ?rst function is smooth, unimodal, strongly convex, and symmetric. F2 (Banana): Rosenbrock’s valley is a classic optimisation function, also known as Banana function. The global optimum is inside a long, narrow, parabolic shaped ?at valley. To ?nd the valley is trivial, however convergence to the global optimum is dif?cult and hence this problem has been repeatedly used to assess the performance of optimisation algorithms. F3 (Step): This function represents the problem of ?at surfaces. Flat surfaces are obstacles for optimisation algorithms because they do not give any information as

to which direction is favourable. Unless an algorithm has variable step sizes, it can get stuck on one of the ?at plateaus. F4 (Foxholes): Function F4 is an example of a function with many local optima. Many standard optimisation algorithms get stuck in the ?rst peak they ?nd. F5: Function F5 is also a multi-modal function which has 40,000 local minimum points in the region when the values of the variables are within [?10,10]. In order to compare the performance of IA, GA and TACO, the simulation results belonging to GA were obtained by using the GA described by Grenfenstette (1986). The control parameter values of the GA were: population size ? 50, generation number ? 200, crossover rate ? 0.85 and mutation rate ? 0.015. In the case of the IA, the number of cells in the network was 20 and the size of both the sub-populations was 10. At each generation, two least stimulated cells were removed from the population. Ten new cells are randomly produced and two of these new cells, which have the maximum similarity with the cells in the population, are inserted into the population. In the case of TACO algorithm, the number of ants in a colony was 50 and the generation number was 200. Each algorithm was run 30 times for each function with different seed numbers to examine their robustness. The number of evaluations was taken as equal to 10,000 for all algorithms and for all functions to compare their performance. The histograms drawn from the results obtained by three algorithms are shown in Figs. 3–7. From histograms obtained for all functions considered in this study, it can be concluded that IA is more robust than GA and TACO because IA more frequently ?nds the global minimum points of functions when the search starts at different initial points. As seen from the histogram drawn for the ?rst function, the local converging ability of IA and TACO is better than GA as expected. Moreover, in terms of the quality of the ?nal solutions, the performance of IA is similar to or

much better than that of GA and TACO as can be seen from Figs. 4–7. In the simulation study, three different ?tness functions were used. These functions are given in Table 3. In the Table 3, c is a very small constant and in this study it was taken as 0.001. 4.2. Digital IIR ?lter design based on IA In this section, IA is applied to the problem of IIR ?lter design and its performance is compared to that of TACO, GA and TS algorithm on this problem. The application of IIR ?lter in system identi?cation has been widely studied since many problems encountered in signal processing can be characterized as a system identi?cation problem (Fig. 8) (Nayeri and Jenkins, 1989; Tang et al., 1996). Therefore, in the

simulation study, IIR ?lters are designed for the system identi?cation purpose. In this case, the parameters of the IIR ?lter are successively adjusted by IA until the error between the outputs of the ?lter and the system is minimised. In other words, IA is applied to minimize the cost function J(w) de?ned by Eq. (4). By constraining the range of the ?lter coef?cients, the stability is guaranteed. The ?lter coef?cients are encoded in the binary string form as shown in Fig. 9. The performance of the solution i in the population is determined by using the following formula: fitnessi ? 1 , 1 ? J i ?w? (5)

where Ji(w) is the cost function value calculated by using the solution i.

The input, x(k), to the system and the ?lter was a white sequence. Since the ?lter order is equal to that of the system, the error surface for this example is uni-modal. Fig. 10 presents the error surface for this ?lter. As seen from the ?gure, there is one local minimum which is also the global minimum. In order to show the robustness of the proposed method, Fig. 11(a)–(d) demonstrates the histograms drawn from the results obtained by four algorithms for this ?lter. To evaluate the convergence speed of the proposed method, Fig. 11(e) shows the evolution of the mean-squared error (MSE) averaged over 100 different runs of IA, GA, TACO and TS algorithms. Each run had a randomly chosen initial w.

Example 2. In the second example, the system was a second-order system and the ?lter was a ?rst-order IIR ?lter with the following transfer functions: H p ?z?1 ? ? 0:05 ? 0:4z?1 , 1:0 ? 1:1314z?1 ? 0:25z?2 a0 H f ?z?1 ? ? . 1 ? b1 z?1

a0

a1

...

aL

b1

b2

...

bM

? 7?

Fig. 9. Representation of the parameters in the string form.

1.0

0.5

b2

0.0

The system input was a white sequence with unit variance. The data length used in calculating the MSE (Eq. (4)) was N ? 2000. Since the reduced order ?lter is employed for the identi?cation, the error surface is multi-modal and it is given in Fig. 12. As seen from Fig. 12, the surface has two local minimum. The global minimum of the surface is at w ? ??0:311; 0:906?T with the value of the normalized MSE 0.277. Fig. 13(a)–(d) demonstrates the histogram produced from the results obtained by IA, GA, TACO and TS algorithms for the ?lter. For this example, Fig. 13(e) shows the evolution of the normalised MSE averaged over 100 different runs of IA, GA, TACO and TS algorithms. As seen from the histograms given in Figs. 11 and 13, the proposed method consistently ?nds the global optimal solution for both examples. For the ?rst example, IA is able to ?nd approximately zero MSE value for about 85 runs while the other algorithms can reduce the error to this level for maximum 35 runs. As seen from the Fig. 13, IA can construct a ?lter which has the MSE of 0.29 or less than for about 70 runs although other algorithms can design a similar ?lter for maximum 45 runs. Figs. 11(e) and 13(e) shows the evolution of the

MSE averaged over 100 different runs of the four algorithms. From these ?gures, it is seen that the convergence speed of the proposed method is faster than that of other three algorithms.0.98 1.4 1.2

0.5

b1

0.01.0 0.99

5. Conclusion An immune algorithm was applied to the digital IIR ?lter design and its performance was compared to that of genetic, touring ant colony optimisation and tabu search algorithms. In the simulations two ?lters were designed for the system identi?cation purpose. Immune algorithm produced better solutions to both the uni- and multi-modal cases than other three algorithms. From the results it is concluded that the proposed method can be ef?ciently used for digital IIR ?lter design.