Never unseen spatial and temporal complexity of systems and tasks How can we drive in heavy traffic? Many components, very complex system. Can classical or even AI systems solve it? Not, as far as we know. But WE, humans can. And we would like to build MACHINES to be able to do the same. Our car, save fuel, save time, etc. 4

Imprecise, uncertain, insufficient, ambiguous, contradictory information, lack of knowledge • How can I get to Shibuya? (Person 1: Turn right at the lamp, than straight ahead till the 3 rd corner, than right again. . . NO: better turn to the left) (Person 2: Turn right at the lamp, than straight ahead till appr. the 6 th corner. . . than I don’t know) (Person 3: It is in this direction → somewhere. . . ) • It is raining • The traffic light is out of order • I don’t know in which building do we have the special lecture (in Building III or. . . )? And at what time? ? (Does it start at 3 p. m. or at 2 p. m? And: on the 3 rd or 4 th of October? ) • When do I have to start from home at at what time? Who (a person or computer) can show me an algorithm to find an 6 OPTIMUM solution?

Finite resources Strict time requirements (real-time processing) • It is 10. 15 a. m. My lecture starts at 3 p. m. (hopefully the information is correct) • I am still not finished with my homework • I have run out of the fuel and I don’t have enough money for a taxi • I am very hungry • I have promised my Professor to help him to prepare some demo in the Lab this morning I can not fulfill everything with maximum preciseness 8

Need for optimization Let’s look ”TIME” as a resource: • The most important thing is to go the Lab and help my Professor (He is my Professor and I have promised it). I will spend there as needed, min. 3 hours • I have to submit the homework, but I will work in the Lab. , i. e. today I will prepare an ”average” and not a ”maximum” level homework (1 hour) • I don’t have time to eat at home, I will buy a bento at the station (5 minutes) • The train is more expensive then the bus but takes much less time, i. e. I will go by train (40 minutes) 11

User’s comfort • I have to ask the way to the university but unfortunately, I don’t speak Japanese • Next time I also want to find my way • Today it took one and a half hour to get here. How about tomorrow? • It would be good get more help • . . 12

User’s comfort Help: • Modeling methods and representation techniques making possible to – handle – interprete – predict – improve – optimise the system and – give more and more support in the processing 13

The most important elements of the solution • Low complexity, approximative modeling • Application of adaptive and robust techniques • Definition and application of the proper cost function including the hierarchy and measure of importance of the elements • Trade-off between accuracy (granularity) and complexity (computational time and resource need) • Giving support for the further processing These do not cope with traditional and AI methods. But how about the new approaches, about COMPUTATIONAL INTELLIGENCE? 15

What is Computational Intelligence? Computer + Increased computer facilities Intelligence Added by the new methods L. A. Zadeh, Fuzzy Sets [1965]: “In traditional – hard – computing, the prime desiderata are precision, certainty, and rigor. By contrast, the point of departure of soft computing is thesis that precision and certainty carry a cost and that computation, reasoning, and decision making should exploit – whenever possible – the tolerance for imprecision and uncertainty. ” 16

What is Computational Intelligence? • CI can be viewed as a corsortium of methodologies which play important role in conception, design, and utilization of information/intelligent systems. • The principal members of the consortium are: fuzzy logic (FL), neuro computing (NC), evalutionary computing (EC), anytime computing (AC), probabilistic computing (PC), chaotic computing (CC), and (parts of) machine learning (ML). • The methodologies are complementary and synergistic, rather than competitive. • What is common: Exploit the tolerance for imprecision, uncertainty, and partial truth to achieve tractability, robustness, low solution cost and better rapport with reality. 17

Computational Intelligence fulfill all of the five requirements: (Low complexity, approximative modeling application of adaptive and robust techniques Definition and application of the proper cost function including the hierarchy and measure of importance of the elements Trade-off between accuracy (granularity) and complexity (computational time and resource need) Giving support for the further processing) 18

How CI works? 1. Knowledge • In real life nearly everything is optimization • (Ex. 1. Determination of the velocity = Calculation of the optimum estimation of the velocity from the measured time and done distance) • Ex. 2. Determination of the resistance = the optimum estimation of the resistance with the help of the measured intensity of current and voltage • Ex. 3. Analysis of a measurement result = the optimum estimation of the measured quantity in the kowledge of the conditions of the measurement and the measured data) • Ex. 4. Daily time-table • Ex. 5. Optimum route between two towns In Ex. 1 -3 the criteria of the optimization is unambiguos and easily can be given Ex. 4 -5 are also simple tasks but the criteria is not unambiguos 20

Optimum route: What is optimum? (Subjective, depending on the requirements, taste, limits of the person) - We prefer/are able to travel by aeroplane, train, car, . . . Let’s say car is selected: - the shortest route (min petrol need), the quickest route (motorway), the most beautiful route with sights (whenever it is possible I never miss the view of the Fuji-san. . . ), where by best restaurants are located, where I can visit my friends, . . . OK, let’s fix the preferences of a certain person: -But is it summer or winter, is it sunshine or raining, how about the road reconstructions, . . By going into the details we get nearer and nearer to the solution Knowledge is needed for the determination of a good descriptive model of the circumstances and goals 21 But do we know what kind of wheather will be in two months?

2. Model • Known model e. g. analithic model (given by differential equations) - too complex to be handled • Lack of knowledge - the information about the system is uncertain or imperfect We need new, more precise knowledge The knowledge representation (model) should be handable and should tolerate the problems 22

Learning and Modeling New knowledge by learning: Unknown, partially unknown, known but too complex to be handled, ill-defined systems Model by which we can be analyze the system and can predict the behavior of the system + Criteria (quality measure) for the validity of the model 23

Iterative procedure: We build a system for collecting information We improve the system by building in the knowledge We collect the information We improve the observation and collect more information 25

Problem Knowledge representation, Model Non-represented part of the problem Represented knowledge Independant space, coupled to the problem by the formalism 26

3. Optimization • • Valid where the model is valid Given a system with free parameters Given an objective measure The task is to set the parameters which mimimize or maximize the qualitative measure • Systematic and random methods • Exploitation (of the deterministic knowledge) and exploration (of new knowledge) 27

• Universe of discourse: Cartesian (direct) product of all the possible values of each of the descriptors • Linguistic variable (linguistic term) [Zadeh]: ”By a linguistic variable we mean a variable whose values are words or sentences in a natural or artificial language. For example, Age is a linguistic variable if its values are linguistic rather than numerical, i. e. , young, not young, very young, quite young, old, not very old and not very young, etc. , rather than 20, 21, 22, 23, . . . ” • Fuzzy set: It represents a property of the linguistic variable. A degree of includance is associated to each of the possible values of the linguistic variable (characteristic function) • Membership value: The degree of belonging into the set. 32

Fuzzy Logic Control • Fuzzification: converts the numerical value to a fuzzy one; determines the degree of matching • Defuzzification converts the fuzzy term to a classical numerical value • The knowledge base contains the fuzzy rules • The inference engine describes the methodology to compute the output from the input 45

Fuzzy inference mechanism (Mamdani) • If x 1 = A 1, i and x 2 = A 2, i and. . . and xn = An, i then y = Bi The weighting factor wji characterizes, how far the input xj corresponds to the rule antecedent fuzzy set Aj, i in one dimension The weighting factor wi characterizes, how far the input x fulfils to the antecedents of the rule Ri. 49

Conclusion The conclusion of rule Ri for a given x observation is yi 50

Fuzzy Inference • Mamdani Type 51

Fuzzy systems: an example TEMPERATURE MOTOR_SPEED Fuzzy systems operate on fuzzy rules: IF temperature is COLD THEN motor_speed is LOW IF temperature is WARM THEN motor_speed is MEDIUM IF temperature is HOT THEN motor_speed is HIGH 52

Planning of Fuzzy Controllers Determination of fuzzy controllers = determination of the antecedents + consequents of the rules • Antecedents: – Selection of the input dimensions – Determination of the fuzzy partitions for the inputs – Determination of the parameters for the fuzzy variables • Consequents: – Determination of the parameters 54

Fuzzy-controlled Washing Machine (Aptronix Examples) • Objective Design a washing machine controller, which gives the correct wash time even though a precise model of the input/output relationship is not available • Inputs: Dirtyness, type of dirt • Output: Wash time 55

Fuzzy-controlled Washing Machine • Rules for our washing machine controller are derived from common sense data taken from typical home use, and experimentation in a controlled environment. A typical intuitive rule is as follows: If saturation time is long and transparency is bad, then wash time should be long. 56

Air Conditioning Temperature Control • • There is a sensor in the room to monitor temperature for feedback control, and there are two control elements, cooling valve and heating valve, to adjust the air supply temperature to the room. Temperature control has several unfavorable features: non-linearity, interference, dead time, and external disturbances, etc. Conventional approaches usually do not result in satisfactory temperature control. Rules for this controller may be formulated using statements similar to: If temperature is low then open heating valve greatly 57

Air Conditioning Temperature Control – Modified Model • There are two sensors in the modified system: one to monitor temperature and one to monitor humidity. There are three control elements: cooling valve, heating valve, and humidifying valve, to adjust temperature and humidity of the air supply. Rules for this controller can be formulated by adding rules for humidity control to the basic model. If temperature is low then open humidifying valve slightly. This rule acts as a predictor of humidity (it leads the humidity value) and is also designed to prevent overshoot in the output humidity curve. 58

Smart Cars 1 - Rules The number of rules depends on the problem. We shall consider only two for the simplicity of the example: Rule 1: If the distance between two cars is short and the speed of your car is high(er than the other one’s), then brake hard. Rule 2: If the distance between two cars is moderately long and the speed of your car is high(er than the other one’s), then brake moderately hard. 59

Smart Cars 4 - Inference – Determine the degree of matching – Adjust the consequent block – Total evaluation of the conclusions based on the rules To determine the control amount at a certain point, a defuzzifier is used (e. g. the center of gravity). In this case the center of gravity is located at a position somewhat harder than medium strength, as indicated by the arrow 62

Neural Networks Neural Nets are parallel, distributed information processing tools which are • Highly connected systems composed of identical or similar operational units evaluating local processing (processing element, neuron) usually in a well-ordered topology • Possessing some kind of learning algorithm which usually means learning by patterns and also determines the mode of the information processing • They also possess an information recall algorithm making possible the usage of the previously learned information 65

Problem building • Selection of the most important features, coding • Fitness function = quality measure (optimum criterium) • Exit criteria • Selection of the size of the population • Specification of the genetic operations 96

Simple genetic algorithms • Mutation (of the bits (genes)) (one or independant) • Reproduction = who will survive and form the next (new) generation – Individuals with the best fitness function • Exit: after a number of generation or depending on the fitness function of the best individual or average of the generation, . . . 99

Example for GAs Maximize the f(x)=x 2 function where x can take values between 0 and 31 Let’s start with a population containing 4 elements (generated randomly by throwing a coin). Each element (string) consists of 5 bits (to be able to code numbers between 0 and 31) 100

Conclusions • The fitness improved significantly in the new generation (both the average and the maximum) • Initial population: randomly chosen • Selection: 4 times by a roulette wheel where ”better” individuals had bigger sectors having bigger chance (the 3 rd (worst) string has died out!) • Pairs: the 1 -2, 3 -4 selections • Position of the crossover: randomly chosen • Mutation: bit by bit with p=0. 001 probability • (the generation contains 20 bits, in average 0. 02 bit will be mutated – in this example none) 103

Anytime Systems – What do they offer? • To handle abrupt changes due to failures • To fulfill prescribed response time conditions (changeable response time) • Continuos operation in case of serious shortage of necessary data (temporary overload of certain communication channels, sensor failures, etc. ) /processing time • To provide appropriate overall performance for the whole system • guaranteed response time, known error • Flexibility: available input data, available time, computational power, balance between time and quality 105 (quality: accuracy, resolution, etc…)

Anytime systems – How do they work? • Conditions: on-line computing, guaranteed response time, limited resources (changing in time) • Anytime processing: coping with the temporarily available resources to maintain the overall performance • “correct”models, treatable by the limited resources during limited time, low and changeable complexity, possibility of reallocation of the resources, changeable and guaranteed response time/ computational need, known error • tools: iterative algorithms, other types of methods used in a modular architecture 106

• • • Shortage indicators Intelligent monitor Special compilation methods during runtime Strict time constraints for the monitor The number and the complexity of the executable task can be very high add-in + optimization 108

Missing input samples Temporary overload of certain communication channels, sensor failures, etc. the input samples fail to arrive in time or will be lost prediction mechanism (estimations based on previous data) example: resonator based filters 109

Temporal shortage of computing power Temporary shortage of computer power the signal processing can not be performed in time Trade-off between the approximation accuracy and the complexity reduction techniques, reduction of the sampling rate, application of less accurate evaluations 110

Temporal shortage of computing time Temporary shortage of computer time the signal processing can not be performed in time Examples: block-recursive filters and filter-banks overcomplete signal representations 112

Anytime algorithms – iterative methods • Evaluate 734/25! (after 1 second: appr. 30 → after 5 seconds: better 29, 3 → after 8 seconds: exactly 29, 36 • We build a system for collecting information We improve the system by building in the knowledge We collect the information We improve the observation and collect more information 113

Engineering view: Practical issues • Well defined mathematical fundation but there is a gap between theory and the implementation • When and which is working better? (the theory can not give any answer or is lazy to think over? ) • How to choose the sizes/parameters/shapes/definitions/etc. ? • What if the axioms are inconsistant/incomplete? (the practical possibility can be 0) • Handling of the exceptions, e. g. the rule for very young overwrites the rule young • Good advises: Modeling, a priori knowledge, iteration, hybrid systems, smooth systems/parameters (as near to the real world as possible) 115

Accuracy problems • How can we handle accuracy problems if we e. g. don’t have any input information? • What if in time critical applications not only the stationary responses are to be considered? • How can the different modeling/data representation methods interprete the other’s results? • New (classical+nonclassical) measures are needed 116

Transients • Dynamic systems: Change in the systems transients • Depending on the transfer function and on the actual implementation of the structure • Strongly related to the „energy distribution” of the system • Effected by the steps and the reconfiguration „route” 117

Transients • Must be reduced and treated: – careful choosing of the architecture (orthogonal structures have better transients) – multi step reconfiguration: selection of the number and location of the intermediate steps – estimation of the effect of transients 118

Is CI really solution for unsolvable problems? • Yes: The high number of succesful applications and the new areas where automatization became possible prove that Computational Intelligence can be a solution for otherwise unsolvable problems • Although: With the new methods new problems have arised to be solved by you Future engineering is unthinkable without Computational Intelligence 119

Conclusions • • What is Computational Intelligence? What is the secret of its success? How does it work? What kind of approaches/concepts are attached? • New problems with open questions 03. 10. 2006 Tokyo Institute of Technology 120