Adaptive algorithms in NDB and in cars

The world is brimming with the ideas of self-driving cars and all sorts ofother concepts where computers are supposed to take care ofdecision making.

This makes a bit worried, for one because I simply like to drive andwould not want a computer to interfere with my driving. I am alreadyquite irritated by many automatic things in cars that don't really workwhen winter is upon in Sweden :)

Anyways this post is not about that, this post is more about the generalproblem of designing adaptive algorithms.

I've been designing NDB software for more than 20 years. During thecourse of these years I have learned a bit about what is optimalwhen executing NDB. Most of the software I write today is aboutputting this knowledge into the NDB software itself.

This is a trend in databases today to automate configuration handlingin a DBMS. In NDB we started this trend in MySQL Cluster 7.4when we implemented a "top" facility inside the NDB data nodes.At the same time we also keep track of lags in writing to disk.

We used this knowledge to design an adaptive algorithm that changesthe speed of writing local checkpoints based on the current CPU usageand IO lag.

We moved on in 7.5 and implemented an adaptive algorithm to controlfrom where sending will happen. This algorithm is also based onkeeping track of CPU usage in the threads in the data node.

The new partial LCP algorithm is also highly adaptive where it decideshow incremental the LCP should be based on the writing in thedatabase.

There is also work ongoing on some adaptiveness in the NDB APIwhere some threads will start up to assist the receive thread in the NDBAPI when it gets overloaded.

There is even more work ongoing to ensure that the checkpoint speedadapts also to conditions where we are starting to run out of REDO log.

Now the idea of adaptive algorithms is clearly a good idea, but, and thereis a big but, there are two problems with ANY adaptive algorithm.

The first problem is oscillation. Adaptive algorithms works by changingthe environment based on input from the environment. When you lookat an adaptive algorithm that works it is actually quite impressive. Bymerely finding the proper conditions based on the input you can get asystem that quickly adapts to any working condition and finds a newoptimal spot to work in.

My original study at the university was mathematical statistics.One important fact in most mathematical statistics is that youhave stable states and you have transient states.

An adaptive algorithm will work fine as long as the frequency ofchanges in the environment is not faster than the time it takes tofind a new stable state.

As an example in the algorithms in NDB, most of them takesdecisions to change the environment about once per second.One important thing to make those adaptive algorithms betterat adapting is to not change the controls to much. If one basethe decision on what to do the next second only on the lastsecond the adaptive algorithm is quite likely toself-oscillate.

Thus it is important to build in some inertia in the adaptivealgorithm. This protects the algorithm from going wild.But it doesn't make it adapt to conditions that changequicker than the change frequency. Adaptive algorithmscannot handle that.

So this is the first problem, to ensure that the adaptivealgorithm is quick enough to change to handle thechanging environment, but not so quick that it starts toself-oscillate.

The second problem is when two adaptive algorithmscrash into each other. As an example in NDB we have aproblem when CPU load is extremely high due toapplication activity while at the same time we arecoming close to the limit of the REDO log. In this casewe have two adaptive algorithms that conflict, one wantsto decrease the checkpoint speed to keep the applicationactivity while the other algorithm tries to slow down thecheckpoint activity to avoid running out of REDO log.

Now in a car the bets are higher, its human lifes involved.Almost the same problem a self-driving car will have tosolve when the driver has decided on the speed he wantsto travel while at the same time the control of the car seesdangers coming up ahead. These dangers could be othercars, cliffs or any other thing.

Sometimes cars even have to make decision on whetherits own passengers should survive or whether the by-standershould survive.

So the software of a self-driving car and any otherself-controlling software suffers from two big problemsto solve.

1) How often should I take input from the environment anddecide to change the controller parameters.2) How should I handle conflicting requirements

Failure in handling 1) will lead to self-oscillatingbehaviour and failure to handle 2) will lead tocrashes.

So hopefully any developer of self-driving cars has read upa lot on adaptive algorithms and know exactly when thealgorithm is safe and when it isn't.

Personally I always feel a bit uneasy about any adaptivealgorithm since I know that it is almost impossible topredict exactly how it is going to behave in all situations.

The mathematics involved in understanding adaptivealgorithms requires a lot of understanding of differentialequations.