Saturday, October 6, 2018

Animation can be a powerful tool. It is one thing to explain a complex topic in words or even in pictures, but visuals in motion have an amazing quality to bring abstract ideas to life. This can be especially helpful in complex areas of computer science like optimization and machine learning.

I recently gave a talk at KotlinConf on optimization and machine learning. Of the several examples, one was the Traveling Salesman Problem (a.k.a. “TSP”). This is such a fun and fascinating problem and it often serves as a benchmark for optimization and even machine learning algorithms. However, explaining some of the algorithms (like k-opt and simulated annealing) is less intuitive without a visual aid. Therefore, I made this an open-source project using JavaFX via TornadoFX.

A lot of folks at the conference and online were surprised by the animated visuals, remarking how “integrated” and slick it looked. The truth is I hacked this application together and JavaFX was instrumental in the heavy lifting. It took care of the animation for me so I could focus on the algorithm itself. This is what I want to blog about in this article.

You can watch the video walkthrough of this application (with a thorough explanation of the TSP) here. I recommend watching this before reading on!

The focus of this blog post will be on the animation and how it was achieved. For an in-depth explanation of the TSP and how it can be solved, please watch the above video.

The Structure

To build this, let’s first lay out the structure of our visual framework. I am going to express this in the Kotlin language, and leverage JavaFX via TornadoFX. Thankfully, TornadoFX does not hide or supresss any of JavaFX’s functionality, but rather augments it with expressive Kotlin DSL’s. So you can do this in Java, Kotlin, Scala, or any JVM language that can use JavaFX.

The first thing I’m going to do in my application is declare a Pane, and place in it an ImageView with a simple map image of Europe. Then from my domain model, I’ll import my City objects and place a red Circle on the x and y screen coordinates relative to the Europe map. Finally, I’ll import the Edge objects from my domain where each one is tied to a City, and bind each one to a Line. Each Edge represents a connection between two cities, and it is initialized with the start point and end point being the same city. Therefore, the Line will initialize by resting inside the Circle as a little dot. The Line will also be bound to the startX, endX, startY, and endY properties of its owning Edge.

When we animate, we will change each Edge’s startX, endX, startY, and endY properties. When we want to connect two cities, for instance, I can change the endX and endY properties to make that line extend to that other city’s coordinates.

Planning the Animation

With this structure in place, I did have to make a few considerations next. Should I animate the algorithm in live time or queue up the animations and make them replayable? Did I want to animate every single thing the algorithm did or filter out the noise and animate only the key events?

These decisions may seem unimportant at first glance, and I even told myself “why not animate everything?”. Of course, this backfired quickly because the animations already slow down the algorithm… and animating unproductive events in the algorithm only added noise. This also made the animation incredingly long and boring.

So what are unproductive events, you ask? Well, the algorithm works by doing thousands of random Edge swaps as explained in the video. When the swap did not improve the solution (or it failed the coin flip in the Simulated Annealing approach), I would undo the swap and put everything back. I learned it was best to not animate these events because most iterations were failed swaps, and it was better to animate successes to show progress rather than every iteration including failures.

Another adapatation I ultimtely made is running the algorithm first, and then animating the results. This had the benefit of being able to replay the results without having to run the entire process again. The key utility I needed in the JavaFX library is the SequentialTransition, which allows me to queue up animations and have them played in order (rather than all at once). I can then have my algorithm add animations to the SequentialTransition and it can be played later when it is done.

I stored each algorithm (“GREEDY”, “TWO-OPT”, “SIMULATED_ANNEALING”, etc) as an enumerable so I gave each one its own SequentialTransition. I also created some convenient extension functions so I could use += operators to add animations.

And of course, I set a speed as a constant that defines how long each animation frame takes.

// animation parametersvar speed = 200.millis

Executing a Path Traversal

On the domain model side, I have Edgeitems that initially belong to one City. However, the startCity and endCity can be mutated and on each mutation, the Edge has an animateChange() function returning a deferred Timeline that will play that change.

But here is the interesting design decision I ended up doing. I created the edgeStartX, edgeStartY, edgeEndX, and edgeEndY to not be synchronized to their respective startCity and endCity. Rather, these are used purely for animation execution. When I decide to animate a change in the startCity or endCity, I call animateChange() to create a Timeline that animates the coordinate changes. It will take the current value in each JavaFX property holding the coordinate values, and animate it by gradually increasing/decreasing to the specified value in that amount of time (which is the speed of the KeyFrame).

Note though this Timeline does not execute, that is up to the function caller on how to use that animation.

This particular function is used to expand an Edge for the first time to another city, which happens in the GREEDY and RANDOM algorithms. Stiching these together in a sequence results in a path traversing slickly to create a round-trip. Here is how the animateChange() function is leveraged in the RANDOM algorithm. Note how when I traverse to each random City, I connect each consecutive Edge pairs by their startcity and endCity respectively. Then I call animateChange() to return a Timeline and add it to the animationQueue.

My UI can then call animationQueue.play() to execute that change when the green play button is pressed.

Executing a Swap

Swaps are a bit more tricky than animating a path traversal. When TWO_OPT or SIMULATED_ANNEALING algorithms select random edges and try to swap their cities (vertices) somehow, sometimes it will fail and sometimes it will succeed. A failure can happen if a swap breaks the tour, and the reverse() function will be called. If it is successful, an animate() function can be called and return a Timeline that waits to be queued or executed.

This can then be used for the TWO_OPT and SIMULATED_ANNEALING algorithms. Note that for both these algorithms I start by cleaning the animationQueue, execute the RANDOM algorithm and take all of its animations, and add them to this algorithm’s animations. For the TWO_OPT, I then attempt 2000 random swaps and only add animations that improve the distance of the tour. Otehrwise I call reverse() and do not animate the swap (as if it never happened).

The SIMULATED_ANNEALING, at least on the animation front, is not much different. I essentially add a further condition to animate inferior moves that pass the coin flip. Swaps that do not are again reversed.

Friday, February 23, 2018

In Part I of this series I introduced binary programming with Kotlin and ojAlgo. In Part II, I introduced continuous variables and optimization concepts. In this section, I am going to present something more ambitious and useful: generating multi-day schedules. This can be applied to scheduling problems such as staffing, manufacturing, transportation, classroom allocation, and even sport event planning.

I started building okAlgo which contains Kotlin idiomatic extensions to ojAlgo. Hopefully I will get a chance to release this soon.

It is one thing to create an app that allows you to input events into a calendar. It is another for it to automatically schedule the events for you. Rather than relying on iterative brute-force tactics to fit events into a schedule (which can be hopelessly inefficent), we can achieve magic one-click generation of a schedule using mathematical modeling.

In this article, we will generate a weekly university schedule against one classroom. We will plot the occupation state grid on two dimensions: classes vs a timeline of 15-minute blocks. If we wanted to schedule against multiple rooms, that would be three dimensions: classes vs timeline vs room. We will stick with the former for now and do 2 dimensions. The latter will likely be another article in this series.

Before I start, I found a challenging but useful Coursera class on discrete optimization. This class is fairly ambitious and I hope to find time to complete it. It goes into different techniques to build optimization models from scratch using Python, Java, or any other platform of your choice. So far I highly recommend this class if you want to commit 10-15 hours a week to this topic.

The Problem

You need a one-click application to schedule university classes against a single classroom. These classes have differing lengths and may “repeat” throughout the week:

Psych 101 (1 hour, 2 sessions/week)

English 101 (1.5 hours, 2 sessions/week)

Math 300 (1.5 hours, 2 sessions/week)

Psych 300 (3 hours, 1 session/week)

Calculus I (2 hours, 2 sessions/week)

Linear Algebra I (2 hours, 3 sessions/week)

Sociology 101 (1 hour, 2 sessions/week)

Biology 101 (1 hour, 2 sessions/week)

Each session must start at the same time of day. The day should be broken up in discrete 15 minute increments, and classes can only be scheduled on those increments. In other words, a class can only start/end on the :00, :15, :30, or :45 of the hour.

The operating week is Monday through Friday. The operating day is as follows with a break from 11:30AM to 1:00PM:

8:00AM-11:30AM

1:00PM-5:00PM

Classes must be scheduled within these times.

OBJECTIVE: Create a discrete programming model that schedules these classes with no overlap and complies with these requirements.

Laying the Groundwork

The very first thing you should notice about this problem is how everything is broken up into “15 minute” blocks. This is not a continuous/linear problem but rather a discrete one, which is how most schedules are built. Imagine that we have created a timeline for the entire week broken up in 15 minute blocks, like this:

Note that the “…” is just a collapsed placeholder since we do not have enough room to display the 672 blocks for the week (672 = 7 days * 24 hours * 4 blocks in an hour).

Now let’s expand this concept and make the classes an axis against the timeline. Each intersection is a “slot” that can be 1 or 0. This binary will serve to indicate whether or not that “slot” is the start time for the first recurrence of that class. We will set them all to 0 for now as shown below:

This grid is crucial to thinking about this problem logically. It will make an effective visual aid because mathematical constraints will focus on regions within the grid.

On the Kotlin side, let’s get our infrastructure set up. First let’s improvise a DSL to make ojAlgo a little easier to work with. Note I am creating an extension to ojAlgo called okAlgo, which will create some nice Kotlin idioms. But for now, this should work.

The repetitionGapDays is the minimum number of days needed between each recurrence’s start time. For instance, since Psych 100 requires 2 repetitions and defaults to a 2-day gap, if the first class was on MONDAY at 8AM then the second repetition must be scheduled at least 2 days (48 hours) later, which is WEDNESDAY at 8AM. All classes will default to a 2-day gap.

The Block class will represent each discrete 15-minute time period. We will use a Kotlin Sequence in combination with Java 8’s LocalDate/LocalTime API to generate all of them for the entire planning window. We will also create a few helper properties to extract the timeRange as well as whether it is withinOperatingDay. The withinOperatingDay property will determine if this Block is within an operating day.

Note I am going to initialize items for each domain object using a lazy { } delegate. This is to prevent circular construction issues.

Finally, the Slot class will represent an intersection between a ScheduledClass and a Block. We will generate all of them by pairing every ScheduledClass with every Block. We will also create a binary() ojAlgo variable which will be fixed to 0 if the Block is not within the operating day.

Coming Up with a Model

In the first article in this series, I showed an approach to capture the necessary contiguous blocks for a given session. I found this approach to scale poorly with ojAlgo, although there are changes in the upcoming release (support for ordered sets) that might work with this approach. I could also drop in a $10K CPLEX license which also might execute a solve quickly.

But I like things to remain free and open-source where possible, so I concentrated hard and came up with a better mathematical model. It is highly abstract but powerful and effective for this particular problem.

Again, we are going to label each Slot as 1 or 0to indicate the start of the first class repetition. Here is one possible iteration the solver may come up with, where the first Psych 101 class starts on MON 9:00AM and Sociology 101 starts on MON 9:45AM. Here it is on our grid:

Study this scenario closely. Do you see a pattern for an invalid case? In the MON 9:45AM block, Psych 101 (which requires four blocks) and Sociology 101 (which also requires four blocks) are in conflict with each other. Visually, you might be able to see the conflict. But how do you describe it?

The sum of scheduled class blocks that “affect” the 9:45AM block must be less than or equal to 1. A sum of 1 effectively means only one class is taking up that block, and 0 means no classes are occupying that block at all (also valid). This particular case fails because the sum of “affecting” blocks is 2.

If we shifted Sociology 101 to 10:00AM, the sum would then be 1 and all is good.

We need to apply this logic to every block across the entire timeline, querying for earlier slots for each class that occupy this block, and dictate their sum must be no greater than 1. This abstract but powerful idea achieves everything we need. Here is what this looks like in practice below, where all slots affecting the 9:45AM block are highlighted in blue. All of these blue blocks must sum to no more than 1.

This can even account for the recurrences too. After all, we put a 1 in a slot to indicate the candidate start time of the first class. If we were looking at the 9:45AM block on Friday, we would query for time slots earlier in the week that would result in this 9:45AM Friday block being occupied (all they way to Monday). Here is a wide visual below. The sum of these blue slots must be no greater than 1.

Okay is your head spinning yet? The power of this model is not so much the math, but the ability for each block to query the slots that impact it and mandate they must sum to no more than 1. That is where the hard work will happen, and Kotlin’s stdlib can nail this effectively. The benefit is we do not have create any new variables, and can constrain the existing slot binary variables with a series of simple sum constraints.

Extracting Recurrences and Affected Slots

Wrangling and transforming data is tedious, and it is the unglamorous part of data science where 90% of the work occurs. It is for this reason Python has rapidly overtook R, but I think Kotlin can serve us type safety-minded folks who also appreciate extensibility and higher-order functions.

What we need to do first is identify the “groups” of slots for each class, and by “group” I mean an entire set of recurrences across the week. The star of this codebase is is going to be this Kotlin extension function, which will accomplish just that:

I will let you dive deep into the implementation on your own later. For now it is more productive to cover what it accomplishes, which is take any List<T> and perform a specialized windowed() operation that injects a gap between each grouping. Note the gap is the number of items between each start of the window. For instance, we can take the numbers 1…20 and break them up in groups of 4, with a gap of 6 between each recurrence start, and have 3 recurrences.

We can use this extension function to handle the class repetitions, and generate all possible permutations within our time planning window of one week. We can then use that to find slots for a particular class that affect a particular block, as implemented with our affectingSlotsFor() function shown below. We will also set our constraints dictating

To finish this off, let’s implement the needed constraints in the ScheduledClass with a addConstraints() function. We will set the sum of slots for each given class must be 1, so that at least one instance is scheduled. We will also limit the model exploring solutions for classes that have 3 repetitions, and say the start of the first class must be on MONDAY for those cases. For 2 repetitions, we will specify the first class must start on MONDAY, WEDNESDAY, or FRIDAY. We will achieve these by saying the sum in these regions must be 1.

We will also create start and end properties that will translate the model’s optimized slots (where one slot is 1), and translate it back to a LocalDateTime.

lNow going back to the Block class, I will add an addConstraints() function. It will query all the affecting blocks for each ScheduledClass and say they must all sum to no more than 1. This ensures no overlap between classes will occur. But if a block is not within an operating day, not only should its slots be fixed to 0, but all of its affecting slots should be fixed to 0.