Improving Heuristics

Table of Contents

(Warning: I had put this on hold in 2015 and revisited it in late 2016. It’s still incomplete, and it’s broken on mobile devices)

For optimizing A* we usually look at the priority queue or the map representation. Often overlooked is improving the heuristic function.

A* uses a heuristic function to get an estimate of the cost of moving from one place to another (also called “geodesic distance”). All other things being equal, the better the estimate, the faster A* runs. Normally in game maps we use straight line distance as the heuristic. This is a great estimate in wide open maps with few obstacles. Here’s a × map of Denerim (Bloodmage HQ) from Dragon Age:Origins with walkable tiles. The straight line distance is but the actual path is . The mismatch means A* has to explore nodes:

{{ the demo here should use a non-grid, maybe with delaunay where nodes are circles and edges are line segments }}

{{ add arrowhead to path, as well as blob & X for start/end, maybe green Δ for landmarks }}

What if we could improve the heuristic from to ? (The best would be the actual graph distance of .) Instead of exploring nodes, A* would explore only nodes:

These are interactive diagrams. You can move the blue and red points around. What are the green nodes?

Suppose we’re finding the shortest path from A→Z. A* will ask for heuristic(n, Z) for many different nodes n, but always to the destination Z. The best heuristic (as explained on the main page) is the exact cost from n to Z. In practice, we can’t use the best heuristic but instead use a distance function.

Let’s suppose for this particular Z we have precalculated cost(n, Z) for all nodes n. What would happen? We could use this as our heuristic, and A* would run fast! But we could only use it when finding a path to Z.

What if we had precalculated the cost(n, L) for some other point L? Can we use that to find the shortest path from A to Z? Yes, but only when Z is on the path from A to L:

AZLcost(A, Z)cost(Z, L)cost(A, L) = cost(A, Z) + cost(Z, L)

In this case we know cost(A, L) and cost(Z, L) because we have precalculated cost(*, L), but we need to know cost(A, Z). If Z is on the shortest path from A to L, then we can calculate cost(A, Z) = cost(A, L) - cost(Z, L).

Can we handle more cases? What if Z is near the shortest path from A→L?

AZLcost(A, Z)cost(Z, L)cost(A, L) < cost(A, Z) + cost(Z, L)

We can’t exactly calculate cost(A, Z) in this situation. However, the triangle inequality[1] adapted for directed graphs, we can construct both lower and upper bounds:

cost(A, L) - cost(Z, L) ≤ cost(A, Z) ≤ cost(A, L) + cost(Z, L)

That’s the key idea here. It’s impractical to precalculate all costs to all locations, but if we’ve precalculated the costs to a specific location, we can use that to help us estimate other costs that don’t involve that location!

{ Heuristic is a lower bound so we normally only care about cost(A, Z) ≥ cost(A, L) - cost(Z, L), but see references — one paper found the upper bound to be useful too }

{ if the graph is directed we can only say this, but if it’s not directed we can use abs(cost(A, L) - cost(Z, L)) which allows for the landmark to be on the other side too }

Let’s generalize this. Instead of having all costs cost(, Z) let’s say there’s a third point L where we have cost(_, L). Point L is called a landmark. If you’re at A and want to get to Z, a friend could say “walk towards L, and you’ll see Z on the way — you can’t miss it!”.

{{ I already generalized in the previous section, so reword this }}

AZLcost(A, Z)cost(Z, L)cost(A, L)

We have cost(*, L) precalculated. We are trying to find a path A→Z, and need to estimate cost(N, Z). We can estimate cost(N, Z) ≥ cost(N, L) - cost(Z, L). This works for any N and any Z, and any L for which we have precalculated cost(_, L).

— EDIT —

This only works when L is behind the route A→Z*. To make it work for many different paths, we need to have multiple landmarks L placed around the map. Then we calculate final heuristic as:

Although I write it as cost(Lᵢ, __), it’s really stored as a precalculated array cost[i][__]. Since distance() is fast and cost[] is precalculated for each of the L points, this heuristic is fairly fast to calculate.

{{ demo: one goal, one landmark, reader moves N to see triangle, costs of each of the three sides, calculated lower bound, and distance heuristic }}

As the name “differential heuristics” suggests, this is a change to the heuristic given to A*, but not a change to the A* algorithm itself.

To use our landmark L, we need to add a map analysis step and modify the heuristic function.

The map analysis, run once per map, is to calculate cost(L, __) for all locations and save those costs in a data structure. My article on A* and Dijkstra’s Algorithm[2] shows how to calculate these with Dijkstra’s Algorithm, and save it in a variable cost_so_far[]. For the landmark L, I’m going to copy that to L_cost[]. This is roughly what we have to do:

What changes with the A* code? Nothing. We only have to change the heuristic function. Previously I set the heuristic to be distance(A, Z), because that was the only lower bound I had. Now I have an additional lower bound, cost(L, Z) - cost(L, A), which has been calculated during the map analysis and saved to L_cost[Z] - L_cost[A]. I need the heuristic to pick the better lower bound. Here’s what the code looks like before landmarks:

If we have just one landmark, where should we place it? A landmark’s location will help some goal locations but not all. Let’s explore this to build up some sense of what landmark locations are most useful. Move the landmark around to see which goal locations are helped:

{{ demo: one Z, one P, all N: show how much the landmark improves the heuristic }}

{{ offline calculation: lots of starts, lots of goals, improvement for each landmark candidate site; no interaction here because there are no free variables anymore }}

{{ conclusion on placement to be written after I have the demo implemented }}

Another idea: do we need complete data? No, we don’t! That means we could use the cost_so_far from the previous few A* runs. They’re already computed, so it’s almost free to keep that around. If we’re finding a lot of paths in the same areas, that data would probably help a lot.

{{ which landmarks help the most? the LPI paper suggests picking the landmarks closest to the start or goal nodes }}

We need to precalculate cost(L, __) to all other points, but what if the map changes? This is not that expensive to calculate occasionally. It’s Dijsktra’s Algorithm starting at L. If all your movement costs are 1, you can use the much faster Breadth First Search. And either way you can spread this out over multiple frames.

There are two problems that happen if you use an outdated cost(L, __) after the map changes:

The new cost is lower than the old cost (you broke a wall). The heuristic will overestimate sometimes, and A* will return a non-shortest path until you’ve updated the cost table. Think of it this way: you broke a wall but the unit doesn’t know right away to take that into account.

The new cost is higher than the old cost (you added a wall). The heuristic will be lower than desired, and A* will take a little longer to run until you’ve updated the cost table. It will still be faster than if you weren’t using landmarks. Think of it this way: you added a wall so the unit might think that area’s safe to walk through but will soon find a path around it.

If you need to recalculate often, consider Breadth First Search instead of Dijkstra’s Algorithm. It runs much faster but will produce worse distance values (but still better than straight line distance). In many maps this will be a good tradeoff.

There’s a different approach in pathfinding also called “landmarks” (e.g. https://pdfs.semanticscholar.org/f918/5ed02848c1cd3e0ccdda16fa7a32f7428a8a.pdf[6]) which is not the same how these points are used. Those landmarks need to be between the start and end points, whereas the landmarks on this page are used on the edges of the map. “If you want to walk to Z, walk to L, and then to Z” vs “If you want to walk to Z, walk to L, and Z is on the way”.

TODO: also try breadth first search with the two-insertonly-queue trick on these large maps. Breadth first search is faster than A* but I’d be scanning the entire map instead of just part of it; I’m not sure which will win overall. Mazes are probably faster with breadth first search, and big open areas might be too.