Metals throws away its navigation index when it shuts down. Next time it starts,
the index is computed again from scratch. Although this approach is simple, it
requires indexing to be fast enough so you don't mind running it again and
again. Also, because we don't persist the index to disk, we need to be careful
with memory usage.

This post covers how Metals achieves fast source indexing for Scala with a small
memory footprint. We describe the problem statement, explain the initial
solution and how an optimization delivered a 10x speedup. Finally, we evaluate
the result on a real-world project.

The work presented in this post was done as part of my job at the
Scala Center.

Problem statement

What happens when you run Goto Definition? In reality, a lot goes on but in this
post we're gonna focus on a specific problem: given a method like
scala.Some.isEmpty and many thousand source files with millions of lines of
code, how do we quickly find the source file that defines that method?

There are some hard constraints:

we must answer quickly, normal requests should respond within 100-200ms.

memory usage should not exceed 10-100Mb since we also need memory to implement
other features and we're sharing the computer with other applications.

computing an index should not take more than ~10 seconds after importing the
build, even for large projects with millions of lines of source code
(including dependencies).

To keep things simple, imagine we have all source files available in a directory
that we can walk and read.

With this index, we find the definition of Some.isEmpty using the same steps
as for Files.readAllBytes in Java:

take the enclosing toplevel class scala.Some

query index to know that scala.Some is defined in scala/Option.scala

parse scala/Option.scala to find exact position of isEmpty method.

The challenge is to efficiently build the index.

Initial solution

One approach to build the index is to use the
Scalameta parser to extract the toplevel classes of
each source file. This parser does not desugar the original code making it
useful for refactoring and code-formatting tools like
Scalafix/Scalafmt.
I'm also familiar with Scalameta parser API so it was fast to get a working
implementation. However, is the parser fast enough to parse millions of lines of
code on every server startup?

According to JMH benchmarks, the Scalameta parser handles ~92k lines/second
measured against a sizable corpus of Scala code. The benchmarks use the
"single-shot" mode of JMH, for which the documentation says:

"This mode is useful to estimate the "cold" performance when you don't want to
hide the warmup invocations."

Cold performance is an OK estimate for our use-case since indexing happens
during server startup.

The Scala standard library is ~36k lines so at 92k lines/second this solution
scales up to a codebase with up to 20-30 similarly-sized library dependencies.
If we add more library dependencies, we exceed the 10 second constraint for
indexing time. For a codebase with 5 million lines of code, users might have to
wait one minute for indexing to complete. We should aim for better.

Optimized solution

We can speed up indexing by writing a custom parser that extracts only the
information we need from a source file. For example, the Scalameta parser
extracts method implementations that are irrelevant for our indexer. Our indexer
needs to know the toplevel classes and nothing more.

The simplified algorithm for this custom parser goes something like this:

tokenize source file

on consecutive package object keywords, record package object

on package keyword, record package name

on class and trait and object keywords, record toplevel class

on ( and [ and { delimiters, skip tokens until we find matching closing
delimiter

There are a few more cases to handle, but the implementation ended up being ~200
lines of code that took an afternoon to write and test (less time than it took
to write this blog post!).

Benchmarks show that the custom parser is almost 10x faster compared to the
initial solution.

At ~920k lines/second it takes ~6 seconds to index a codebase with 5 million
lines of code, a noticeable improvement to the user experience compared to the
one minute it took with the previous solution.

Evaluation

Micro-benchmarks are helpful but they don't always reflect user experience. To
evaluate how our indexer performs in the real world, we test Metals on the
Prisma codebase. Prisma is a server implemented in
Scala that replaces traditional ORMs and data access layers with a universal
database abstraction.

Can the indexer become faster? Sure, I suspect there's still room for 2-3x
speedups with further optimizations. However, I'm not convinced it will make a
significant improvement to the user experience since we remain bottle-necked by
dumping sbt build structure and compiling the sources.