Thursday, November 22, 2012

The Only Way to Benchmark Dart Async Code

Yesterday I used the benchmarking package for Dart to measure my current data-load scheme in dart dirty. Today, I would like to be able to do the same for a better approach. The problem is that a better approach is asynchronous and I am unsure of how to benchmark that.

Hearkening back to my efforts with async testing in Dart, it would be really nice if some of those unittest methods blocked. That is, if the expectAsync0 method blocked until the async method in question was called, then I could do something like wait until my DB has finished loading:

But that just falls straight through without hitting the expectAsync0 (expect an async method to be called with zero arguments). The problem is that none of the expectAsync methods in unittest will block until expected async method is called. If the functions inside the test have completed execution without invoking the expected callbacks, then the test fails. Actually, it seems to simply stop all tests as my test suite returns nothing:

Unless I am mistaken, Dart benchmarking suffers from a similar problem. It expects only a run() method to be defined. In other words, there is not a startTimer() / endTimer() pair of methods I could define when the database load first starts and when it is complete.

For this stuff, I think that I am going to have to drop back to writing my own timer. Happily, Dart includes a microsecond stopwatch:

What I do here is start the stopwatch before I open the DB stream. By virtue of defining an onLine handler for whenever new data is available to be read, data is read from the file. When there is no more data to be read, the onClosed handler is invoked, stopping my timer and reporting back to me.

It finished in 11 seconds! Whoa. That was faster than my supposed optimization. I still might have memory issues with this approach, but speed is definitely not a bottleneck. And this is why I wanted to benchmark my code.

There still may be some things I can do to improved the time of my data load. Now that I have some concrete numbers to compare with, I can get started on that tomorrow.