Turns out I did not have @Test annotation on a method I was trying to run as a JUnit test. The interesting thing I only got that error when other methods in that test class were annotated with @Test. If there were no other methods it would throw a
"No runnable methods exception". Completely trivial but I did not immediately connect the error message to the actual cause.