To be able set UncaughtExceptionHandler per thread we need to create a Java ThreadFactory, which will create threads and set the handler for each Thread. Then we just stop ExecutorService and print all errors collected by the exception handler:

Pretty straightforward. The only thing to remember is that when we want to set UncaughtExceptionHandler per thread we have to provide own ThreadFactory, in which we do that. Another approach would be to use static Thread.setDefaultUncaughtExceptionHandler(UncaughtExceptionHandler) to set the default handler for all threads. Chosen solution depends on what you need.