We have been doing some aggressive profiling and tuning of our application and we have noticed that the highest amount of processing time is spent doing validation. We push a lot of data through, so this isn't a big surprise. But we are surprised when we look at what it appears to be doing during this time.

Depending on the type of workload we are testing, we are seeing 8-50% of execution time spent in this stack trace.

Can someone help us understand what might be going on here? Our validation is defined through static xml files that are bundled into our jars, and the java classes are created through Thrift schema definitions and code generation.

This stack trace seems to be on some boundary between Hibernate Validator and other libraries such as ejb and bytecode enablement. But it doesn't look right to us that the app is spending so much time just checking isLoaded and other methods in this stack trace over and over again for the same few classes that we are validating.

I don't think you are doing something wrong, apart from using a very old version of Hibernate :)

Any chance you could try with a current release (5.1/5.2)? The bits around bytecode enhancement have been largely reworked since then, so chances are that this issue is gone by now. Also many bug fixes and performance improvements have been done since then, so I highly recommend that upgrade.

If you can't do that for whatever reason, it's a bit trickier because Hibernate ORM 4.1 isn't maintained anymore. If you are on a supported platform (JBoss EAP), we could look into a fix in the supported 4.x version. Otherwise you might open an issue in the community tracker (https://hibernate.atlassian.net/browse/HHH) and discuss with the maintainers of Hibernate ORM whether they'd be willing a fix of yours for this issue in 4.x.

One other option I can see is that you plug in a custom TraversableResolver implementation which essentially is a copy of the existing implementation and those bits around instrumentation state detection but keeps a cache of instrumentation state by class, avoiding the repeated calls to getInterfaces().