The Time It Takes To Find A Bug

How much time passes between writing a line of RTL or testbench code and knowing that it works. I know it varies, but on average how long would it take? A line of code is written, time passes, then a test is run that either verifies it’s correct or discovers a bug. Does that time pass as minutes? As days? Maybe weeks? Months?

I’m preparing for a talk at DAC next week in the Verification Academy booth and in going through my material I’ve started asking myself that question. It’s the 3rd year in a row the folks at Mentor have invited me for a session in the booth. The last 2 years have been a lot of fun and I’m hoping for the same this year. This year’s session is called Add Unit Testing To Your Verification Toolbelt. As the name suggests, the focus will be on how we can use unit testing and test-driven development to improve the quality of the hardware we build.

I believe quite strongly in the value of unit testing and test-driven development because the quality of what I produce as a verification engineer is noticeably better because of them. I believe so strongly in TDD in particular that I know I get a little preachy at times. It’s the most important engineering skill I’ve ever learned and I don’t shy away from telling people about it.

I always think of rigour first when it comes to benefits. Testing code as you write it a few lines at a time is a rigorous process to say the least. But during the run-up to DAC I’m starting to think it’s timing that’s the real key. With TDD, the amount of time that passes between writing and testing a line of code is minutes, sometimes less. A design and test cycle measured in minutes is almost too short a time to think about anything else. I focus on getting exactly 1 thing right. There’s no thinking back, no back-and-forth with a teammate that found a bug, flipping through my logbook for clues as to what I’ve done or editing bug reports. In fact, with TDD there’s hardly any context switching at all. I write a line or 2, test them, if they’re broken I fix them, then I move to the next line. Most of the bugs I create are killed immediately; which coincidentally is the best time to kill them.

I’ve seen a lot of people talk about how bugs become more costly to fix the longer they live. For me, TDD has been the most effective way to respond.

If you’re at DAC next week in Austin and using TDD to kill bugs fast sounds good – or great! – I hope to see you at Mentor’s Verification Academy Booth at 11am on Monday. It’ll be a half hour presentation with lots of time for discussion. We’ll talk about TDD, unit testing, SVUnit and all the frustration that magically vanishes because of them!