With deep learning, it would seem like self-driving vehicles would continuously improve the level of safety of their operations and "learn from their mistakes." How then do you set a pass/fail threshold that may be ever-changing?

Douglas Farrell: One of the things that you have to worry about with machine learning is that you’re not sure even what the things are that you're going to have to test, because the software is constantly evolving. You're going to need define scenarios, and expected outcomes for each scenario. You'll need to make sure that the vehicle is meeting whatever expected outcome you had.

As you mentioned, these outcomes could be changing and improving over time, and you’re going to have to be mindful and cognizant of that. We’ll have to be heavily involved in making sure that we've defined our test cases and scenarios, and the expected outcomes of these scenarios, and that they're always being modified according to how the latest state-of-the-art technology is saying that these vehicles should be operating.

We will always know that a vehicle should stay in its lane no matter what — unless you’re changing lanes. That's a pass/fail scenario that is easily stated: If the vehicle swerves out of its lane, that's a fail.

As we start talking about failures interoperating between systems, it's going to be a lot more complicated. We're going to have to look at how different components of the vehicle think that we passed or failed the system. For example: Did multiple sensor types detect the same issue in the road, or do we think that that was a failure in one of the subsystems? We're going to have to look at how the holistic vehicle defines the pass/fail scenarios for how the vehicle is behaving. Again, that's an argument for the need for system integration testing.

Articles: Test & Measurement

Tech Briefs Media Group

Sign In

By submitting your personal information, you agree that Tech Briefs Media Group and carefully selected industry sponsors of this content may contact you and that you have read and agree to the Privacy Policy.