Saving the Day with the Thread Monitor Framework

Systems have a tendency to work just perfectly while on the bench but once deployed into the field, it is possible for systems to experience lock-up, single event upsets or have a thread run off into the wild. Developers have often employed the watchdog timer to swoop in and save the day but there are many instances where teams don’t implement their watchdog properly. A few simple examples that I have witnessed repeatedly in industry is for watchdog timers to be disabled in production and allowing the watchdog reset to be handled by a timer interrupt that continues to function long after the system has crashed and burned. Implementing a watchdog strategy can be complicated which is why the Renesas Synergy™ Platform includes a Thread Monitor Application Framework to simplify the process.

The idea behind the Thread Monitor Application Framework is simple. If a thread stops executing or executes too many times within a given time period, half the watchdog timer refresh period, the thread monitor resets the processor and recovers the system. How does the Thread Monitor detect if a thread has run amuck? Each thread is registered with the Thread Monitor Application Framework and periodically increments a counter as it executes. The thread monitor will check the thread counter and compare it with minimum and maximum count values set in the configuration. If the count doesn’t fall within the desired range, the thread monitor will then invoke the watchdog and reset the system. An example sequence with two separate threads being monitored can be seen in the diagram below.

At this point, one might wonder how on Earth a developer is supposed to determine how many counts to expect from a thread in a given time period. One might be tempted to just guess and hope for the best but thankfully, the Thread Monitor Framework has a profiling mode. The profiling mode allows a developer to monitor the thread behavior and determine the minimum and maximum counts. In profiling mode, the selected watchdog will be refreshed even if a thread stops executing or runs off into the weeds. WARNING! Make sure that your software is tested and verified before running the profiler for production. Developers can use first pass values during development but running the profiler on untested and potentially buggy code could lead to incorrect minimum and maximum counts.

Adding the Thread Monitor Application Framework to a Synergy Project is simple. From within the Synergy configurations threads tab, select or create a new thread that the monitor will be added to. In the Thread Stacks window, click add and navigate to Frameworks > Services > Thread Monitor Framework on sf_thread_monitor. The result will be similar to the screen shot below:

The Thread Monitor component requires a Watchdog Timer driver. Developers a choice to select either the Watchdog Timer (WDT) or the Independent Watchdog Timer (IWDT). Once the timer has been added, a developer can set the clock rate, whether it is windowed or not and a few other parameters that will then set the refresh rate used by the Thread Monitor Application Framework to check in on the running threads.

Next time we will explore the ADC Framework and how it can be used to sample analog signals in the background.

Live long and profit!

Professor_IoT

Hot Tip of the Week

If you have not already found the wealth of information available on the Renesas Rulz Synergy Forum, now is a good time to check it out because there is a post related to the Thread Monitor that includes a simple, but useful, Getting Started Guide. It will help you dig into more of the details behind adding a Thread Monitor Framework to your project.