Everything seems working fine until we noticed few records executing twice causing inconsistency.

The issue was random and unable to reproduce consistently.

Reason –

Culprit was System.Timer’s Elapsed event

Assume the timer’s duration is 3 seconds, so every seconds Elapsed event kick in and executes the business logic. What happens if previous elapsed event would not complete before next Elapsed event ticks in.

This is exactly what happened in our case and caused duplicate execution of records.

Fix –

We restricted overlap of Elapsed event using a Static variable as below