Description

There is a race condition in run record corrector:
1. Program is running according to run record (and it actually is).
2. Next, it checks live-info to see if the program is still running. By this point, the program could have stopped/completely successfully. We don't see it in the live-info as running, so run record corrector things it has to correct the run record.
3. It logs "Fixing RunRecord <runId> in program...", and attempts to update the run record to ERROR status.https://github.com/caskdata/cdap/blob/release/3.4/cdap-app-fabric/src/main/java/co/cask/cdap/internal/app/services/ProgramLifecycleService.java#L649-L653
4. Fortunately, it is using a compareAndSet method, to handle this race condition, so the run record isn't actually updated. I can see from the UI that it is in COMPLETED state. However, the log statement is very misleading.

The fix could be to have compareAndSet return a boolean value, with which we could use to avoid logging the statement saying that the run record is being corrected.