EC2 shows this CPU utilization during the time of the crash (in the middle, before 13:00 UTC).

So there is nothing even visible during that time period. None of our other EC2 instances crashed and I can find no evidence that it was an application error. In fact, this happened on our tools server (apache, mongodb, and redis). Monit was also running, but there are no suspicious logs at the time of the crash.

What could have caused this crash and what do the ^@'s mean in the syslog?

1 Answer
1

There was a problem with the underlying hardware that caused the underlying system to crash. The artifact of the syslog lines, could be just a representation that the system was in mid write when it crashed, or the lines could have come from the the cron process that was running on the previous line.