Software is hard. When writing a Java program, you're already so far up the Jenga tower that is a modern computing system that when a lower brick causes you to question your sanity, it's something else.

It is possible for the file metadata to be updated before the file payload is updated accordingly. Let that sink in.

Consider a program's log file that is constantly being appended, rotated, etc... The Apache commons-io Tailer object can be constructed in such a way that it will continually reset itself to the head of the file because it thinks it has been truncated or overwritten with the exact same data. If you pass in a zero second delay (tail's default is 1.0s as is the Tailer's) it may read that the file has been updated but by the time it gets to the length check, it is the same and resets to the beginning of the file.

while (getRun()) {
final boolean newer = FileUtils.isFileNewer(file, last); // IO-279, must be done first
// Check the file length to see if it was rotated
final long length = file.length();
if (length < position) {
// File was rotated
listener.fileRotated();
// Reopen the reader after rotation
try {
// Ensure that the old file is closed iff we re-open it successfully
final RandomAccessFile save = reader;
reader = new RandomAccessFile(file, RAF_MODE);
// At this point, we're sure that the old file is rotated
// Finish scanning the old file and then we'll start with the new one
try {
readLines(save);
} catch (IOException ioe) {
listener.handle(ioe);
}
position = 0;
// close old file explicitly rather than relying on GC picking up previous RAF
IOUtils.closeQuietly(save);
} catch (final FileNotFoundException e) {
// in this case we continue to use the previous reader and position values
listener.fileNotFound();
}
continue;
} else {
// File was not rotated
// See if the file needs to be read again
if (length > position) {
// The file has more content than it did last time
position = readLines(reader);
last = file.lastModified();
} else if (newer) {
/*
* This can happen if the file is truncated or overwritten with the exact same length of
* information. In cases like this, the file position needs to be reset
*/
position = 0;
reader.seek(position); // cannot be null here
// Now we can read new lines
position = readLines(reader);
last = file.lastModified();
}
}

We enter the else block if the length is not less than position indicating a file rotation. If the length is not greater, indicating more lines to process, it assumes that the file must be truncated or overwritten to the exact size. Moving from a zero time delay to 10 milliseconds alleviated this problem. I suppose the lesson here is to not assume atomic operations on your file system OR just use the defaults? Happy coding!