Contact Us

Eat My Data: How Everybody Gets File IO Wrong

Too many applications don’t deal with important files in a safe way. Along with that, they don’t check error codes properly or do things like blindly rely on fsync() actually sending data to disk (which becomes a problem if you intend your application to be portable). Write to temp file, close, rename is not a safe operation. We’ll discuss why—along with the various behaviors of different file systems and platforms and all the different ways you can expect your important data to get mangled.

More importantly, we’ll talk about how to write code using the POSIX file IO api and get it right—keeping your application portable, fast, and most importantly—respecting the data that users trust in it.

We’ll discuss:

when data hits disk

what causes data to hit disk

how you can ensure that data is on disk

how applications get this wrong

how a file system is not a database (and vice versa)

Stewart Smith

Percona

Stewart Smith works for MySQL AB as a software engineer working on MySQL Cluster. He is an active member of the free and open source software community, especially in Australia. Although Australian, he does not dress like Steve Irwin—although if he wrestled crocodiles he probably would. He is a fan of great coffee, great beer, and is currently 39,000 feet above sea level.