Monthly Archives: November 2014

In Python, you can read from and write to files without import any modules. Python has built-in function “open” which can be used to view and manipulate file objects. Let us see two ways of opening a file for reading/writing, for instance –

fp_in = open('/etc/hosts', 'r') # default is 'r', we can omit it.
fp_out = open('/tmp/hosts', 'w')
for line in fp_in:
fp_out.write(line)
fp_in.close()
fp_out.close()
with open('/etc/hosts') as fp_in:
with open('/tmp/hosts') as fp_out:
for line in fp_in:
fp_out.write(line)
# No need to close file, it is automatically closed at end of block.

One of the most common reasons given why you have to close the file object in the first case is to free up resources. But there is a second reason why you should always use ‘with’ keyword. After writing to a file object, and before closing it, the whole content from the source file might not appear in the destination file. This is because write uses buffering, and the changes will not be reflected until you run flush() or close() on the file object. Here is the help page for ‘write’ –

write(...)
write(str) -> None. Write string str to file.
Note that due to buffering, flush() or close() may be needed before
the file on disk reflects the data written.

Let me demonstrate this by copying the /var/log/messages file to /tmp/message, the bigger the file, the more likely you will witness the effect of buffering. First i will take a copy of /var/log/messages to /var/log/messages.orig, and work with messages.orig as the former will most likely change in size as work along.