The cause of your error doesn't have anything to do with writing to a file. It appears that link_tag.find('a').contents sometimes returns an empty list and gives an error when you try to get the first item. You can try something like this: