Python Programming Language

writing to a file

as i understand there are two ways to write data to a file: using f.write("foo") and print >>f, "foo". what i want to know is which one is faster (if there is any difference in speed) since i'm working with very large files. of course, if there is any other way to write data to a file, i'd love to hear about it

montyphy@gmail.com wrote: > as i understand there are two ways to write data to a file: using > f.write("foo") and print >>f, "foo".

well print will add a '\n' or ' ' if you use ',' after it

> what i want to know is which one is faster (if there is any difference

there shouldn't be any noticable difference

> in speed) since i'm working with very large files. of course, if there > is any other way to write data to a file, i'd love to hear about it

and probably there are other obscure ways, but the intended way is obviously f.write

nsz

montyphy@gmail.com schrieb:

> as i understand there are two ways to write data to a file: using > f.write("foo") and print >>f, "foo". > what i want to know is which one is faster (if there is any difference > in speed) since i'm working with very large files. of course, if there > is any other way to write data to a file, i'd love to hear about it

You should look at the mmap-module.

Diez

On May 30, 1:41 pm, "Diez B. Roggisch" <d@nospam.web.de> wrote:

> montyphy@gmail.com schrieb: > > what i want to know is which one is faster (if there is any difference > > in speed) since i'm working with very large files. of course, if there > > is any other way to write data to a file, i'd love to hear about it

> You should look at the mmap-module.

Yes, memory mappings can be more efficient than files accessed using file descriptors. But mmap does not take an offset parameter, and is therefore not suited for working with large files. For example you only have a virtual memory space of 4 GiB on a 32 bit system, so there is no way mmap can access the last 4 GiB of an 8 GiB file on a 32 bit system. If mmap took an offset parameter, this would not be a problem.

However, numpy has a properly working memory mapped array class, numpy.memmap. It can be used for fast file access. Numpy also has a wide range of datatypes that are efficient for working with binary data (e.g. an uint8 type for bytes), and a record array for working with structured binary data. This makes numpy very attractive when working with binary data files.