On Thu, 2007-03-01 at 09:22 -0800, ramzi abboud wrote:
> Is R file IO slow in general or am I missing
> something? It takes me 5 minutes to do a load(MYFILE)
> where MYFILE is a 27 MB Rdata file. Is there any way
> to speed this up?
>> The one idea I have is having R call a C or Perl
> routine, reading the file in that language, converting
> the data in to R objects, then sending them back into
> R. This is more work that I want to do, however, in
> loading Rdata files.
>> Any ideas would be appreciated.
> Ramzi Aboud
> University of Rochester
Here are some timings on my system, which runs Linux on a 3.2 Ghz P4
with 2 Gb of RAM and a 7200 rpm HD. I typically get around 28 Mb/sec
throughput on this drive, which is about 15% lower than normal, as it is
an encrypted partition using 256 bit AES.
> Vec <- 1:15000000
> system.time(save(Vec, file = "Vec.RData"))
[1] 33.297 0.565 38.889 0.000 0.000
# File is ~29 Mb
> file.info("Vec.RData")$size
[1] 30112009
> system.time(load("Vec.RData"))
[1] 5.607 0.167 6.575 0.000 0.000
Not terribly burdensome...
You might want to be sure that you are not low on RAM, resulting in a
lot of swapping to disk, or perhaps just a slow drive.
HTH,
Marc Schwartz