perlquestion
perl_gog
Hello perl-monks, <br>
I am trying to: <br>
a) download a xml file <br>
b) do some basic processing on this xml file and <br>
c) finally save it. <br>
<br>
I was wondering what is the best option to do this: <br>
<br>
option 1. <br>
<code>
system("wget -O - 'http://host/getFeed.xml' > /tmp/myfeed.xml"); //download + save as tmp file..
open FH, "</tmp/myfeed.xml";
while(<FH>) {
//manipulation steps here..
}
close FH;
</code>
<br>
Option 2: <br>
Is there a way to process the file, as its being downloaded? (instead of having to save it temporarily, and then reading it to process it) ?<br>
<br>
I dont know how to implement option#2. Do you have any suggestions? <br>
The xml feed can be quite huge, (~150Gb max.) As, the feed is huge, I am hoping there's a better option compared to option #1, as writing this file first to some tmp file and then saving it again would mean more disk-activity. <br>
<br>
thanks! <br>