Physical system memory: 160Mb
Perl version: 5.6.0
Relevant bits from the indexing output:
Using DB_File...
...
Crawler finished(8659 files, 77760 terms)
...
Illegal division by zero at ./indexer.pl
I don't know if anybody has used perlfect-search on such a large data
set (8659 files) so I might be running into some system limitations
here. The indexing works just fine if I use a smaller number of data
files, but I haven't played with it to see at what point it starts
breaking down.
The error occurs while calculating the weights at the following
statement in the "weights" sub:
$weight = $tdf{$doc_id} * log ($DN / $df);
$df is 0 at this point. I added some print statements to see what's
happening and here's what I found:
In the "index_file" sub, the last time $df_db{$_}++ is executed with
$_ being set to 4488, $df_db{4488} == 9. $df_db continues to be
updated for other terms after that. In the "weights" sub, when we now
read $df_db{4488} we get 0.
Any ideas? I searched the archive but did not find any similar
problems reported.
Thanks.
--
Gabriel Burca