I am getting an Internal Server Error from Apache when attempting to run a
search that returns a very large number of results (i.e. 25,000 pages and
up). Same search run at the command line executes perfectly. When the
number of indexed pages was less (ca. 80,000) the search worked, but now
that the number of pages is over 110,000 the Error occurs. The messages
file says "Premature end of script headers" and that is it.
Running the search at the command line takes around 8 or 10 seconds; it
appears that Apache is giving the error after timing out waiting for a
response. I have set this timeout as high as 3 or 4 minutes. The machine
is not under a whole lot of strain via http.
Question: Should sending the results from HTDIG to Apache be taking
several minutes longer than the search itself at the command line?