CPANdeps efficiency improvements

CPANdeps has been getting more heavily used recently, and all of the users hitting my shonky code have been occasionally DOSsing the server. That's because the old code would eat up to 134MB of memory to generate a set of results. And that's because I was using Parse::CPAN::Packages to convert from module names to package names. That reads the entire 02packages file into memory and converts it into a very large data structure.

The new version of the code stores that in the database instead, so instead of using 134MB I now only use 25MB per request. This means that I'm hitting the database harder, but even so, things should still run a lot better because even serving 25 requests in parallel (the maximum it's configured to do) it will only need 625MB, so you won't all be fighting over swap space.

I would like to thank Jonathan Rockway once again for writing the excellent CPANdeps test suite which lets me be confident that the site still works just as it did before. His test suite is available on the CPAN as Angerwhale:-)

The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
Without JavaScript enabled, you might want to
use the classic discussion system instead. If you login, you can remember this preference.