Older versions of the CPAN module will do that. Fetch a fresh CPAN.pm tarball and install it manually, then you'll be able to use the CPAN shell from there on without it pigheadedly insisting on upgrading your Perl.