You can set $Data::Dumper::Deepcopy = 1 and Data::Dumper will dump the full contents even if they are just a copy of some other part of the structure already dumped. The only exception is if a structure contains a reference to itself, then it has to show a variable, otherwise it would loop forever.

I wouldn't worry about the memory use of Data::Dumper.

Comment on Re: In need of a Dumper that has no pretentions to being anything else.

That sounds like a serious bug in DD. In order to catch circularity it only needs to keep a 4 byte hash key and a short string for every reference in the structure. Unless your structure is full of almost empty arrays, hashes and scalar refs, this should take less memory than your structure.

Update: Just dug in DD and I see it's storing a 2 element arrayref for each ref it find. That's still not very much and unless you have an unusual structure, it should be negligible. The only other thing is that it stores a copy of the hash key in that 2 element array. If your keys are very big then that could be a problem however you'd still only be at most doubling things.

Add whatever Dumper options you like. Prior to the Dump, this hash with somewhat under 500,000 keys and a smallish array for each value consumes ~ 177 MB of ram.

Attempting to dump it pushes that memory consumption (transiently on Win32) to well over 700 800 MB (and still going and consumption still climbing after 1/2 3/4 hour!).

My real hash has close to a million keys and nested arrays. It consumes over 500 MB to start with. Trying to dump it blows 2GB of virtual memory before it crashes Perl--and the time taken even before swapping starts is measured in the half-lifes of Plutonium. I'd like to avoid both. I just need to be able to dump the structure to a file. Preferably in a reasonably compact format.

Except that there are thousands of arrays at varying depths of nesting.

I can write one myself, perhaps based around Data::Rmap or similar, but I thought I look and see if there is an existing one available. My search didn't turn up anything promising, but it seems a reasonably simple enough requirement that someone might know or have one already written?