On Tue, Jan 31, 2012 at 12:19 PM, Steve Severance
<sseverance at alphaheavy.com> wrote:
> The webpage data was split out across tens of thousands of files compressed
> binary. I used enumerator to load these files and select the appropriate
> columns. This step was performed in parallel using parMap and worked fine
> once i figured out how to add the appropriate !s.
Even though advertised as parallel programming tools, parMap and other
functions that work in parallel over *sequential* access data
structures (i.e. linked lists.) We want flat, strict, unpacked data
structures to get good performance out of parallel algorithms. DPH,
repa, and even vector show the way.
-- Johan