Just for fun, I was able to filter a 1 Terabyte stream through a
Haskell program, using Data.ByteString.Lazy, with a constant 2M
heapsize, in
$ time ( repeat 100 ; cat /home/dons/data/10G | ./lazy > /dev/null )
12752.37s user 4069.35s system 58% cpu 8:03:11.42 total
8 hours.
Just so we can say it is possible :)
-- Don