I have two large storage arrays from different vendors, each with about 10Gb/s of bandwidth available for read/write. I need to move multiple TB's of files from one to the other. Each speaks NFS and CIFS. Currently using cp or scp or rsync, the intermediate host's GbE interface becomes the bottleneck, since I have to read from one array and write to the other. There's no host on either array that I can log into to cut out the middleman. I have bunches of hosts that could provide additional bandwidth, if only I could do something like:

If the two storage arrays will not talk directly to each other, you're best option would be to build a purpose-built machine with high-bandwidth connections. Even a relatively inexpensive new workstation with a pair of 10GbE NICs can pump out a lot of bandwidth if the supporting network and storage arrays can take it.

You should talk to your storage vendors to determin if the controllers can talk directily to each other. If not build or rent a high bandwidth intermidate node. My guess is that there is a way to make the two arrays play nice and run the clone full bore on the controllers.

If you can't find a way for the controllers to talk to each other (as others have mentioned), you can try doing this:

On your destination server, run the following command:

$ nc -l -p 9999 | tar xvzf -

Then, on your source server, run the following command:

$ tar cvzf - <SOURCEFILES> | nc -q 1 <DST_SERVER> 9999

The advantage to this is it avoids any encryption overhead that SSH/rsync gives, so you'll get a bit of a speed boost. This also compresses and decompresses on the source and destination servers in-stream, so it speeds up the transfer process at the expense of some CPU cycles.