Copying a Filesystem between Computers

If you need to transfer an entire filesystem from one machine to another, for example, when you get a new computer, do the following steps.

1) Boot both PCs with any Linux live CD (for example, Knoppix), and make sure they can access each other via the network.

2) On the source machine, mount the partition containing the filesystem to be copied, and start the transfer using netcat and tar:

cd /mnt/sda1
tar -czpsf - . | pv -b | nc -l 3333

3) On the destination machine, mount the partition to receive the filesystem, and start the process:

cd /mnt/sda1
nc 192.168.10.101 3333 | pv -b | tar -xzpsf -

The nc (netcat) command is used for any kind of TCP connections between two hosts. The pv (progress viewer) command is used to display the progress of the transfer. tar is used to archive the files on the source machine and un-archive them on the destination.

Comments

Comment viewing options

I found it necessary to invoke netcat on the source side of the connection with the -p switch to specify the port, that is

tar cvf - . | nc -l -p 3333

instead of

tar cvf - . | nc -l 3333

This may be due to version differences--I'm using Ubuntu 8.10's "netcat-traditional" package, the description for which notes that it is different from the more featureful "netcat-openbsd" package. YMMV.

My recent experience of migrating data was when updating PC due to over zealous cleaning of the old box's cpu fan with a vacuum cleaner...
Hence no original PC to connect to. I put the original hard drives in an enclosure with usb connection and then used rsync to copy the data.
Our household computer has several users. An important thing to remember to do when setting up the users on a new system is to specify the same numeric user and group IDs as on the previous system (write them down!). This ensures the permissions preserved by "rsync -a" make sense.
Finally, any suggestions on life cycle management of user data? The way "home" is used for application and user data, and lack of robust migration tools for data, results in an eclectic mess. I copied the old hard drives' data to "/archive" for now....ouch. Manage a home network over a period of a decade or more and the real issues are way deeper than just copying the files!

I see many drawbacks to the use of tar+nc+pv:
- hard links are not conserved;
- it is difficult to filter out some files (exclude file) or add extra files (include file);
- connection is not encryted, nor compressed (default with rsync, but not necessary);
- rsync can also be used for updating a [sub]partition, without having to copy everything, just the modified files.
- rsync is used without connecting to both computers, either in sink or feed direction.

The only drawback of rsync is the large number of options. The documentation is well done, rather clear, but most options are never used. A typical use of rsync would not take more space than the introduction to tar+nc+pv above, and would be more usefull, in particular to newcomers. Some thing like:

Having just performed one of these mirrorings (personally, I used 'tar ... | ssh', yet another way of accomplishing it ;) ) I stumbled across one more little "gotcha!" -- if you don't generate new host keys on the mirrored machine, you will have two machines sharing one set of access tokens. That's probably not desireable...

Albeit more indepth: I married a geek so now when Jeff gets a new laptop, the missus gets the same one. For the longest time I would get new hardware, drop Linux on it, spend time figuring out all the stuff to install, hardware tweaks, etc, then have to repeat the process on her box. Now, I get the first computer, burn it with whatever distro I want, trick it out so everything is cool, then add her account. Then I boot each machine off of a pair of el-cheapo 256M usb keys, basically do the netcat trick to move it over the gigabit network but I us dd as the in and out on each side. The drives are the same since we always buy two of the same (well, the odds are in our favor). I start the process before we crash for the night and in the morning she reboots into her new system. The one thing you want to remember to do is set the new hostname to something else.

/boot /home or /var other other partitions can be done the same way, and attributes are preserved. Bind mounting will keep rsync from trying to copy /proc /dev or /sys from your original / partition, too. Only / has to be bind mounted before copying.

This method won't preserve any extended attributes of the filesystem in question. For that matter, dump can be used, although not supported by all filesystems. For ext2/3, there is dumpe2fs which works just fine.