Note – I technically don't need archive_mode, as I'm not going to use wal archiving, but it makes pg_basebackup warn. In case you wonder – why it warns – well, you should have archive_mode or some non-zero wal_keep_segments, but using archiving is safer, and it's possible to setup compressed archive. On the other hand – this is just test setup, so I don't care about archive itself, just trying to suppress the warning.

Then, I created new role in master postgres, so that I can run basebackup and replication on it, and not on superuser:

(if you want to say that it's ugly – don't waste your breath/keyboard – I know. But it's just test situation. And I checked that it's filehandle 3.)

So. Results for just sending the backup without any compression, in plain format:

sendto : 0
recvfrom : 48201

Which means that virtually no data was transferred back to master (which is just a sanity check), and total size of data transferred from master to slave is slighly more than size of datadir. But not much more – just 805kB. Which is 1.7% of PGDATA size.

Afterwards, in test-copy, I found base.tar.gz file, which was 5560 kB in size. Strace showed, that this time size of transmission on the net was … 48201 kB.

Which makes some sense. Some in terms – compression is clearly done on the receiving end, and master is not being overloaded with this task. On the other hand – if your $PGDATA is large, you might still to prefer to make your base backups using normal means (tar/omnipitr-backup-master/rsync), to preserve network bandwidth.

All in all – it looks like a definitely cool tool, and I thank authors for it.

3 comments

Just a FYI – compression is something we discussed, and something we eventually want to add (as an option, of course), but given how little time there is left before 9.1 freezes, it was pushed back. But it’s definitely on the radar for the future!

@Magnus:
that’s great. One question though – when done, will it be done by calls to zlib library, or external program? reason for question – i usually have lots of cpu power left, while io and network are much more problematic. and with ability to define my own compressor program, I would be able (for example) to use pigz (http://www.zlib.net/pigz/)