If this is your first visit, be sure to
check out the FAQ by clicking the
link above. You may have to register
before you can post: click the register link above to proceed. To start viewing messages,
select the forum that you want to visit from the selection below.

Slow streaming throughput for a very large filesystem

Hi,

I'm trying to use Amanda to backup 1.4 Terabytes of data with an HP 1/8 Ultrium 960 Autoloader over the network. The network maximum throughput is 100Mb/s. It is measured by copying a 20GB tar file from the host client to the backup server. The host client has a SATA raid 5 array holding the 1.4TB data and a single SATA as its system drive. The backup server has a holding disk of 230GB on a SATA drive. I don't use any software nor hardware compression.

I want to backup the data of 1.4TB in 48 hours which is not unreasonable. However, so far it takes more than 4 days to backup 1.2TB of data. The log statistics is at the bottom of this message. It backups at around 3MB/s.

I do a backup test of 86GB of data. It can backup at round 8MB/s.

I'm using SUSE Linux 9.3 for the host client and the backup server.

Could you please help me explaining the cause of this problem? What can I do to solve this problem? Am I doomed?

I understand that backing up more than 1.4TB is a challenge, but it should be possible to do it in less than 60 hours. Please see my amanda statistics log below.

-- Log for 1.2TB backup.

These dumps were to tapes WeeklySet-4-01, WeeklySet-4-02, WeeklySet-4-03, WeeklySet-4-04.
The next 4 tapes Amanda expects to use are: WeeklySet-1-02, WeeklySet-1-03, WeeklySet-2-01, WeeklySet-2-02.
The next new tape already labelled is: WeeklySet-2-03.

NOTES:
planner: Adding new disk host_client:/data1/tiff.
driver: WARNING: This is not the first amdump run today. Enable the usetimestamps option in the configuration file if you want to run amdump more than once per calendar day.
taper: no split_diskbuffer specified: using fallback split size of 10240kb to buffer host_client:/data1/tiff.0 in-memory
taper: tape WeeklySet-4-01 kb 368373792 fm 35975 writing file: No space left on device
taper: continuing host_client:/data1/tiff.0 on new tape from 368373760kb mark: [writing file: No space left on device]
taper: tape WeeklySet-4-02 kb 368373792 fm 35975 writing file: No space left on device
taper: continuing host_client:/data1/tiff.0 on new tape from 736747520kb mark: [writing file: No space left on device]
taper: tape WeeklySet-4-03 kb 368373792 fm 35975 writing file: No space left on device
taper: continuing host_client:/data1/tiff.0 on new tape from 1105121280kb mark: [writing file: No space left on device]
taper: tape WeeklySet-4-04 kb 50347488 fm 4917 [OK]

These dumps were to tape WeeklySet-3-04.
The next 4 tapes Amanda expects to use are: WeeklySet-4-01, WeeklySet-4-02, WeeklySet-4-03, WeeklySet-4-04.
The next 2 new tapes already labelled are: WeeklySet-2-04, WeeklySet-2-03.

NOTES:
planner: Adding new disk host_client:/data1/dat.
driver: WARNING: This is not the first amdump run today. Enable the usetimestamps option in the configuration file if you want to run amdump more than once per calendar day.
taper: tape WeeklySet-3-04 kb 89379296 fm 86 [OK]