However any time I try to access any of the ZFS filesystems (all in the pool "data") the system just hangs.

So a
$ ls /corporate
or
$ cd /corporate

will hang the system. Obviously I can't get to my data although it appears to be there.

I can perform operations ok on anything on ad4, the system disk.

I added these lines

vm.kmem_size="1536M"
vm.kmem_size_max="1536M"

to my /boot/loader.conf file as per the instructions at http://wiki.freebsd.org/ZFSTuningGuide
and the system is 64 bit with 4G RAM.
All the server has been doing is receiving nightly rsyncs which it did ok for about a month.

I'm new to freeBSD and have kinda run outta ideas of what to try next.

$ dmesg |grep ZFS
WARNING: ZFS is considered to be an experimental feature in FreeBSD.
ZFS filesystem version 6
ZFS storage pool version 6

Though it complains about root being improperly unmounted:
$ dmesg |grep -i "warning\|error"
WARNING: / was not properly dismounted
WARNING: /tmp was not properly dismounted
WARNING: /usr was not properly dismounted
WARNING: /var was not properly dismounted
WARNING: ZFS is considered to be an experimental feature in FreeBSD.
nfsd send error 32

And if I do
$ ls /
it hangs.

Which I don't understand - if / were unmounted wouldn't the system be inoperable?

don't know about ZFS.
for a "usual" bsd:
..............
boot to single-user
swapon -a
fsck -p (that takes a long time!)
mount -u /
mount -va
shutdown -p now
.................................
way back in 5.x that would result in lost files.
haven't had lost files with that procedure for years.
..................................
on a post-it on the monitor for easy reference

It doesn't. But it does have a built-in "scrub" command that checks every bit on every disk against the checksums and repairs any files that don't match (if the pool is mirrored/raidz'd or has copies set to >1).

What's the output of zpool list?

What's the output of dmesg | grep -i zfs or dmesg | grep -i ad?

Can you run top at the console, and ssh to the system from another system, and do the ls and see what's happening in top?

Do you have background fsck disabled in /etc/rc.conf? If not, disable it, reboot into single-user mode, and run fsck -y on all your partitions on ad4. Background fsck can cause all kinds of issues and system lockups. You want fsck to run in the foreground if the system reboots after a lockup.

dmesg as I said above isn't showing anything that grabs my attention:
$ dmesg |grep -i zfs
WARNING: ZFS is considered to be an experimental feature in FreeBSD.
ZFS filesystem version 6
ZFS storage pool version 6

Top runs ok - although there's x4 rsync processes running (which were probably from backups being sent that failed) along with a pile of sshd instances.

It's a headless server so I'm connecting over ssh - which works fine.
I get the feeling that if I reboot it'll come good - but I wanna figure out what happened cos this is my backup server and i get kinda nervous when all the data _just drops out_

Quote:

Do you have background fsck disabled in /etc/rc.conf?

rc.conf didn't have anything explicitly mentioning fsck so I added
background_fsck="NO"
in case the default was YES.

Ok I rebooted and ran fsck. Fsck threw up a whole bunch of errors on the system disk, which it fixed.

And it all came good.

I think the problem stems from another problem I have where FreeBSD is only successfully booting every third or fourth boot. I was having to kill the power and reboot multiple times to get the system to come up - obviously that's thrown up some filesystem errors.