Mounting files via loopback doesn't really present an expansion problem. You would simply extend the underlying file and then use resize2fs to grow the filesystem. I think using LVM is a better solution, but the one you proposed does not have the limitations you think it does.
–
larsksJan 7 '11 at 16:12

If you don't rely on group permissions, you can use a different UNIX group for each "quota directory", then set sgid bit on each directory (so created files and directories will belong to the group of the directory instead of the primary group of the creator user), and use group quotas.

I've actually wanted to do this for a while because I didn't want to have to make system users for all of my virtual mail users. ZFS filesystems with quotas would be great, and zfs-fuse makes progress every day, but what I wanted was a very lightweight solution. Finally I decided to write a FUSE filesystem that I could mount as a layer over another filesystem (any base directory will work actually). It has a utility for managing quotas that can be scripted with easily and since quota values are just stored as xattrs on directories or files, one mountpoint can provide support for an arbitrary number of quotas.

I've been testing it on a mail server for a while and recently decided it had reached adolescence and was ready to release into the wild. If you're curious, you can check it out at http://code.google.com/p/fusequota/. I would greatly appreciate any feedback.

New filesystem for each shared folder is IMHO overkill. Just make new group for each shared folder, set owner group of shared folder to this group, set a sticky bit to group (so that every new file and directory has this group as owner )and for permissions on files and folders use acl lists. Then set quotas for these groups.