You can create a volume (e.g. with LVM) and mount it there. Then the folder is a separate filesystem and cannot grow beyond the filesystem size.

quota

A less forceful approach (i.e. the users could circumvent it): You could create a group which is used for this folder only (which cannot be easily enforced). You could use the SGID bit on the folder to make all new files (just the created ones, not the moved ones, though) and define quota for this group.