The problem:
At boot, the devfs.rules rules are not loaded and if I drop into the jail and do a ls in /etc, I can see everything the host machine has. However, if I manually restart the jails, the devfs rules are loaded.

I tried putting the rules in /etc/defaults/devfs.rules too (and removed them from /etc/devfs.rules) but didn't work. The behavior is the same.