It is of course important to realise that other, older robots may not support these newer mechanisms.
For example, if you use "Disallow: /*.pdf$", and a robot does not treat '*' and '$' as wildcard and
anchor characters, then your PDF files are not excluded.