New rules should be set to a warn level. Developers will expect new changes to NOT break their existing workflow, so unless your change is extremely usefull, default to warn. Expect discussion if you choose to use error.

Look at existing rules and test structures when deciding how to name your rule. no_foo.coffee is used for many tests designed to catch specific errors, whereas foo.coffee is used for tests that are designed to enforce formatting and syntax.