In Chrome 18, this unpacked extension fails to load, displaying an error:

If I change 'unsafe-inline' to 'self', the extension loads fine, but alert() does not work, and the option page's console contains an error:

Refused to execute inline script because of Content-Security-Policy.

In Chrome 16, using 'unsafe-inline' lets the extension load fine and alert() works, too. However, in Chrome 16, replacing 'unsafe-inline' with 'foo' lets the extension load, but of course does not let alert() work, so perhaps Chrome 18 is stricter than 16, but...

Is default-src 'unsafe-inline' actually invalid, or is this a bug? What CSP value can I use to make alert() work in Chrome 18?

Based on the accepted answer below, inline scripts no longer work in extensions in Chrome 18. alert() will need to be placed in its own JavaScript file.

There really is no other way than moving all your javascript into js files and point to them with a <script src>.

There's, though, the option to do Eval and new Function inside a sandboxed iframe, for instance with the following lines in the manifest:

"sandbox": {
"pages": [
"page1.html",
"directory/page2.html"
]
},

A sandboxed page will not have access to extension or app APIs, or direct access to non-sandboxed pages (it may communicate with them via postMessage()). You can further restrict the sandbox rights with a specific CSP

Thanks to Google, there's a lot of extension rewriting in the lineup :(

EDIT

It is possible relax the security policy for REMOTE scripts. But not for inlines.

The policy against eval() and its relatives like setTimeout(String), setInterval(String), and new Function(String) can be relaxed by adding 'unsafe-eval' to your policy:
"content_security_policy": "script-src 'self' 'unsafe-eval'; object-src 'self'"

However, we strongly recommend against doing this. These functions are notorious XSS attack vectors.