WLID Authentication page Unauthorized (?)

I'm trying to get the WLID going on an EBE site.
When I try to sign in I receive a 401 on the _layouts/liveauth-handler.aspx page. This happens when attempting form the Internet (extended) Site. I get redirected fine (to LiveID) when accessing from the Intranet site, but I'm of course authenticated there.
Seems like there's some anonymous setting missing somewhere (?)