Ok I've fixed it, I'll post up what I did in case it saves somebody else the headache I've had trying to get this to work

I think there was two things I did wrong, first I think I was supposed to set 'unitName' on the persistence context, not 'name':

Secondly I packaged the application wrong, I had an ear file, with a WAR, an EJB JAR, a plain JAR and a persistence unit JAR, all in the root. I changed it so that only the WAR and EJB JAR were in the root, and the plain JAR and persistence unit JAR were in the lib subdirectory of the EAR.