The band self-released their debut album Unicorns Are People Too in March 2003 through their own record label Caterpillars of the Community, with a limited press run of 500 copies.

In October of the same year they went back into the studio, recording several new songs as well as reworking a number of songs from their debut. The resulting second album, Who Will Cut Our Hair When We're Gone?, was released in November 2003 on the label Alien8 Recordings, and was a notable departure for a label that has traditionally released albums by Avant-Garde and highly experimental artists such as Acid Mothers Temple...