Regarding the subject matter of the tracks that comprise All We Got Is Each Other, the group had this to say on their Bandcamp:

"[These] songs are all about our friend Sam. She's not with us anymore. They are also about mental illness..."

Considering the personal weight of the statement, it should come as no surprise that Ghost Mice put a particularly heavy emphasis on their lyrics. Their combination of a highly lyrical focus with an energetic folk punk styling strikes me as something that might please fans of last year's Andrew Jackson Jihad album.