Hersh has written a memoir, titled Rat Girl, which was published in the USA by Penguin Books on August 31, 2010 and is the co-founder of a non-profit, open-source software project for recording artists, CASH Music.

Born in Atlanta as Martha Kristin Hersh, she was raised in Newport, Rhode Island. She learned guitar at age nine from her father, and started writing songs soon after. As a teenager, she formed Throwing Muses in the early 1980s with stepsister Tanya Donelly and other high school friends.

Hersh began singing and writing most of Throwing Muses' songs in changing tempos, with Donelly also singing and writing some of the songs. Early in the band's career, Hersh became friends with Betty Hutton while both were students at Salve Regina University. Hutton attended several Throwing Muses concerts.[1]

The group was signed by the British 4AD Records label in 1986 and, after one album, signed a U.S. deal with Sire/Reprise Records in 1987. They began touring around the U.S. and Europe while recording critically acclaimed rock albums, with Hersh writing most of the songs.

Throwing Muses became a trio when Donelly left the group after 1991's The Real Ramona. In 1994, Hersh began an additional career on Sire/Reprise and 4AD as an acoustic solo performer, beginning with Hips and Makers, an album sparely arranged around her vocals, guitar, and a cellist, in contrast to the volatile, electric sound of her band work. Michael Stipe of R.E.M. made an appearance on this first solo album.

Hersh's solo songwriting style focuses some of the relationship subject matter on her family. While Hersh's work reflects her personal experience, she has said that she writes from a point of view outside of her personality. Stating that "songwriting is about shutting up instead of talking", Hersh has said that songs that appeal to her are those that "say things that I don't know yet and tell stories I may not have lived yet".

The New York Times pointed to Hersh's explorations of "rage, aggression and mental chaos" as evidence that there were at least a few female rock music artists by the early 1990s pushing against gender role boundaries to express "more than simply vulnerability or defiance" in their work.

Hersh, whose early publicity at times portrayed her as a tortured artist "channeling" her songs from her psyche, has mentioned that the "angry young woman" fascination of some writers in reviewing the work of female performers has at times led to cartoonish stereotypes, rather than three-dimensional portraits respecting their intelligence. By the mid-1990s, journalists acknowledged that the breadth of her "fierce, quirky, and imaginative" lyrical style included explorations of "emotional and physical love" combined with "elliptical puzzlement".

After receiving some airplay and major media coverage for Throwing Muses album University in 1995, Hersh moved to Rykodisc for her 1996 Throwing Muses album, Limbo, and her 1998 solo album, Strange Angels. In order to better control her career and the distribution of her recorded material she created the ThrowingMusic label with husband/manager Billy O'Connell in 1996. This enabled her to co-release certain Hersh-related projects, including an ongoing download subscription service called Works in Progress (WIP) for releases available only through the label's ThrowingMusic website.

In 1999, Hersh also participated in Throwing Muses drummer David Narcizo's Lakuna solo project album release, Castle of Crime.

In 2001, she released the Sunny Border Blue solo album, on which she again played nearly all instruments. She has described this album as having even more intensity than her previous works, as she continued her pursuit of songwriting as being in part a way to transform "ugly feelings" into art.

In 2003, she released The Grotto, an acoustic solo album of song sketches with personal lyrics set in Providence, Rhode Island, with Andrew Bird on violin and Howe Gelb on piano. On the same date a self-titled album by her Throwing Muses group was also released, the first since Limbo. Both were recorded at Steve Rizzo's studio in Rhode Island.

Also in 2003, she formed a power rock trio called 50 Foot Wave, when Narcizo was unable to tour on a full-time basis due to other commitments. Her touring appearances and recording efforts in 2004 and 2005 centered around both 50 Foot Wave and her solo career.

On November 26, 2007, Hersh announced the opening of CASH Music.[3] The subscriber-based, direct-to-consumer model had its first year-long project in the form of what was supposed to be an album called Speedbath, which was released one song per month for free at Kristin's CASH website. 50 Foot Wave also released an EP titled Power+Light through the CASH organization. January 2009 began another series of one track per month for free on the website, and the tracks wound up cohering without the song "Speedbath" at all; the new record, Crooked, was released in 2010. It was available to buy as a hardback book which included essays on the songs and a link to download the album and related tracks.

Hersh has written and illustrated a children's book called Toby Snax.[4]

A second collection of Appalachian folk songs, The Shady Circle, is expected to be released. Live recordings of the songs have been available since late 2008.

In 2010, Hersh released her memoir, Rat Girl, which covers events from early 1985 through early 1986, when she was in her late teens. The UK version of the book, released in early 2011, is entitled Paradoxical Undressing. The book covers topics such as the meaning behind many of her songs and the early stages of Throwing Muses.

Hersh's music is known for its chords, sonic treatments, and a vocal style ranging from softly melodic singing to impassioned screaming. Some of her signature contributions to popular music include addressing the complexities of life through impressionistic, sometimes hallucinatory lyrics about everyday feelings and varying mental states. A few of her songwriting subjects have included childbirth ("Hysterical Bending"), love ("Tar Kissers", "Lavender"), surreal vignettes ("Delicate Cutters", "Fish"), death ("Limbo"), emotional anguish ("The Letter"), loss of custody of her first son ("Candyland"), and the shedding of a relationship's anxiety ("Snake Oil").

Hersh has used images such as apples, water, diamonds, eyes, the sea, snow, ice, rain, fire, the sun, parking lots, sand, and cowboys. On occasion she has used historical figures like anorexicsuicideEllen West as metaphors in depicting a state of mind. Eccentric characters encountered in her family's travels have made occasional appearances in songs such as "Ruthie's Knocking"; a 2005 live solo set list included a then-untitled song ("Under The Gun") about a "parrot lady" character she met while visiting Lake Michigan.

Some interviews have described Hersh's early drive to perform as due to hearing sounds in her mind so that her songs began to "write themselves", becoming at times their own separate presences in her life, inner voices haunting her. She has stated that hearing these "pieces of songs" clanging together in her mind compelled her to take the pieces apart and craft songs from them. "If I don't turn ideas into songs, they can get stuck in me and make me sick," she said in a 1995 interview with AOL's Critics' Choice electronic music magazine. "That's the way a song hits you right here, right here [she motions to the heart and gut] instead of in your brain because the words themselves are all real sweaty, color, action words, so they just go bangbangbang. They're not supposed to make you think and try to figure out some puzzle. People think that I'm trying to trick them, that I have some thing I could write down and I haven't done it and I've just given them a bunch of poetry instead. I find it to be the clearest way to talk. It's like the way little kids talk because they have no filler words and no overriding thoughts to color your impression of what's happening in a song."

Her parents' Lookout Mountain heritage influenced her to record a solo album of Appalachiangothicfolk songs in 1998 – Murder, Misery and Then Goodnight. Performing traditional songs was a rare covers excursion for the prolific songwriter, although she was no stranger to these tunes, having heard some of them played by her father when she was a child. In fact, Hersh has cowritten two songs with her father, "Uncle June and Aunt Kiyoti" and "Houdini Blues", which appeared on other solo releases; she also recorded a third, "Sinkhole", that he wrote on his own.

Hersh and her family have moved every few years to live in a different locale. Her experiences in each location have sometimes influenced the emotional landscape of her songs. Living for a period near Joshua Tree, California, affected some of the atmosphere and lyrical imagery of Sky Motel, a 1999 solo album on which she played most of the instruments. Time spent in the New Orleans area while recording Limbo in 1996 at Daniel Lanois' Kingsway Studio had similarly influenced songs like "Ruthie's Knocking", inspired by Ruthie the Duck Girl, an offbeat character well known to locals for her antics in the French Quarter.

Hersh has also said in an interview that she writes many of her songs which feel to her as though "they come from New Orleans or Providence, Rhode Island". She and her family additionally spent time in New Orleans when recording Throwing Muses' 1994 Bright Yellow Gun EPs, Throwing Muses' 1995 University album, and the Sky Motel solo album. University, Limbo, and Sky Motel were recorded by Grammy-winning engineer Trina Shoemaker, the latter album co-recorded by engineer Ethan Allen; Allen later worked on band tracks with Hersh in the 2000s. Hersh spent part of the 1990s in Portsmouth, Rhode Island, recording several solo albums and a few band tracks at Stable Sound studio with engineer Steve Rizzo.