It's a real challenge for a punk rocker to generate anything resembling actual shock nowadays. Thirty-five years after the Sex Pistols sang God Save the Queen, what seemed transgressive now looks practically quaint. It's been almost as long since sex-change surgery could work up much interest either. Even the tabloids have wearied of "gender bender" headlines about butch army officers turning into foxy chicks.

Last week Against Me! were playing in Oslo, and when I arrive I by chance run into the rest of the band in the hotel lobby. Heavy set, swaddled in black, a little bit socially awkward, they talk in amiable grunts punctuated by the word "awesome", a textbook rockers-on-tour ensemble. After some slightly stilted small talk, I get into the lift, step out on to a roof terrace, and come face to face with the most sweet-natured, articulate, nakedly vulnerable punk you are ever likely to meet.

Grace doesn't look like a woman, but then she only began taking hormones a month ago. There's a subtle feminity in her posture, though, and in the way her features soften as she talks. The twin impressions of apprehension and an eagerness to please are for a moment disconcerting – until you see just how extraordinary it must be to start telling a total stranger about the very thing she's been keeping secret for most of her life.

Grace's first intimation of gender dysphoria came when she was just five years old. "The first time I had that moment, when I knew, was seeing Madonna on a televised concert. And I thought: 'Why not me?'" The son of a military man, she grew up on army bases feeling lonely and confused, bullied at school and bewildered by an indelible sense of toxic otherness.

"It's a feeling of great existential dread. Peripherally, being aware of the way you are in your body and feeling cognisant of the fact that I'm male – like looking down and seeing male features – and feeling, internally, 'but I'm not'. You feel a detachment, and you feel hyper-aware of everything that's around you. Then, at the same time, you feel extreme feelings of shame and guilt and confusion, so that all works into a nice little cocktail in your head."

Punk rock and recreational drugs got Grace through adolescence, and at 18 she founded Against Me!, throwing herself into an ultramasculine frenzy of mosh pits, tour buses and lyrical rage. "When you're in a punk band you're framed as an angry white male. That's you: angry; white; male. And you're screaming at the top of your lungs, and you're angry about this and that." Which, in one way, suited her. "But you become a parody in some ways, and the more and more I saw that, the more and more frustrated I felt."

When the internet arrived, gender dysphoria websites confirmed who and what she really was. "If you're a transsexual, it's an incredible resource." Secretly, she would sneak off to hotel rooms to dress up as a woman – but the guilt and shame that always followed were overwhelming. "Though I guess, in a way, I'm thankful for the secrecy now, because it helped to develop an imagination. Creatively, you live in your own world, so it focused me in that way, and I'm thankful to be the person that I am from that."

Grace resolved to quit cross-dressing for good when her band was signed by Warner Bros. "So you pile everything up into a trash bag, and go behind a store and chuck everything into a dumpster. You just get rid of everything, and you're like: 'That's it.' Because you feel like you're a deviant, and there is this danger of being caught, and you're terrified." She believed it was simply a question of willpower. "You believe you have a choice in the matter. But it becomes apparent to you, after a while, that you really don't have a choice in the matter." And so she would relapse. "And that's the feeling of guilt that comes with all this."

Reading this on mobile? Click here to view.When Grace married, she kept her secret to herself. But three years ago, her wife Heather became pregnant, Warners dropped the band, and the compulsion became uncontrollable. She began to go a little bit crazy. "You're thinking, 'I'm going to be a father, and I'm going to have a daughter,' and you can't help but examine that as you get older – just having less patience with all the bullshit. Just thinking about the role model that you're going to be for your child, and what's the most important lesson you can teach them? Wanting to be honest was an example that I wanted to set."

Early this year, Grace arrived at what sounds less like a decision than a surrender. "I kind of reached a point where there was nothing else that I could really write about and focus on other than transsexual topic matter – whether those are stories of fiction or my own emotions writing it – I couldn't really write about anything else." So she sat down with her wife, and told her the truth.

Did it mean, Heather asked, Grace would leave her? Once reassured, Heather's support was so unconditional – so miraculous, so transformative – that three days later Grace found herself blurting out her secret to the band. "And straight away, I was completely terrified. What the fuck did I just do? Why did I do that? But I think I was just on a roll. I called my wife and said: 'I just totally told everyone, and I don't know why.'"

The band were stunned. The news was quite literally awesome. "But they were completely cool with it."

The problem Grace faced was what to do next. To leap from a lifetime of secrecy into the pages of Rolling Stone might look a little drastic, but to Grace the strategy made a lot of sense.

"There's a certain amount of normalising that happens when you do it in a fashion like that. I felt like the first time we played on a late-night TV show in the States – for my parents that was a moment when they were like: 'Oh, that's what you do.' It legitimised what you did. So when you have something like Rolling Stone that you can hand to someone and be like, 'If you have a question after you read this, feel free to ask me,' was a great thing to have. As opposed to having a million and a half conversations."

Grace knew she had to tell her parents, who divorced during her childhood, before the whole world found out. "That was the scariest of all. They're your parents; you're terrified of the way that they're going to react – that they're going to reject you. You live with that feeling of terror, that you're going to be rejected by your parents, for the way that you are and you can't control the way you are – you live with that feeling for your whole entire life. And now you're about to test it."

Her mother took the news well. Her father was less sanguine. Grace wrote him a long, emotional email – and received a three-line "cryptic and manipulative" reply. They haven't spoken since. Grace's 26-year-old younger brother was the easiest to tell. "It was really casual. I don't know if he necessarily even got it, really. I was like, 'So I'm a transsexual and this is what's happening', and he was like, 'OK, whatever you say, I love you; you're my brother.'" Grace starts to laugh. "It's like: 'Well, but actually I'm telling you I'm your sister, so you're going to have to flip that around; you just got to rewire the way you're thinking about me.'" She laughs again. "But at the same time, I appreciate his earnestness."

Reading this on mobile? Click here to view.She was much more worried about the reaction of her conservative, God-fearing Florida neighbourhood, but says that so far everyone's been lovely. Against Me!'s fans were an even bigger worry. "They were pretty high on the pecking order, just because there's the very real worry of: 'What if someone attacks me? What if I'm on stage and someone tries to do something to publicly humiliate me?' There's that total fear, because I've been hit in the face with bottles in the past. So there's that fear, for sure." But so far they've all been lovely too.

Grace thinks she first fell in love with punk because "the aggression is a great outlet when you have feelings of frustration", so I wonder whether gender reassignment may diminish its allure. "I don't know," she admits. "I'd love to get to that point, though, to tell you." For the first time in her life she is performing stone cold sober, so clarity is quite a novelty in itself. "Coming out has a slightly terrifying edge to it, so being focused and feeling like you're a little bit grounded is a lot better than feeling like you're out of control." She can't yet say exactly how singing as a woman feels different, but it definitely feels like a relief. "Of course, you're trying to unlearn all the mannerisms that you've picked up from posing as a man. But I'd reached a point where I felt like every time I spoke into the microphone, my voice was coming out like this dude voice – really husky – and just completely uncomfortable. So this just feels like you're more in the moment."

Only one worry threatens Grace's newfound composure, and when she talks about it her voice begins to crack. "Well, I still worry about the way that people will treat my daughter, and the way that this will affect her. I know that kids can be cruel to each other. I want to be an involved parent in my daughter's life and do the things that other parents do, like go to the PTA meetings. When I go to her ballet recitals me and my wife go, and there's already a real detachment from the people there, just when you're that 'rocker family', standing there all in black with tattoos. And when you throw 'I'm a transsexual' on top of it, I worry what they'll say about me, what their kids will pick up on and what they'll then repeat to my daughter."

Grace and Heather haven't yet begun to explain anything explicitly to their two-year-old. "But I've already noticed that – even just on that last tour we were on – she, on her own, started to pick up on the female pronouns and started to refer to me as 'she' or 'her'. Then, also, I noticed she has started to morph 'Daddy' into like a 'Dadda', 'Daddo', like she's searching in the way she's saying it for some kind of twist that's going to fit her."

The person I find most fascinating, of course, is Heather. She is not bisexual, and joked to Rolling Stone, "I have no vagina experience", but intends to remain married to Grace, who will in due course become a physical woman. Heather had panicked that Grace would leave her – but surely Grace must have been more worried about Heather leaving her?

"Of course, yeah. It's something I still worry about. I'm very early on in my transition – I still blur a line – but I know the changes are going to be coming, and I fear how she's going to interpret them and whether that will affect if she's physically attracted to me is terrifying. But ideally you marry someone because you hope that they're your soulmate, and that's something that's beyond gender."

Reading this on mobile? Click here to view.I want to ask Grace how her marriage can be gender-blind, when gender dysphoria has been the defining affliction of her life. If a soul should ideally transcend gender, why does she need to become a woman? Afterwards I wonder why I didn't. It's partly, I think, because I liked her so much – and partly because of the unspoken code that seems to prohibit close questioning. It's odd that a transsexual punk can still command headlines, when Grace appears to inhabit a world more relaxed about the matter than almost any I can think of. The approved emotional response to her story is nonchalance – and if it's what she wants and needs, I'm happy to oblige. But it feels like a curiously inadequate response to an anguish so acute that only radical surgery can relieve her agony.

Grace's grateful trust in other people's support is both charming and disarming, and she is quite baffled by British cynicism. Has anyone suggested that this whole thing might be a publicity stunt for the band? She looks astonished. "No!" Has she ever wondered whether all the generous declarations of support might conceal unspoken reservations? "No," she says, "I guess at this point I'm just blissfully naive."

And yet she has a formidable strength. "Saying 'I'm a transexual' completely puts the power of the conversation in your corner," she explains. "Because after that, what's the worst anyone can do to you?"

I'm curious about her first experiences of living publicly as a woman, so I ask if she now finds herself scrutinising other women's beauty, longing for looks that measure up to impossible ideals?

"That's not 'now'," she laughs. "That's always been there. That's always been part of when I'm out and about, or I'm looking at magazines. That's what I'm thinking. As opposed to most men, who are thinking: 'Oh I want to fuck her.' I'd be thinking: 'Oh I wish I had her shape' or 'She has beautiful hair.'"

In a perfect world, who would she like to look like? Grace laughs. "I'm big on hair. I love Julianne Moore's hair. That's all I'd like: Julianne Moore hair."