Welcome to the 70s … Tales of the City. Photograph: Kevin Berne Images

One day in 1991, when Jake Shears was 13 years old and so far from being "Jake Shears of the Scissor Sisters" that he was Jason Sellards and as yet unaware he was gay, he was hanging out with a gay couple, who had taken the youngster under their wing. "I think they knew I was gay before I did," he recalls. "So, you know, they would turn me on to cool music." One of them handed him a book, saying: "I think you'll like this."

It was Tales of the City, Armistead Maupin's much-loved saga set in 1970s San Francisco, involving a hugely diverse group of characters who are all (often unknowingly) linked, and many of whom live in a large guesthouse run by the mysterious Mrs Madrigal. The book is full of stories of bath houses and break-ups, all told in Maupin's genial tone. "It was the first thing I'd ever read that had a positive gay perspective," says Shears, "and two years later, I came out. Make of that what you will."

Twenty years later, Shears is repaying his debt to Tales. As he relates this story, backstage at American Conservatory Theater in San Francisco, rehearsals are going on across the hall for the musical version of Tales of the City, for which he and fellow Scissor Sister John Garden have written the music and lyrics. The libretto is by Jeff Whitty, who won a Tony for Avenue Q, and it's directed by Jason Moore, who also worked on the coming-of-age puppet parable. "Yes, your name has to begin with J to work here," the press officer dryly confirms.

The four Js make a charmingly symmetrical double double act, with the bright-eyed and loquacious Shears and the quieter Garden in one corner; and the adorably excited Whitty and the calmer Moore in the other.

"It's been extraordinary to see the books come to life on stage," says Maupin, "but the really moving thing has been to see how well they all get along – they are 28 Barbary Lane." This was the address of Madrigal's house, where the characters meet, fall in love and form lifelong friendships.

In fact, the genesis of the musical could have come from the pages of Tales itself. Whitty came up with the idea five years ago on a flight to London. He called Moore who instantly said yes. "Jeff was passionate about it, and that's all I needed." Whitty then made a mixtape of songs that sounded like the kind of music he and Moore wanted; the only contemporary piece was by the Scissor Sisters, so he called up Shears. How did he get his number so quickly? "Oh, we met about 11 years ago when we were both go-go dancers," Shears recalls airily. "He enjoyed pulling his clothes off and dancing on bar tops. The first time we met in New York, I was probably off my face and we both had half our clothes off."

"That," Whitty says, "pretty much describes it. And we've turned it into art! No regrets ever!"

The other person Whitty had to convince was his literary idol, Maupin. "I read Tales when I was 21, when I first moved to New York," he smiles. "I was so lonely. Those characters were my company." So, nervously, he flew to San Francisco, where the writer lives. But instead of the "big box of crazy" he was worried about finding, "Armistead was so welcoming. We just got stoned within the first five minutes and that was it."

Maupin puts it somewhat more euphemistically: "Oh, we instantly clicked and spent about five hours gabbing." Maupin took to Shears right away, "although it took a while to get past the eyes". Had he been a Scissors fan? "Well, I'd heard Filthy/Gorgeous before, which I thought was grabby and fun."

On the day we meet, Shears happens to be wearing a loose vest top featuring a design by Tom of Finland, inventor of the macho gay image; he looks more like a Tales characters than the currently dressed-down actors do. "It's always been my ambition to make a musical," he says. "When Jeff said Tales, my heart started racing, and I thought, 'I don't know what I'm doing but let's do it.'" He then grabbed his bandmate Garden and said: "Get your keyboard – we're writing a musical!' They sat down and, in one day, wrote a song that's still in the show.

Later, we go to the rehearsal room to watch the opening of the second act. Here's Mary Ann fighting with creepy Norman; there's Mrs Madrigal and Edgar enjoying a romantic moment. The songs are excellent: melodic, emotional and catchy; the only one that sounded recognisably Scissor Sisters-esque is the one from the disco scene. The four Js watch carefully, and Shears even puts down his iPhone for Mrs Madrigal and Edgar, visibly moved.

Even though it's their first musical, as they chat over a snack, Garden politely eating a salad, Shears chomping on a chicken burrito as big as his thigh, the two men come across like a modern-day Rodgers and Hammerstein, talking about "making use of the 'real estate' of a song to tell the story" and "sacrificing top-drawer ballads for the rhythm of the show".

Shears is, characteristically, devoid of self-doubt. No nerves about taking on a book with which he has such an emotional connection? Never. Is he worried how San Franciscans will react, since it is the unofficial book of the city? Of course not. "But then, maybe I'm too confident," he muses, making Garden splutter with laughter.

One need only look to the SpiderMan musical in New York to know that it's not enough just to take a popular story and a successful pop group and assume it will all work out. And this project presented very specific difficulties: there are as many plotlines as there are characters – and there are a lot of characters. What's more, Whitty didn't make things easier by mashing together Tales of the City and its sequel More Tales of the City. "There were some things I just couldn't leave out," he says.

'The show queens out MAJORLY'

Maupin originally wrote the books as newspaper articles, so the books have an episodic feel, rather than the flowing rhythm that musicals need; and, even though it's set in the 70s and is full of gay characters, they didn't want to make a camp pastiche. "That said," Whitty adds, "at times, the show does queen out MAJORLY."

And how much do sex and drugs feature in the musical? "About 38%," says Garden.

If the show goes well, the plan is to bring it, not to New York, but to London. At the opening last week, according to the San Francisco Examiner, the "audience went wild. After all, they were privy to plenty of inside jokes as well as captivated by the broad and witty characterisations." It added: "This love letter to countercultural San Francisco is neither deeply emotionally engaging, nor nuanced – it's just out-and-out good, rousing fun, with some poignant moments, such as Mouse's affecting coming-out song, Dear Mama."

Whitty and Moore remember how audiences in London took to Avenue Q, and there's a big Scissor Sisters fanbase there. Maupin, too, feels "the books were really discovered in England", thanks to Patrick Janson-Smith, the British editor who was a champion of Tales. Plus, Whitty points out, "It took Channel 4 to make a TV show out of Tales. American networks wouldn't take a chance on it. London audiences are much less shockable than American ones, they're less prudey-sue. We had to make the sex scenes in Avenue Q filthier for the London audiences. That could be fun with Tales."