"Holly came from Miami F-L-A, Hitchhiked her way across the USA, Plucked her eyebrows on the way, Shaved her legs and then he was a she ..."

That's how Lou Reed made Holly Woodlawn, the Warhol Factory superstar and legendary drag queen, famous in his 1972 song Walk on the Wild Side. Here's Woodlawn's expanded version: "I was 15 years old and failing at high school in Miami Beach because I was too busy partying. I was supposed to go to summer school to catch up and really didn't want to, so I joined some of these Cuban queens to go to New York. I hocked some jewellery and we made it all the way to Georgia, where the money ran out and we had to hitchhike the rest of the way.

"Atlanta, Georgia, of all places - you could expect to be tarred and feathered and murdered in those days! But we survived and I remember the first time I saw New York: the Emerald City. I thought the sidewalks were made of diamonds because of the specks of mica in the asphalt. It was 1962. Marilyn had just died. I lived on the streets like everyone does when they run away. I met some girlfriends who took me in and we found a place in Queens. I was really lucky. I met this guy who fell in love with me and asked me to be his girlfriend. I started taking hormones for a sex-change and lived as his wife, working in the days as a clothing model at Saks Fifth Avenue. Oh, the things I did! And for six or seven years they never knew I was a boy. Not a clue!"

I meet Woodlawn at her apartment in West Hollywood, Los Angeles' gay village or ghetto, on a sweltering hot day. In a few weeks she'll be in the UK to promote an exhibition of paintings of herself by the British artist Sadie Lee, showing her in a less glamorous guise than usual. "I said, 'Why don't you paint me as everyday me for a change, instead of all peaches and cream?'" she says.

As we sit on her balcony talking, we're favoured with an ambient soundtrack that, appropriately, seems more redolent of Manhattan than of sleepy California: a fire nearby means that we're constantly interrupted by screaming sirens. "All right, already," howls Woodlawn. "Find the fucking fire and shut up. I swear, West Hollywood is breeding pyromaniacs today."

The Holly Woodlawn of 2007 is a far cry from the sweet-voiced cross-dresser who made her first splash in the film Trash in 1970, fake-masturbating with a Miller beer bottle to considerable acclaim. Back then - during what we must inevitably call her 15 minutes of fame - she was one of the many drag queens and hustlers at the lower end of the Warhol social scene, congregating around his Factory studio and at hangouts like the bar Max's Kansas City. "The mole people," Factory manager Billy Name called them. "The amphetamine people." At the other end were the rich, famous and powerful: Jim Morrison, Yoko Ono, Janis Joplin, author George Plimpton.

The 61-year-old man who answers the door today is out of drag, bent and frail, though indefatigably cheerful, using a Zimmer frame because of various slowly fusing discs in his spine that, he says, are unimaginably painful and incurable. "Oh no, this is IT, honey, downhill all the way from here on!"

He rises at six; by 11 his painkillers have slurred his speech a little and fogged his memory. The outrageous spark is still there and the stories are as funny as ever, but delivered with a weariness and frustration he blames on the pills. I play nursemaid a little, fetching coffee and cigarettes from the nearby market, and getting the phone for him when it rings, treading carefully around his untidy, sparsely furnished apartment, with its bed and carpets covered in fag ash, and its one shrine-like photo of Andy, Candy, Holly and others in its most visible corner.

I ask him about his alter ego, the boy born Haroldo Santiago Franceschi Rodriguez Danhakl. "I don't even know who he was," he says. "When I was younger, I was extremely shy and living in what's now Miami Beach. My father had a nice job. I guess we were middle income. I had good schools. I just was unhappy because I didn't know who I was. I didn't associate with the other kids in school, the suburban-minded ones. Plus I came out very young. I was raised in Puerto Rico for the first few years of my life, where the culture is more Caribbean. Everyone's naked, it's hotter, you come out earlier. I was having sex when I was seven and eight in the bushes with my uncles and cousins - of course, they were only 11 or 12 themselves. I was raised in a house full of women and my uncle was gay. We lived in a little tiny town, so those were my role models. Then Miami Beach. All the Cubans arrived after Castro took over, and that's where I really came out, on 21st Street in Miami Beach."

The same month that Woodlawn hitchhiked north, July 1962, Warhol had his first major art show not 10 blocks from where we're talking, at Irving Blum's Ferus Gallery. Although the Campbell's Soup paintings didn't sell well at the time, Warhol had arrived. The night the show closed, Marilyn Monroe died three miles away in Brentwood, causing Warhol to work a poster image of her from the 1953 movie Niagara into his famous Marilyn screenprints - which in time turned him into, in the words of the American critic Wayne Koestenbaum, "our greatest philosopher of stardom".

To some extent, Woodlawn was a product - or an exemplar - of his ideas. Although she didn't become a full-fledged Factory insider until 1969, she was very much on the same scene. She'd decided against the sex-change by this point, though: "Honey, once they cut it off, it's OFF!" And she didn't get to know Lou Reed until after A Walk on the Wild Side came out, but nonetheless saw many early shows by the Velvet Underground and Nico.

"I was just one of the audience," he says. "I used to go to the Exploding Plastic Inevitable at the Dome, with all that colour and insanity, with Gerard Malanga [Warhol's assistant] brandishing a whip and Mary Woronov from Chelsea Girls dancing. So I was very happy when I gradually became a Warhol superstar. I felt like Elizabeth Taylor! Little did I realise that not only would there be no money, but that your star would flicker for two seconds and that was it. But it was worth it, the drugs, the parties, it was fabulous. You live in a hovel, walk up five flights, scraping the rent. And then at night you go to Max's Kansas City where Mick Jagger and Fellini and everyone's there in the back room. And when you walked in that room, you were a STAR!"

Trash, a film improvised and shot in 1969 in the basement apartment of its director, Warhol's manager Paul Morrissey, was the nearest Woodlawn came to broader fame. A kindly, soothing presence on screen, Woodlawn certainly had acting ability: her horny, drug-happy character is the film's highlight. "That beer bottle scene is to my career what eating dogshit was for Divine in Pink Flamingos!" The gay director George Cukor is said to have tried to get Woodlawn nominated for an Academy award, but the issue floundered, perhaps predictably, on whether Woodlawn belonged in the Best Supporting Actor or Actress category.

Since then, Woodlawn has published her autobiography - the toothsome and scandalous A Low Life in High Heels - and made a cult career in drag. Although based in New York until a couple of years after Warhol died in 1987, Woodlawn has lived on the west coast ever since. And she'll be in London shortly. "It's all blossomed into this week of Holly Woodlawn. I'll be busy every day, at parties and shows. Who knows? Hopefully I'll come home with a whole bra-full of money!"

'I was worried that she might be a diva'

Sadie Lee on the kiss that led to her Woodlawn paintingsThe first time I met Holly, I went down on one knee and kissed her hand. I was at a festival in Amsterdam four years ago; she turned up surrounded by about 100 people, with a gold lamé dress and fantastic auburn hair, pouting and waving. During a screening of Trash, I stepped out and saw her sitting alone in the bar. She clocked my smart suit and beckoned me over. I bought her a glass of Chardonnay and knelt down. It was a real Hollywood moment.

She has been photographed all her adult life. Everybody has always gone for her glamorous, artificial drag-queen side. I wanted to paint her as she is now, without any airbrushing. I worried that she might be a diva, and refuse to be pictured like this. But she said she trusted me.

I spent a week in her LA apartment, working intensively. I'd arrive early and put a tape recorder in the middle of the room to record our conversations, just like Warhol used to do. Then we'd break for a drink and a rest. It was just like spending time with an old friend.

Holly is a complex person, so I could only ever hope to capture a small part of her. She's not as strong as she used to be, but she's not a victim, either. Despite her failing health, she's proud and dignified. I hope that comes through in the paintings.

· And Then He Was a She, paintings of Holly Woodlawn by Sadie Lee, is at the Drill Hall, London, from Monday to November 11. Woodlawn and Lee are in conversation at the same venue this Sunday. Details: 020-7307 5060 and www.drillhall.co.uk.