These days, the Lower East Side crack dealers are more subtle and most of the prostitutes have been cleared from Times Square – but the scent of cigarette smoke hasn’t yet cleared and it is still impossible to go an entire day without hearing a car horn.

I think one of the reasons I live in New York is cause I know my way around New York. I don’t know my way around Paris. I don’t know my way around Denver. I don’t know my way around Maui. I don’t know my way around Toronto … it’s almost by default. I don’t know very many people in New York who don’t also say ‘I’m leaving.’ I’ve been thinking of leaving for 35 years now, I’m almost ready.

John Cale’s apartment: 56 Ludlow Street

When Lou Reed met John Cale, a fellow founding member of The Velvet Underground, Cale was living in this apartment on Ludlow Street in New York’s Lower East Side. Victor Bockris, who wrote a biography of Reed, described the space as a “bohemian slum dwelling”.

This dwelling is where Reed, Cale and Sterling Morrison rehearsed and recorded six songs in 1965, creating the foundation of their first album The Velvet Underground and Nico.

When we rehearsed, we weren’t too loud. We only had acoustic instruments and the most basic amps. We stole electricity from other apartments, so wires snaked all over the place. Besides, the people downstairs were always blasting music on the radio, so no one in the building heard us or complained.

The Factory: 231 East 47th Street

This was the site of Andy Warhol’s Factory from 1963 to 1968. Reed’s Walk on the Wild Side is about the time he spent hobnobbing with New York hipsters in The Factory:

Holly came from Miami, FloridaHitch-hiked her way across the USAPlucked her eyebrows on the wayShaved her legs and then he was a she

She says, “Hey, babeTake a walk on the wild side”She said, “Hey, honeyTake a walk on the wild side”

That space is now a parking lot in the well-groomed Midtown district of Manhattan which is almost always vacant by 10pm, any night of the week.

A large group of people were gathered outside the area on Monday, enveloped in the meaty scent of what seems to be the neighborhood’s most popular food cart.

Andy Warhol’s first Factory location was razed in the late 1960s. Now, it is a parking lot. Photograph: Amanda Holpuch for the Guardian

Dom: 19-25 St Marks Place

The East Village bar and club Dom played host to the early stages of Andy Warhol’s mixed media show with The Velvet Underground and Nico – Exploding Plastic Inevitable. This cavalcade of rock music, avant-garde film and dancing started at this nightclub in April 1966 and eventually toured throughout the US.

A pedestrian walks by the former site of the rock club The Dom, where the Velvet Underground was the house band. Photograph: Spencer Platt/Getty Images

The neighborhood gentrified considerably since I’m Waiting for the Man appeared on The Velvet Underground’s 1967 debut – but this corner maintains the trademark New York grit evinced in Reed’s music. On any given day, a person may have to step over a flattened rat carcass to cross 125th Street and get a $1 slice of pizza or hotdog across the street.

It’s serving as the city’s defacto memorial for Reed – with flowers and lit candles at the doorway, below the plaques honoring the hotel’s famous guests, including Leonard Cohen and Dylan Thomas.

Janis Joplin, Patti Smith and Iggy Pop have also spent time in the 250-unit hotel, which is now a registered New York City landmark. Some of their tales were published by Vanity Fair this month.

Chelsea Hotel was purchased by a real estate developer in May 2011 and stopped taking reservations in August of that year. It’s been mired in lawsuits, management restructuring and renovations ever since.