Yeah, there are loads of movies from that era which really focused on that seedier side. Besides the ones already mentioned my favourite ones centred on NYC would be Mean Streets; Taxi Driver; Basket Case; Midnight Cowboy; The French Connection and almost anything by Paul Morrissey or Abel Ferrara. A lot of blacksploitation movies from that era also have a lot of that atmosphere.

For a similar sleaziness but from a more European perspective, some Fassbinder and very early Almodovar are great. Also, albeit more recently, Gary Oldman's Nil By Mouth has that atmosphere in spades.