It was one of the largest, most lauded office complexes in Europe when it opened in the 1960s, and for Ernö Goldfinger, Alexander Fleming House was a crowning achievement. For others, the concrete high-rises in south London were a horrible place to work and represented one of the worst examples of soulless postwar development.

The four blocks in Elephant and Castle housed the then Ministry of Health for 25 years, but are now residential flats. They have never been to everyone's taste but on Tuesday the culture minister, Ed Vaizey, said he was "pleased now to be able to give it the recognition it deserves".

He added: "Goldfinger considered this to be his most significant work. Acclaimed when it was first built, it subsequently divided opinion but has now – 25 years after first being put forward for listing – clearly passed the test of time."

The decision followed advice from English Heritage, whose chief executive, Simon Thurley, called it "an excellent example of high-calibre postwar architecture". He said he was pleased that the minister agreed, adding it was "an accomplished building, it has proved highly influential for designs which followed".

The listing covers the four main blocks and linking bridges which were built in two phases, 1959-62 and 1964-67.

An inspector's report said Goldfinger, a Hungarian émigré who moved to London in 1934, considered Alexander Fleming House his major work that "endorsed his postwar interpretation for the city". It won him the Riba bronze award for best building of the year in London in 1964 and led, in January 1963, to the journal Architectural Design devoting its issue to Goldfinger's career.

The ministry moved out of Alexander Fleming House in 1989 – ironically, the place was often cited as having sick building syndrome – and the blocks remained empty until developers took them on, giving them a name considered more attractive for young urbanites wanting to buybuyers.

Today, a one-bedroom flat in Metro Central Heights might set you back £300,000.