When Labour launched its manifesto last week, it chose a brand new building as a backdrop. This was the Queen Elizabeth hospital, Birmingham, where the first phases of a new £545m super-hospital will open in June. The forecourt where the cabinet gathered to brandish their paperless manifesto memory sticks looked somewhat bleak, but never mind. We were invited to admire the scale of the investment behind the V-formation of grinning ministers.

Labour has been an enthusiastic builder. It has embarked on a huge hospital building programme and has promised "to rebuild or renew nearly every secondary school". It has celebrated construction which flourished in the prolonged boom. In his first conference speech as prime minister, Brown promised 240,000 new homes a year, a target that has shrivelled in the merciless drought of recession.

It also, in its early days, proclaimed the importance of architecture and design, to an extent never before heard from a British government.

In 1997, Sir Richard Rogers, soon to be ennobled, had the ear of Tony Blair, and his long, lonely campaign to better and beautify British cities was suddenly favoured. An Urban Task Force, chaired by Rogers, was set up. The government swept away the Royal Fine Arts Commission, a clubby, oak-panelled, leather-upholstered institution led by the fragrant Tory ex-minister Lord St John of Fawsley. In its place, it made the Commission for Architecture and the Built Environment, or Cabe, an organisation with expanded powers for influencing the quality of British architecture.

Ample construction plus good design should equal a transformation of the country, comparable to the Victorian wave of town halls, libraries and museums, and some of it has happened.

There are indeed parts of British cities better and brighter than they were before. There are some lovely school buildings such as Kingsdale school in south London and examples of decent housing such as Adelaide Wharf in east London.

A generation of young architects has grown up and been given the opportunity to prove their worth. British architecture was stagnant in the early 90s and now it's not, for which some of the credit is due to Labour.

But Labour has also presided over some of the poorest and most ill-considered housing of modern times, thanks mostly to the explosion of buy-to-let developments. In big cities, especially London, it has permitted grossly exploitative development with only the flimsiest pretence of being good architecture or planning.

Many of the new schools and hospitals are at best very ordinary, at worst soul-destroying. Their problem, as in other fields, has been the New Labour belief that you could hand over the delivery of social benefits to the private sector.

And so the ideals of Rogers's urbanism were corrupted. He always argued for a combination of increased urban densities and better designed environments, as in his beau ideal city of Barcelona. More people in cities, he said, would mean a more active and cared-for public realm, better use of public facilities and less use of cars and greenfield sites. Developers gleefully seized on the density part of his argument, always happy to maximise returns on their land. To cram a site could now be portrayed as a public service.

The "good design" part was put in the hands of Cabe and planning authorities, which would often do a good job of mitigating monstrosities and steering proposals in more intelligent directions. But the definition of good design became enfeebled. The concept arose that a development was good if it was "iconic" and was designed by a "world-class architect", with the latter notion being ever more flexibly interpreted. Coteries of consultancies grew up to verify what good design was, while architects of skyscrapers, sitting on Cabe's design review panels, would take it in turns to applaud each other's works.

By the mid-noughties, a type had emerged: an office or residential development straining at the limits of its site, extravagantly styled so as to earn the adjective "iconic", with "public realm", in the form of highly monitored aprons of granite, at its foot. Such developments would rarely work with their neighbours to form coherent urban fabric, which should surely be part of good architecture. They were individualistic and egotistical.

Some of the worst are still unbuilt. These include the Vauxhall Tower in London, pushed through by John Prescott against the advice of his planning inspector. Also the Pinnacle, or Helter-Skelter, the putative tallest building in the City of London, which completely ignores a neighbouring design by Richard Rogers. To get an idea, however, of misdirected urbanism in practice you can go to Stratford High Street in east London. Here, strung along a howling road, you get stacks of undersized, poorly oriented, cheaply built flats, each shrieking with its own design gimmicks and contributing not at all to a shared public space. That they are within spitting distance of the Olympic site, billed as the greatest regenerative gift of all time, is entirely typical.

As for the schools and hospitals, these are mostly being delivered by the infamous private finance initiative, which asks private contractors to carry the debt and risk of public building projects.

The drawback is that the contractors also get to call most of the shots, with the result that they interpret their obligations as minimally as possible. Another drawback is that PFI contracts are vastly complicated, requiring generous payments to lawyers, accountants and other consultants. At the same time, there are paltry margins left to pay architects to use their imaginations or for specifying any finish or detail above the most basic. The end product is places of learning and healing that look like escapees from a business park or Ricky Gervais's domain in The Office.

In the planning of office blocks and blocks of flats, and in the building of schools and offices, the same thing happens. Worthy public aims are stated. Ambitious targets are set. The private sector is entrusted with achieving the aims and targets which, as it will, put its own interests first. So the worthy aims are compromised.

What is lacking is the belief that public authority can state clearly and firmly what it thinks is the public good and then see that it is achieved. This might mean a meaningful plan for places like Stratford High Street or the ability to describe what a beautiful school might be.

British government, national and local, used to be able to do this, for example when the handsome London Board Schools were built in the 19th century. To achieve this again would be a fitting task for the next government, whoever it may be.