Between endless building projects and Thames Water's efforts to replace leaking Victorian water pipes, there are times when central London resembles one big obstacle course. A new exhibition at Sir John Soane's Museum suggests there's nothing new in that. It presents a small fraction of the work of an extraordinary, but largely unknown, Bavarian artist who made London both his home and his principal subject matter after arriving in England in 1816.

A wanderer as a young man, George Scharf had reached England via Munich, moving across Europe in the train of the armies fighting the Napoleonic Wars, painting portrait miniatures of officers from both sides of the conflict. He eventually enlisted in the Royal Engineers, later claiming to have seen action at Waterloo. Once in London, he married his landlady's sister and based himself in the artistic quarter of St Martin's Lane. London was then a thriving centre for lithography – the new printing process in which a smooth stone was divided into ink-receptive and ink-repellent areas through the application of oil – and Scharf was to enjoy success with mostly topographical views and genre scenes that could be transformed into prints.

Scharf viewed the teeming and evolving metropolis with the fascinated eye of the outsider, homing in on the details and paraphernalia of everyday London life – the chaos of building works, the throng of tradesmen and itinerant entertainers. An 1828 watercolour showing the construction of Robert Smirke's new British Museum focuses most of its attention on the workers, their tools and materials, rather than the scaffolded classical buildings in the background. A sketch of 1841 depicts the bustle of working-class families as they hurriedly carry their lunches home, having purchased them from the local inn.

It's this ability to offset precisely drawn objects with the bustling unpredictability of human life that makes Scharf's work so engaging. There's a particularly delightful drawing of a stage-coach being loaded up with passengers and luggage in Piccadilly, in which the driver peers behind worriedly as a man, arriving late with his family, struggles to pull on his overcoat. Another drawing shows oil being piped from barrels into a shop on the Strand, counterpointed with a woman's annoyance at discovering a drop has landed on her gown.

If this emphasis on the anecdotal suggests Scharf as a kind of Regency Posy Simmonds – touchingly humane but with an eye for the comical – there was also another side to his work, in the form of illustrations for a number of London's scientific institutions, such as the Zoological and Geological Societies and the Royal College of Surgeons. This work brought him into contact with leading scientists, including Robert Owen and Charles Darwin. In fact, it was a row with Darwin over the pricing of drawings of South American fossil bones – Darwin thought he was being ripped off – that helped stem this lucrative source of income.

It also marked a decline in Scharf's fortunes. The artist's last years were rather abject: living apart from his family, he was reduced to trying to sell his London drawings to the City Corporation, who turned him down. He even solicited minor German royalty for a pension in exchange for all of his work, but was again rebuffed. Following his death in 1860, his wife, Elizabeth, sold over a thousand drawings and watercolours to the British Museum, from whom many of these works are loaned. Sadly, this unique visual chronicle of London's transformation into a modern metropolis is rarely displayed – all the more reason not to miss this exhibition.