Richard Dadd's great paintingThe Fairy Feller's Master- Stroke shows a leather-clad person, axe raised to cleave a hazelnut, perhaps to make a coach for Queen Mab, as Mercutio describes her in Romeo and Juliet. He is intently watched by a grotesque crew of beings of all shapes and sizes. Octavio Paz, in The Monkey Grammarian, says that the sense of the work is a terrifying anticipation. Time is suspended until the axe falls, and the axe is eternally about to fall. Dadd, says Paz, has painted "the vision of the act of vision, the look that looks at a space in which the object looked at has been annihilated".

Dadd's life is a strange combination of violence and inaction. He was a successful young painter in the early 1840s when he accompanied a patron on a long tour of the Middle East, complaining that they never stopped long enough for him to draw what he saw. On his return he showed signs of mental disturbance. His father, who had a gilding business, went with him to Cobham, where the family roots were. They went on a walk in Cobham Park in the evening. There Dadd stabbed and killed his father – the attack was obviously premeditated, as Dadd had prepared his flight out of the country. In a coach in France, he attacked a fellow passenger with a razor, and was apprehended. He claimed to be controlled by demons, and that his father was not his real father.

He returned to England and was tried for murder. He was found to be a "criminal lunatic" and shut away in Bethlem, as one of those grimly defined as "Pleasure Men" – those detained at the Queen's pleasure. In Bethlem he painted some amazing paintings, including The Fairy Feller, on which he worked from 1855 and which was left behind unfinished when he was moved to the new asylum of Broadmoor in 1863.

In his book Richard Dadd: The Artist and the Asylum Nicholas Tromans is interested in the nature of asylums in general, and in the effect of Bethlem and Broadmoor on Dadd and his work. He is writing, as he says, in the wake of Foucault's history of asylums, Madness and Civilization, and Erving Goffman's criticism of mental institutions, as well as the anti-psychiatry movement of RD Laing, who believed in schizophrenia-inducing mothers, and allowing patients to "work through" their madness. Tromans nevertheless shows a somewhat unfocused need to be suspicious of the asylum authorities – including those who bought Dadd's work and encouraged him to paint. These authorities knew nothing of the causes of madness, though they believed in treating patients kindly. The enlightened William Charles Hood in 1862 listed among the causes of insanity Fright, Jealousy, Sudden Prosperity, Coup de Soleil and Onanism. Tromans is interested in whether schizophrenia is hereditary and gives a chilling account of the fates of Dadd's siblings. His brother George was admitted to Bethlem when Richard was fleeing after killing their father. His sister Maria married in 1844 but was insane by 1853, and in an asylum by 1859. Another brother was said to have a private attendant. Tromans also considers the possibility that all this was not hereditary but the result of poisoning by the mercury used in the gilding process.

This book is superbly illustrated with reproductions of Dadd's work. There are the early "fairy" paintings shown in London in 1841-42 – in a genre then very fashionable in which the fairy folk are of different sizes, from tiny to buxom, and dance and fly in animated crowds among vegetation that dwarfs them. Dadd's three paintings Puck (1841), A Fairy – Sunset (1841-42) and Come unto these Yellow Sands (1842) are elegant and precise – the Puck is a baby, sitting on a mushroom in moonlight under a columbine dripping with dewdrops, among grasses also beaded with water, and watches much smaller naked dancers cavorting below him. These fairies have sharp, mischievous features, quite different from the later fairies of Bethlem.

I remember the extraordinary moment on The Antiques Road Show when the magical Halt in the Desert, painted in 1845, was identified as by Dadd. It is a watercolour of a camp in the dark, lit by a fire round which the travellers are crowded, and by a small pale moon in a rack of lemony, silvery clouds. Tromans shows views of places such as Jerusalem and Venice, painted in the asylum from memory and sketches. There are several portraits of Dadd's patron, Sir Thomas Phillips, a magistrate knighted for putting down a chartist protest. Many of the portraits are in Arab costume. Phillips stares out of them with mad, suspicious pale blue eyes. He did indeed become worried about Dadd's odd state of mind on their journey, and it was suggested that he was affected by sunstroke. Dadd later painted a series of imaginary portraits of allegorical "Passions" – Treachery, Recklessness, and so on – who also have this unnerving, fixed mad stare. On his Middle Eastern journey he was apparently taken by the sturdy beauty of the women: "the water-carriers (women) are very capital subjects for the brush; and they rush along with great celerity under pitchers of no small size."

There are three extraordinary paintings at the centre of this book. One is Flight Out of Egypt (1849-50). Then there isContradiction: Oberon and Titania (1854-58), depicting the quarrel over the Indian Boy, which was painted for William Charles Hood at Bethlem; and The Fairy Feller, painted for George Henry Haydon, also at Bethlem. All three are crowded with figures, and all seem to bristle with occult meanings. Tromans does not offer any deep analysis of the subject matter of these paintings. Flight Out of Egypt has a huge crowd of figures at what seems like a desert oasis – to the left there is a rhythmic forest of plumed lances held by horsemen and camel riders, to the right tents and groups of Arabs (including a tambour dancer based on an image at Pompeii). There is a group in the foreground of pale-skinned people who in some ways represent the flight into Egypt – a woman with a swaddled baby, a bearded Joseph figure, a sinister child with a bow and arrow, and an even more sinister child battling a nasty goat next to a spilled water vessel. The other rhythm in the painting is repeated images of water vessels, on veiled heads, being dipped into the stream. In the very middle is a group of mailed soldiers, helmeted and red-cloaked. The central one, bejewelled and in a leopard skin, is drinking from a metal vessel that obscures his face. It is not the Israelites leaving Egypt – Moses and Aaron are nowhere to be seen. It is some sort of allegory but its meaning – to me at least – is totally obscure. Dadd held beliefs about Egyptian myths as opposed to Christian ones. It is possible that the looming sense of danger and disaster are simply part of his state of mind. He is said to have explained that chess pieces could be possessed by devils and be against you when you played. This picture belonged to Sacheverell Sitwell, who expressed frustration, in Narrative Pictures, with its elusive meaning.

Contradiction is easier to read and amazing to look at. Oberon is clothed in vaguely Middle Eastern robes, bearded and crowned. Titania is unexpected – a hefty woman, in a Greek dress and fantastic crown, wielding a long wand and apparently crushing to death one of a group of minuscule fairies who are flying around her feet. Her face is truculent; she stares up and away from Oberon, who is apparently being restrained by a sharp-faced Puck. The whole work is crammed and crowded with the Dadd mixture of intricate plant life and small figures on all scales – a miniature bacchanal with satyrs, and a dead deer on a pole that rushes along above Titania's head. A mysterious green egg shape hangs from a pagoda–like object, next to which is a tiny hanging green fairy on the summit of an odd triangular gilt construction. There are, as in The Fairy Feller, variegated grasses wreathing randomly over the surface of the work, and delicately depicted lilies of the valley, about as tall as the fairy queen herself. Everywhere you look, a tiny face peers at you, in the stem of a leaf, in a seedpod. The whole thing is undergrowth-green, sage green, with Titania's bright green shawl and the odd egg standing out brightly.

Dadd left a 24-page description of The Fairy Feller's Master-Stroke. It is written in excruciating doggerel verse, with appallingly irritating rhymes and shapeless rhythms. Its title is "Elimination of a Picture and its subject – called The Feller's Master Stroke." Tromans does not comment on the odd word "Elimination". I think it is part of Dadd's predilection for double-speak and dangerous puns: "Elimination" contains the word "limn" which is a good word for painting, but also is part of Dadd's habit of decrying painting as pointless and worthless. There has been scholarly speculation about the double–entendre of explaining his murder of his father as caused by "sun-stroke", playing with sun and son. The word "feller" has a bland meaning of "good fellow" and a more dangerous one of "striker-down". Critics often see the unfinished stroke of the axe as a reference to Dadd's own striking-down of his father. There is a sinister onlooker at the centre of the painting called the Patriarch, with a triple crown who leans on a "large little club".

If on a sudden it descendsOn fairy sconce, its revel endsAnd then you know poor little fartUnto another private realm he will depart.In many ways the most interesting part of the "Elimination" is the long description of how the painting grew from the contemplation of a prepared canvas, "by paint smeared o'er & fit for use / Not smooth but of a texture loose", rather like Leonardo seeing subjects and forms in uneven walls. (Terrry Pratchett describes the work in The Wee Free Men, and comments that it gives the impression that the artist was seeing, rather than inventing, the complex scene.)

One of the characteristics of Dadd's fairy paintings is the way grasses and tendrils are apparently randomly interposed between the onlooker and the world in the painting. There are particularly energetic ones winding across The Fairy Feller, which Dadd describes in "Elimination":

Turn to the Patriarch & behold Long pendents from his crown are rolled, In winding figures circling round. The grass and such upon the mound. They represent vagary wild And mental aberration styled. Now unto nature clinging close Now wildly out away they toss, Like a cyclone uncontrolled.

This passage comes near the end of the "Elimination", which concludes, sadly: "You can afford to let this go /For nought as nothing it explains / And nothing from nothing nothing gains."

The late, great Sigmar Polke chose this painting to describe on the Tate Gallery's website. He compares the painting to Dürer for botanical precision, and also to Bosch and Peter Bruegel, in the way the whole canvas is covered with figures. He also compares it to Jackson Pollock's "all-over" style, with an absence of horizon. He describes the wild grass as "holding everything together optically", combined with the patterning of spherical forms, "distributed throughout … hazelnuts, prickly fruits or white ruffles on a shirt". He writes of the scattered daisies as a device for "setting the stage for the artist's curious treatment of scale". Polke's own work, like Dadd's, as he says, relies on radiating circles and spheres. In 1986 he made a wonderful series of Schleifenbilder (curlicues or whorls), in which he made images from a series of abstract whorls on a Dürer woodcut of a triumphal carriage. Polke explains the significance of the unpainted parts of the canvas – the foreground and the axe itself – which Octavio Paz read as a stone axe. The unpainted foreground, he says, reads as bright earth and "generates a bit of distance, the necessary breathing space". And he says that the axe is unpainted because the Patriarch's signal for it to fall "is delayed for all eternity by never allowing the picture to be completed". Paintings are always in a sense outside time. The arrested masterstroke makes that precisely clear. And the incarcerated painter was also in a sense outside time.

Sign up for the Guardian Today

Our editors' picks for the day's top news and commentary delivered to your inbox each morning.