There is a portrait in this marvellous show at the National Gallery complex in Edinburgh that shocks you with its flashing intensity. It shows a dark-eyed Spaniard staring out of the shadows with such penetrating directness that the past becomes the present in his glance. Only the archaic collar, stiff as a porcelain dish, returns the picture to the 17th century.

A Spanish Gentleman carries all the mystery of Velázquez's art. The soft hair, the inner tension, the level-eyed look of intellectual empathy: every brushmark is quick with life and descriptive power until you get closer, whereupon they lose legibility and it is nearly impossible to see how the portrait was made. It is an additionally precious work, too, for it surely reveals Velázquez's one true friend at the Spanish court – the chamberlain Nieto, that dark figure silhouetted in the doorway at the back of Las Meninas, hovering between this world and the next.

To see this portrait is to feel the live connection between artist and sitter. To learn how it came to be in Britain is to sense the close connection between art and war. Joseph Bonaparte stole it from the Spanish court during Napoleon's invasion of Spain; Wellington's troops found it rolled in the baggage of the fleeing French after the Battle of Vitoria. Wellington tried to return it; a grateful Spain refused. Which is how it comes to reside at Apsley House, Wellington's home, otherwise known as No 1 London.

This star-studded show of Spanish art in British collections is full of such piercing crosscurrents and encounters. Goya paints Wellington, frail and dazed, with a look of wide-eyed exhaustion. The portrait is small, almost as small as the etchings from the artist's terrifying Disasters of War hanging on the opposite wall – images of bravery and horror that both Goya and Wellington have seen.

One plate shows a Spanish woman letting off a cannon against the French at Saragossa. Alongside is David Wilkie's colossal vision of the same scene, about 30 times bigger, though no more powerful. The Scottish painter made the arduous trip to Spain in 1827 and came back revolutionised by his experience of both the country and Velázquez.

Yet only a few years later the British diplomat Sir Edmund Head lamented that Velázquez was practically the only name we knew: "Of art in Spain … we are almost totally ignorant." And when it did begin to arrive in this country through all sorts of peculiar back routes, Spanish art was not always welcome. Ruskin excoriated Murillo, absurdly picking upon his innocent street urchins as inappropriately degenerate subjects for art. El Greco was thought intolerably alien.

And when the shattering image of a cowled monk holding a skull by that great mystic Francisco de Zurbarán was bought for the nation, critics were convulsed. "Why would we have something so black and repulsive in our national collection?"

Time passes, tastes change and now the crowning glory of this show is the blackest room of all, seething with the glittering darkness of Golden Age Spain. Zurbarán's outlandish imaginary portraits of the sons of Jacob show them facing straight into sepulchral gloom. The eggs coalescing from translucent liquid to white flux in Velázquez's tavern scene are a feat of illusionism conjured out of darkness. El Greco's transfigurations occur at dead of night.

And the drama of Zurbarán's hyper-real St Francis, that came across as Catholic sensationalism to 19th-century England, now looks like the very emblem of Spanish painting: all pictorial austerity, sombre and mysterious black light.

The masters of many British masters were Spaniards; consider Velázquez's influence on Gainsborough, Whistler and Millais. In fact one gallery here is devoted to making this possible. It has some very weird and dubious offerings, including William Hazlitt's portrait of Charles Lamb as a latterday Nieto, but also a self-portrait by Whistler in the pose of Velázquez's famous Pablo de Valladolid. Whistler spent the last three years of his life returning to the picture, abjectly trying to catch some trace of the Spaniard's spirit in this faltering seance.

Inevitably, the show loses its own spirit somewhat whenever the Spaniards are off-stage. There is undue stress on minor British painters to illustrate the prevalence of Spanish influence on our art. For every brilliant attempt to get the high blue hardness of a Toledo sky by John Phillips, say, there are half-a-dozen mediocre paintings of bullfights or señoritas. And there is a long stretch of insipid pastiche before you get to the final room, with its Bombergs, Moores and Picassos, representing the Spanish Civil War.

But this is unimportant, except perhaps to anyone intent on getting their art history pedantically straight. What matters is the magnificent array of Spanish paintings united in one building. The pulling power of the Scottish National Gallery is tremendous, from the National Gallery's Goyas and El Grecos to the Tate's Weeping Woman and Zurbarán's imaginary portraits from Bishop Auckland. This is a feat. The opening of British eyes to Spanish art, it turns out, is both the show's theme and its own vivid achievement.