Wallmoden was an illegitimate son of George II of Great Britain by his mistress Amalie von Wallmoden. She was married to Adam Gottlieb, Count Wallmoden (1704−1752), but for a payment of 1000 Ducats the Count was prepared to defer his claims on his wife to George, and was finally separated from her in 1740.

On the death of Queen Caroline in 1737, the Prime Minister, Robert Walpole, suggested that Amalie be brought over from Hanover to Britain to take her place as maîtresse en titre to George II. In the mean time Lady Deloraine, a loquacious but not very intelligent courtesan, with whom George had a distant relationship, functioned as a stopgap. Thus the young Johann Ludwig came to be conceived in England and grew up at St. James's Palace and Kensington Palace. As an illegitimate son of the king, he received a comprehensive education, after which he went on a Grand Tour to Italy, where he acquired an extensive collection of classical statues, busts, and reliefs. On his return he entered the Hanoverian army and rose to the rank of major general.

Collection of Wallmoden-Gimborn in the exhibition The Hanoverians on Britain's Throne 1714-1837, in the Palace of Herrenhausen, 2014

Around the year 1700, several noblemen's country estates had been established in the former flood plain of the Leine. In 1768 Wallmoden acquired some of these gardens and merged them into the Wallmodengarten (later to become the Georgengarten). In 1782 he built the Wallmoden-Schloss to house his collections of antiquities. In 1782 he bought the ReichsherrschaftGimborn in Westphalia from Prince Johann I. of Schwarzenberg, and on 17 January 1783 was raised to the nobility of the Holy Roman Empire by the emperor Joseph II, with the title Wallmoden-Gimborn and with a corresponding augmentation of his coat-of-arms to Imperial count.

On 5 July 1803, as Oberbefehlshaber (commander-in-chief) of the Hanoverian army, he signed the convention of Artlenburg and thus capitulated before the Napoleonic troops arrived.

After Wallmoden-Gimborn's death, his nephew King George III acquired his collections of antique sculpture and books, over 8000 volumes. The collections are still in the ownership of the Welfs and since 1979 have formed a collection of the Archeological Institute in Göttingen.

Friedrike Eleonore Juliane (12 July 1776−18 February 1826), who married Ludwig Friedrich Count von Kielmansegg and was the mother of Eduard von Kielmansegg (de)

Wallmoden's second marriage, on 3 August 1788 in Bückeburg, was to Baroness Luise Christiane von Lichtenstein (1763−1809), a daughter of Baron Friedrich Karl von Lichtenstein by his marriage to Charlotte Ernestine von Berckefeld, and with her he had three further children:

Karl August Ludwig (4 January 1792−28 February 1883), an Austrian Privy Councillor and Lieutenant-General. In 1833 he married Zoe, Countess von Grünne, daughter of Philipp Ferdinand von Grünne (de), and from him is descended the Oberhaus Wallmoden line.