Gyllenhaal has been severely criticized in letters and memoirs. He appears under nicknames such as "The lion of Gota," "The Devil of the Customs" and "Bustler Busybody," and is frequently criticized for his repeated visits to the antechamber of the royal palace. Crusenstolpe writes, "Next before the office room, and second after the roads of the country, there is no place on the globe with more founded pretensions to brag over this Baron's personal presence." A couple of years after Gyllenhaal's death, Biographiskt Lexicon sums up: "Rarely has a high civil servant had such bad words said about him as the Director General of the Customs Gyllenhaal." These bad words came also from those of the same political opinion and relatives. A contribution to this was perhaps that, during the middle of the 19th century, when there were disputes about the Customs, he openly supported more liberal trade, against the majority of the conservatives, with the explanation that the many prohibitions had a demoralizing effect and taught the Swedes to commit perjury and other crimes. His first period as head of the Customs (Department) was prosperous. He carried through reforms in conformity with the times that were appreciated even by Crusenstolpe. But soon there was talk about his openness to bribes in connection with appointments, his peculiar way of using the Customs fund for extraordinary expenses, and his accounts for travelling expenses. During the parliament of 1854 his promotion and administrative system were the subject of a well-prepared attack from the liberal wing of the clerical state. The King reacted slowly, but when he wanted to rearrange his Cabinet in 1856 in a liberal direction, J. A. Gripenberg demanded the elimination of Gyllenhaal as a condition for him to accept the post as Lord of the Exchequer. King Oscar I then gave in and Gyllenhaal was granted dismissal the same day as the opening of the parliament of 1856. He remained, however, as Governor of the palaces (mentioned above). As such he angered the general public through certain changes in the Park of Haga and by excluding Beylon, which had been a popular destination for excursions. Later, when Ulriksdal became a summer residence for the future Karl XV, Gyllenhaal got the opportunity to make himself useful to a third generation of Bernadottes. He was interested in music and tried when possible to help young musicians such as, e.g., Adolf Lindblad.

Fritz von Dardel describes Gyllenhaal as a reckless old roué without scruples. At this time, when he became "Grand Maitre of the order of Amaranth," Crusenstolpe is enraged and says that Gyllenhaal's greatness was only in his corporal size. One of his subordinates (accountant Magnus Hollertz) gives a different picture of his boss: "It is for sure, that he has the best heart, the most happy mind, perhaps a somewhat too French liveliness, and familiarity with good society, which either shows in an exhilarating contempt, or in an almost royal, but never hautain dignity. At present he is 68 years old, but strong as a man of 30 years age, agile as a voltigeur and elastic as a young lieutenant."

Travel Around a Very Small Part of the Earth, by Magnus HollertzHollertz wrote this book about an inspection trip on which he accompanied Gyllenhaal

Gyllenhaal died in the aftermath of cholera at Beylon, to which he moved after his resignation from the Customs Department. Earlier he had shown an extensive and costly hospitality in a 17-room apartment at Holländargatan.