“Is it really believable that someone who had a Wall Street career before coming to Washington five years ago was so politically naïve that he didn’t see the potential for scandal in that information and give the White House a heads-up?” Commentary’s John Steele Gordon wondered Tuesday.

“Sooner or later this [question] will have to be answered,” tweeted Fox News senior political analyst Brit Hume, “What was the ex-IRS chief doing at the White House all those times?”

Shulman’s integrity has come under fire in recent weeks as he — in the face of congressional investigators — failed to recant his March 2012 testimony, during which he insisted that the IRS was not targeting conservative groups.

Public White House records are incomplete, with records only showing visits after September 15, 2009. The White House releases several months of records at a time. The last few months should be released later this year.