Now, the New York Post is offering further details of his behavior allegedly contained in the diary of former Late Show staff member Stephanie Birkitt – identified as the woman at the center of the Letterman scandal. According to the newspaper, Birkitt, Letterman's former personal assistant, wrote that she continued her affair with Letterman even after moving in with Robert "Joe" Halderman, the CBS producer now charged with attempting to blackmail the late-night host.

Last fall, according to the Post, Letterman, now 62, and Birkitt, 35, hiked together at his expansive Montana ranch, where Letterman and longtime girlfriend Regina Lasko, the mother of his son, Harry, were married this past March. The newspaper quotes a source saying that Halderman, 51, remained home in Connecticut while Birkitt and Letterman were hiking. Birkitt reportedly told Halderman she was Letterman's "best friend" and they two shared "a platonic relationship."

Among the other details obtained by the Post: at least once a week Letterman drove Birkitt to the Norwalk, Conn., home she shared with Halderman from 2004 until recently, and that after Birkitt graduated from Benjamin Cardozo Law School, Letterman offered her a job as his personal lawyer.

Halderman read Birkitt's diary in December, sources told the paper, and grew angry upon learning she and Letterman were still carrying on what the Post labels a steamy affair.

Speaking with PEOPLE Tuesday morning, Halderman's attorney, Gerald Shargel, would not comment on how his client found the diary. He did say, in light of the events of the past few days, his client is doing "as well as can be expected."

As for the extortion charge against Halderman, the attorney said, "'I'm going to try the case in the courtroom. The D.A. said it was an open-and-shut case, and he embraced Letterman's story … I want to be clear it's not an open-and-shut case at all. There are a whole bunch of facts I'm pursuing, and I want the public to know that."