Training book used for VA: 'How To Lie With Statistics'

A book used to train hundreds of employees discusses ways to present data

New VA Secretary Robert McDonald has ordered the VA to discontinue the use of the statistics textbook “How To Lie with Statistics” for employee health-care analytics training.(Photo: Chip Somodevilla/Getty Images)

Story Highlights

A book entitled How to Lie with Statistics came under fire during a House VA Committee hearing.

Book doesn't encourage readers to lie, but the committee questioned the use of lessons in the book.

As the U.S. Department of Veterans Affairs struggles to redeem its reputation after a data-manipulation scandal, it faces new scrutiny over a book used to train up to 500 employees a year: "How To Lie with Statistics."

The popular statistics book, published in 1954, was penned by Darrell Huff, an author of several other "how to" books who later worked to debunk the surgeon general's statistical links between cigarettes and cancer.

"This book is a sort of primer in ways to use statistics to deceive," Huff writes in his introduction. "It may seem altogether too much like a manual for swindlers."

Used as recently as August to train VA employees, the book came under scrutiny during last week's contentious congressional hearing over the validity of the VA Office of Inspector General's investigative report into the Phoenix VA Health Care System.

A House Committee on Veterans' Affairs member grilled a senior Phoenix VA official over the use of the book and questioned a graph submitted by the Phoenix VA that he said employed one of the misleading techniques discussed in the book.

It's a volatile time at the VA in the aftermath of the scandal. New VA Secretary Robert McDonald has promised a change in the institution's culture, and House VA Committee members have shown him support. Against that backdrop, the use of Huff's book in VA-sponsored data-analysis training found a natural place in the ongoing debate over the integrity of VA data and the agency's credibility.

McDonald acknowledged the negative perception and ordered employees late last week to discontinue using the training material.

"I have not read and am not commenting on the merits of the book. The implication was that some VA employees may be using this book to mislead the public," McDonald wrote in a memo to employees.

"Let me be clear — anything that is contrary to our mission of serving veterans, in perception or practice, or which does not align with our I-CARE core values — Integrity, Commitment, Advocacy, Respect, and Excellence — will not be tolerated in the open and accountable culture we want in this new VA," McDonald wrote.

"The book is an entertaining series of examples. ... It's not any kind of statistical manual. It's a book that people give to students to read to serve as an entertaining example of how to be careful when you're presenting data," said Andrew Gelman, a Columbia University professor of statistics and political science.

Despite Huff's introduction, the book does not explicitly encourage readers to lie. The title is intended to be ironic and reflects the cheeky, almost farcical, tone Huff carries through the book to show readers the many ways statistics are manipulated.

The VA course is held three to four times a year, with about 150 employees in each session. Sixteen employees from the VA Southwest Health Care Network, which includes Phoenix, attended the course this fiscal year. The VA pays $1,000 per employee for the program.

The training began because VA facilities wanted to learn how to better translate data into actions, regional VA spokeswoman Jean Schaefer said. The voluntary program is intended for managers, analysts, nurses and other staff members who want to learn how to effectively analyze health-care data, Schaefer said.

The book was removed from the course syllabus per McDonald's order.

Rep. Tim Huelskamp, R-Kan., a former statistical analyst, drew a connection between the book's titular dogma and the committee's continued frustration with conflicting VA data and investigators' findings that the agency repeatedly manipulated statistics.

He drew attention to a graph that compared the growth of outpatient visits with the number of full-time employees at the VA. It was submitted by the Phoenix VA in a July congressional briefing. At a glance, the graph appears to show that growth in outpatient visits far exceeded the increase in number of employees from fiscal 2010 to 2014.

But a closer look shows the two measures in the graph were created on far different scales, inaccurately comparing the growth in outpatient visits with the growth in employees.

A graphic on outpatient-visit growth in relation to full-time equivalent employees.(Photo: House Committee on Veterans’ Affairs)

In an interview with The Arizona Republic, Huelskamp said he "thumbed through" the book and looked at examples in the book that showed ways to mislead people with data.

The book created a bad perception and underscored his frustration with information the VA has been providing the committee, he said, including the graph submitted in July.

"We've received misleading, misleading numbers again and again," Huelskamp said.

McDonald's order was the correct move, Huelskamp said: "If we're going to get the VA fixed, we need a good, open relationship and good, accurate, real numbers from the VA. And the secretary, I think, is going to provide those and move forward and work together."

Nebraska Methodist College Vice President for Academic Affairs Paul Savory said he was not surprised by the committee's suspicions with the book, given its title. He said the book was used to caution the reader about misleading data and was integrated into data-analysis projects in the class.

"We've never focused on any aspect of that (lying) in terms of what we've been doing," Savory said. "It has a controversial title, but it's been around for 60 years. It's the most widely sold statistics textbook out there. ... But I understand from the VA's perspective that it's a perception issue."

ON THE BEAT

Michelle Ye Hee Lee is a government accountability reporter and is one of the reporters covering the VA scandal.