Friday, June 22, 2012

Much of the criticism of PolitiFact from liberals is lame. On occasion, however, a liberal makes a pretty good point, and Jared Bernstein makes one of those in commenting on a PolitiFact rating about median income in Massachusetts while Mitt Romney served as governor.

The claim came from Ed Gillespie, who was defending Romney's record in Massachusetts by pointing out that median income in Massachusetts rose by $5,500. The problem, as Bernstein pointed out and PolitiFact acknowledged in the story, comes from the fact that family income actually dropped slightly after adjusting for inflation.

Saying that income went up by thousands of dollars while it dropped in terms of real dollars is flatly misleading. Bernstein has a good foundation for a sense of outrage.

But even though Bernstein's basic point is accurate, he oversteps a bit by assigning the statement a rating of "Mostly False." That's because PolitiFact gives each of the ratings a definition. Here's the definition of "Mostly False":

MOSTLY FALSE – The statement contains an element of truth but ignores critical facts that would give a different impression.

Gillespie's statement did not simply contain an element of truth. It was true and misleading. The "Mostly False" rating serves as an ill fit for claims like Gillespie's because the rating itself is only half true.

When PolitiFact started out, the "Half True" rating appeared to fit statements like Gillespie's:

HALF TRUE – The statement is accurate but leaves out important details or takes things out of context.

Some of PolitiFact's state operations may still have this definition of "Half True" posted on their websites (confirmed for Texas).

Thursday, June 21, 2012

In an otherwise okay article about adapting journalism to the Internet, PolitiFact editor Bill Adair ended up making his usual pitch for pretending that journalists' selection bias does not affect fact check stories:

We’ve done the same thing with the Obameter and the other meters we use for tracking campaign promises. The unit of journalism is the promise and then we write updates and rate whether the promise is kept or broken. The promises also get tallied so you can see how the politician is doing.

PolitiFact's "two new forms" of journalism encourage readers to accept a non-scientific process as science. PolitiFact and Bill Adair continually sell the falsehood that its candidate report cards serve as a useful guide for judging whom to support in an election.

PolitiFact writes fact check stories its left-of-center staff thinks are relevant and interesting. Left-of-center staffers find stories favoring their left-of-center views more interesting than others. This naturally results in more fact checks of conservatives since left of center staffers question conservative ideas more than those jibing with their own. Should PolitiFact experience concerns that its fact checking looks overly focused on conservative claims, it picks a few claims by liberals to help balance the ledger--claims that were in the first place less likely to warrant a fact check by left-of-center lights. As a result, to no one's surprise, liberals tend to fare better on the report cards. Liberals feel no surprise because of confirmation bias. Conservatives feel no surprise because they've learned to expect biased reporting from journalists.

PolitiFact's defenders like to suggest that conservatives perceive a non-existent bias. The simplest explanation, some of them say, is that conservatives simply lie more.

Number two: The story selection process--indeed, PolitiFact's entire process--might as well be designed to reinforce liberal bias. There's no check on selection bias, and the three-editor panels PolitiFact uses to assign grades help ensure that the liberal majority in the newsroom decides the ratings by popular vote on any occasions where the voting isn't done only by liberal judges.

PolitiFact's process requires those who understand its workings to suspend disbelief when looking at its report cards and collected numbers.

Given that PolitiFact says Romney's numbers check out, how the heck did PolitiFact then conclude Romney's statement is "mostly false"? Well, they did what fact checkers habitually do whenever they find something factually correct but politically disagreeable—kick up a bunch of irrelevant contextual dirt and lean on some biased sources. Which is why PolitiFact's own language here is absurd: "We found that though the numbers are accurate, their reading of them isn’t" and "The numbers are accurate but quite misleading." I would also note that my friend Glenn Kessler, the fact checker at the Washington Post,evaluated the same claim and deemed it "TRUE BUT FALSE." I do hope that if media fact checkers expect to retain any credibility to evaluate basic empirical claims, they're aware that this kind of Orwellian doublespeak is going to make them a laughingstock.

Read the whole thing, because Hemingway's just warming up with the above.

The above point, that PolitiFact appears absurd for ruling a true statement "Mostly False" probably can't receive enough emphasis. PolitiFact's rating system provides no description fitting this type of rating. If the results make it look like PolitiFact isn't categorizing claims according to whether they fit some type of established objective criteria, it's probably because that's the way it is.

Addendum:

PolitiFact's response to the complaint from the Romney campaign deserves a closer look:

We considered the complaint and interviewed four other economists, none of whom have formal or financial ties to any campaigns. Our additional reporting found no reason to change our ruling, which remains at Mostly False.

Two words: Fig leaf.

The point is that the original reporting didn't justify the ruling. If PolitiFact can't see that then it's no surprise that additional reporting fails to sway its made-up mind.

Monday, June 11, 2012

It seems that Sauron, Satan and Darth Vader (not necessarily in that order) are all aligned against noble Mr. Nelson (bold emphasis added):

I just got a new potential Republican opponent from the extreme right in the Florida Senate race. Perhaps more
significantly his campaign is being run by two of the top operatives
behind Rick Santorum's presidential bid -- the same Rick Santorum who
said that President Kennedy made him want to throw up.

Did Rick Santorum say that President Kennedy made him want to throw up?

Thursday, June 7, 2012

I was intrigued to see the latest "In Context" article at PolitiFact minutes ago. I was intrigued initially because it claimed to provide the context surrounding Bill Clinton's remarks that Republicans have construed as opposed to President Obama's policy stance on the future of the Bush tax cuts. Those cuts are set to expire after this year.

I was intrigued even more because I had referenced Clinton's statement in a discussion on PolitiFact's Facebook page about the wisdom of raising taxes to the levels existing during Clinton's presidency.

And I was especially intrigued when the "In Context" story did not include the quotation of Clinton I used, which was from September of last year.

Here's what I referenced from Politico:

“I personally don’t believe we ought to be raising taxes or cutting spending, either one, until we get this economy off the ground,” Clinton told Newsmax in an interview Tuesday. “This has been a dead flat economy.”

This "In Context" feature seems designed to let Clinton make us believe the economy has taken off. It leaves out the context of Clinton's clearer statements on tax increases.

We arrived at this conclusion by looking at the net monthly change in jobs all the way back to 1970. Since Romney was referring to total jobs, rather than private-sector jobs only, we used total jobs as our measurement. And since Romney was talking about job creation patterns during a recovery, we looked only at job creation figures for non-recessionary periods, as defined by the National Bureau of Economic Research. Finally, we excluded the current recovery.

The Kessler/PolitiFact method is entirely wrongheaded.

Romney didn't claim that 500,000 jobs created per month was a normal figure during a recovery. I can imagine the furrowed brows of Kessler, Louis Jacobson and other mainstream media fact checkers. Aren't they the experts? What am I talking about?

It's actually pretty simple.

The size of the economy changes. If country A enters a recession losing 1 million jobs then it takes two months to regain the lost jobs at a rate of 500,000 per month. If country B experiences a recession losing 10 million jobs then it takes 20 months to regain them at a rate of 500,000 per month.

Not only does the size of the economy vary, but so does the depth of the recession. The rate of recovery for lost jobs needs to account for both factors. Neither Kessler nor PolitiFact gave any apparent consideration to those critical criteria. It's like comparing prices between now and the 1950s without adjusting for inflation.

The Romney campaign has made a number of statements like the 500,000 jobs claim, and they probably relate to the following chart or one like it from the Bureau of Labor Statistics:

Romney's claim almost certainly derives from the fact that post-war recoveries usually replace lost jobs much faster than the present recovery.

So how does one check that claim? It's not that hard. Take the bottom point of employment, then count the number of months that it takes to get employment back up to the peak level. Divide the number of jobs lost by the number of months it took to return to return to the employment peak at the start of the recession. Do the same for each of the post-war recessions, then average the numbers to obtain the average job recovery time. After all of that, divide the number of jobs lost from the 2007 recession by the averaged job recovery time.

Why didn't the Washington Post or PolitiFact do anything remotely resembling the fact check I just described? It could be gross incompetence. It could be ideological bias. Or it could be both.

Monday, June 4, 2012

This looks like another classic case where PolitiFact (PolitiFact Rhode Island in this case) applies a hyper-literal interpretation to a statement that is plainly true when given normal charitable interpretation.

Specifically, Lisa Blais of the Ocean State Tea Party In Action group appeared on WHJJ 920's Helen Glover Show. During that appearance, Blais said ""As you know, we came out against Deepwater. And everybody is now paying for it in their electric bills."

Blais' comment came in the context of a different case of potentially wasteful government intervention. Taken in context, Blais is saying that the Deepwater wind turbine project receives its financing from electric bills. Blais was not communicating the message that persons paying electric bills today are directly paying for the Deepwater project. Find Blais' comments at about the two minute mark of the following audio.

PolitiFact ruled based on the idea that the listener would conclude that today's electric bill directly pays for the Deepwater project and gave Blais a ruling of "Barely True." The PolitiFact story concedes that the project indirectly increases utility bills for today's energy consumer:

In 2009, the legislature and Gov. Donald Carcieri modified that law, requiring National Grid to actively encourage renewable energy projects; one of them was to be an undersea power line from Block Island to the mainland.
(...)
National Grid has racked up some legal costs dealing with that power line and those costs are being passed on to consumers as part of a 0.007 cents per kilowatt hour surcharge on their bills. For the typical residential customer -- who uses about 500 kilowatts per month -- that surcharge translates to no more than 3.5 cents per month.

Even taken hyper-literally Blais' claim was partly true, but the real crime is PolitiFact's decision not to give credit Blais' statement with a normal interpretation.

Blais appeared again today on the Helen Glover Show and gave PolitiFact an appropriate and well-delivered pushback. Blais point out how PolitiFact went easy on a liberal in a different fact check when the literal interpretation was egregiously wrong. She correctly observes that PolitiFact's standards abuse the term "standards."

Chartier provided the following update:

Permit me to be more specifical as to how PolitiFact is miss-serving the public and the truth in the case of Lisa's statement. The headline of today's rating is

However deplorable, not everyone delves into an article or an issue. On any given day, many people are only going to skim headlines to pick up the news - I'm guilty of that myself.

And in skimming mode, the all-important modifier "now" in the PolitiFact headline is not likely to register. What will certainly register, however, is the "False". So people are going to come away from this rating thinking, "That's that offshore windmill project, isn't it? I thought it was going to be funded by our electric bills. But PolitiFact says it isn't. That's good!"

Of course PolitiFact is acting hypocritically. Blais' statement might mislead people. That's bad, and it deserves a "Mostly False." PolitiFact similarly misleads people. That's good, because it gives people an easy way to find the truth in politics without reading every word of a fact check.

What should be done about the economy? Republicans claim to have the
answer: slash spending and cut taxes. What they hope voters won’t notice
is that that’s precisely the policy we’ve been following the past
couple of years. Never mind the Democrat in the White House; for all practical purposes,
this is already the economic policy of Republican dreams.

Sunday, June 3, 2012

On May 23, PolitiFact published an embarrassingly shallow and flawed fact check of two related claims from a viral Facebook post. One of the claims held false a claim from Mitt Romney that President Obama has presided over an acceleration of government spending unprecedented in recent history. The second claim, quoted from Rex Nutting of MarketWatch, held that "Government spending under Obama, including his signature stimulus bill, is rising at a 1.4 percent annualized pace — slower than at any time in nearly 60 years."

PolitiFact responded with an article titled "Lots of heat and some light," quoting some of the criticisms without comment other than to insist that they did not justify any change in the original "Mostly True" rating.PolitiFact claimed its rating was defensible since it only incorporated part of Nutting's article.

(O)ur item was not actually a fact-check of Nutting's entire column. Instead, we rated two elements of the Facebook post together -- one statement drawn from Nutting’s column, and the quote from Romney.

I noted at that point that we could look forward to the day when PolitiFact would have to reveal its confusion in future treatments of the claim.

PolitiFact gives some background for the criticisms it received over its rating. There's plenty to criticize there, but let's focus on the central issue: Was PolitiFact's "Mostly True" ruling defensible? Does this defense succeed?

The biggest reason this CYA fails

PolitiFact keeps excusing its rating by claiming it focuses on the Facebook post by "Groobiecat", rather than Nutting's article, and only fact checks the one line from Nutting included in the Facebook graphic.

Government spending under Obama, including his signature stimulus bill, is rising at a 1.4 percent annualized pace — slower than at any time in nearly 60 years.

This claim figured prominently in the AP and Washington Post fact checks mentioned above. The rating for the other half of the Facebook post (on Romney's claim) relies on this one.

PolitiFact tries to tell us, in essence, that Nutting was right on this point despite other flaws in his argument (such as the erroneous 1.4 percent figure embedded right in the middle), at least sufficiently to show that Romney was wrong.

A fact check of the Facebook graphic should have looked at Obama's spending from the time he took office until Romney spoke. CBO projections should have nothing to do with it. The fact check should attempt to pin down the term "recent history" without arbitrarily deciding its meaning.

The two claims should have received their own fact checks without combining them into a confused and misleading whole. In any case, PolitiFact flubbed the fact check as well as the follow up.

Spanners in the works

As noted above, PolitiFact simply ignores most of the criticisms Nutting received. Let's follow along with the excuses.

PolitiFact:

Using and slightly tweaking Nutting’s methodology, we recalculated spending increases under each president back to Dwight Eisenhower and produced tables ranking the presidents from highest spenders to lowest spenders. By contrast, both the Fact Checker and the AP zeroed in on one narrower (and admittedly crucial) data point -- how to divide the responsibility between George W. Bush and Obama for the spending that occurred in fiscal year 2009, when spending rose fastest.

I'm still wondering why PolitiFact ignored the poor foundation for the 1.4 percent average annual increase figure the graphic quotes from Nutting. But no matter. Even if we let PolitiFact ignore it in favor of "slower than at any time in nearly 60 years" the explanation for their rating is doomed.

PolitiFact:

(C)ombining the fiscal 2009 costs for programs that are either clearly or arguably Obama’s -- the stimulus, the CHIP expansion, the incremental increase in appropriations over Bush’s level and TARP -- produces a shift from Bush to Obama of between $307 billion and $456 billion, based on the most reasonable estimates we’ve seen critics offer.

The fiscal year 2009 spending figure from the Office of Management and Budget was $3,517,677,000,000. That means that $307 billion (there's a tweak!) is 8.7 percent of the 2009 total spending. And it means before Obama even starts getting blamed for any spending he increased spending in FY 2009 over the 2008 baseline more than President Bush did. I still don't find it clear where PolitiFact puts that spending on Obama's account.

(B)y our calculations, it would only raise Obama’s average annual spending increase from 1.4 percent to somewhere between 3.4 percent and 4.9 percent. That would place Obama either second from the bottom or third from the bottom out of the 10 presidents we rated, rather than last.

PolitiFact appears to say its calculations suggest that accepting the critics' points makes little difference. We'll see that isn't the case while also discovering a key criticism of the "annual spending increase" metric.

Reviewing PolitiFact's calculations from earlier in its original story, we see that PolitiFact averages Obama's spending using fiscal years 2010 through 2013. However, in this update PolitiFact apparently does not consider another key criticism of Nutting's method: He cherry picked future projections. Subtract $307 billion from the FY 2009 spending and the increase in FY 2010 ends up at 7.98 percent. And where then do we credit the $307 billion?

If we attribute that $140 billion in stimulus to Obama and not to Bush, we find that spending under Obama grew by about $200 billion over four years, amounting to a 1.4% annualized increase.

Neither does PolitiFact:

(C)ombining the fiscal 2009 costs for programs that are either clearly or arguably Obama’s -- the stimulus, the CHIP expansion, the incremental increase in appropriations over Bush’s level and TARP -- produces a shift from Bush to Obama of between $307 billion and $456 billion, based on the most reasonable estimates we’ve seen critics offer.

That’s quite a bit larger than Nutting’s $140 billion, but by our calculations, it would only raise Obama’s average annual spending increase from 1.4 percent to somewhere between 3.4 percent and 4.9 percent.

But where does the spending go once it is shifted? Obama's 2010? It makes a difference.

The graphic embedded to the right helps illustrate the distortion one can create using the average increase in spending as a key statistic. Nutting probably sought this type of distortion deliberately, and it's shameful for PolitiFact to overlook it.

Using an annual average for spending allows one to make much higher spending not look so bad. Have a look at the graphic to the right just to see what it's about, then come back and pick up the reading. We'll wait.

Boost spending 80 percent in your first year (A) and keep it steady thereafter and you'll average 20 percent over four years. Alternatively, boost spending 80 percent just in your final year (B) and you'll also average 20 percent per year. But in the first case you'll have spent far more money--$2,400 more over the course of four years.

It's very easy to obscure the amount of money spent by using a four-year average. In case A spending increased by a total of $3,200 over the baseline total. That's almost $800 more than the total derived from simply increasing spending 20 percent each year (C).

Note that in the chart each scenario features the same initial baseline (green bar), the same yearly average increase (red star), and widely differing total spending over the baseline (blue triangle).

Some of Nutting's conservative critics used combined spending over four-year periods to help refute his point. Given the potential distortion from using the average annual increase it's very easy to understand why. Comparing the averages for the four year total smooths out the misleading effects highlighted in the graphic.

We have no evidence that PolitiFact noted any of this potential for distorting the picture. The average percentage increase should work just fine, and it's simply coincidence that identical total increases in spending look considerably lower when the largest increase happens at the beginning (example A) than when it happens at the end (example B).

Shenanigan review:

Yearly average change metric masks early increases in spending

No mention of the effects of TARP negative spending

Improperly considers Obama's spending using future projections

Future projections were cherry-picked

The shift of FY 2009 spending from TARP, the stimulus and other initiatives may also belong on the above list, depending on where PolitiFact put the spending.

I have yet to finish my own evaluation of the spending comparisons, but what I have completed so far makes it appear that Romney may well be right about Obama accelerating spending faster than any president in recent history (at least back through Reagan). Looking just at percentages on a year-by-year basis instead of averaging them shows Obama's first two years allow him to challenge Reagan or George W. Bush as the biggest accelerator of federal spending in recent history. And that's using PolitiFact's $307 billion figure instead of the higher $456 billion one.

So much for PolitiFact helping us find the truth in politics.

Note:

I have a spreadsheet on which I am performing calculations to help clarify the issues surrounding federal spending and the Nutting/PolitiFact interpretations. I hope to produce an explanatory graphic or two in the near future based on the eventual numbers. Don't expect all the embedded comments on the sheet to make sense until I finalize it (taking down the "work in progress" portion of the title).

Jeff adds:

It's not often PolitiFact admits to the subjective nature of their system, but here we have a clear case of editorial judgement influencing the outcome of the "fact" check:

Our extensive consultations with budget analysts since our item was published convinces us that there’s no single "correct" way to divvy up fiscal 2009 spending, only a variety of plausible calculations.

This tells us that PolitiFact arbitrarily chose the "plausible calculation" that was very favorable to Obama in its original version of the story. By using other equally plausible methods, the rating would have gone down. By presenting this interpretation of the calculations as objective fact, PolitiFact misleads their readers into believing the debate is settled.

This update also contradicts PolitiFact's reasons for the "Mostly True" rating:

So the second portion of the Facebook claim -- that Obama’s spending has risen "slower than at any time in nearly 60 years" -- strikes us as Half True. Meanwhile, we would’ve given a True rating to the Facebook claim that Romney is wrong to say that spending under Obama has "accelerated at a pace without precedent in recent history." Even using the higher of the alternative measurements, at seven presidents had a higher average annual increases in spending. That balances out to our final rating of Mostly True.

In the update, they're telling readers a portion of the Facebook post is Half-True, while the other portion is True, which balances out to the final Mostly True rating. But that's not what they said in the first rating (bold emphasis added):

The only significant shortcoming of the graphic is that it fails to note that some of the restraint in spending was fueled by demands from congressional Republicans. On balance, we rate the claim Mostly True.

In the first rating, it's knocked down because it doesn't give enough credit to the GOP for restraining Obama. In the updated version of the "facts", it's knocked down because of a "balance" between two portions that are Half-True and completely True. There's no mention of how the GOP's efforts affected the rating in the update.

Their attempts to distance themselves from Nutting's widely debunked article are also comically dishonest:

The Facebook post does rely partly on Nutting’s work, and our item addresses that, but we did not simply give our seal of approval to everything Nutting wrote.

That's what PolitiFact is saying now. But in the original article PolitiFact was much more approving:

Neither Nutting or the original Groobiecat post claim Obama had the "lowest spending record". Both focused on the growth rate of spending. This spending record claim is PolitiFact's invention, one the fact check does not address. But it sure looks nice right next to the "Mostly True" graphic, doesn't it? Sorting out the truth, indeed.

The bottom line is PolitiFact's CYA is hopelessly flawed, and offensive to anyone that is sincerely concerned with the truth. A fact checker's job is to illuminate the facts. PolitiFact's efforts here only obfuscate them.

Bryan adds:

Great points by Jeff across the board. The original fact check was indefensible and the other fact checks of Nutting by the mainstream media probably did not go far enough in calling Nutting onto the carpet. PolitiFact's attempts to glamorize this pig are deeply shameful.

Correction 6/4/2012: Corrected one instance in which PolitiFact's $307 billion figure was incorrectly given as $317 billion. Also changed the wording in a couple of spots to eliminate redundancy and improve clarity, respectively.