Government on the scale, cont'd

By
Ezra Klein

A reader who spends a lot of time delivering metrics to the government was not happy to hear Sen. Kent Conrad and others saying that there are no metrics or evaluation components behind most government programs, and in particular for job-training programs. His e-mail follows. I'm interested in this subject, by the way, so if anyone else wants to write in on it, feel free:

I am a grant-writer and program manager at a nonprofit that receives many federal grants. WE LIVE AND DIE BY METRICS! Almost to the point, in my opinion, of spending too much money for compliance and performance-measurement systems, and not enough money on the actual training. In rough terms, we hire one full-time compliance and performance staff person for every 10 staff who actually work with people. All of the federal job training programs have performance metrics. In fact, many have started using the so-called Common Measures, a simplified, standardized system for measuring outcomes across different agencies. The three Common Measures for adult training programs are:

- Percent placed into employment.
- Percent retained in employment for six months.
- Earnings change at six months.

For youth programs, the Common Measures are:

- Placement in employment or higher education.
- Attainment of a degree or an industry-recognized credential.
- Increase in reading and/or math scores among youth who have skill deficiencies.

Different agencies use additional measures, and yes, those get complicated. And I'm not saying these systems are perfect. But the federal government has become increasingly better, ever since the passage of GPRA, at using metrics. It drives me a little nuts to spend so much time out here in the field collecting, analyzing, and reporting performance metrics, and then to have a politician claim that "there were almost no metrics on any of them." No, simply not true. They're just ignorant of the data that is available. As one example, check out:

www.doleta.gov/Performance

One last note, in case you really want to dive into this issue: In some programs, future funding gets tied to performance, creating an unhealthy obsession with the metrics. For example, programs start "creaming," or accepting only clients that are somewhat more likely to succeed. Because the metrics are not weighted to encourage job training for, say, high school dropouts, programs have a very hard time serving that population. They want to help the high school dropout, but they know the chances of success are so low, and their jobs depend on meeting these metrics, so they opt to enroll instead an unemployed person who at least has a high school diploma. This is inside baseball, as Obama would say, but that's the real struggle in the field. Grantees like my organization are trying to serve people who are most in need, and we're caught between these daunting skill deficiencies in our clients and the rigorous performance standards we must meet in order to keep our jobs. For someone like Conrad to claim that there are no metrics -- he misses the point entirely.

Not clear to me why the performance metrics wouldn't be able to handle high school dropouts.

You would expect that the metric's baeline would utilize outcome expectations based on the demographics of the individuals served;

which is to say, if on average 40% of dropouts get re-employment and 85% of high school graduates within 6 months, re-employing 60% of dropouts should be considered more successful than getting 80% of grads. 'Creaming' should still be expected, but not on easily quantified demographic aspects.

There could well be a difference between the metric requirements for groups that receive federal grants and programs undertaken by federal agencies directly. Was Conrad referring to outside job training programs?

No metrics??? I would expect someone writing a column on the federal government to at least be familiar with the basics. The Government Performance and Results Act (GPRA), signed in 1994, requires five year strategic plans, annual performance plans and annual performance reports based on quantafiable measurements. President Bush linked GPRA to a Program Assesment Rating Tool (PART) as one component of an effort to examine and grade every program in government on a regular basis. Obama did away with PART but continues to complie with GPRA. Anyone, including Senators, who tries to maintain the federal government does not evaluate programs based on quantifiable metrics just does not know what they are talking about.

1) yet another big-government progressive policy has negative unintended consequences, in that some of the intended beneficieries are ignored as agencies game the metric-reporting system, and

2) this sounds like a wonderful job creation opportunity, using progressive logic. If 1 full-time compliance position is created for every 10 actual staff positions working with people, then we should increase the federal budget 10-fold!!! This would erase the 10% unemployment rate, everybody would have a job, and progressive-utopia would arrive at last. Oh, Happy Day!!!

There is also an extensive evaluation process that OMB uses to gauge IT investments by agencies througout the government. The form submitted by agencies for their programs is called the Exhibit 300 -- a couple of relevant links are

I love talking about federal grants and contracts with conservatives. There's outrage - OUTRAGE - that our taxpayer dollars go to nonprofit companies that compete for contracts and then strenuously report stats and metrics quarterly to ensure the contract terms are being met. They think nonprofits and social service agencies should be able to solicit donations to pay for their operations, and if they can't solicit these donations independently, then obviously their missions are not worthy and their agencies should cease to exist.

The question then becomes, what about the people who need job training? What about people who need help finding housing? What about the poor, hungry, sick, old, weak, etc.? The answer appears to be some mix of up-by-the-bootstraps 80s-era rhetoric or die-cold-and-hungry-on-the-streets 1800s-era reality.

Presumably "job training" is associated with some job somewhere. Why should the government (using money stolen from the citizenry) subsidize the hiring expenses of business? If a business needs a welder that it can't find, badly enough, won't the business train a welder on its own?

I use the "welder" example because my little local government is trying to use taxpayer monies to set up a welding school to train welders for the TVA (a government chartered monopoly), while up in Ohio, there are government trained welders sitting around looking for jobs (and there is no guarantee that the TVA will want any of the locally trained welders.)

I was unaware that "creaming" of good applicants takes place in Federal Job training programs but it makes sense.

The Armed Forces already has tighted its "creaming" methodologies in avoiding high school dropouts as applicants. So why are we surprised that federal job programs are doing the same thing?

Creaming or selecting the best possible candidates makes perfect sense in the corporate world. But culling out high school dropouts from federal programs for "better analytics" makes poor public policy.

If high school dropouts had a political voice I'm sure they wouldn't enjoy the concept of "creaming" in federal programs.

My knowledge of high school dropouts and their concerns comes from my website at http://onlinegedsite.com/ .

Here's hoping politicians do the math and realize that each high school dropout costs society approximately $300,000 during their lifetimes (they pay less taxes, are more apt to be on welfare, tend to be less healthy, have a higher probability of going to jail).

We encourage users to analyze, comment on and even challenge washingtonpost.com's articles, blogs, reviews and multimedia features.

User reviews and comments that include profanity or personal attacks or other inappropriate comments or material will be removed from the site. Additionally, entries that are unsigned or contain "signatures" by someone other than the actual author will be removed. Finally, we will take steps to block users who violate any of our posting standards, terms of use or privacy policies or any other policies governing this site. Please review the full rules governing commentaries and discussions.