Print to PDF

07/24/2012

The Oasis of Survey Information Shrinks Some More

The bad news is there are fewer pay surveys for reliable guidance. The good news is there are fewer pay surveys to complete. Last year, I mourned the disappearance of surveys and whined about how few are still standing. A quick glance at that list shows that some of those few remaining survivors are now already gone, discontinued or absorbed by competitors. Independent sources of salary survey information continue to die out. In addition, the situation has worsened. There is a new and disturbing trend towards flashy internet style truthy analytics replacing old-fashioned rigorous real surveys with actual observations by verifiable SME sources. Academic theories or public restroom wall postsI mean unreliable internet offerings that can’t meet court testimony standards or DOJ/FTC safe harbor rules may displace actual pay surveys.

Although estimates can be justified when real data is missing, actual surveys are being dropped and replaced by ersatz imitations. This is a worrisome development.

Two of the biggest and most notable national pay surveys in America will be converted into economic models: NACE and BLS NCS.

NACE has decided to discontinue its survey of employment offers from college placement departments and campus recruiters. From 2012 forward, they will simply update BLS and Census data via a secret algorithm.

Reasons are easily discovered. Actual surveys are difficult, time-consuming and expensive… and they can reveal uncomfortable truths that undercut your business model. In light of the negative results shown by more granular, precise studies of college placements, institutions of higher learning may have a vested interest in obscuring or even distorting the real picture that faces graduates.

So, how do you hide bad information? You don’t report it. You change your “surveys” into “estimates,” replacing actual observations that can be subjected to regression analysis to determine normative current patterns into projections from historical data that can be manipulated to say what you want them to say. Instead of reporting the standard deviation of actual observations from the average, you construct a pretty pixelated picture drawn from the rosy past and project it into the future. Without reliability statistics, it may not be precise or even accurate, but it can be made to appear convincingly truthy.

This shell game is not isolated to Higher Education (one of the biggest businesses in America). Government employs even more people than Education, and they are doing the same thing with the survey that controls federal pay. The BLS NCS survey is likewise being altered into an analytic model less dependent on (or responsive to) actual data. Judging from the explanation, it will be a mess that only an academic labor economist could love. Job evaluation via the existing FES, supplemented by an additional new job family grading system, will be used to make guesses about as much as 81% of the modeled data. Do you see smoke? Notice the flash of mirrors?

Granted, it is hard and expensive to conduct real surveys according to rigorous statistical reliability procedures. Accordingly, OPM endorses this path as ”being less costly, and being less burdensome on respondents.” It will be applied to set federal pay. No comment on that (not enough room here).

Bottom line: if you do not participate in as many wage and salary surveys as you can, they may be displaced by an economic model that imitates the reality. Those who pass up surveys and do not choose to give feedback that will influence their own destinies will find plenty of others willing to do it for them… or to them.

What do you think?

E. James (Jim) Brennan is Senior Associate of ERI Economic Research Institute, the premier publisher of interactive pay and living-cost surveys. Semi-retired after over 40 years in HR corporate and consulting roles throughout the U.S. and Canada, he’s pretty much been there done that (articles, books, speeches, seminars, radio/TV, advisory posts, in-trial expert witness stuff, etc.) and will express his opinion on almost anything.

Comments

You can follow this conversation by subscribing to the comment feed for this post.

Jim - Perhaps the diminishing salary survey data is due to the changing workplace environment, where job descriptions, if they exist at all, tend to be short lived. The combination of challenging economies and rapid change may have created a 'what the market will bear' pay philosophy, and being 'competitive' is whatever someone says (thinks) it is. Frankly, many organizations don't have a pay philosophy to guide their reward practices, which makes survey data less meaningful. Moreover, the speed with which business models are changing require current, up to date information, and traditional salary surveys can't compete in that environment. The internet is quick, easy and mostly free, so despite the questionable veracity of the data, many will use it simply because its there.

All of these dynamics will require comp professionals to take a fresh look at how they manage their pay systems. To be relevant they must respond to the business and employees they support, and employ highly creative practices to address the compensation needs of their current and future workforce.

John makes a good point about traditional salary surveys not being updated quickly enough to match the demand for new data (especially in the uber-competitive tech space). Certainly this requires HR and Comp professionals to think on their feet about how to supplement mid-year as business situations change and/or new job families are created/evolve.

By the same token, I'm a firm believer that the traditional surveys are STILL the backbone any solid compensation program. There are times when I can’t buy every survey on my wish list. When that happens I still make a concerted effort to supply my data, however time-consuming. Obviously this helps maintain the health of the overall survey database… and it allows me to claim a discount later in the year if it turns out there’s some extra money in my budget. Some surveys are pleased enough with this dedication to offer a free cut of data in exchange for my efforts. (This doesn’t happen often, but it’s always appreciated.)

John and Windsor both make good valid points. Yes, there are many contributing causes behind the decline of credible surveys, but that simply places more weight on those few still surviving. "Free" open-sourced data have severe limitations, because anonymous numbers cannot withstand any serious challenge. Furthermore, remember the old saying: "there is no good price for bad data."

I thought that EVERY survey that solicits participation offers a participant price discount. But I guess there are still some around that don't.