The author is a Forbes contributor. The opinions expressed are those of the writer.

Loading ...

Loading ...

This story appears in the {{article.article.magazine.pretty_date}} issue of {{article.article.magazine.pubName}}. Subscribe

A recent study from the job search website Bright.com had some pretty serious allegations in it. They claim that working in the fast food industry and other low-wage retailers can permanently harm you on the job market. The methodology, however, is fatally flawed. I'll go into the details below, but suffice it to say that you can't learn anything like what they claim from their analysis.

The basic point of the study is that if you work for one of these low-wage employers it means you have almost no change of getting a good paying job at any time in the future. What Bright.com did was look at their database of 8 million resumes and find anyone whose current or previous position was at a handful of low-wage companies. In particular, they focused on companies on the National Employment Law Project's list of "50 largest low-wage employers". They then look at the salaries that those people go on to earn, and show that for some of these companies the percent who earn $70,000 or more "ever in their career" is very low.

The biggest problem with this is that there is an obvious massive selection bias. You're not looking at people who have ever worked at , Walmart, Wendy, and so on, but people who are still listing them as the current or most recent position. Like many people, I worked at low-skilled jobs when I was a teenager, including a grocery store, a carwash, and a video rental store. But do I put these on my resume still? Of course not, I have more relevant and recent positions on there. In fact if you did observe a man in his early 30s with a Phd in economics who was putting his high-school entry-level jobs on his resume it would tell you something awfully worrrying. The best case scenario is that I was locked in an institution of some sort for a decade or more, like a prison or mental hospital.

I emailed the author of the study, who is also a Phd, to ask whether he had a high school job and whether he still put it on his resume. He didn't answer the question, and instead tried to claim the study didn't imply causality. Unfortunately, despite this obviously fatally flawed approach, the study clearly does imply causality. You can see it right in the headline of their report which says "Working for these companies could hurt your earning potential forever". Ok, well maybe someone else put that headline on it. This happens sometimes. But causality is again implied in the very first sentence, which says "A new analysis by Bright reveals that working in the fast food industry could permanently damage your health — your financial health." Sorry, but this does not even go with the usual shoddy science weasel words of "linked to", but uses the crystal clear phrase "permanently damage".

Since this study gives readers what they want in bashing the always unpopular low-wage employes, it is of course a popular read. Even where people have tried to interpret these findings cautiously they are failing to grasp the flaws. Lydia DePillis for example tried to inject some skepticism to the coverage of this story at Wonkblog, writing:

Now, there's no reason to believe that working low-wage jobs necessarily torpedoes your chances of earning a decent living down the road. It's more likely that the collection of circumstances that propel people into those jobs -- lack of education, geographic mobility, or better jobs -- continue to hold them back for the rest of their lives.

While this is a good start, the problems with this study requires far more than the usual "correlation is not causality" warning, but instead has a really bad sample selection that means the observed correlation is not even real. So DePillis is incorrect when she closes with:

The data also calls into question the validity of what we usually consider to be "entry-level" jobs. If a job isn't an entry into a career that moves in a more prosperous direction, it's unclear what real value it really has.

No, the data does not call into question anything about entry level jobs, because the massive selection bias inherently does not tell you about the average career paths of people who start with these jobs. Entry level jobs are important, and it's bad to be scaring people away from them with shoddy studies.

This is a badly done study, and attempts at skepticism failed to convey how deeply flawed it was. So my message to Bright.com is this: it sounds like you have some neat data; if you would like to pay someone to pull interesting and credible social science tidbits out of it please email me next time.