Salary Transparency at Stack Overflow

We believe (and developers tell us) that job seekers should be empowered with as much information as possible when looking for a job – especially salary. So we ran an experiment on Stack Overflow Jobs to see if the evidence would support it.

Remarkably, we learned that job listings which include a salary range got 75% more clicks than job listings that don’t. With this experiment, we’re even more convinced that transparency isn’t just the right thing to do, it’s good for companies, too.

Along with much of the tech community, we were impressed with Buffer’s boldness and leadership in salary transparency. So…

Introducing the Stack Overflow salary calculator

We’ve created a salary and skills calculator for Stack Overflow’s engineering, design and product roles. This has been transparent internally for a while; now it’s transparent with you.

Those who know Stack Overflow know that we work hard to work in public. This is a continuation of that tradition.

What we hope

We hope that moves like this will inspire other employers to greater transparency.

A lack of transparency is what economists call an information asymmetry: it’s in companies’ interests to keep these numbers close to the vest. Individuals are uncomfortable talking about salary sometimes, too.

We believe that conventions can change. If more companies become open on salary, perhaps openness will become expected.

Work in progress

Our salary calculator doesn’t cover every role at Stack Overflow. It doesn’t include equity, and only describes US salaries. (International employees use the same system but it’s not merely a currency conversion.) In the spirit of “default public”, we would rather share an incomplete system than not share at all.

The experimental details: 75% more clicks

For our experiment, we redesigned the Stack Overflow Jobs ads to display salary ranges. We were curious: just how much effect does this information have?

We ran an A/B test, where for a random half of our users we hid the salary information from the ads they were shown, and measured the difference in clickthrough rate. Visually, it looked like this:

We expected to see an improvement, but we were surprised by the size: a 75% average increase in clickthrough rate (CTR) when we showed a job’s salary range.

What if a job has a relatively low salary – is it still worth showing? Generally speaking, yes: we found that showing any salary range led to an increase in CTR, though higher salaries led to a greater bump. For American jobs, we saw roughly a 60% increase for jobs with salary ranges centered below $100K, and about a 100% increase (doubling) for salaries above $100K.

Is this just an effect of novelty, where users were surprised to see salary? Unlikely – we didn’t see any decline in the effect, and it has been consistent in the months since. We’ve tried many other changes to ads and have never seen anything this dramatic.

Clickthrough rate isn’t everything, but it’s an encouraging sign that advertising a salary range will help draw developers to a position.

Further reading

Author

Matt Sherman

Engineering Manager

Matt speaks on engineering and engineering management. He specializes in recruiting, getting developers talking to salespeople, and the Go programming language. He is also the founder of Alikewise, a dating site based on books.

Not surprised by this result. When you’re looking, you need to know if you’re looking at jobs in the right ballpark. There’s no way to know this until they tell you. The scenario involving it being hidden results in you not knowing until way down the line in the interview process which is a potential waste of time both for prospective employer and employee alike. Being up front about it saves everyone the hassle.

The A/B test is interesting and relatively convincing. If I were evaluating this, I’d like to see a couple of other variants to get a better measure of the true effect.

First, make the ads the same size whether or not the salary is displayed. The most obvious way would be to just leave whitespace where the salary would be shown. (There is undoubtedly some UI effect from just making the ad more noticeable by increasing the size alone.)

Second, have an additional treatment where you keep the text color for the salary the same as the gray used for the company name and location. (The green is also a UI effect that will attract the eye regardless of what you put there, as will going from 5 figure salaries to 6, as the plot also seems to intimate.) Third, stratify users by some coarse geolocation data. Is the effect larger for higher salary ads when users are from areas not in the same geo region as the ad, but from one with a lower cost of living?

Rudy Kroska’s comment is also one to pay attention to. There is, undoubtedly, a curiosity factor at play: “What does it take to make $120K+?” By giving the salary range upfront, you might reduce the quality of the traffic (e.g., in terms of the advertiser finding suitable candidates for every n resumes submitted).

Second that. Another interesting point is what happens when the salary is low or high compared with benchmarks for the jobs offered instead of merely in absolute numbers. An Angular.JS & Rails engineer in New York at 85 to 120 K may be lower or higher than expected for that job in that area. Comparing with external data from benchmarks could establish whether those numbers are average, below-33 percentile or above-66 percentile. Would love to see that graph, too. Otherwise, great post.

Listing the salary range saves us both time. I can filter out jobs I don’t want, because of low salary, or jobs I’m not qualified for, based on high salary. That way we don’t waste time just to find out that I don’t want an entry level position after being in the field for 10 years. If a salary is not listed I just assume it’s because it’s not competitive.

Agreed – Expecting someone to go through an entire recruiting process without a salary range will end up wasting time for both the company and the candidate if the salary doesn’t meet your minimum expectations.

Why? As a candidate applying for a job, you should have a figure or range you are happy with. If a company doesn’t have a salary range, but your expectation is above what the company is willing to pay, then the process stops there and no one’s time is wasted.

I always like to see salaries when considering job ads. It gives me a better idea of what level of experience and qualification they are expecting, whether it is worth me considering them, and whether they would be likely to consider me. It also reassures me that it won’t be an onerous negotiation task if I am offered the job.

I’m a developer and If I see a job advertised without a salary or I am contacted by a recruiter about a position and salary range is not provided, I will more than likely not have any interest in it unless the job description sounds amazing and then I might consider asking what the salary is.