Engineering

First 100 Days of Yelp's Public Bug Bounty Program

Martin Georgiev, Software Engineer

Dec 20, 2016

One hundred days ago we launched
Yelp’s public bug bounty program on HackerOne.
Since launching the program, we received over 564 reports from 512 reporters.
The distribution of the reports was as follows:

Resolved: ~ 7%

Informative: ~ 36%

Duplicate: ~ 31%

Not Applicable: ~ 26%

Looking back on the first 100 days of our program, we fixed 39 vulnerabilities
and paid out $13,850 in rewards. We maintained less than 24 hours response time
and less than 1 month resolution time. The distribution of bug-bounty payouts
over time is shown in Chart 1.

Chart 1: Distribution of bug-bounty payouts over time

Chart 2 shows the distribution of “New” reports over time. The large volume of
“New” reports in the first few days after the public launch is (anecdotally)
consistent with what other companies experience with their public bug bounty
programs. For instance, in the first week after the launch we received 239
out of the 361 reports received in September.

Chart 2: Distribution of new reports over time

Below, we list the top three (by impact and category) bug reports from the past
100 days. All issues reported here have been fixed. The fixes were verified
both internally and by the respective bug reporter.

The 0x02 List

0x00

Background: Users on Yelp can write reviews, upload images, engage in
discussions, etc. Further, they can receive compliments in one of several
categories (e.g., “Good Writer”, “Great Photo”, “Like Your Profile”, etc).
Internally, each compliment has an id and a type.

Vulnerability:@mlitchfield
analyzed this functionality of Yelp and reported a Denial of Service
vulnerability at the application layer. The vulnerability stemmed from lack of
proper error handling when checking compliment ids. Effectively, an attacker
could crawl Yelp, collect user ids and then issue compliments to those users
referencing a malformed compliment id. When the recipient of a compliment logs
in, the server tries to process and notify the user of all newly received
compliments since the user’s last login. However, if the server encounters a
malformed compliment id, it throws an error. The error was not handled properly
and hence bubbled up as an HTTP 500.

0x01

Background: Yelpers can check in and broadcast their location whenever they
visit a local business. Users can subscribe to receive check-in notifications
from their friends. Two users are considered friends on Yelp when they both
confirm their friendship via the app.

Vulnerability:@vinesh1989
analyzed the Friend check-in notification feature and found that the friendship
was not properly verified. A malicious user could subscribe for and track
check-in notifications from anyone, not just his friends.

0x02

Vulnerability:@insomniac
thoroughly tested this feature and discovered that logged-in sessions and
password reset tokens do not expire even after the email used for the login is
removed from the list of authorized emails. An attacker, who could obtain
access to a victim’s account via some means (e.g., managed to guess the
victim’s password), could issue herself a password reset token. Even if the
victim were to change his password, the attacker could use the previously
generate password reset token to take over the victim’s account.

Private vs. Public Bug Bounty Program

Prior to launching Yelp’s public bug bounty program, we ran a private bug
bounty program with a small number of invite-only bug reporters for
approximately 2 years. The private program helped us both handle rate of “New”
reports (e.g., by controlling the number of hackers in our program) and
allocate enough time to systematically fix common vulnerabilities (e.g., XSS).

Table 1 shows a comparison of our private bug bounty program to the first 100
days of our public bug bounty program.

The public bug bounty program has resulted in a significant increase in bug
reports. Yet, our stats show that bugs have gotten harder to find and exploit.

Recommendations

Based on our experience running bug bounty programs, we recommend that
companies leverage private bug bounty programs before launching any public
ones.

Private programs are ideal for “testing the waters”. They help bring the
Engineering and Security teams together and get them used to validating
and systematically fixing new issues. The most common classes of security
vulnerabilities are often found and fixed through private bug bounty programs.

Public programs (when pipelined after private programs) are great for finding
remaining security loopholes. We suggest that companies open their
private programs to the public only when they can devote sufficient engineering
resources to resolving issues found. They should also be prepared for the high
volume of reports in the first few days of the public program launch.

Conclusion

The first 100 days of Yelp’s public bug bounty program have been a great
success. We collaborated with hundreds of bug hunters on HackerOne and as a
result have made significant improvements in our bot detection, API-abuse
prevention, spam identification, and suspicious user-activity detection.
When compared to our private program, the public bug bounty program shows that
high-severity security and privacy vulnerabilities have become harder to find.
We attribute these favorable results to the issues addressed during our private
program.

We’d like to thank all bug reporters for their participation in our program and
helping us improve the security of our sites, apps and infrastructure. We look
forward to continuing our collaboration with the security community and
ensuring the security of Yelp’s users.