Simulating the Secretary Problem with Java

You might have noticed that I like reading books. I have recently read “Algorithms to Live By: The Computer Science of Human Decisions” which absolutely fascinated me! The book mentions a famous optimal stopping (Wikipedia) problem called Secretary Problem. In this blog post I will explain it and then we will have some fun simulating it with Java. Let’s see if we can find a solution by brute force!

Secretary Problem Defined

Imagine that you need to hire a secretary. Imagine now that you have 100 candidates that you are going to interview. Because you are a perfect interviewer, you can compare every single person against everyone else that you have seen so far. After the interview, you have to either hire the person or reject. If you reject, you can’t change your mind. You win if you have managed to hire the best candidate out of the whole lot.

The applicants, if seen altogether, can be ranked from best to worst unambiguously.

The applicants are interviewed sequentially in random order, with each order being equally likely.

Immediately after an interview, the interviewed applicant is either accepted or rejected, and the decision is irrevocable.

The decision to accept or reject an applicant can be based only on the

relative ranks of the applicants interviewed so far.

The objective of the general solution is to have the highest probability of selecting the best applicant of the whole group. This is the same as maximizing the expected payoff, with payoff defined to be one for the best applicant and zero otherwise.

Why is this fascinating? Because it is not trivial and not too dissimilar from other decisions we make in life! Buying a house, finding a life-partner, staying in your career… All of these can be viewed as optimal stopping problems, or even as a variation of the Secretary Problem.

Simulating the Secretary Problem with Java

I decided to have some fun and model the situation in Java. I will start by creating a SecretaryProblem class:

Now, I know that the optimal solution to this is to keep interviewing candidates, skipping everyone and at some point make a decision that we are ready to commit. Once we are ready, we will hire the next person that is better than everyone seen so far. This is implemented in my SecretaryProblemSolver class:

Secretary Problem with Java – the results

After tunning the code for 100 different commit-point, I came up with the following success rates:

The highest value is found at 37 – 0.37243

I have found that committing after the 37th candidate works best and gives us about a 37% chance of finding the best candidate!

What does the math say?

Secretary problem is solved and well understood. We know that we should always commit after seeing about 37% of candidates and that this would give us about a 37% chance of success! Great to see that our little Java brute-force confirms that (it means we did it right).

Why 37%? Well… 1/e equals about 0.367879. Why is this the magical stopping point? If you have stomach for some hardcore mathematics, I refer you to Wikipedia here!

It is worth noting that this ~37% rule for stopping point and the rate of success holds for any sizes of candidate pool- be it 100 or 1000000!

The really fascinating thing is that our adventure with the secretary problem does not have to end here. We can easily modify the problem and look at how this impacts the solution and resulting curve. Modeling can often reveal insights that can take longer to discover with a strictly mathematical approach.

Summary

I hope you enjoyed this little write up on the Secretary Problem. I am planning to return to it in the future and see how we can model modified versions of the problem.

In the meantime, if you like problems like that, I recommend you to check out “Algorithms to Live By: The Computer Science of Human Decisions” which is so far my favorite book I read this year! I will definitely be borrowing more from it and this is not the last attempt at modeling that you will see on this blog.

Post navigation

Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
To find out more, including how to control cookies, see here:
Cookie Policy

About

E4developer is a place where I share my open and honest views on software development, technology and working with people. The name – e4 comes from a chess move, this is how I start most of my games. Follow me on twitter – @e4developer