About

Precision versus Recall

Precision -> of the number you thought were positive.... what fraction are actually positive. (think: what fraction of darts hit targets)

True Positives / (True Positives + False Positives)

Recall -> what fraction of cats were predicted as cats? (think: how many targets were hit)

True Positives / (True Positives + False Negatives)

If you just blindly say everything is a cat we get 100% recall, but really low precision (a tonne of non-cat photos we said were cat). Machine learning papers often show a "precision recall curve" which shows that as you adjust a threshold to guess more "cats".... if we only pick 99.9% confidence of a cat, it's probably almost always right (100% recision), but terrible recall. But as we go down to 80% sure of a cat, we get better recall (we capture a higher fraction of cats), but the trade off is that we're probably selecting more and more non-cats.

Said Another Way

Your boss asks:

What percent of your predictions were correct? "accuracy"

What percent of the positive cases did you catch? "recall"

What percent of positive predictions were correct? "precision"

Table of Confusion

A table of confusion (sometimes also called a confusion matrix), is a table with two rows and two columns that reports the number of false positives, false negatives, true positives, and true negatives.