Calibration for the (computationally-identifiable) masses

March 29, 2018

4:30 p.m. - 6:00 p.m.

Doherty Hall A302

Dr. Omer ReingoldStanford University

Abstract

As algorithms increasingly inform and influence decisions made about individuals, it becomes increasingly important to address concerns that these algorithms might be discriminatory. The output of an algorithm can be discriminatory for many reasons, most notably: (1) the data used to train the algorithm might be biased (in various ways) to favor certain populations over others; (2) the analysis of this training data might inadvertently or maliciously introduce biases that are not borne out in the data. This work focuses on the latter concern.

We develop and study multicalbration as a new measure of algorithmic fairness that aims to mitigate concerns about discrimination that is introduced in the process of learning a predictor from data. Multicalibration guarantees accurate (calibrated) predictions for every subpopulation that can be identified within a specified class of computations. We think of the class as being quite rich, in particular it can contain many and overlapping subgroups of a protected group.

We show that in many settings this strong notion of protection from discrimination is both attainable and aligned with the goal of obtaining accurate predictions. Along the way, we present new algorithms for learning a multicalibrated predictor, study the computational complexity of this task, and draw new connections to computational learning models such as agnostic learning.

Bio

Omer Reingold is a Professor of Computer Science at Stanford University. Past positions include Samsung Research America, the Weizmann Institute of Science, Microsoft Research, the Institute for Advanced Study in Princeton, NJ and AT&T Labs. His research is in the Foundations of Computer Science and most notably in Computational Complexity and the Foundations of Cryptography with emphasis on randomness, derandomization and explicit combinatorial constructions. He has a keen interest in the societal impact of computation. He is an ACM Fellow and among his distinctions are the 2005 Grace Murray Hopper Award and the 2009 Gödel Prize.

“There is very little success where there is little laughter.” Andrew Carnegie

“I think great artists and great engineers are similar, in that they both have a desire to express themselves. ” Steve Jobs

“We are what we repeatedly do. Excellence, then, is not an act, but a habit. ” Will Durant

“I hear and I forget. I see and I remember. I do and I understand” Chinese Proverb

“Education is the most powerful weapon we can use to change the world.” Nelson Mandela

“Design is not just what it looks like and feels like. Design is how it works.” Steve Jobs

“There's nothing I believe in more strongly than getting young people interested in science and engineering, for a better tomorrow, for all humankind.” Bill Nye

“At the heart of science is an essential balance between two seemingly contradictory attitudes—an openness to new ideas, no matter how bizarre or counterintuitive they may be, and the most ruthless skeptical scrutiny of all ideas, old and new.” Carl Sagan

“Creativity...involves the power to originate, to break away from the existing ways of looking at things, to move freely in the realm of imagination, to create and recreate worlds fully in one's mind—while supervising all this with a critical inner eye.” Oliver Sacks

“I just believe that the way that young people's minds develop is fascinating. If you are doing something for a grade or salary or a reward, it doesn't have as much meaning as creating something for yourself and your own life.” Steve Wozniak