01CH740: Experiment Design in Computer Sciences

Summary:

The field of computer sciences unfortunately does not yet have a culture
that requires new discoveries to be backed by rigorous experimentation,
as is the norm in physics and the biological sciences. It is very common
to see CS papers with biased comparisons, incomplete scientific
evidence, unreproducible results or simply incorrect statistical
analysis of experimental data. I believe that most of these errors are
not due to malice, but to the lack of teaching computer scientists the
merits of rigorous experimental methodologies.

In this course we will review the concept of experimentation, and what
is its role in Computer Science research. We will discuss how to
properly plan and setup a scientific experiment, and how to correctly
analyze the data generated by it. Topics include statistical techniques,
experiment design procedures and the reasoning behind them.