Researchers have realized that humans are the “last mile” to designing secure systems; technically secure systems may still be exploited if users behave in unsafe ways. Thus, researchers have begun studying human behavior as it relates to privacy and security decisions to design more user-centric systems. These research studies generally fall into two categories: controlled laboratory experiments and large-scale measurements in the field. The former allows researchers to directly ask participants questions and observe how they interact with various types of security mitigations, all while controlling the environment to eliminate confounding factors. The latter allows researchers to better estimate attack rates and understand how users behave in their natural environments. However, both methods suffer from shortcomings: laboratory experiments do not take place in users' natural environments and therefore may not accurately capture real world behaviors (i.e., low ecological validity), whereas large-scale measurement studies do not allow the researchers to probe user intent or otherwise gather explanatory data for observed behaviors, and offer limited control for confounding factors.

We fill this gap in the literature through the Security Behavior Observatory (SBO), a panel of participants consenting to our observing their daily computing behavior, so that we can understand what constitutes “insecure” behavior. On a technical level, the SBO consists of a set of “sensors” monitoring various aspects of participants’ operating system and applications (e.g., browser, network traffic, file system), which report a comprehensive overview of user activity to a secure server. The SBO has data from over 500 users, with about 200 users sending their data at any given time.

We have used the SBO to study a variety of security and privacy behaviors including software updates and computer maintenance, susceptibility to phishing attacks, password reuse, malware infections, and use of private browsing modes.