Computer Science Department Technical Reportshttp://hdl.handle.net/1903/22252017-09-26T21:33:54Z2017-09-26T21:33:54ZA Summary of Survey Methodology Best Practices for Security and Privacy ResearchersRedmiles, Elissa M.Acar, YaseminFahl, SaschaMazurek, Michelle L.http://hdl.handle.net/1903/192272017-06-30T03:38:34Z2017-05-03T00:00:00ZA Summary of Survey Methodology Best Practices for Security and Privacy Researchers
Redmiles, Elissa M.; Acar, Yasemin; Fahl, Sascha; Mazurek, Michelle L.
"Given a choice between dancing pigs and security, users will pick dancing pigs every time," warns an oft-cited quote from well-known security researcher Bruce Schneier. This issue of understanding how to make security tools and mechanisms work better for humans (often categorized as usability, broadly construed) has become increasingly important over the past 17 years, as illustrated by the growing body of research. Usable security and privacy research has improved our understanding of how to help users stay safe from phishing attacks, and control access to their accounts, as just three examples.
One key technique for understanding and improving how human decision making affects security is the gathering of self-reported data from users. This data is typically gathered via survey and interview studies, and serves to inform the broader security and privacy community about user needs, behaviors, and beliefs. The quality of this data, and the validity of subsequent research results, depends on the choices researchers make when designing their experiments. Contained here is a set of essential guidelines for conducting self-report usability studies distilled from prior work in survey methodology and related fields. Other fields that rely on self-report data, such as the health and social sciences, have established guidelines and recommendations for collecting high quality self-report data.
2017-05-03T00:00:00ZHow Well Do My Results Generalize? Comparing Security and Privacy Survey Results from MTurk and Web Panels to the U.S.Redmiles, Elissa M.Kross, SeanPradhan, AlishaMazurek, Michelle L.http://hdl.handle.net/1903/191642017-06-30T03:35:19Z2017-02-21T00:00:00ZHow Well Do My Results Generalize? Comparing Security and Privacy Survey Results from MTurk and Web Panels to the U.S.
Redmiles, Elissa M.; Kross, Sean; Pradhan, Alisha; Mazurek, Michelle L.
Security and privacy researchers often rely on data collected from Amazon Mechanical Turk (MTurk) to evaluate security tools, to understand users' privacy preferences, to measure online behavior, and for other studies. While the demographics of MTurk are broader than some other options, researchers have also recently begun to use census-representative web-panels to sample respondents with more representative demographics. Yet, we know little about whether security and privacy results from either of these data sources generalize to a broader population.
In this paper, we compare the results of a survey about security and privacy knowledge, experiences, advice, and internet behavior distributed using MTurk (n=480), a nearly census-representative web-panel (n=428), and a probabilistic telephone sample (n=3,000) statistically weighted to be accurate within 2.7% of the true prevalence in the U.S. Surprisingly, we find that MTurk responses are slightly more representative of the U.S. population than are responses from the census-representative panel, except for users who hold no more than a high-school diploma or who are 50 years of age or older. Further, we find that statistical weighting of MTurk responses to balance demographics does not significantly improve generalizability. This leads us to hypothesize that differences between MTurkers and the general public are due not to demographics, but to differences in factors such as internet skill. Overall, our findings offer tempered encouragement for researchers using MTurk samples and enhance our ability to appropriately contextualize and interpret the results of crowdsourced security and privacy research.
2017-02-21T00:00:00ZA Comparison of Transfer Learning Algorithms for Defect and Vulnerability DetectionWebster, Ashtonhttp://hdl.handle.net/1903/191622017-06-30T03:34:53Z2017-02-08T00:00:00ZA Comparison of Transfer Learning Algorithms for Defect and Vulnerability Detection
Webster, Ashton
Machine learning techniques for defect and vulnerability detection have
the potential to quickly direct developers' attention to software
components with faulty implementations. Effective application of such
defect prediction methods in practical software development environments
requires transfer learning algorithms so that models built using
existing projects can recognize defects as they emerge in a new project.
Up until this study, comparing the efficacy of transfer learning
algorithms was challenging because previous studies used differing data
sets, baselines, and performance metrics. By providing open source
implementations and baseline performance metrics for several transfer
learning algorithms on two different data sets, our project offers
software engineers the tools to objectively compare methods and readily
identify top performing transfer learning algorithms in the domain of
both vulnerability and defect prediction.
2017-02-08T00:00:00ZIdentifying Fixed Points in Recurrent Neural Networks using Directional Fibers: Supplemental Material on Theoretical Results and Practical Aspects of Numerical TraversalKatz, GarrettReggia, Jameshttp://hdl.handle.net/1903/189182017-06-13T10:41:41Z2016-12-12T00:00:00ZIdentifying Fixed Points in Recurrent Neural Networks using Directional Fibers: Supplemental Material on Theoretical Results and Practical Aspects of Numerical Traversal
Katz, Garrett; Reggia, James
Fixed points of recurrent neural networks can represent many things, including stored memories, solutions to optimization problems, and waypoints along non-fixed attractors. As such, they are relevant to a number of neurocomputational phenomena, ranging from low-level motor control and tool use to high-level problem solving and decision making. Therefore, global solution of the fixed point equations can improve our understanding and engineering of recurrent neural networks. While local solvers and statistical characterizations abound, we do not know of any method for efficiently and precisely locating all fixed points of an arbitrary network. To solve this problem we have proposed a novel strategy for global fixed point location, based on numerical traversal of mathematical objects we defined called directional fibers [2]. This report supplements our results in [2] by presenting certain technical aspects of our method in more depth.
2016-12-12T00:00:00Z