Crawling the internet Data Science Within a Large Engineering System

Bill Richoux, a Data Scientist at Google, graduated from Massachusetts Institute of Technology with a PhD in Electrical Engineering in 2010. Since then he has been working at Google and is currently focusing on Fleet Deployment and Optimization, which works for end-to-end planning and delivery of Google's production compute infrastructure. He and the team are automating the planning and management of resources as well as ensuring they are running optimally.

Presentation Description

Critical decisions are being made continuously within large software systems. Often such decisions are the responsibility of a separate machine learning (ML) system; but there are instances when having a separate ML system is not ideal. In this talk we describe one of these instances — Google search deciding when to check if web pages have changed. Through this example, we discuss some of the special considerations impacting a data scientist when designing solutions to improve decision-making deep within software infrastructure.