Make your own spreadsheet

Students will be expected to copy this spreadsheet and use it as a baseline for creating a sort of "data-gathering diary". In the first weeks of class, many of the scraping tasks may seem impossible. That's OK, what you need to do is triage the things you can do now, and keep track of the things you need to learn how to do.

SSS is less an exercise about programming and more about how to strategically break down an otherwise overwhelming task.

Note: Do NOT use Beautifulsoup if you can help it. I've left this up as an example but use lxml as shown in Freebie #30 below

importbs4importrequestsresponse=requests.get('http://www.data.gov/')soup=bs4.BeautifulSoup(response.text)link=soup.select("small a")[0]print(link.text)# Reminder: Do NOT USE BS4; use lxml, as in the below example

Note: The csv library won't just let us turn a string into a data structure. So in the example above, I save the file to disk, and re-read/open it with csv.reader(open(fname)). There's a more graceful way to do this with io.StringIO. But the effect is the same: