A blog about search, search skills, teaching search, learning how to search, learning how to use Google effectively, learning how to do research. It also covers a good deal of sensemaking and information foraging.

Thursday, September 27, 2018

How many pianos in Salzburg... ... can you Ferminate and answer the Challenge? Well... okay... let's just jump into it. The Challenges from last week, please... 1. As you know, Wolfgang Amadeus Mozart had a famous sister--also a performing prodigy. Maria Anna Walburga Ignatia Mozart, called Marianne and given the nickname "Nannerl, was his older sister. But in nearly every picture of her, she's got a remarkable head of hair. See this picture below:

Nannerl and Wolfgang Mozart playing two-handed duets at the piano

The question I had was How do you sleep with hair like that? Or, I suppose, the other way to think about it would be How much time do you spend on hair like that? Any ideas? This is a pretty open-ended question, and it's worth noticing that while I asked the question about Nannerl, a simple image search for: [ 18th century big hair ] quickly tells us that florid, extravagant hair was the rage in the late 1700's. Here's what you get if you look at the images from that query:

Sure, some of these are modern recreations, but clicking through some of the obviously older images quickly leads to images like this, for both men and women:

One of the blogs that appeared with this query was Two Nerdy History Girls with their great post on this topic. In part 2 of their post, they spoke with hair dressers at Colonial Williamsburg living museum who actually practice this craft and asked the relevant question, "How long does it take?" Their answer? Not as long as you think--around 10 minutes for an "ordinary" pile of hair as seen in some of the images above. In reading their blog, I learned a new term: pomatum (a kind of scented pomade applied to hair to give it some structure and scent). Using this in a query, such as: [ pomatum 1700..1799 hair style ] leads to books about fashion, and to the inevitable Wikipedia article about hair styles. All of which agree: These hairstyles can be relatively straightforward (around 10 - 15 minutes) to build a relatively simple coif, but the larger, more impressive and elaborate hair-dos can take a long, long time. And yes, you don't sleep in these things. You take them down in the evening and comb out the hair. (Whew!) 2. When I visited the Mozart residence at No. 8 Makartplatz (the so-called "Dance Master's House," aka Tanzmeisterhaus), I was struck by the odd appearance of one of Mozart's pianos--the key colors are reversed! The accidentals are white, while the naturals are black. Here's what I mean; Mozart's piano keyboard is on the left, while a current piano keyboard is on the right.

So... when did the colors change from black naturals to white naturals... and why? I started this Challenge by looking for a history of the piano. I figured any such account would probably cover the key color shift, and give some references to boot. I started with this query: [ history of the piano key color ] All of the resources found on this SERP told me that the piano was invented by Bartolomeo Cristofori, and that his original pianos had black keys with white accidentals (those keys that are shorter and raised above the keyboard). That's certainly the case for Mozart's pianos (see above). As with many things, there are a number of stories about why the colors changed. Even on the Question-Answering (QA) site Quora, there are a number of hypotheses on the post Why are colors of piano keys reversed on certain Viennese pianos:

a. Cristofiori's friend convinced him to change the color.

b. It was cheaper to do it that way. Elephant ivory became cheaper than the black ebony wood that was being used for the black keys. Since a keyboard is composed of 52 type naturals and 36 accidentals, it was cheaper to use the expensive ebony for the accidentals.

c. It's easier to see. No accidentals are adjacent, but there are many places where naturals are adjacent. If the naturals are white, it's easier to see the gap between them because it's dark. See below:

(As a piano player myself, I will also add that ivory keys are very robust and have a great touch--they have just the right amount of friction and absorb sweat well. They're just much nicer to play than wood.) Reading more of the results from this single query, I found that: Piano maker Yamaha believes, Living Pianos, and the Joshua Ross website believe (c). And I as read through many of the results, that seems to be the dominant belief. More importantly, I could not find a solid justification for the color switch. Without finding a design document or standard (that says something like "henceforth, all accidentals shall be black and the naturals shall be white"), and given that so many early pianos, clavichords (a precursor instrument), and harpsichords have black naturals, it's more likely that it's just the fashion of the time that turned into standard practice. Not all questions have clear, unambiguous answers--and this might be one such question. We'll leave this Challenge marked as "open for more research." After looking at at least 100 different sites (including history books about pianos), this is as far as I got. Maybe we'll find a better answer in the future!

3. Speaking of Fermi Estimation and Mozart's piano (we were, weren't we?)... What would you estimate as the total number of pianos in Salzburg during the year when Mozart left Salzburg for Vienna? How would you do this estimate?

Austrian pianos over the years

This is another Challenge that could consume a LOT of time. I know I had to keep resisting the urge to answer this definitively... Our goal is to estimate the number of pianos. I started with a basic question: How many people lived in Austria in 1781, the year Mozart left Salzburg for Vienna? By checking the Wikipedia entry on Salzburg, I know that Salzburg had 27,858 in 1869. But by searching for [ Salzburg population 1770..1800] I found a book (surprisingly entitled Pilgrimage to Crete) that tells us that the population in 1771 was around 16,000. Now, how to estimate the fraction of the population with a piano?? (Not organs, clavichords, or harpsichords, but honest-to-God pianos.) With a search in Google Books for [ piano social history Salzburg ] I found the book Men, Women and Pianos: A social history, which points out that in Vienna, during Mozart's time, around 2% of the population had a piano at home. If that number holds for Salzburg (only a couple hundred miles away), 2% of 16,000 would mean there were approximately 320 pianos in private homes. If we add in a few for schools, churches, etc -- that would bring the number to around 350. Things changed a few years later when piano manufacturing started in Salzburg and Vienna on a large scale, but at this time, in the late 18th century, there weren't that many!

Search Lessons

There are a couple of things to learn from this Challenge. 1. A simple query often leads to a number of results which are collectively quite good. In the "big hair" Challenge, not all of the results were superb... but when reading them en masse, you can learn a bit from all of them--in particular, terms of art like pomatum that let you dig more deeply and precisely into the topic. That is, the collection is more valuable than any one item. 2. While it's tempting to dig deeply into a research topic, when estimating, you want to find just the essentials to make your case. In the Challenge about numbers of pianos in 1781 Salzburg, all I wanted to do was to find the population of Salzburg AND a reasonable estimate of the number of pianos in an Austrian city. I found the number for Vienna, and transposed that to Salzburg. Just be careful when doing an estimation search as opposed to finding a definitive answer. 3. Sometimes... you can't find a definitive answer!Yes, my friends, there are some Challenges for which a definitive answer lies somewhere in the mists of history. So why DID piano keyboards reverse their colors? We might never know. We have hypotheses, but I wasn't able to find a clear cut answer.

Like Google's Image search, Bing also has the camera icon, which lets you upload an image to Search-By-Image. When I uploaded this image, the UI looks like this:

Bing's search-by-image UI.

And.. as you can see on the right-hand side, this didn't work either. HOWEVER... if you roll your cursor into the image area, you'll see a "crop symbol" popup in the upper right corner. (Pointed to by the yellow arrow I added to the screen grab.)

When you select the crop tool, the UI changes to let you do live cropping.

At this point, you can drag the corners of the sub-image box to create a cropped search of the image. The really nice thing about this is that when you let go of the corner widget, Bing re-runs the query with the cropped area. This means you can very quickly search through many cropped versions of the image! (You can read Bing's announcement of this--they include a nice demo.) With Bing's cropping tool, I played around with this for about 5 seconds before I found the right crop that located a near-match of the image! Look at the image in the upper left of these results.... THAT's the image I was looking for... Now that I've got a matching image, it doesn't take long to look at that image and figure out that it's in Detroit, Michigan, just outside of the downtown YMCA.

Here's a closeup--you can see it's clearly a picture of the same statue.

It's a great tool... Highly recommended when you need to explore the possibilities of cropped images in a short amount of time.

Wednesday, September 19, 2018

Salzburg was Mozart's home ... ... in his pre-Vienna years. It's where he was born, lived, and launched his career. When I was visiting earlier this year, I visited several of the places that Mozart lived and performed. As you'd expect, as I looked around, I noticed a few things--things that struck me as funny/odd. Of course, being a curious fellow, I had to look these things up with a bit of online research. Can you answer these oddities as well? Here are three of the things I had to look up. I was surprised at the answers to each of these. 1. As you know, Wolfgang Amadeus Mozart had a famous sister--also a performing prodigy. Maria Anna Walburga Ignatia Mozart, called Marianne and given the nickname "Nannerl, was his older sister. But in nearly every picture of her, she's got a remarkable head of hair. See this picture below:

Nannerl and Wolfgang Mozart playing two-handed duets at the piano

The question I had was How do you sleep with hair like that? Or, I suppose, the other way to think about it would be How much time do you spend on hair like that? Any ideas? 2. When I visited the Mozart residence at No. 8 Makartplatz (the so-called "Dance Master's House," aka Tanzmeisterhaus), I was struck by the odd appearance of one of Mozart's pianos--the key colors are reversed! The accidentals are white, while the naturals are black. Here's what I mean; Mozart's piano keyboard is on the left, while a current piano keyboard is on the right.

So... when did the colors change from black naturals to white naturals... and why? 3. Speaking of Fermi Estimation and Mozart's piano (we were, weren't we?)... What would you estimate as the total number of pianos in Salzburg during the year when Mozart left Salzburg for Vienna? How would you do this estimate?

Austrian pianos over the years

As always (and I hate to sound like an 8th grade math teacher), show your work! At least tell us how you answered each of the Challenges! (And note that you don't need to answer all three of the Challenges. If you don't have enough time, just do one. But be sure to tell us HOW you figured out the answer. I'll write up what I found next week. Until then, Search On!

Thursday, September 13, 2018

Being a good estimator... ... requires two kinds of knowledge: (a) facts about the world, and (b) basic math estimation skills. Those skills let you work from the basic facts, combining the data and doing quick math estimates to get to some numbers that give you the information you seek.

How tall was Enrico Fermi, the physicist? How would you estimate? Assume he doesn't have extraordinarily long legs.

Fermi Estimation needs both kinds of knowledge--along with a bit of practice in figuring out how to go from point A to point B. Let's practice with a true Fermi estimation!

1. Can you estimate how tall Enrico Fermi was?

In this example, I give you a piece of data: Fermi's head is 9.4 in (23.9 cm) tall.

How would you estimate his height just from this information? Ask yourself, What other information do I need to know?If you're an artist, you probably learned that when you sketch a person, their body proportions are often expressed in terms of head-heights. Example: An ordinary person is about 7.5 heads high--a tall, slender, elegant person is about 8 heads high.

If you don't know that bit of data, you could look the relationship up quickly: [ body proportion in heads ]

Later edit (Sep 14, 2018): In the original post I messed up the calculation with a typo... I wrote that 24 * 7 was 164, which is clearly wrong. Once I had that typo, all kinds of errors followed. Bottom line, check your math. Didn't your sixth grade teacher say that?

So know you know Fermi's head is 23.9 cm tall. To Fermi estimate, let's round that up to 24 cm. You can probably guess that 24 * 7 = 168 cm. If you then add in the half-head height (12), you can add 168 + 12 and get 180 cm. Make sense? In this case, I did the mental math in metric because that value (23.9) could be easily rounded up to 24 without causing many estimation problems, whereas the English measurement (9.4 inches) was kind of a pain--if you round up OR down, you lose a lot of accuracy. Plus, I know how to convert from cm to inches pretty easily. I know (off the top of my head, OTTOMH) that 10 cm ~ 4 inches. (Here, the ~ means "close to" in value.) SO... to convert 180 cm into inches, I would divide 180 by 10 and multiply that by 4. My mental math was 180/10 gives me 18 --I then multiply 18 by 4 to get your Fermi estimate of 72 inches, or 6 feet.

When I read that, it SOUNDS really complicated... but it's not. Here are the steps laid out in a diagram.

1. I know 10 cm~ 4 inches. So... I just have to divide by 10, then multiple the result by 4 to do that conversion.

2. 180 cm /10 = 18

3. 18 * 4 = 72 inches...

At that point, most English-unit-using people can convert that into feet/inches without any trouble: 6 feet tall. (Yes, I know that's a weird skill; so be it. We grew up with it. Point is, it's not that hard. So yes, you have to know a basic conversion fact (10 cm ~ 4 in), and know how to break up a complex mental multiplication into parts.(If you want to learn more about how to do this kind of mental math for estimation, there are many online videos that will walk you through the process. One that's pretty nice is from the Khan Academy: Estimating values.)

2. Can you estimate (without looking up the answer!) how many people in the United States are over 80 years old? (For extra credit, how many people worldwide are over 80 years old?)

This is an interesting question: How would you go about estimating this? Again, you need to know a few things to start.

1. The population of the US. (I know this: It's around 325M people.) 2. The population age distribution.

Once upon a time, I remember seeing a chart like this. They're sometimes called "age pyramids" or "age distribution" charts. When I look at this chart in my mind's eye, the top couple of slices of that chart constituted around 3% of the whole. To check this, I also remembered that the average life expectancy is around 78 (which is lower than in some countries, higher than in some). So 3% off the top sounds like a reasonable guess. So... 3% of 325 is easy. Divide by 100 (3.25) and multiply that by 3. That's 9.75 million people who are older than 80 in the US. When I look up the actual numbers to check myself, I see that there are 328.3M people in the US, and this is the age distribution:

And, surprisingly, my estimate of 3% is pretty close to the actual value. Even more surprisingly, I also happen to know that the US is slightly above average for ages > 80 years throughout the world. Since I know that the population of the Earth is 7.5B, 3% of 7.5B is 225M people, which is a lot of elder wisdom.

3.To do Fermi Estimates you actually need to know a few basic facts (e.g., about how many people live in the US). This brings up a great meta-question for Fermi Estimation and sensemaking of data that you see presented in the news... What facts do you need to know to be a good Fermi Estimator? (There's no perfect answer for this; just tell us what facts you've used to do your own Fermi Estimates!)

When I start thinking about how you do estimates in your everyday life, I think about all of the times I do the Fermi estimations on news stories that I read. Usually, these days, these are stories about income, wealth, distribution, immigration, and science stories. To do any kind of Fermi estimation you need to know a few things. I've been writing down some of what I'd consider core knowledge over the past week, just taking notes when I did my own estimates to see what core knowledge I used. Here's my list (yours is probably different): - population of the US in 2017:327M - population of the world in 2017: 7.5B- area of California: ~100M acres- conversion from inches to cm: 1 inch = 2.54 cm - number of stars in our galaxy: 300B starsAnd so on... The reality is that nobody can take the time to spend 5 minutes doing online research for everything. But it's pretty simple to do Fermi estimates to see if what's being said actually makes sense. It's a really good skill to have. I highly recommend you practice this whenever you see something that sounds a little off. It just might be. But with your Fermi estimator skills, you can see through the mistakes. Here's a short video by my friend Jevin West talking about Fermi estimation in his class at the University of Washington. It's well worth watching--his examples are great! (Video link.)

Search Lessons

A few lessons spring out at me... 1. You still have to know basic facts about the world. Dates, places, quantities, names, sizes, durations... For instance, how important are plastic straws as a source of plastic pollution? (Can you estimate what fraction of the total plastic trash they are?) Once you know how to do Fermi estimates, your reading and understanding of news stories / current events changes. Beyond just knowing facts--e.g., the population of Japan is around 127M--once you know things like this, you know what kinds of information can be derived from them. Example: While 3% is a good estimate of the number of people world-wide who are older than 80, you might also know that Japan is an outlier in the age pyramid--they have many more older residents than most countries, so you should Ferminate a higher fraction for folks > 80 years of age. 2. Realize that you CAN do estimates that are pretty good, and use them to do basic fact-checking of a story. If it doesn't add up, you have more research to do.This more of an attitude about research than anything--but it's important. Many people don't know how to do estimates, and so they don't know they can guesstimate what they're reading or hearing about. And that makes them more susceptible to incorrect data and news stories. 3. Of course, online search is handy to get to the basic facts and formulas.This is really true for breaking stories where there simply isn't any good information yet. Even if you can't determine a particular value for something you've read, there's a good chance that you can estimate it by looking up other information and combining the data together into a fuller picture. Keep estimating. It's a great personal skill to have! Search on!

"There are many thousands of data repositories on the web, providing access to millions of datasets; and local and national governments around the world publish their data as well. To enable easy access to this data, we launched Dataset Search, so that scientists, data journalists, data geeks, or anyone else can find the data required for their work and their stories, or simply to satisfy their intellectual curiosity."

For people looking for online data, this is a godsend.To use it, visit: toolbox.google.com/datasetsearchand do a search. Here's one of the first things I tried to do... (Naturally, I checked local data that I probably would recognize..)

Notice that Dataset Search, like regular Google Search, uses Autocomplete. This is a wonderful behavior that will let you search a data space very quickly. (Caution: It doesn't seem to reflect ALL of the possible completions, so use this feature carefully. ) And, naturally, when you get a dataset, read the metadata carefully. (We discussed this a while ago, Feb 15, 2014--"Read metadata carefully.") The next search I tried was for Stanford's dataset of global warming information, I've always wanted a copy of it so I could do my own analysis. I did the obvious search, and found not only three different providers of the datasets, but other, related datasets as well.

On the left hand side you'll see a scrolling list of related datasets (that is, other data sets that match the query, but like regular Google results, are not ranked quite as high as the first hit). You can use Control-F to do a text search within the page, but notice that the scrolling list of related datasets might go on-and-on-and-on... You can't trust Control-F to search everything in that list. (It's a "scroll on demand" list; Control-F only searches what's "visible" and not the entire contents of the list.)

Note that you can ALSO use site: to restrict the sites that are searched for data. A search tool like this one is only as good as the metadata that data publishers create. Maybe some enterprising SRS folk will publish a data set or two. (I certainly will try!) If you publish data and don't see it in the results, visit our instructions on our developers site which also includes a link to ask questions and provide feedback. Learn all about how to publish your own datasets here at the dataset publishing guidelines page. We'll have some future Challenges that will use Dataset search, I'm sure. Search on (for data)!

Wednesday, September 5, 2018

How tall was Enrico Fermi, the physicist? How would you estimate? Assume he doesn't have extraordinarily long legs.

An important skill in a lot of skilled reading (or fact checking, or just being a skilled SearchResearcher) is being able to do quick estimates of values by just doing a bit of thinking about them. This estimation technique is often called Fermi Estimation, after the famous physicist who was known for his ability to good approximate calculations with little or no actual data. These are sometimes called "back-of-the-envelope calculations," but great Fermi Estimators don't actually do this without any data--they need to start somewhere... but they know a few key facts, and then work forward from what they know towards an estimate. Fermi estimationg problems typically involve making reasonable guesses about quantities, their variances, the upper and lower bounds, and how to combine guesses together to lead to a useful insight. This came up for me recently when the shopper in front of me at the grocery store insisted that she wanted 1.00 pounds of ground hamburger. When the butcher dropped a lump of ground beef onto the scale, she insisted on NOT paying for the 1.05 pounds, but she really wanted 1.0o pounds. The butcher rolled his eyes and removed a tiny bit of meat to get it exactly to 1.00 pounds. I immediately thought of this as a Fermi Estimation question: About how big a lump of meat is 0.05 pounds? Would it be the size of your hand? Would it be the size of your little finger? If you assume that ground beef is about the same weight as water, then if we know how many ounces are in 0.05 pounds, then we can estimate the volume of meat that represents. Here's the Fermi Estimation I did in my head: In the US, 1 pound is 16 ounces. In particular, 1 pound of meat (which weighs about the same as water) is 16ounces. First I converted that number into a ratio I could work with. I realized that 0.05 of 1 pound is half of 0.1 of a pound... and that's half of 1.6 ounces, or 0.8 ounces. Luckily, an ounce of water weighs the same as 1 fluid ounce of water. So.. 0.8 ounces is just under one fluid ounce of water. 1 fluid ounce which is 2 tablespoons of water, so 8/10ths of 2 tablespoons is close to 1.5 tablespoons of water... around the size of my little finger! (Of course, doing this in grams is SO much easier. Imagine if the shopper wanted exactly 500 grams, and not 530 grams of ground meat. About how big is 30 grams of meat? 1 gram of water is 1 milliliter, so we'd be looking at 30 ml of meat..)

As Enrico Fermi might say: non tanto! (not much!) This leads us to our Challenges for this week--the first is a true Fermi estimation?

1. Can you estimate how tall Enrico Fermi was?

2. Can you estimate (without looking up the answer!) how many people in the United States are over 80 years old? (For extra credit, how many people worldwide are over 80 years old?)

3.To do Fermi Estimates you actually need to know a few basic facts (e.g., about how many people live in the US). This brings up a great meta-question for Fermi Estimation and sensemaking of data that you see presented in the news... What facts do you need to know to be a good Fermi Estimator? (There's no perfect answer for this; just tell us what facts you've used to do your own Fermi Estimates!)

What I want you to think about is how you do estimates in your everyday life. When you read an article that claims something that seems excessive, what kind of thinking should you be doing in order to sanity-check the assertion? Have you seen any recent examples of assertions that fall apart under a Fermi Estimation? What do you have to know in order to Fermi-check the claim? Let us know in the comments! Search (and estimate!) on!