This blog discusses current weather, weather prediction, climate issues, and other topics

Wednesday, March 26, 2014

Google Could Greatly Improve Weather Forecasts: Will It Take the Necessary Steps?

Google could make a huge contribution to weather prediction, undoubtedly saving many lives and billions of dollars of economic costs due to severe weather in the U.S. alone. It is something that is well within their technical capability and would cost little. And if Google can't help, Facebook or Twitter would be almost as good.

What am I talking about? Collecting millions of pressure observations each hour from smartphones.

The skill of weather forecasts, and particularly predictions of
smaller-scale features, such as severe thunderstorms or strong fronts,
depends on having detailed observations of the atmosphere. And even
today, meteorologists often don't have enough. As we secure more computer power and run our models at finer and finer resolution, we need the data to describe small-scale structures that can lead to tornadic thunderstorms, flash floods, downslope windstorms, and other serious threats to life and limb.

Surface pressure is a uniquely valuable observation. It is the only surface parameter that reveals information about the whole depth of the atmosphere (since pressure is dependent of the weight of the atmosphere above). Recently, atmospheric scientists demonstrated they could determine the entire structure of the atmosphere with pressure observations alone. Pressure observations can be taken inside or outside of a building, in your pocket or purse, in the the sun or out. It is a forgiving observation to make.

And initial research, like that done by my graduate student Luke Madaus, has revealed that additional pressure observations can substantially improve forecasts of small scale storms and weather features. For example, using only a handful of smartphone pressure observations substantially improved the forecasts of thunderstorms over the eastern slopes of the Cascades (see figure).

The radar echoes on the eastern side of the Cascades (thin solid lines) compared to the predictions of max thunderstorm radar returns from a collection of forecasts (colored dots). Left side, without smartphone pressures, right side, with smartphone pressures. The right side is much better!

Forecasting the initiation of major thunderstorms in the Midwest and Great Plains depends on knowing the details of wind, temperature, and pressure. A dense network of smartphone pressure might greatly enhance such forecasts.

In several past blogs (here and here), I told you about the excellent pressure sensors that are in a number of Android smartphones (e.g., Samsung Galaxy S3, 4, 5;Samsung Nexus) and a few others (e.g., Nokia 1020). Today, there are probably 10-20 million Android phones in North America alone that are taking pressure measurements-- measurements folks in my professions need.

During the past two years I have been talking to two energetic, young start-up firms that are collecting pressure observations from smartphones through two free apps (Pressurenet and OpenSignal). The principals of these firms have shared the pressure data with me to evaluate the potential of the phone pressure sensors. Combining both groups' collections, which I get in real time, results in about 25,000 pressure observations per hour across North America (see graphic, each dot is one pressure observation)

In some metropolitan areas there are so many observations that the figure turns black, but there are plenty of areas of poor coverage, including regions (like the Great Plains) with terrible storms. In reality, the current collection is only securing data from one thousandth of the smartphones out there. Imagine, what a million observations per hour would look like. I suspect the eastern half of the U.S. would turn black and coverage over the west would get far better. Remember, farmers and ranchers have smartphones and so do folks traveling the interstates and other roads.

But now we get to the problem and why Google is essential. Only a very limited number of people have loaded the free pressure apps and we probably will never get the density we need following this route. We need to put the code for collecting the pressure observations and the position of the smartphone in software that millions of folks have. And because most of the smartphones with pressure are Android base, the software has to work on Android phones first and foremost.

What one company has the ability to do this? You know who: Google.

Imagine if GoogleMaps, which is surely used by millions, collected pressure information. That app is already transmitting position to Google to allow determination of car speeds. Pressure would be a relatively trivial addition.

Or even better, what if the Android operating system itself collected the pressures (after giving the user a chance to op out, of course.)

Google could do this and have a tremendously positive impact on society, hopefully sharing the data with the National Weather Service, the research community, and the U.S. weather industry.

I have talked to my friends in Google and several have tried to talk to contacts at Google's Mountain View headquarters. I have emailed the head of GoogleMaps. Nothing has happened. No response at all.

Perhaps one of you is well connected with someone high enough in Google management to get someone's attention. If so, maybe you can help. Or maybe someone in Google management reads this blog. Google is working on lots of technological advances, like autonomous cars. Why not improving weather prediction? If invited, I am ready to fly down to the Bay Area to talk or give a presentation on this issue.

If Google won't do this, the next obvious possibilities would be Facebook or Twitter, both with huge installed bases of app users. Anyone in those organizations interested n helping? Perhaps Mark Zuckerberg is a weather enthusiast. And if any of you know of any Android app used by millions of people that I missed, let me know about it....I would love to talk to the principals of that group. And, of course, the greatest home run would be if Samsung included the software in every phone.

It is frustrating to see all the pieces of a technology that could really contribute to the safety and economic interests of people all over the world, yet they have not come together. Maybe, with the right connections, we can realize the potential of the pressure observations folks are already taking every day without realizing it.

Curious how this would actually work--Google Maps is useful mostly when one is driving. Even with pretty good latitude and longitude it seems you wouldn't generally have an accurate enough altitude to make the pressure reading useful, and when the smartphone is moving, the delta-P readings also wouldn't be meaningful. What am I missing?

LVDLM I am explain. First, phones are often not moving. Then we can easily assimilate pressure change. Second, we do have GPS heights---good to roughly +- 10 meters...or 1 hPa, which is still very useful. And we can use the pressure sensor to aid the height calculation. We also know the roadway height so when traveling on roads we can do better. Anyway, I will spare you more details, but there is a lot of useful information available from the phones...cliff

While I am sure Google can help you out in this I'm not myself sure how. Building this functionality directly into the OS doesn't sound like a good option to me. I'd be surprised if they were to add it to the Google Maps app, but who knows. It's still worth a try.

If you are already going down the path of lobbying companies to include this functionality, how about lobbying either the wireless carries (T-Mobile, Verizon, Sprint, ATT, ...) or the manufacturers (HTC, Samsung, LG, ...) to include Pressurenet and/or OpenSignal as a bundled app. They already include all sorts of crap I don't want, why not include something that I do want. If it were me, I guess I might start with someone like T-Mobile, or really any carrier where I (you in this case) might have a personal contact.

Cliff - You might try talking to the folks that make the Waze app. It's a GPS nav app with crowd-sourced traffic alerts -- so they're already collecting data -- and they're a relatively small company, so you'd probably only have to engage one individual to get things moving. Finally, they recently got bought by Google, so it's likely useful functionality in Waze will eventually get integrated into Maps.

I can tell you why it's going to be tough to go down this road - privacy concerns. Even for something as useful and innocuous as this. People are freaking out about the NSA, Google, Facebook and the rest sharing their personal information. I'm not sure the big tech companies are going to get behind something as altruistic as this because people are just waiting to jump on them for anything that looks like "invading privacy."

Hi Cliff. Just wanted to let you know that a lot of people inside Google have seen your post and are talking about it. Sorry you didn't get a reply to your earlier attempts to get in touch. Of course I'm not promising anything, I'm just a typical engineer on Maps... but I hope something can be done, I've been wanting Google to help make weather prediction better for a long time!

I believe the data bus on OnStar always sends data even if the car owner is not subscribed. Am wondering if the data bus is capturing atmospheric data as like fleet vehicles? Have a look at www.inrix.com as an example of a company that makes use of that data.

Mr. Borasky, There are a lot of smartphones in rural areas, both from residents and those driving through. My map shows that the eastern U.S. gets a lot of coverage. I plotted an hour of twitter feeds...the eastern U.S. turns black..and thing about Europe and China...cliff mass

A vast majority of the people would rather have a root canal before they let Google install an app to snarf even more isnfo on them. What Google does make the NSA intrusions seem no more harmless than a glass pressed against the wall ... of a house on the next block. Google is just evil. Period.