Visualizing Data

Many of us are data professionals that deal with the acquisition, transforming, storing, querying, and ultimately, display of data. We often spend a lot of effort on that last one, trying to find ways to help our clients easily understand and use their data. In most of my career, the use of text reports was the primary way that people would visualize the data under my stewardship. I remember many ours spent producing tables of data that would print on (first) greenbar paper and (later) standard A4 paper.

As technology has advanced, and become much more powerful, various graphical and other visual representations of data, or based on data, are becoming more prevalent. SQL Server added Reporting Services with SQL Server 2000 and since then our tools have grown with the various "Power" tools, notably Power BI, but also with the ability to integrate our reports into various applications with all sorts of tools.

This week I saw a few items that talked about how we use data to achieve some purpose, and then create some visualization to influence others. I could see this technology being used in lots of ways, some good, and some bad, depending on your perspective. While there are no shortage of business applications, there are others that can help change the way the world works.

There's a new technology to create screens out of store windows, or at least for now, display cases. How long before that becomes personalized in some way, maybe similar to what Minority Report showed us as personal advertising. I could see that the need for real time data integration with customers, or perhaps with the data they share through some Bluetooth broadcast, an RFID customer card, or something else. This is one that gets into the scary category for me because marketing people often can't help but ask for too much data and overuse it.

I am fascinated by maps, and in some sense, I miss the older physical maps that helped me navigate across the country. These days they're not convenient, as up to date, or useful as the digital copies that are much less satisfying. However, the best maps of the future may be for cars, self-driving or not. Whether you like the idea of autonomous cars, having more detailed maps, updated by cars themselves as they drive is interesting. This also presents an interesting data challenge. A multi-mode replication, everyone is a subscriber and publisher, perhaps even voting by multiple nodes to determine what data that is shared to other nodes. I could see interesting visualizations that might point out data useful to drivers, such as potential safety issues (safe speed limits, potholes, condition changes).

Speaking of real time data, we see more and more cases where we use data to respond in crisis. The mining and analysis of data after Hurricane Sandy was used to help responders, and is being used to plan for future issues. I expect that some of us will work in other situations where we need to quickly surface data and help our organization make use of the information, perhaps by quickly altering the gathering or analysis of the data. There are challenges here, and certainly opportunities for those that have the skills to build and alter visualizations quickly.

One of the most popular topics in the technology world of data today is machine learning. It seems that so many vendors and industries are trying to find ways to make sense of their data and turning to various AI type technologies to help them. One interesting one I saw this week was the use of machine learning to help guide students and suggest classes for them. One of the things machine learning algorithms need is lots and lots of data, and I expect more of us to learn how to build, train, and deploy models in various environments. This might be one of the more exciting, but also frustrating since we don't really program the model. We decide what data to feed it and then evaluate its performance, doing our best to produce a better outcome than other efforts. Not how most data professionals want to work, but this will be a way that more and more of us will actually surface information from our troves of data.

We have more and more ways to showcase our data outside of text displays, with more options coming all the time. I see more and more custom visuals for Power BI, new mapping tools, better graphical analysis, and more. I hope that some of you will consider other options the next time a business person asks you to help them analyze some bits.