Tag Archives: data visualization

As the calendar turns over to a new year, it’s useful to look back and see what the last 365 days have been all about. Looking back is always easier when you have something to look back on, and, no surprise here, self-tracking is a great help for trying to figure out how things went. That’s what makes this time of year so interesting for someone like myself. I spend a good deal of my time trying to track down real-world examples of people using personal data to explore their lives. Sometimes it’s easy, and sometimes it’s hard finding people willing to expose themselves and their data. However, when late December rolls around, I perk up because this is the time for those yearly reviews.

I’ve spent the last few weeks gathering up some great examples from individuals from all over the world. I hope the following examples inspire you to track something new in 2015 and maybe share it with the QS community in person at a local meetup, at our QS15 Global Conference, or in our socialchannels. Okay, let’s dive in!

2014: A Year in Review with iPhone Pedometer Data by Geoffrey Litt. I really enjoyed this very thorough exploration of a year’s worth of pedometer data gathered from the Argus app (iOS). Not satisfied with just looking at his total step count for the year, Geoffrey ran a series of data explorations. Among my favorite, his visualization of his daily rhythms:

2014, Quantified by Sarah Gregory. Sarah does an amazing job of capturing and showcasing her 2014 activities in this beautifully simple post. With a balance of pure quantitative information and qualitative insights I found this review especially compelling. (It was also nice to see that she used our “How to Download Your Fitbit Data” tutorial.)

2014 in Numbers by Donald Noble. Speaking of our Fitbit data download tutorial, here’s a short post about a year’s worth of steps – 4.15 million steps to be precise.

Three Years of Running Data: 1,153km with Nike+ and Mind by Todd Green. As you can see from the title, this post details three years of running, but as a runner myself I always like peeking into other runner’s data. (Todd also has a fantastic post from early 2014 about tracking every penny he spent in 2013.)

Food, Glorious Food by Peter Chambers. A fun post detailing what Peter and his family ate for dinner nearly every day of 2014. One juicy bit – the most common meal? Chili – Peter’s favorite!

2014 in Numbers by Jill Homer. With the help of her Strava app, Jill details her cycling and running from 2014. Click for the numbers, stay for the gorgeous photos.

I wrote every day in 2014: Here’s an #infographic by Jamie Todd Rubin. It’s great fun following Jaime’s blog. He’s relentless on his journey of daily writing (and is quite the active Fitbit user as well). What was 2014 like for his writing? Over 500,000 words – almost enough to take on Tolstoy’s War and Peace. Plus, the visualization is great (click through for the full version):

2014 Stats by Dan Goldin. Amazing data gathered from a self-designed Google spreadsheet that includes mood, sleep, food, and drink.

Tracking My Life in 2014 by Mike Shea. Mike tracks his life using his own custom designed “Lifetracker app.” This includes his rating on six aspects of his life, daily activities, media, and location. In this post he turns his 8,400 rows of data into elegant visualizations and interesting analysis:

Tiny Preview By Lillian Karabaic. If her previous work is any indication this year’s review is going to be great. Keep in mind this is just a place holder until the full post is up.

Why #DIYPS N=1 data is significant (and #DIYPS is a year old!) by Dana Lewis. Along with her co-investigator, Scott Leibrand, Dana has been on a journey to better control, understand, and generate knowledge about her type 1 diabetes through augmenting CGM data, devices, and alerts. What started as project to make alarms more clear and useful has morphed into a full on DIY closed loop pancreas. In this post, Dana explores what they’ve learned over the last year of data collection. Truly inspiring work:

2014 Year in Webcam and Screenshots by Stan James. We’ve featured Stan and his great LifeSlice project here on QuantifiedSelf.com before. It’s an ingenious little lifelogging application that tracks your computer use through webcam shots, self-assessments, and screenshots. Check out this post to see a fun representation of his data.

2014 Personal Annual Report by Jehiah Czebotar. Coffee, travel, Citi bike trips, software development, laptop battery life, and webcam shots – all included in this amazing page. Presented without narrative or explanation, but meaningful nonetheless. The coffee consumption visualization is not to be missed (click through for the interactive version):

My Q4 2014 Data Review by Brandon Corbin. While not a full “year in review” here, I still found this post compelling. Brandon created his own life tracking application, Nomie, and then crunched the numbers from the 60 different things he is tracking. Some great examples of learning from personal data in here.

20140101 – 20141231 (2014). Noah Kalina started taking a photo of himself on January 11, 2000. On the 15th anniversary of his “everyday” project he published his 2014 photos.

Reading
When I was spending late nights searching for variations on “2014”+”data”+”my year in review” I stumbled upon quite a few posts detailing reading stats. Here’s a good selection of what I can only assume is a big genre:

2014 Reading Stats and Data Sheets by Kelly Jensen. A great place to start if you want to track your own reading in 2015. Kelly provides links to three excellent spreadsheet examples.

My Year in Reading by Jon Page. Short and to the point, but a great exploration of format, genre, and authors.

Well, that it for now. Special thanks to Beau Gunderson, Steven Jonas, Nicholas Felton (and many others) for sending in links and tips on where to find many of the above mentioned work. If you have a data-driven year in review please reach our via email or twitter and we’ll add it to the list!

If you’re interested in learning about how people generate meaning from their own personal data we invite you to join us for our QS15 Global Conference. It’s a great place to share your experience, learn from others, and get inspired by leading experts in the growing Quantified Self Community. Early bird tickets are on sale. We hope to see you there.

If you’ve made it this far here’s a fun treat: Warby Parker made neat little tool you can use to generate a silly personal annual report.

In 2009 Tim Ngwena switched on Last.fm and he’s been running in across all his devices ever since. Earlier this year he decided to take a deep dive into his listening data to see what he could learn.

I realized that I was listening to the same old thing and I began to think about changing what I was listening to. But how can I change? Where can I start? I also wanted to learn something about my music, what I was listening to and who was behind the sounds. I decided to focus on music because it was doable.

In this talk, presented at the London QS meetup group, Tim explains how he was able to make sense of almost five years of data and learn more about himself and his listening habits.

What Did Tim Do?
Tim explored his music data along side additional information such as location data from Moves to learn about his musical tastes, listening habits, and explore new visualization and data analysis techniques.

How Did He Do It?
Tim exported his data, used the Last.fm API and some data cleaning and organizational tools to create a simplified and extensive database of his music listening history and associated data. He then visualized that data using Tableau.

What Did He Learn?
Tim learned a lot about himself and what the music he listens to says about him. He describes a few of the most interesting below,

Basically 80% of my listening comes form 10% of the artists that I have in my library.

I’ve listened to Erykah Badu for over a week (7.2 days). It led me to ask what is she saying to me?

Monday is my jam time. I’m listening from the morning into the evening.

I listen to music mostly when I’m walking.

Tim also learned a lot through the process of designing and creating his data visualization. The visualization, which you can explore here, made him think about being able to see the big picture when he has so much linked data.

I think context is important and you need to see all that information in one place and the tools I’m using allows me to do this.

Two weeks ago we announced the release of the QS Access App so you could access your HealthKit data in tabular format for personal exploration, visualization, and analysis. In that short period of time, we’ve seen a good number of downloads and positive feedback.

We know from our experiences hosting in-person and online communication about personal data that seeing real-world examples of what is possible is what inspires people to engage and ask questions of their own data. With that in mind we’re excited to announce our QS Access Visualization Showcase.

We are looking to you, our amazing community of trackers, designers, and visualizers, to show use what you can do with data gathered from using the QS Access App. Make heatmaps in D3, complete analyses and visualizations in Wizard, or just make meaningful charts in Excel. If you’re visualizing your QS Access data we want to see it.

We also know that data visualization design and creation is not trivial work. To support the community and help expose the visualization work we’ll be awarding free tickets to our QS15 Global Conference & Exposition to individuals who use QS Access to create unique and interesting visualizations. We’ve earmarked two tickets (a $700 value) for outstanding work. If you’re selected, we’ll also work with you to showcase your work at the QS15 Conference and Exposition so other community members and attendees can explore and learn from their own data.

Example Visualizations

HealthKit is still new and the number of apps that integrate with it is growing by the day. At QS Labs we’ve done a bit of work making simple visualizations that are meaningful to us.

Steps and Sedentary Activity

Gary has an iPhone 5s which has native step tracking. We used the QS Access app to export his hourly step totals and made these simple line graphs in Excel. You can read more about what he learned from these simple data visualizations here.

How Much Do I Run?

Ernesto is an avid runner and enjoys running along the quiet trails in Los Angeles. He was interested to see how often he actually runs and if there’s any pattern to his running. Using a well-designed D3 template he was able to make a calendar heatmatp of his running distance.

Example Data

If you don’t have any HealthKit data to work with, or just want to play with some example data we’ve created a few files that you can use as examples. Download the files below from our GitHub account and make sure to read the documentation to understand where the data is coming from. Descriptions of the data files and sources are available in our QS Access Data Examples repo on Github.

At our 2013 Quantified Self Global Conference we were excited to share a variety of beautiful and insightful data visualizations from our community. In the months leading up to the conference we asked attendees to send in their own personal data visualizations along with a short description. In our 6 years of hosting Quantified Self meetups and events, as well as running this website, our forum, and social channels, we’ve seen the power of data visualization as a story telling medium. We exist in part to help people tell their stories – about the data they collect, the changes they create, and the insights and new knowledge they’re excited to share.

Today we’re sharing a few of our favorite visualizations from past conferences. The images and descriptions below represent a wide a variety of tracking experiences and techniques, and we hope to showcase eve more unique personal data projects at our upcoming QS15 Conference & Exposition.

Tracking Sleep by Anita Lillie

This is concatenation of screenshots from my sleep app. Most sleep apps don’t let you zoom out like this and still see daily/nightly detail, so I just made it myself. I like that it shows how almost-consistent I am with my sleep, and made me ask new questions about the “shape” of a night of sleep for me.

2.5 Years of My Weight by Mette Dyhrberg

I gained a lot of insights from this heat map. The most obvious weight gain was no surprise — that’s when I periodically don’t track. In any case, the big picture patterns are easily identified with a heat map. Realized looking at this heat map that the point of no return was mid-April 2012 — my data shows that was when I switched protein shakes with an egg based breakfast. I have since experimented and seen that protein shake in the morning seems to keep my blood sugar more stable and as a result my weight under control!

One Month of Blood Sugar by Doug Kanter

This is a visualization of one month of my blood sugar readings from October 2012. I see that my control was generally good, with high blood sugars happening most often around midnight (at the top of the circle).

Tracking Productivity by Nick Winter

Six Months of My Life by David El Achkar

This is my life during the past six months. Each square = 15 minutes. Each column = 1 day. This picture represents 138 days or 3,000+ activities.

My Thesis Self Portrait by Sara M. Watson

Here’s a period of a few days of webcam images taken using Stan James’ LifeSlice during the final days of editing my thesis on Quantified Self uses of personal data. Serious business!

Sleep and Meaningful Work by Robby Macdonell

In an average work day, I don’t consider communication (email, instant message, etc) to be terribly meaningful work. I’d much rather be working on building software. Getting more sleep the night before increases the amount of meaningful work I’m likely to do in a day.

“Instead, Beane and his front office have bought in bulk: They’ve brought in as many guys as possible and seen who performed. They weren’t looking for something that no one else saw: They amassed bodies, pitted them against one another, were open to anything, and just looked to see who emerged. Roger Ebert once wrote that the muse visits during the act of creation, rather than before. The A’s have made it a philosophy to just try out as many people as possible—cheap, interchangeable ones—and pluck out the best.”

Build Great Models . . . Throw Them Away by Mark Ravina. A digital humanities researcher makes the case for using data and statistical methods of modeling not to answer questions, but to come up with better questions. Really enjoyed the great examples in this post.

9-Volt Nirvana by RadioLab. This episode of the always interesting RadioLab tells the story of a journalist who was hooked up to a tDCS device for a sniper shooting exercise. The device helped her accuracy in the simulation, but then there was an unexpected after-effect. For three days afterward, the voices of self-doubt and self-abnegation receded from her consciousness. She talks about that experience directly on her blog. (Thanks to Steven Jonas for sending this one in!)

Tracking Sleep With Your Phone by Belle Beth Cooper. A great roundup here of iOS and Android apps you can use to track sleep. I especially appreciated the nice discussion of the current limitations of using mobile apps to track and understand sleep.

Basis to Roambi by Florian Lissot. Florian wanted to explore his Basis data. After using Bob Troia’s great data access script and some additional tools to aggregate multiple files he was able to create some great visualizations with Roambi and learn a bit more about his daily patterns of activity.

VisualizationsHow We Move in Cities by Human.co. It seems that making heatmaps based on movement is all the rage these days. Human has gone one step further than previousentries in this category by including motorized travel alongside cycling, walking, and running data. Don’t forget to check out the amazing GIFs as well.

“[...] visualizations are not the data. The data is not the sum of the experience. We’ve been inappropriately using data visualizations as the basis for statements and conclusions. We’re leaving out rigorous statistical analysis, and appropriate qualifiers such as confidence intervals. It’s exciting that we’ve become more and more a society of pattern-seekers. But it’s important that we don’t become lazy and cavalier with what we do with those observations.”

Reﬂections on How Designers Design With Data [PDF] by Alex Bigelow, Steven Drucker, Danyel Fisher, and Miriah Meyer. Researchers from Microsoft and the University of Utah sought out to understand how designers go the process of understanding data and creating unique visualizations.

Rain Ashford is a PhD student in the Art and Computational Technology Program at Goldsmiths, University of London. Her work is based on the concept of “Emotive Wearables” that help communicate data about ourselves in social settings. This research and design exploration has led her to create unique pieces of wearable technology that both measure and reflect physiological signals. In this show&tell talk, filmed at the 2013 Quantified Self Europe Conference, Rain discusses what got her interested in this area and one of her current projects – the Baroesque Barometric Skirt.

We’re back with another great set of articles, show&tells, and visualizations for you.

ArticlesHow to Make Government Data Sites Better by Nathan Yau. Government entities are some of the largest holders of interesting data. Nathan focuses this article on the difficulties of accessing and making sense of data from the United States Centers for Disease Control and offers some good ideas on how to make it better.

Sitting is Bad for You. So I Stopped. For a Whole Month. by Dan Kols. As a past frequent user of a treadmill desk and a sedentary behavior researcher I found this article intriguing. Yes, a bit silly in nature, but an interesting look at what happens when you go to the extreme. I especially enjoyed the integration of personal tracking in the piece.

Show&TellAnalyzing Squash Performance Using Fitbit by Ben Sidders. Ben sought out to see if he could learn anything from his step data to improve his squash playing. In this post he explains how he used R to access his data and plot it against his squash records, which he also records.

My Life As Seen Through Fitbit. Reddit user, VisionsofStigma, plots a year and a half of Fitbit data to find out what is related to the rise and fall of his activity.

VisualizationsFreeing My Fitbit Data by Bonnie Barrilleaux. Bonnie used our instructions for accessing Fitbit data in Google Spreadsheets then used Python to visualize her data. I especially like the histogram pictured at left. If you want to visualize your Fitbit data, she’s included her code in the post.

Iconic History by Shan Huang. As part of a the Interactive and Computational Design class at Carnegie Mellon University, Shan created a Chrome browser extension that visualizes your browser history. More about the project here.

Visualizing Last.fm History by Andy Cotgreave. Andy has been using Last.fm since 2006 to track his music listening activity. As a data scientist he was interested in what he could learned from all that data. In this four-part series, he explores his data along side data from eight of his friends. (If you want explore your Last.fm data you can export it using this awesome CSV export tool.)

In the Quantified Self community we focus on projects and ideas that help people access and get meaning out their personal data, including the information you can collect with your smartphone. If you have an iPhone, Android, or Windows phone you’re already have carrying of the world’s most sophisticated self-tracking tools. The GPS, accelerometer, the microphone, all of these tiny sensors make up a great set of tools you can use to understand how you move around the world.

I’m going to focus this short “how to” on geolocation data and mapping your movement, specifically using data gathered by the Moves application. Moves is a passive activity and location tracking tool available for the iPhone and Android. We’ve written a bit about it in the past and had a chance to interview their CEO, Sampo Karjalainen. I’ve been using it since May, 2013 and I wanted to share some neat tools and methods for getting a bit more out of the data Moves collects.

I find that visualizing my data on a map to be incredibly powerful. It might by my inner cartographer, but seeing my patterns of movement (or lack there of) in reference to known places and landmarks is a great mechanism for inducing recall and reflection on where I’ve been and what I’ve done. Hopefully you’ll use one of the tools or methods below to map you data and learn something new!

Moves Connected Apps
Like many self-tracking applications and devices, Moves has a API that many different developers have built services on top of. Here are just a few of the services that allow you to see your data on a map. Be advised that each of these services has access to your data. Make sure to read their Terms of Service before agreeing to the data transfer.

WebTrack. This is by far the most utilitarian data mapping tool. However, you shouldn’t get discouraged by the lack of fancy design because it gives you an very unique data view. When you use Moves on your phone you typically only see the “storyline” and the detected places you’ve spent time at. However, Moves is constantly pinging and recording your location when it detects movement. WebTrack allows you to see all those movement points by hovering over the associated timestamp.

Fluxtream. You might know Fluxtream as Friend of QS and a great open-source data aggregation tool. They’ve set up a “Moves Connector” that allows you to import and visualize your Moves data. Because Fluxtream is set up as an aggregation and visualization tool you can also map other interesting data sets. Want to know where you were tweeting last week? Fluxtream will map it for you. (You can see me tweeting on a CalTrain ride between San Francisco and Palo Alto below.)

Zenobase. Another interesting data aggregation service here. Zenobase treats your Moves data bit differently. Rather than importing all the movement geolocation data it focuses on your place data and visualizes those locations. I like the high-level view it start with, but make sure to keep zooming in to see more specific place data.

Resvan Maps. This mapping application adds a unique twist to the typical mapping visualizations. It will plot your places, paths, and categorize paths depending on the activity (transport, walking, running, and cycling). Additionally, you can create “analysis cirlces” and have the application compute the time you spent in a certain location you bound (it aggregates to hours:minutes per day).

MMapper. This method for mapping your data, developed by Nicholas Felton, is by far the most technical, but it produces some really neat visualizations. You’ll have to download Processing and follow the instructions Nicholas provides on the Github repository page here. The great thing here is that the mapping and data access is all happening locally.

Move-O-Scope. Another great mapping application here from the folks at Halftone.co. They’ve probably completed the most thorough mapping and exploration tools for your Moves data. After linking your Moves account you can explore maps by activity type, day of the week, and custom data ranges. Additionally, they’ve implemented a neat feature for exploring place data. You can see how many times you’ve visited a specific place, where you’ve come from and where you go next, what days you typically visit, and your typical time of day at that place. (See this post for background on why they created this nifty tool.)

Map It Yourself!
If you don’t want to trust your data to a third party, but you still want to explore your movement maps there is really great option for you. Our friend and co-organizer of the QS LA Meetup, Eric Blue, recently published a method for easily exporting your data: the Moves CSV Exporter. You’ll have to login and use the Moves pin system in order to download your data, but Traqs isn’t storing your data, just providing a way for you to access it. The tool allows you to download and explore your activity, summary, tracks and place data. We’ll focus on the place data for creating maps. You can also use your full tracks history for mapping all the geolocation points Moves collects.

Because this data is based on latitude/longitude coordinates there are many different methods available that you can use to map your data. I’m going to focus on two here: Google Fusion Tables and CartoDB (if you know of others share them in the comments or our forum).

Google Fusion TablesFusion Tables are a new Google Drive tool that you can use to store, analyze, and visualize many different types of data. Once you download your Moves places.csv file you can upload it to a new Google Fusion Table. Once you upload your data, which takes about 2 minutes, you’ll see a menu bar and three tabs: Rows, Cards, Map of longitude. Just click on the “Map” tab and you’ll see your data already placed on a map. If you want to see a heatmap rather than a point map just navigate to Tools -> Change Map and you’ll see an option for a heatmap on the lefthand side. This is just the tip of iceberg for mapping fusion table data. You can learn more about different mapping methods and tricks here.

CartoDBCartoDB is a visualization and analysis engine for geospatial data. I’ve been using it to play around with a few of the different geolocation datasets that I have (I actively keep three). Although it is paid service, they do offer a free plan for smaller datasets, which is perfect for your Moves data. Again, you’ll have to upload your places.csv file to a new table once you set up your account. Once the data is uploaded there are quite a few different map visualization wizards you can use to view your data in different ways. Pesonaly I like playing with the “Torque” visualization that gives you a real feeling of space-time to your data.

TileMillTileMill is an interactive map design tool from the folks over at Mapbox. If you’re looking to create custom maps with your data that you can format, style, and share then this is a wonderful tool to use. At first glance it’s a little daunting because it looks like a mashup of a CSS editor and map tool. That actually gives it the unique power to drive customization. Don’t be afraid, it’s not too hard to get started with. Mapbox has provided a great “crashcourse” to get you started with importing data, saving it as a new layer on your map, and then manipulating how it looks on your screen. If you want to go just a bit farther you can also add legends and informative popups to describe your data points. Mapbox also offers a free hosting plan if you want to share your interactive maps on a webpage. For example check out my MovesMap here, where I added a quick styling to manipulate the point size in relation to the time spent at a location.

Hopefully you’ve learned something new from this. If you map your Moves data (or any other geolocation data) we want to see it! Leave a link in the comments, post it in the location mapping thread on the QS Forum or get in touch on twitter!

We’ve all come face to face with tracking some aspect of our life only to realize that we’re not quite sure how to get started. Enrico Bertini encountered this roadblock when he began thinking about tracking the amount of time he spends engaging in “focused work.” As an information visualization researcher at NYU he decided on a simple rule that would give him the most accurate data that represented his interests: if it wasn’t tracked then it wasn’t focused work. In this talk, given at the New York QS meetup group, Enrico explains his process and shares his findings (including some great visualizations).

(Editor’s Note: Enrico also co-hosts a great podcast on data visualization and information design called Data Stories. I highly recommend listening. If you’re looking for a place to start try Episode 17: Data Sculptures.)

As you may know, we get excited when someone in our community uses interesting data visualizations to help tell their self-tracking story. Jana Beck is no exception. As a woman living with Type 1 diabetes she’s constantly learning how to better understand what her Dexcom data is telling her. In this talk, Jana follows up on her previous show&tell presentation with some new visualization techniques she’s using. If you’re interested in Jana’s methods be sure to check out her Github repository and her work with Tidepool.org.

(Editor’s Note: I very interested in Jana’s use of Chernoff faces for multivariate data visualization. If you’re using this type of visualization for your own data I would love to see it. Get in touch.)

Partners

Sponsors

Friends of QS

We recently started a program to invite QS Toolmakers to contribute directly to funding our events. We call this program Friends of QS. If you would like to participate we invite you email us to learn more.