Tuesday, January 18, 2011

Garmin FIT activity splitter to eliminate large time gaps

I've gotten useful code on my third project for processing FIT files using Kiyokazu Suto's Garmin::FIT package for Perl. First, I described fit_to_cols, which extracted selected data from FIT files and formatted it in a space-delimited file. Next I described fit_filter_motor_segments which attempted to identify segments where the Garmin was accidently left running in a fast car or train. That project's still being refined, as I described last post.

Here is perhaps the most useful of the three, fit_split_on_gaps, which finds gaps of some specified minimum duration (default 8 hours, or otherwise specified with the -tgap option) and splits the FIT data into multiple sub-files at any gaps found which meet or exceed this threshold.

It's fairly common in my experience to forget to "reset" my Garmin between activities, despite having set it to warn me at the start of a ride if I have not. GoldenCheetah, for example, contains some nice code to deal with this problem, allowing rides to be split at selected gaps. However, since I've not been using a power meter since last summer, I've been using Strava a lot more than GoldenCheetah. And Strava lacks and ride-splitting capability.

Enter fit_split_on_gaps. Its default behavior is to take each FIT file and, if it finds any gaps between meaningful data (ignoring the "power down" and "power up" records which sometime pollute idle periods), it partitions the original file into sub-files, naming each according to the Garmin convention of "yyyy-mm-dd-hh-mm-ss" with a "_split.fit" suffix (although the suffix can be changed) in the same directory. Alternately the files can be written to a separate directory with or without (default) a suffix.

There's a few options. The most important is "-h" which prints help. I won't try and describe the other options, as the code may still be modified and these may change. However, so far there's two primary modes:

To take one or more input file, and produce output files (by default only if gaps are found) of individual segments.

To write one file, or standard input, to standard output, retaining only one segment, and discarding the others

The project took longer than I expected, and honestly I wonder how robust it is. Presently I assume every record without a time stamp is a "definition" and should be included in each split file. So these are always written. Records with time stamps are included in a given file only if the record falls between the gaps which delimit the segment. It's assumed the records are in chronological order, so if gaps are found, everything in between is written to the same file (assuming it's not discarded as a "fragment").

Thanks again to Kiyokazu Suto for doing such nice work on Garmin::FIT.

8 comments:

I've run into the same problem of forgetting to reset my Garmin Edge 500 between commuting to work and commuting home.

Previously I'd used Ascent to load a .fit file and then write each lap to its own .tcx output prior to uploading them to Strava.

In searching for an easier solution I'd actually come across Kiyokazu's perl module before but never done anything with it. Glad that you took the time to write your script and post about it. I Just used it and it seemed to do the right thing!

Thanks, Ben! It's encouraging when other people can get stuff to work. I've been reading through the FIT SDC docs from Garmin and there's a lot there, so I'm still refining the code a bit. For example, I'm just now putting in a check that the file type = 4 ("activity") to avoid confusion from other file types (for example, course descriptions).

Thanks for the comment! I've actually got to fix these codes... the distance fields in the records need to be adjusted to account for the deleted segments, then in the lap and final ride data fields, the totals need to be adjusted. Honestly when I look at Garmin:FIT it's far from obvious how to do anything, even something as simple as adjusting distance.... so much indirection. Hopefully I can produce an updated version reasonably soon.

Really, I'd hoped Strava would adapt the same algorithm. Not yet, obviously.

Thanks -- I noticed the latest Garmin firmware update broke this. I need to check to see if it runs with the latest update of Garmin::Fit. Honestly I've never been comfortable with the Garmin::Fit Perl module. I'd prefer to deal directly with the FIT SDK.