Tuesday, August 9, 2011

Strava Suffer Score decoded

Strava recently debuted its "Suffer Score", which attempts to quantify in one way how hard an activity was. Suffer score, they claim, is based on heartrate: ride further or harder and suffer score is higher.

Suffer score is a good move for Strava. Strava's market segment is informal competition. Other web sites have logged how far people ride, and where, but Strava really locked into the demand for competition: competition via social networking. To date these rankings have been based primarily on speed over "segments": routes defined by users, for example on roads or trails, typically up climbs. They then added contests for volume: most miles ridden in a week and most feet climbed, for example. But with Suffer Score, they combine the two: a metric combining both quantity and quantity.

Why's this important? Of course not everyone can climb fast enough to compete for KOMs on popular climbs. But a heartrate-based metric levels the playing field to a large extent. Anyone who's been riding regularly for long enough can go out with the goal of generating an "epic" suffer score for the day.

Suffer score is an obvious correlary with the Coggan "training stress score" (TSS). TSS is an attempt to quantify the training stress of a ride: it's effect on fatigue and its stimulus for physiological adaptation. That both of these can be captured with the same simple metric is far from obvious, but TSS is a blunt instrument, and its popularity among hardcore power meter fans proves people find it useful.

But Strava's focus isn't on hard-core training junkies. It's on the typical Mission Cycling or SF2G rider: people who like to ride, and when they ride like to ride quickly, and as a consequence of this quick riding tend to acquire fitness. However, the goal of every ride is still the joy of the ride, not strictly to improve fitness. So "suffer score" is a "don't take me quite so seriously" number by which long-hard-rides can be compared. "Pain is good", it suggests, not only for it's "training stress".

The big question, however, is how Suffer Score is calculated? To test this, I used a Perl script to generate ride data at constant heartrate. My "rides" consoisted of perfect 1 km radius circules in the Bonneville Salt Flats ridden at a perfect 10 meters per second. Each ride had a different heart rate, perfectly constant, or a different duration.

First I tried using the GPX file format for this purpose, but Strava wouldn't digest the heartrate numbers. So I switched to TCX. This worked nicely.

Here's one of my rides:

From a few tests, I was able to guess the following formula was used for Suffer Score:

Suffer score = K1 t1 + K2 t2 + K3 t3 + K4 t4 + K5 t5

where:
t1 = time in hours in zone 1
t2 = time in hours in zone 2
t3 = time in hours in zone 3
t4 = time in hours in zone 4
t5 = time in hours in zone 5

I like this approach, although I'd be tempted to increase K5, for example to 200. Time in Z5 is very, very painful. Of course this depends on having a good number for maximum heart rate, which may not be the case. In fact a geometric progression would have worked well here: 12, 24, 48, 96, 192. I think this would have helped the criticism I've been reading that it's too endurance-focused.

38 comments:

Whoops -- I'll fix the blog (I don't know where "bias" came from -- doing too many semiconductor device simulations for the day job).

To be fair, then did mention TRIMP in the support forum. So they're not practicing active deception (benign neglect if anything). Not sure about the TSS descriptions.

I still need to check the formula against time gaps in the ride to see if they're doing any time averaging. So far I checked linearity only against rides of constant HR. Spent too much time debugging my ride data generation script.

Hi Andrew, this is Mark Shaw from Strava. You're absolutely right, we should make reference to TRIMP in our "What's this?" section, as it is correct to say we're using a modified TRIMP score. It wasn't our intention to hide this fact and it was an oversight not to mention it directly on the site. As Dan points out, our intention is to be open about this, as we do discuss it on support forums. We've been getting a lot of feedback from our users which we're using to help make adjustments to the score and the language around it. We'll add a mention of TRIMP to our site - thanks for the heads up.

Hi Andrew, this is Mark Shaw from Strava. You're absolutely right, we should make reference to TRIMP in our "What's this?" section, as it is correct to say we're using a modified TRIMP score. It wasn't our intention to hide this fact and it was an oversight not to mention it directly on the site. As Dan points out, our intention is to be open about this, as we do discuss it on support forums. We've been getting a lot of feedback from our users which we're using to help make adjustments to the score and the language around it. We'll add a mention of TRIMP to our site - thanks for the heads up.

Ps.: what I wrote earlier might be good to rate the toughness of the ride, but it should not be mixed with TSS values. I did not mean that. Now that you wrote about it I read a bit into these things, and I have found lots of interesting stuff, thanks again!

Great post. Since going premium I've been confused by the suffer score. I do a very, very hard race of 1.5hrs with almost all my time in Z4 but I get a greater SS on a 4 hr club ride spent in z2 & z3. The race caused a lot more "suffering"

Why can't Strava just give us a TSS and IF score. And ideally all the analysis you get in WKO+ for a "super premium" membership fee?

The terms "TSS" and "TSB", etc, are trademarked, so they can't be used without license. But I think Strava views its market as different: not focused on power meters, but more the average rider who just likes to ride hard without formal metrology. So it's intended to be a "fun" number. They've upgraded their metrology to include a "red zone" score, which is the component of "suffer score" from Z4-Z5. Actually, it would make a good API project to extract a TSS-like statistic from Strava data. I've got a bunch of projects on my plate but that wouldn't be too hard.

There's no doubt Strava is a broad mainstream product. I can understand that as a business strategy at start up and beyond but there's a huge gap in the market for a single cohesive system.

There's currently very little you get from Strava premium. Most people I know pay cos they think they should rather than for what Strava provides. If they want to drive premium membership then they need to be looking to add significant functionality for the growing numbers of club cyclists. To my mind these are the people who will pay...if you just enjoy riding your bike then why go premium?

My ideal product would be trainer road, something akin to WKO+ all integrated with Strava.

Its funny the comment about no one being on Strava in Europe. It seems to be hitting a tipping point here in the UK.

GoldenCheetah has some Strava integration. I've not followed it, but I suspect it does what you want. Try that out...

Strava has a simple API which makes it easy for apps to upload or download rides to Strava accounts. This makes it fairly easy to implement alternative front-ends. It will become simpler when version 3 of the API comes out within the next few months.

For example, I'd like to do an app to extract critical power curves from Strava rides. It's easy to do for one ride, harder if you want some sort of global curve. But GoldenCheetah already offers this capability.

Awesome Post Dan, thanks for the reas up. Recently I have been perplexed with the idea of renewing my subscription of Strava as I love the Golden Cheetah program. The Suffer score I am sorry to see only works with a heart rate, they dont include it in a power meter which I believe should be. I hope to see more meaningful updates to Strava for the people that pay for a subscription.

I fully agree: there's no reason they couldn't do a suffer score with power meter data. Perhaps they wish to limit the computational burden, since they're dealing with their exponential increase in users as their user base has expanded from it's previous focus in the San Francisco Bay area. In running they use pace for analysis.

Thanks for the INFO! Very few of us on STRAVA in Saigon, it helps with the motivation. I moved my max HR up by 2 beats, found that I am in much better shape, but get home from a 130km ride and I get a "tough".... would have been "extreme" (nearly 50 points more) for similar rides this time last year.

Any ideas about the Out and Back issues? I have emailed the company and there seems to be no fix in sight. Instead, people cut the course short and get outragous speeds. We really have about 80km or good riding in Ho Chi Minh City where we can really just let it out. So, this error is a bit of a shame!

Thanks, Tina! I have been to Saigon and really liked it. I didn't try cycling there, but other places in Vietnam, like De Lat 300 km to the north, seemed like excellent cycling locations.

On the out-and-back problem I have spoken to Strava in person about that and I expect it will be solved eventually. The problem is that the matching between the start and end point is relatively loose. So if you do an out-and-back, and match the start and finish (the same point), then if you have visited 80% of the points in between (80% of the route, cutting short 20%), then it might match you. What they need to do is to add in special checkpoints which must be matched in addition to the start and finish. For an out-and-back that would be the turn-around point. I had expected they would have done this by now but I think their priorities have been elsewhere, for example on handling the enormous number of new users.

I recently ran a 10 km race and it gave me credit for the 10 km race segment using only 8 km of my data: after 8 km I passed close to the finish line and it concluded I had matched the segment. So my time for that segment was extremely good... same problem.

I still not understand!so if k1= 12/hrand hr means the time spent in the zone and then i multiply for t1 that is time spent in zone 1... the in the final i have just 12+24+... because the hr cut t and just leave the score for each zone??I know that some of my cal is not correct, but what??? pls give an example!

K1 t1 + K2 t2 + K3 t3 + K4 t4 + K5 t5

where:t1 = time in hours in zone 1t2 = time in hours in zone 2t3 = time in hours in zone 3t4 = time in hours in zone 4t5 = time in hours in zone 5

With coefficient roughly equal to the following (may be off by 1/hr):K1 = 12/hrK2 = 24/hr

No: (12/hr) is a factor with units per hour. So if you multiply (12/hr) by (2.5 hr) then the hours cancel, and you get 30. If you multiply (12/hr) by (250 minutes) you need to multiply by (1 hr / 60 minutes) and now the hr, minutes both cancel and you are left again with 30. You do this separately for each HR zone, counting the number of individual seconds spent in each, and you end up with total suffer score.

I am not certain how power is calculated, but unless rider weight + bike weight on Strava = the weight of everything together, including water bottles, water, food, clothes, helmet, shoes, tools, etc, the power calculation will be biased low. The issue here is I suspect most users use their bare bike mass and their naked body mass, so power estimates tend to be too low.

Not much, Franz. TIme is limited for these projects, unfortunately. Most of my blog writing is done on crowded Caltrain commutes.

But the training score, fitness, and fatigue are very similar to CTL, TSB, and ATL from TrainingPeaks. Those parameter names are trademarked. GoldenCheetah has similar metrics. The issue is you need to use a power meter every ride. I didn't use one during the Berkeley Hills Road Race, a considerable physical stress, and so that shows up as a total rest day in my metrics. It takes months to flush that omission from my scores.

At GoldenCheetah I proposed an algorithm that a regression could be done on distance and climbing of existing rides to estimate these scores for riders without power data. Strava could also use estimated power. Anything is better than assigning zero score to a ride without power meter data, assuming there's no data from car or train rides, for example, contaminating these.

t1 is the time in zone 1. I say "in hours" to be easier for people to understand, but really that's redundant. Time has units of "hours" but it can also have units of "minutes" or "seconds", etc. Suffer score has no units. So I need to multiply t1 by something which has units of 1/hours. Then the hours will cancel (one in numerator, one in denominator) and I'll be left with something with no units, which is what I want.

If I were to specify time in minutes instead of hours then I'd need to multiply by a unit conversion factor, for example (1 hour / 60 minutes).

This is common in physics. For example, if I have a velocity (units distance/time) and multiply it by a time (units time) the times cancel and I'm left with distance, which is what I want for riding a particular speed for a particular time. When I put the multiplier in units 1/hr, you're guaranteed to get the correct result.

It's always important to keep track of units. Suffer score, in order to be simple to understand, is presented without units: you can't say "my suffer score was 100 hours" or "my suffer score was just over 4 days" or "6000 minutes".... it's just 100.

But I'd say Strava calculates SS differently for running, see for example this workout:http://www.strava.com/activities/215099305/heartrateOnly 25 minutes in Z4 and 15 in Z3 got me SS of 130.Could You please also try to decode running SS?

Thanks for the comment! If I run the cycling calculation for your run I get 46.03, but Strava reports 129. So you're right -- it needs to be recalibrated! This could be done easily enough with several runs with different HR distributions. But that rate of inflation, which is a factor of 2.8 in your case (and this may be all there is to it: an inflation by a factor 2.8) seems high. Consider Ironman results: the average of the top 3 was 4:27 on the bike and 2:51 on the run, a ratio of 1.56. If Ironman is balanced then the ratio should be closer to this, assuming average heartrates are somewhat similar. I'd need to look at a few activities to isolate coefficients though (it's a simple algebra problem).

Hi! I was wondering for some time, how I can get SS of over 200 in an hour when running (my best was 247/245pir in 51minutes in 10km race), but biking uphill and giving my best would only get me about 120/100pir in about two hours (of which about an hour was working at full going up).I'm also pretty sure I have zones set about right. I'm estimating my 1 hour max HR(FTP HR) to about 161 in it is in the middle of Strava's Z4(156-167), and getting into Z5(>167) is very hard (on bike I've been there only few times). Getting over 170 for me is "suffer as hell" :)

I think "suffer score" is fairly arbitrary -- it's whatever Strava decided it should be based on their participants. I'm not sure comparing running to cycling is valid on any objective basis other than they took some sort of average of their cycling and running activities.

Well the way I see it - suffer score is (or rather should/could be) an estimate of how hard You were working (whatever that might be) based on Your HR.It doesn't really matter whether You were running or cycling or doing something else - You got Your "pump" (Your heart that is) working hard. In this regard different activities could potentially be compared in terms of how much stress they caused to You heart.Unfortunately Strava obviously doesn't have the same view on SS, because it is a lot harder (in terms of heart stress) to get high SS in cycling then it is in running.Some kind of normalized SS would be much more informative if You could (provided You have Your zones set correctly) only get SS of maximum around 100 in one hour - that would correlate with FTP power/Lactate Treshold.You could get Your heart working harder for shorter time, of course, or maintain hours of less stress, but getting over 100 in an hour would mean, You got Your zones set too low...In fact according to my experiences getting SS for Cycling over 100 in an hour is almost impossible for me, but for running i can get over 250 in less then an hour...

I have been running into a slightly different issue in relation to Suffer Score... If I do an exercise that has GPS enabled, for instance RUNNING, Suffer Score is based on "Moving Time" NOT total elapsed time (even if you manually change workout to RACE or WORKOUT)

I would like to be able to train and go for a run and do static exercises (e.g. Squat Jumps) and then continue running. However, currently my heart rate for the period of time spent doing squat jumps is not contributing to my suffer score :(