RunKeeper – Lincomatic's DIY Blog

Accuracy of Pyle PSWGP405BK vs RunKeeper Revisited

This morning, I repeated my Eaton Canyon loop with the Pyle PSWGP405BK and RunKeeper on the iPhone 4S. This time, both of them were quite a bit more accurate than on the last hike.

Pyle PSWGP405BK

This time round, the Pyle tracks the shapes of the switchbacks fairly accurately, albeit with an offset. There is a fair amount of overshoot, however, in the last switchback.

RunKeeper on iPhone 4S

RunKeeper on the iPhone 4S again does much worse than the Pyle, but still much better than in my previous recording. Out of the various iPhone tracking apps I’ve tried, I still find RunKeeper to be the most accurate. I would like to try this route with an Android phone sometime, to see if it does a better job with outputting accurate location data. Here’s the tabular data:

	Distance (mi)	Calories
RunKeeper	3.47	491
Pyle PSWGP405BK	3.31	593
Pyle PSWGP405BK track on RunKeeper.com	3.43	475

Interestingly, the disparity between RunKeeper’s distance calculation of its own track vs its calculation of the Pyle’s track was much smaller today, but this can be explained by the fact that RunKeeper didn’t meander as badly today, so the tracks lined up more closely.

On the other hand, disparity between the results I got today and my previous post show how GPS accuracy can vary, even in similar weather conditions, at a similar timeof day on the same route. I would conjecture that the increased accuracy today has to do with a more optimal geometry of the satellites in the sky. Unfortunately, since the devices I used don’t output detailed satellite signal and skyview data, there is no way to verify for sure.

This evening, I again pitted RunKeeper against the Pyle, walking on a very flat route, composed mostly of straight lines. Here are the surprising results:

	Distance (mi)	Calories
RunKeeper	3.38	381
Pyle PSWGP405BK	3.22	355
Pyle PSWGP405BK track on RunKeeper.com	3.27	354

Like this morning, the RunKeeper and Pyle tracks were a lot more similar than on previous days. RunKeeper’s distance was .11mi longer than its calculation of Pyle’s track, which makes sense, because its track was more wiggly, as usual, which adds distance. But what’s inexplicable is that on this track, the difference between the Pyle’s own distance calculation versus RunKeeper’s calculation of its track is only .05mi, only 1.5%. Contrast that to this morning’s hike, where the difference was .12mi or 3.6%.

Accuracy of Strava vs RunKeeper vs Pyle PSWGP405BK vs Garmin Forerunner 10

I went out for a hike today, with Strava and RunKeeper simultaneously running on my iPhone 4S, and the Pyle PSWGP405BK on my wrist. Unfortunately, my wife had the Garmin Forerunner 10, so I have to use an old track, recorded last week, at approximately the same time of day. I know that it’s not an ideal comparison, but it will have to suffice for this article. The route was in mountainous terrain, through a canyon … a much more challenging for a GPS than the flat route that I used in my last comparison. Here’s RunKeeper’s take on the track: Strava’s take, running simultaneously on the iPhone with RunKeeper: Strava’s exported GPX file, imported to RunKeeper.com: What’s revealing is that given the same data, RunKeeper.com and Strava calculate essentially identical distances. This becomes important when we compare them to the Pyle. Strangely, RunKeeper.com shows a 13 second shorter duration, but identical average pace. The data downloaded from the Pyle and displayed in GPS Master are identical to what displayed on the watch: I used the Pyle’s included wireless heart rate monitor on this hike, and its connectivity was flawless during the entire recording. I didn’t have to moisten the contacts in order to get a good reading, which was also nice. GPS Master shows my average heart rate was 121, and the max was 155. The plots above are speed, altitude, and heart rate. The GPX file exported from GPS Master and uploaded to RunKeeper.com: A comparison table gleaned from the above data:

	Distance (mi)	Calories
Strava	3.7	N/A
Strava track on RunKeeper.com	3.71	600
RunKeeper	3.53	422
Pyle PSWGP405BK	3.27	526
Pyle PSWGP405BK track on RunKeeper.com	3.40	555

Once again, the Pyle PSWGP405BK and GPS Master displayed identical data, and the calculated distance was shorter than RunKeeper’s calculation using the Pyle’s exported GPX data. I am convinced that the Pyle’s distance calculation is too conservative… 3.27mi vs 3.40mi calculated by RunKeeper.com on the same data. Strava calculated 3.7 mi, which is too long, due to its poor filtering of the iPhone’s noisy data stream. The Google Earth plots below will reveal more information. Here is a Google Earth plot of Strava (green), RunKeeper (red), Pyle (blue), and Garmin FR10 (yellow) in the most challenging part of the track: Strava (green) vs RunKeeper (red) Viewing the track reveals why Strava calculated the longest distance. Strava’s filtering algorithm is clearly inferior to RunKeeper’s in handling the iPhone’s poor quality location data stream. Pyle (blue) vs Garmin (yellow) Again, not an entirely valid comparison, since the Garmin track was recorded on a different outing, but it’s quite obvious that the Garmin outperforms the Pyle in accuracy. The Garmin is the only device that was able to trace all of the switchbacks on the trail. Even in the less challenging parts of the track, the Pyle wanders a bit. Still, the Pyle’s accuracy far exceeds that of the apps running on the iPhone. I am very disappointed that several reviewers said that the Pyle is very accurate. Clearly, they didn’t do their homework. The Pyle PSWGP405BK consistently calculates distances too short. The Forerunner 10 kicks Pyle’s ass. But the Pyle has a compass, navigation, and heart rate monitor for about the same price, and is not that bad, as long as you don’t mind the fact that it miscalculates distance, and have to use other software in order to extract the correct distances from its track. If you want to play with my data, you can download the tracks an view them in Google Earth yourself: eckml.zip Strava doesn’t export KML, but Google Earth can easily import its KML file. I have also included full size image files of the screen shots I showed above. In case you decide to compare the gory details of the tracks, when I got to the top of the winding trail above, I took a little break to fix my socks in a tree-covered area. The FR10 track doesn’t show me deviating from the trail, because I didn’t take the detour during that outing.

Previous Related Post: Garmin Forerunner 10 vs Pyle PSWGP405BK

Next Related Post: Accuracy of Pyle PSWGP405BK vs RunKeeper Revisited

Accuracy of Garmin Forerunner 10 vs RunKeeper vs MapMyRun

Even though I’m the engineer in the family, my wife is more of a gadget freak than me. This year, she beceame interested in tracking her workouts. She started out with a FitBit, and found that to be rather useless, so we returned that. Lately, she’s been into training for a half marathon, so we started using GPS workout tracking apps on an iPhone 5 and iPhone 4S.

We played with several different apps: Kaiser EB Walk, Nike+, MapMyRun, NeoTreks, and Runkeeper. It’s interesting how you can run several apps simultaneously on an iPhone, and get wildly varying metrics from them. The distance and pace vary quite a bit between apps, and I think the main reason is that the realtime geodata coming out of the iPhone is crap. Therefore, the app developers implement different filtering strategies on the data, leading to the different calculated metrics.

I find it amazing that the iPhone has an onboard GPS, accelerometer, and compass, and still manages to output such bad location data. Granted, the GPS antenna is quite small, but GPS watches are pretty small these days, and work much better. My guess is that Apple needs to do some work in iOS’ sensor data processing code. If only they would give us access to raw GPS data, rather than forcing us to use their garbage location API’s…

Following the recommendations of friends, my wife recently bought a Garmin Forerunner 10 (FR10) GPS watch. I was skeptical that it was worth the $130, price tag, and was very interested in finding out how much better it would perform than the iPhone. After using it for only a few days, I must say that I am very impressed. Unlike my 10 year old Garmin eTrex GPS, the FR10 gets a fix very quickly, and manages to keep the fix even under moderate tree cover and in canyons.

Below, I will compare the simultaneously recorded outputs of RunKeeper, MapMyRun, and the FR10. I chose RunKeeper and MapMyRun as the iPhone apps, because they offer easy access to the logged track data in the form of GPX files, and both websites allow you to upload/edit track data. Note that the MapMyRun and RunKeeper data were recorded simultaneously on my iPhone, so they share the same sensor data.

Route plotted on Google Maps:

Calculated distance: 1.3mi

RunKeeper

Despite the fact that the route is largely in a flat area, with very little tree cover, notice how the track wiggles and meanders.

MapMyRun

The MapMyRun track, as expected, also wiggles quite a bit.

MapMyRun Track on RunKeeper.com

The above plot is taken from a GPX file exported from MapMyRun.com of the workout, as recorded by MapMyRun. Notice how RunKeeper.com, using the same track data, calculates the distance/pace/calories differently. I have found that MapMyRun consistently calculates the longest distances, and highest calorie count of the bunch.

Garmin FR10 on Garmin Connect

Note how the Garmin track is very clean and smooth, and tracks the roads reasonably closely.

Garmin FR10 Track on RunKeeper.com

The above plot is of GPX data exported from Garmin Connect, and then uploaded to RunKeeper.com. The FR10 stores the track data in a proprietary FIT format, but luckily, their website is able to output GPX. Again, notice that RunKeeper.com used the same data, and yet, calculated different results.

Comparison

	Distance (mi)	Calories
RunKeeper	1.40	153
MapMyRun	1.51	178
FR10 (Garmin Connect)	1.28	81
MapMyRun Track on RunKeeper.com	1.44	147
Garmin FR10 Track on RunKeeper.com	1.29	121
Google Maps	1.3	N/A

Below are some data from a couple of other routes that I tested:

Euston Loop

	Distance (mi)	Calories
RunKeeper	2.53	279
MapMyRun	2.71	306
FR10 (Garmin Connect)	2.44	156
MapMyRun Track on RunKeeper.com	1.44	147
Google Maps	1.3	N/A
Manual Plot on RunKeeper.com	2.55	N/A

Winston Loop

	Distance (mi)	Calories
RunKeeper	1.79	186
MapMyRun	1.83	205
FR10 (Garmin Connect)	1.66	103
Manual Plot on RunKeeper.com	1.73	N/A

Discussion

While my sample size is small, some trends emerge. The Garmin FR10 consistently calculates the shortest distances and calories. MapMyRun consistently calculates the longest distances and highest calorie counts. There is a difference between the calculation methodology between RunKeeper.com and MapMyRun.com, because given the same GPS track, MapMyRun calculates a longer distance (1.51 vs 1.44mi) and a higher calorie burn (147 vs 178cal) than RunKeeper.com. On the other hand, Garmin Connect calculates shorter distances and calorie burns than RunKeeper.com, given the same track.

I have tested several other times, and the above trends hold. Furthermore, the Garmin FR10 gives the smoothest, most accurate GPS track, especially when in a canyon, or under heavy tree cover. Below is a canyon track, Garmin FR10 in red, RunKeeper in blue:

Notice the vastly improved accuracy of the Garmin FR10 track vs RunKeeper running on the iPhone.

Finally, a canyon track which was recorded on my iPhone 4S, running RunKeeper and MapMyRun simultaneously. RunKeeper is in red, MapMyRun in blue:

The image above shows the vast differences which sometimes occur between the way that RunKeeper and MapMyRun interpret the same location data stream. Neither track closely follows the actual path. It’s pretty obvious that the Garmin FR10 is a worthwhile upgrade from using the iPhone as a sensor.

Next Related Post: Garmin Forerunner 10 vs Pyle PSWGP405BK