My Karoo Test Journal

Chat about Hammerhead's Karoo in general.
Post Reply
User avatar
drisotope
Posts: 137
Joined: Thu Mar 08, 2018 4:12 pm
Location: SoCal, USA!

Hammerhead: neither lucky nor good

Post by drisotope » Sun May 06, 2018 5:47 pm

Statistically, if you were to give a blindfolded man an unlimited supply of darts and faced him to a dartboard, he would hit a bullseye... eventually. I guess that's our hope for the guys at Hammerhead as well, because their software "tweaks" are about as well-aimed as a blindfolded man throwing darts. Whatever they fixed in the latest update (anything, anyone?) broke most of the basics. It's absolutely absurd. I'm gradually moving from "I don't actively recommend the Karoo to people who ask, simply because it's not a finished product," to "How is it even conceivably possible for anyone to mess anything up this thoroughly?"

Edge 520 / Karoo

Moving Time
1.44.46 / 1.47.38
Elapsed Time
1.56.53 / 1.56.48
Elevation
686ft / 630ft (Strava corrected to 821ft)
Weighted Avg. Pwr
178W / 174W
Total Work
894kJ / 908kJ
Avg. Cadence
76rpm / 76rpm

Basically, every time I would come to a stop, the time would keep running for 8-11 seconds, and the elevation smoothing has fallen back to about 3 software updates ago. The Edge consistently shorts me, and now the Karoo is reading about 10% lower than the Garmin. Uncorrected, my wife recorded 758ft over the same route, to the Karoo's 630ft. That's around 20% lower.

Hey Hammerhead guys, STOP BREAKING STUFF. You've solidly established an inability to add features without breaking existing ones. And I mean the most basic ones. :|

JohnJ80
Posts: 223
Joined: Tue Apr 03, 2018 4:55 pm

Re: My Karoo Test Journal

Post by JohnJ80 » Sun May 06, 2018 7:48 pm

Well put. I'm basically sitting here with a unit that is, at best, a buggy $250 Edge 520 that I paid $299 for.

User avatar
drisotope
Posts: 137
Joined: Thu Mar 08, 2018 4:12 pm
Location: SoCal, USA!

Re: My Karoo Test Journal

Post by drisotope » Sun May 06, 2018 9:24 pm

I just realized I hadn't checked the numbers from Saturday's ride-- here they are, and they're equally bad. Worse, maybe.

Edge 520 / Karoo

Moving Time
1.45.25 / 1.51.25
Elapsed Time
2.22.11 / 2.22.12
Elevation
541ft / 496ft (Strava corrected to 630ft)
Weighted Avg. Pwr
171W / 158W
Total Work
807kJ / 834kJ
Avg. Cadence
71rpm / 72rpm

A full 6 minutes of non-paused time. The shown average speed on Strava is 13.7mph, average in ride analysis is 14.3mph. Average power drawn way down due to the laggy auto-pause as well. This whole update is a head-shaker.

User avatar
drisotope
Posts: 137
Joined: Thu Mar 08, 2018 4:12 pm
Location: SoCal, USA!

Not all speed sensors are created equal

Post by drisotope » Mon May 07, 2018 9:02 pm

So I decided to be proactive. The Karoo moves back and forth between two bikes, which have their own sensors. One bike has a Wahoo SPEED, the other has a Garmin magnetless speed sensor. I had never really thought to compare the two sensors directly against one another... until now. Today was the day for the Wahoo. I had both the Garmin and the Karoo "forget" the speed sensor, then re-paired it to both computers, and manually input the exact same circumference into both units. Did it help?

Edge 520 / Karoo

Moving Time
2.36.27 / 2.37.49
Distance
51.23 miles / 51.63 miles
Avg. Speed
19.6mph / 19.6mph
Elapsed Time
2.45.24 / 2.45.21
Elevation
1,198ft / 1,191ft (Strava corrected to 1,468ft)
Weighted Avg. Pwr
219W / 220W
Total Work
1,790kJ / 1,792kJ
Avg. Cadence
83rpm / 84rpm

It's all very interesting. At the 70-minute mark, the moving times were identical down to the second. Then all of a sudden, there was a 15 second gap. The difference in total distance is also interesting, even though it amounts to less than half a percent. What's interesting is that the Karoo measured 0.4 miles farther and 1m22s longer, which ends up with the exact same average speed. I would watch the Karoo's behavior pausing and unpausing. It would generally lag 3-4 seconds behind the Garmin at each stop-- but it lags both ways, starting as well as stopping. So it would stop 3-4 seconds after the Garmin, and then not start recording again until 3-4 seconds after the Garmin. So as far as I can tell, it's not 100% due to the sensor, but rather to however the Karoo is smoothing the data.

And it's not just speed. If you really dig into the data, and look at moments when you are completely stopped-- say for 10 or 15 seconds, the Karoo will have an average recorded speed between 0.2 and 3.0mph, and be showing the last recorded power zone as well. Basically, pause is just momentarily suspending the activity. I had ~20 wait at a stoplight, and the data analysis shows 0.2mph and 313W power for the full 20 seconds. It then goes through a whole 'nother round of smoothing when it uploads to Strava-- the gap between Karoo time and Garmin time sat @ 14 seconds when I got home. The upload swelled it to 1m22s.

So tomorrow, usually my "no ride" day, I will be doing the same test with the Garmin speed sensor-- delete and re-pair, I might even throw in a new battery just to see if that's not the problem. I'm going to do short neighborhood laps with lots of complete stops to see how the Karoo manages auto-pause with the Garmin sensor compared to the Wahoo sensor. Previous performance with the Garmin sensor has been much, much worse on Karoo than with the Wahoo sensor. And hey, worst case scenario I have to buy another Wahoo speed. My son's bike is going to need a cadence sensor, maybe I'll just buy the combo. :P

JohnJ80
Posts: 223
Joined: Tue Apr 03, 2018 4:55 pm

Re: My Karoo Test Journal

Post by JohnJ80 » Mon May 07, 2018 11:53 pm

Your power numbers are a lot closer than mine are.

Today I did a ride with my Elemnt in my jersey pocket and the Karoo on the mount. When I compared both rides on Strava, the Karoo was HALF the work of the Elemnt and therefore, half the caloric burn.

I'm using a Stages power meter (one leg) that has been flawless for a couple of years.

After the navigation fiasco on the last update, I did a factory reset. When I repaired the sensors, I forced it to ANT+ as suggested in the Facebook Live session to improve battery power. I was having drop outs from the PM to the Karoo and the numbers and curve are completely different from the Karoo to the Elemnt. I'll try repairing it through BT, but I think the update whacked the power data as well. So right now, for me, I have broken navigation and broken power.

J.

User avatar
drisotope
Posts: 137
Joined: Thu Mar 08, 2018 4:12 pm
Location: SoCal, USA!

Re: My Karoo Test Journal

Post by drisotope » Tue May 08, 2018 1:38 am

Yeah, I think they've mentioned in the update notes that they still don't have Stages fixed, even though Stages is probably the most common PM out there. I was actually amazed that they fixed power2max as quickly as they did.

JohnJ80
Posts: 223
Joined: Tue Apr 03, 2018 4:55 pm

Re: My Karoo Test Journal

Post by JohnJ80 » Tue May 08, 2018 4:06 am

Yeah, except they said they fixed it. Funny definition of the word “fixed.” I’m not sure they have any QA at all.

Brad
Posts: 23
Joined: Fri Mar 09, 2018 9:54 am

Re: My Karoo Test Journal

Post by Brad » Tue May 08, 2018 10:28 am

drisotope wrote:
Tue May 08, 2018 1:38 am
Yeah, I think they've mentioned in the update notes that they still don't have Stages fixed, even though Stages is probably the most common PM out there. I was actually amazed that they fixed power2max as quickly as they did.
Generally speaking, the Karoo does not work properly with a majority of PM.

On my side :

Garmin Vector 1 pedals (dual-sided, ANT+) : not working (too many dropouts, unusable data),
Powertap P1 pedals (dual-sided, BTLE) : not working (only one pedal is taken into account for power measurement, displayed power = 50% real power, average power at the end of the ride = 50% of the real average power)),
Powertap G3 hub (ANT+) : working pretty well but there are still some dropouts.

All tests and comparisons performed in parallel with a Garmin Edge 1000 with which all PM mentioned above operate flawlessly.

User avatar
Steve
Site Admin
Posts: 516
Joined: Tue Mar 06, 2018 2:56 pm
Location: Surrey, UK

Re: My Karoo Test Journal

Post by Steve » Tue May 08, 2018 10:50 am

JohnJ80 wrote:
Tue May 08, 2018 4:06 am
Yeah, except they said they fixed it. Funny definition of the word “fixed.” I’m not sure they have any QA at all.
This seems to be the heart of the problem. Whatever sort of QA they think they have in place, it sure isn't working. The impression given is that they have a guess at what might be a way to fix the problem, then compile the code and ship it straight out without trying it first. Maybe this isn't what they do, but it's the impression I get from the quality of their supposed "fixes".

Surely they must realise that something is very wrong here, and that they need to have a big re-think? I'm also sure that no-one would mind if the frequency of updates slowed down but they were of higher quality.

jeffjsmith33
Posts: 72
Joined: Sat Mar 10, 2018 3:06 pm
Location: Reading, UK

Re: My Karoo Test Journal

Post by jeffjsmith33 » Tue May 08, 2018 11:34 am

I wouldn't actually mind being an official beta tester - since it feels like we are all a bit like that now anyway.

But I would want a 'revert' option to get out of any really bad builds.

dthempel
Site Admin
Posts: 275
Joined: Tue Mar 06, 2018 3:35 pm
Location: Cape Cod, MA
Contact:

Re: My Karoo Test Journal

Post by dthempel » Tue May 08, 2018 12:24 pm

jeffjsmith33 wrote:
Tue May 08, 2018 11:34 am
But I would want a 'revert' option to get out of any really bad builds.
Or a second "dev" device that gets the updates from their daily build instead of the bi-weekly "all users" build. :)

User avatar
drisotope
Posts: 137
Joined: Thu Mar 08, 2018 4:12 pm
Location: SoCal, USA!

After 1.5.19, Garmin Speed NOT working

Post by drisotope » Tue May 08, 2018 4:28 pm

I mean, it works-- it's just whatever nav modifications they made in an attempt to fix TBT wrecked the auto-pause when used in conjunction with a Garmin speed sensor. I did five 1-mile laps around my neighborhood, coming to a complete stop 6 times per lap at pre-selected spots, taking note of the changes in moving time between the Karoo and the Garmin. It's really bad.

New battery in sensor, sensor re-paired and wheel circumference reset. For lap 5, I took the Garmin speed sensor off of my wife's bike as a second "control."

Lap 1, speed sensor on rear wheel: Karoo +31 seconds
Lap 2, speed sensor on front wheel: Karoo +33 seconds
Lap 3, speed sensor disabled (GPS for speed, "control") : Karoo +7 seconds
Lap 4, speed sensor re-enabled, front wheel: Karoo +29 seconds
Lap 5, second Garmin sensor, rear wheel: Karoo +44 seconds

That's 2 and a half minutes of added time over just 24 minutes or so. Which is bad. But Strava uses their own math when they import, so it gets WAAAAAYY worse.

Edge 520 / Karoo

Moving Time
0.24.15 / 0.29.19 (Strava corrected to 0.30.46)
Distance
5.44 miles / 5.23 miles (Strava corrected to 5.38 miles)
Avg. Speed
13.5mph / 10.7mph
Elapsed Time
0.34.59 / 0.37.05
Elevation
197ft / 121ft (Strava corrected to 199ft)
Weighted Avg. Pwr
162W / 164W
Total Work
204kJ / 231kJ
Avg. Cadence
62rpm / 63rpm

That's 5 minutes of differential over just 5 slow-moving miles. The distance disparity is due to the lag when the Karoo re-initiates motion-- making about ~30 stops over 5 miles, and accounting for a 2-3 second lag every time, easily adds up to 0.2 miles less distance. Then Strava just adds that time back on, so the speed tumbles down ever farther. Best case scenario, the Karoo adds just over 1 second per stop with no speed sensor. Even on GPS, this could very easily add on 3-4 minutes over the course of a ~100km ride.

Please note: this is all since the 1.5.19 update-- it was working perfectly last week. Doubly concerning, because the Wahoo speed works mostly fine, also on ANT+. It added about a minute and a half over 2.5 hours, less than 30 seconds per hour of normal riding. Still, the difference between 20mph and 19.6mph, but it's better. I don't think we can really hold out for perfect.

User avatar
drisotope
Posts: 137
Joined: Thu Mar 08, 2018 4:12 pm
Location: SoCal, USA!

Up and down the hills in the drizzly rains

Post by drisotope » Sun May 13, 2018 12:39 am

Sticking exclusively to the Wahoo Speed sensor, as it's a bit better accuracy-wise (with relation to auto-pause, at least) than the Garmin or the Karoo GPS. Today was a wet and chilly climb up Glendora Mountain Road in SoCal, followed by a descent down Mt. Baldy Road, which is a screamer, and only a bit frightening on wetted roads with wet... everything. So how did the Karoo do?

Well, the battery got ate up-- I turned adaptive brightness back on, because in two rides with it off, I didn't really see any improvement in battery life. I had WiFi and SIM off, because most of the ride is in Out of Service areas. The battery killer was indeed the adaptive brightness, but not for the usual reason-- rain kept pooling up on the screen, and when raindrops get on the light sensor, the thing goes bonkers. I could watch the brightness pulsing bright/dim/bright/dim for minutes at a time. In two hours, battery was down to 50%. Wiped the screen down as best as I could, and in the next half hour lost another 8%. At the halfway point (Mt. Baldy Village) I plugged it into the backup. The internal battery probably would have made it through the last half of the ride as it was dry... but now I know. If you're going out the rain, be sure to turn adaptive brightness OFF.

Edge 520 / Karoo

Moving Time
3.14.35 / 3.16.28
Distance
44.74 miles / 45.55 miles
Avg. Speed
13.8mph / 13.9mph
Elapsed Time
5.16.38 / 5.16.39
Elevation
4,396ft / 5,039ft
Weighted Avg. Pwr
222W / 207W
Total Work
2,035kJ / 1,920kJ
Avg. Cadence
71rpm / 71rpm

Once again, this is with the same wheel circumference set on both computers, so the difference boggles my mind. That elevation difference is ridiculous. The other guys I rode with are within 5% of my Karoo number, and I did a couple of extra miles/feet looping around when we lost one of our riders on the descent. So the Karoo is likely very close to the actual, and that Garmin is just crap. I've known that the Garmin has been screwing me out of elevation for years. And here it goes again.

So C'MON <HH> GUYS!! Fix this software so the 520 can go in a drawer where it belongs!

nordog
Posts: 166
Joined: Thu Mar 08, 2018 7:32 pm

Re: My Karoo Test Journal

Post by nordog » Sun May 13, 2018 8:31 am

Not in the draw drisotope, in the the bin or better still to a charity that will have it stripped to make money.
From what elevation do you start your ride from? I would not get to your highest point from my elivation of 198 feet, I would be free wheeling back the way I came at 1,500.

User avatar
Steve
Site Admin
Posts: 516
Joined: Tue Mar 06, 2018 2:56 pm
Location: Surrey, UK

Re: My Karoo Test Journal

Post by Steve » Sun May 13, 2018 2:17 pm

Interesting that your elevation is quite good, drisotope. Mine has been terrible ever since the last upgrade. Apparently today I did 72,000 feet of climbing and 63,000 feet of descending. That's despite me manually setting the correct altitude right at the start.

User avatar
drisotope
Posts: 137
Joined: Thu Mar 08, 2018 4:12 pm
Location: SoCal, USA!

Re: My Karoo Test Journal

Post by drisotope » Sun May 13, 2018 5:13 pm

Starting point was 1,070ft ASL. One of the guys I rode with had the current elevation on his Garmin within 15 ft of mine the whole way up the mountain. I was watching the ascent on both the Karoo and the Garmin (not much else to do on a 2 hour climb @ 9mph) and there was a period of about 20-25 minutes where the Garmin just stopped ascending. The elevation was still going up in the Current Elevation, but ascent was just... frozen. And while climbing at ~2,000ft/hr, 20 minutes robs me of that 700ft. The guy I spent the most time riding next to on the day ended with 4,953ft and I had ~2 miles more total distance at the end of the day, so now I think my 5k feet is spot-on.

It was actually kind of refreshing to see the Garmin actively being an idiot, and the Karoo doing well for itself. Also an excellent test day as I learned a valuable lesson about water and adaptive brightness.

dthempel
Site Admin
Posts: 275
Joined: Tue Mar 06, 2018 3:35 pm
Location: Cape Cod, MA
Contact:

Re: My Karoo Test Journal

Post by dthempel » Tue May 15, 2018 1:20 pm

drisotope wrote:
Sun May 13, 2018 5:13 pm
It was actually kind of refreshing to see the Garmin actively being an idiot, and the Karoo doing well for itself. Also an excellent test day as I learned a valuable lesson about water and adaptive brightness.
The Karoo screen isn't as water-tolerant as their "droplet test" video would have you believe. That's not a real-world case anyway. In my experience last Sunday marshalling a road race in downpours, the Karoo gets *very* confused by water droplets on the screen, to the point of the screen becoming almost unresponsive to touch and scroll. I could probably make this better by angling the Karoo/mount in such a way that more water rolled off. I wish I took a video of it in action but it was so wet, I didn't want to bring out my phone and didn't have my GoPro... :(

User avatar
drisotope
Posts: 137
Joined: Thu Mar 08, 2018 4:12 pm
Location: SoCal, USA!

Re: My Karoo Test Journal

Post by drisotope » Tue May 15, 2018 2:08 pm

I didn't really have any issues with the operation of the touchscreen while wet, but the light sensor went bonkers. I did have the pulldown menu go nuts for a little while, but I believe that was also due to the light sensor being very confused.

More concerned about all the guys on the FB group having really bad water intrusion into the body or under the screen. And the increase in number of broken "bracket feet." Up to now, the hardware had been superb... but I'm seeing a lot of people for whom that is not the case.

Scummer
Posts: 15
Joined: Wed May 09, 2018 8:07 am
Location: Bavaria, Germany

Re: My Karoo Test Journal

Post by Scummer » Tue May 15, 2018 3:42 pm

Steve wrote:
Sun May 13, 2018 2:17 pm
Interesting that your elevation is quite good, drisotope. Mine has been terrible ever since the last upgrade. Apparently today I did 72,000 feet of climbing and 63,000 feet of descending. That's despite me manually setting the correct altitude right at the start.
I have the same issue with the climbing and descending amount. But I see that as the error from the altimeter going to 0 once the auto pause hits. I have looked with Golden Cheetah at the FIT file and it clearly shows the altitude going to 0m once stopped. So, that's a drop from 1100m to 0m in less than a few seconds and back up to 1100m once I started moving again.
Take a look at this:
viewtopic.php?f=2&t=200&start=20#p1244

User avatar
Steve
Site Admin
Posts: 516
Joined: Tue Mar 06, 2018 2:56 pm
Location: Surrey, UK

Re: My Karoo Test Journal

Post by Steve » Tue May 15, 2018 4:19 pm

Scummer wrote:
Tue May 15, 2018 3:42 pm
I have the same issue with the climbing and descending amount. But I see that as the error from the altimeter going to 0 once the auto pause hits. I have looked with Golden Cheetah at the FIT file and it clearly shows the altitude going to 0m once stopped. So, that's a drop from 1100m to 0m in less than a few seconds and back up to 1100m once I started moving again.
On that same ride (where I supposedly climbed 72,000 ft), at one point I noticed the altimeter reading -5,000 ft for my current altitude. Needless to say, I wasn't actually in a deep ocean trench.

Post Reply