Sigh. So, this is something that bothers me but I sorta blame myself for being a poor explainer. People do repeated trials because they misunderstand statistical inference, and where the cause of imprecision comes from. Because of that, they end up doing inefficient testing because they're focusing on the wrong things. Here's the key tell: if riders are following a protocol that tells them to hold their speed constant during a run, then you know they're doing things inefficiently.

Doing lots of constant speed laps or out-and-backs makes the analysis easier but the data collection harder. That's silly. I prefer to do more, harder, trickier analysis in return for easier data collection. I do the analysis at home while sitting in a comfortable chair, sipping either a cold brew or a hot coffee.

We use a protocol of out and backs for fitters, coaches, and federations in order to make identification of a test area easier (400 m stretch of road is all you need), safer (finding a place without any road traffic may be easier and the operator can often see the rider at all times), more likely to encounter a single road surface condition (less variability in Crr within the analysis), and guess what, we nail down a near "perfect" elevation profile vs position for the course through our internal algos.
Downsides of this approach are asymmetry in the encountered wind, and other statistical analysis aspects that can be carried out slightly different if you do a different protocol.

Nothing in our algos requires constant speed, power or anything like this. Performing a standing 1000+W sprint won't help the analysis, but it does not make it any less valid.

---

Chris Morton, PhD
Associate Professor, Mechanical Engineering
co-Founder and inventor of AeroLab Tech
For updates see Instagram

Yep. What surprises me is the number of earnest trying-to-be-helpful people who recommend constant speed runs and lots of trials. Not only is that time inefficient, it's fatiguing, and rider fatigue is likely to introduce other kinds of imprecision. (Sometimes there are ways to handle that, sometimes not).

 
I had never seen that article from MTM. Maybe he can chime in on it, but this is what I read. "Get to know the device", figure out where it works well and where you need more protocol and data to compensate for some situations. But I would think that if you do 7 runs but repeatedly see consistency after 3, why do 7. I think a big part of that article is "get to know your device"

In my opinion, having to over protocol or over repeat is exactly the opposite of what these devices are trying to do. No way you should have to keep constant speed. Having to do loops will compensate for sub optimal sensors or conditions.

Friday I went to Toronto to compare two products head to head. Product A is intended to be more "ride once, get a number" with a lot of sensor technology". Product B is lots and lots of repeats. Problem was we were only able to test 4 configurations (and simple ones) before the rider said F'it. It was compounded by very erratic wind that made it chilly, but also made product B require even more runs to get to a statistically meaningful number.

For your custom mount, you may want to check with the Notio guys what the pros and cons may be. They suggest (or use to), far out on the extensions, but still below the hands. You can place it closer to the heat tube with different "challenges". D Bingham, puts in near the brake. That has specific advantages but also challenges in specific conditions. I can think of one situation where your use case could (not sure if it will) impact somewhere else in the system, although yes, you will get cleaner air.

I like that.

Only part of that concerning is that the device I have is the one he was using in that article. I mean it's an article on the Notio page so I would assume it has to be with the Notio. But I would hope I have more to find as an average joe than a seasoned pro. He said himself he's trying to sniff out those last little bits.

Eff it. I'm going to try it on my street and see what I get. I'll do something obvious first I have old data for from doing VE. If it matches well, I'll proceed. The pavement is just too good and the short stretch just too damn perfectly graded (like minus 0.3%) towards the house. No bumps, no ridges, same CRR. For 1/8mi you can plump a string on it.

Rchung has it though, you can't tire yourself out with exhaustive field work. I'm not MTM. So it's an ask of me as a rider to spend so much time pushing pedals that hard.

Not only that, the convenience of knowing if I forget a part/tool to changeover items would be a loop into the driveway to change it out. Not going "well shit" and driving back home.

Let me be clear. MTM knows the device inside out/upside down. He is or at least was sponsored by Notio and has extensive experience. I would trust his comments 1000%.


Ride, look at the results, but more importantly look at some of the underlying data. If that is wrong, then go "hmmmmmm". Having analyzed data from multiple of these devices, they all have their strength and weakness.

Multiple runs are for averaging out things that vary from one run to the next. Wind and position are the main things that come to mind, but many other variables will not be constant or precisely accounted for by your measurements. Repeats will not help with all of them, but some (the random ones)...

If you are doing A vs B testing of something that is easy to swap, and you are not concerned with the absolute value then calibration of your sensor during your runs isn't necessary. Otherwise it is, for good results. That means lots of runs, too. Getting the aero sensor to be immune to any change you care to test is mandatory in the first place; this task (testing for CdA in windy conditions) is much too difficult to throw in silly unnecessary variables.

The best way to test at race speed and avoid fatigue is to find a spot with enough of a rise at the beginning/end of each pass so you can use it to get up to speed with little effort.

Unless Notio has improved their system lately, they don't leverage elevation mapping of the test course nor precise position (you need both), so... ya, I wouldn't pay for it.

Errors in an estimate can come from random and systematic sources. Wind and position changes aren't random so averaging the runs doesn't typically average out the error. Wind and position changes are systematic, so if you don't account for them you're omitting things that matter, and you get omitted variable bias. That is, even when you average, you get a biased estimate. This is why doing multiple runs and averaging them can often make things worse, not better. If that's the situation (and the more runs you make, the more likely there will be systematic changes in conditions that you don't observe) then there's no number of extra runs you can make that will improve the MSE of your estimates. You can do a thousand extra runs and averaging over those thousand runs ain't going to make things better.

There are both random and systemic components to wind measurement and position. As I mentioned only the random parts (which are not small) are helped by repeats.

Errors due to omitted variables aren't random. Averaging runs doesn't force omitted variables to magically appear so they can be canceled out. They're still omitted.

What variable is omitted?

For instance position; the rider is trying to hold the an identical posture for each run, but this is of course impossible. Are you saying that the effect this has on his test will not be considerably reduced if he has 10 runs with each configuration rather than only 1?

This is why I brought this up.

The probability of a disturbance to any single data point gathered at the data collection rate is a coin flip if the disturbance exists. Same for the mentality of multiple runs. It's a coin flip for that also. So you in theory can end up with 10x head or 10x tails while the tester is hoping instead for 5x heads 5x tails so heads and tails average out across different setups.

For rider position I can see the coin flips averaging out as you can control the rider versus controlling the wind. I don't think a rider needs 10x to average out how they sit on the bike. I think 3 is probably sufficient.

If wind were random then you wouldn't need a wind sensor: all you'd need to do is a lot of repeats.

The same goes for position: changes in position don't vary randomly with duration or fatigue or maybe temperature, so simply doing lots of runs doesn't guarantee that errors will cancel.

We typically do controlled experiments (and that includes statistically rather than experimentally controlled) because they make it easier to detect systematic patterns in the errors. So rather than ask, "which variables are being omitted?" the right question is, "are there systematic patterns in the errors?" If so, then you can figure out what to do about them.

Doing lots of runs doesn't guarantee anything unfortunately... except that the random component of imperfectly measured or controlled variables *can* be reduced, via averaging. The effect of wind or position on the A vs B question, can be most definitely reduced with more data points. A decent measurement of airspeed is better than none, but I don't know of any field sensor that is close enough to perfection to take one run and be done.

Certainly, looking for systemic patterns is one reason for collecting many data points. Also, it's best if possible to use a protocol that minimizes the effect of systemic variables. An A-B-A-B-A-B protocol is preferred so that the effect of both day to day and time in test variables can be reduced.

Sure you can flip 10 heads for configuration A and and 10 tails for configuration B... but the odds of that happening are a whole lot less than the chances that a single flip will be heads with A and tails for B. That's why there is uncertainty, no matter how many times you test... but increasing the number of runs reduces the uncertainty from random variables.

I don't know, we are having a funny PhD party,

anyway, as minor the CdA differences between two set-ups become as closer in time the test runs should be and as minor the changes betweem the runs should be and as shorter the mean filter time should be in order to avoid systematic errors.

Sorry, I try to avoid the mumbo-jumbo stuff but sometimes it leaks out.

I'll try to do better.

Haha, the bit by the house is laughably too short. I forgot how time flies distance wise at that speed.

No problem. At least I tried.

Would this be the reason that I am seeing a gradually increasing CdA calculated in Notio GC when I simply roll down a hill in a fixed position? Actually, the calculated CdA decreases between 4kph and ~28kph (0.34 going down to 0.26) then gradually increases up to my max test speed of ~40kph (0.26 going up to 0.34). I’m also puzzled by the average CdA for this test run then calculated as 0.201 (see screenshot).

I saw this same pattern on every test run (over 20) in 3 different positions.

Can anyone shed any light on to why this would occur?

Without knowing anything about the Notio/GoldenCheetah version, I would say, according to your screenshot, that the CdA values during first seconds are all except accurate (it needs at least 30-45 seconds to have enough data at your speed to have an accurate average CdA with a good precision). But it doesn't explain the difference between 0.201 and 0.34.

---

Blog | Twitter| Bike CdaCrr app

Thanks bugno - so are you saying that I can never look at the values in those CdA graphs as accurate at any one time, or I can but most ignore the first 30-40secs?

Notio Golden Cheetah version details:

I don't know how plotted CdA is computed into GoldenCheetah Notio version. If we suppose that is an CdA average (or a rolling average), then the data of the first seconds are not enough to compute an accurate value. So yes, may be after the 30-45 seconds, the value may be a good guess, even though speed is too low at the beginning of the lap and the elevation difference is also not good, as precision on total mass may seriously alter CdA accuracy in this last case, depending on the slope/speed. Doing loops (zero elevation) is a protocol which suppresses total mass/measured elevation errors from CdA estimation.

---

Blog | Twitter| Bike CdaCrr app

I haven't used a Notio in a while so maybe things have changed. You can check with Notio support if what I am suggesting is still valid.

They use to use a rolling CDA that had a configurable window. The default was 60 seconds. In other words, for example at Time=5minutes, the CDA plotted was calculated calculated with the data from 4.0 to 5.0 minutes. Your laps seems to be 45 second long. It would be interesting to know if data before the start of the lap is included in the first 60 seconds.

I am guessing that if you used a smaller window, you would see the CDA rise to even higher than it currently does and possibly go lower. Showing the window before the start of the lap would also help.

There may be a few things going on. My guess, and this is pure speculation, is it is underestimating the down slope. So the real slope is say -10deg and it is seeing -7deg, it's saying wow, this guy is moving for a -7 deg, he must have a really low CDA, hence goes down to .2x. But it can be other things like bad CRR, bad weight......

But there is something weird in there. That .201 is weird. I can think of a few things but would need to see data to confirm. If you share the JSON file (export from GC to JSON), it's pretty easy to figure out.

Their version of GC to allows to do Virtual elevation to figure things like this out. You can easily prove/disprove my theory above. Compare VE to barometer elevation.

Every product has it's strengths/weakness but the ability to diagnose what's wrong with the data is extremely valuable (IMO) and they do a decent job of this.

Is this in TT position or road bike ?

I got the same thing yesterday. I tried just running it in workout mode so I could learn about the sensor and post-ride data review more.

Identical thing as this dude. I had a .75mi out and back piece I was using. I don't even think that was enough for the sensor to get going.

This has been why I'm hunting big time for a circular lap. It's a royal PIA to find a good enough road to satisfy this thing's sensibilities for out/back and the averaging.

Same thing for me, I would start the lap or u-turn at the end and the CdA would plummet from the obvious out of aero turn around down below 0.200 then steadily rise well beyond a reasonable CdA approaching the other way. Pretty flat road with just a little tiny 1% grade bump up at the start and u-turn points.

I'd prefer taking samples of the data from a period of constant CdA for 30sec. Not having an average of a moving CdA.

I re-weighed myself this morning. I also measured my tire rollout with a fabric tape that reads in cm. Next up is another baseline cal.

I personally think if you're not a pro that can do a couple hours of sweetspot power in a ride, a loop, huge traffic circle, something may work best. Then, instead of finding a road long/good enough to out/back and dealing with the averaging BS each time.........once up into good data each lap is good stuff.

It's just tough going well over 25mph finding something big enough circle or gentle enough turn to work well. I found two. Just gotta drive out to try them.

Next up.........how the hell did I accidentally delete the CdA tab in the GC-Notio? I don't see that graph tab in the selection either to add it back! Do I now have to delete/re-install?

Shoot, I mean I'm about to point of trying to get the cal factor right in the app then working in Excel. I don't like the chart zoom and axis options in GC. I like to resize charts and axes how I damn well please.

I am going by memory here, it's been a while but try this.

Create a new athlete. I suspect the chart will be there.

Then you can "export" the chart under the "More" at the top left of the chart. Put it somewhere on your file system

Then go back to your original athlete and simply drag (from the file system) and drop the chart into GC.