The gold standard, though still should be outdoor results, if you are racing outdoors. Saying wind tunnels are advantageous does not change this - I can tests blood pressure medications in highly controlled laboratory conditions with highly controlled subjects, using parameters that have shown excellent correlation to how the med should function in the real-world, but until I actual provide that real-world testing data and results, it's all theoretical.

This is a core concept in Phase1-Phase4 clinical trials. No matter how well the drug performs in the lab or in selected controlled human subjects, you cannot skip the gold standard testing of actually showing the results after you give the drug to a range of people in the real world, if you wish to claim that your lab results hold up in actual practice.

I'm also ok with using 'next-best surrogate test' (like CFD) if real-world testing is too cost-prohibitive (jet engine testing) or difficult. But as you yourself show with your own studies, this is eminently NOT the case in outdoor bicycle frame speed testing, even if it's not absolutely perfect.

without letting this go too much further into rabbit-hole/hijack territory...



not because there's something to hide, but because it wasn't the purpose of the testing. the point was to establish relevance of aero data even in drafting situations, period...not to establish whether or not one frame performed better than another in a given drafting situation. seemed like a prudent thing to investigate, since there's a chunk of the road market in particular which gets hung up on "but it all goes away in the real world when you're drafting"...conveniently forgetting that in the real world you're still moving through air.

<steps back into black helicopter>

It actually very might go away in the real world. If the external variable forces are enough to render the small gains of the aero frame immeasurable, those vaunted wind tunnel gains are in fact 'gone' on race day. It is critically important to interpret the magnitude of the speed gain relative to the expected variability of 'real world situations.'

This would be analogous to having a blood pressure drug, that in the lab, with humans, absolutely lowers blood pressure by 1mm HG in the lab, but has no measurable effect on BP out of the lab, due to the variability of human physiology/BP which makes a 1mmHG difference clinically useless. There is no clinician that would say that this was an effective hypertension drug worth prescribing to reduce real clinical hypertension, despite the rock-solid data that it really does lower blood pressure (by a measly 1 mmHG.)

They're NOT "gone on race day"...they're still there, it's just that unmeasured variables are making it difficult to compare results from separate runs. If you measure those variables and account for them, the differences stand.

And again, if I just simplified everything to 'it's there, you just can't detect it', I would right now be a billionaire several times over by investing in blood pressure and other medications that lower your BP by <1mmHG (clinically insignicant), but saying 'just because you can't measure it, it's still there so you should take this.'

Fortunately for the general public, the FDA folks are smart enough to know that doing this would be a bunch of bull.

I still stand by that if you can't even reliably measure the speed gain outdoors with a reasonable trial (like the article you link to above), you better seriously question whether you're gaining any advantage with it while racing, and seriously consider not spending any more time/energy looking at that variable compared to the other much more influential variables that DO show significant and measurable effects outdoors.

Why should the manufacturers test more outside?

Those studies (along with logic and physics) demonstrate that drag values obtained within a wind tunnel match extremely close to the data measured outdoors, and to a level well capable of teasing out differences between frames. The correlation is there.

However, wind tunnel testing is significantly faster, easier, and more consistent.

They aren't going to test outside just because you don't get that...that would be a waste of time and resources.

I'm saying that you cannot just choose to focus on the potentially insignificantly small effect of your aero frame, and just use idealized wind tunnel data, and then claim 'my frame is the fastest as per wind tunnel testing', when in outdoor practice, normally encountered variability can render that small measured effect irrelevant and unmeasurable.

It is absolutely NOT a waste of time and resources to test bikes outside the wind tunnel. That's something so easily done as per the article you linked to above, that it SHOULD be done, every time. Costs a HECK of a lot less than a wind tunnel, and is much closer to race situations.

We encounter this sort of 'avoid the-gold-standard test' approach all the time in biology and drug development. Companies might know full well that their drugs don't work as well as they claim in the fuzzier real world situation where real patients take the meds, but they try and pass off selected laboratory highly controlled studies with good results as surrogates measures. If someone is purposefully avoiding providing data that should otherwise easily be measured (and I consider the bikeradar 'experiment' a pretty easy one, at that), you must first ask why they would go through so much trouble to not do the easier outdoor experiment.

If bikeradar did that same trial with 5 bike frames, and couldn't measure a reliable difference, I would conclude 'no significant difference in aero advantages' for those frames, regardless of how well one of the performs in the wind tunnel. Likewise, if they showed that the Cervelo was reliably faster outdoors in those tests, but was last in the wind tunnel, I'd first make sure they weren't screwing up their methodology, but if the methods were sound, I'd go with the Cervelo as fastest.

Here is a different one: Let's talk about Dimond Bikes.

Dimond has a huge opportunity here. They own the fastest bike split at Kona. Not Cervelo, not Trek, not Specialized, not Felt- Dimond. A little, brand new start-up company with a decidedly unconventional looking bike relative to the rest of the competitive set.

So, we have a different story to tell. Dimond bikes are likely pretty aerodynamic. I don't know for sure. But they may also offer an additional sales feature entirely absent from the rest of the competitive set. They are a beam bike. They have different ride characteristics. Could it be that part of the reason they are faster is that they do have different ride characteristics? What if there is some measurable physiological benefit from having the rider sit at the end of a beam as opposed to on top of a nearly vertical seat tube? If every major and some minor brands can argue a case that their bike is the most aerodynamic, when all of them except one are wrong, then it isn't much of a task (relatively speaking) to construct an argument that beam bikes are measurably faster. And, since Dimond is really the only beam bike widely available (isn't there another that looks like the bat-bike?) then it becomes the "desired outlier". It becomes the De Soto two-piece wetsuit of the bike world.

I would absolutely love to see Bikeradar do the exact experiment you suggest, outdoors, with P3C vs P2k with same rider and equipment.

If I had to place a bet on the outcome, I'd bet that there would be no measurable advantage, meaning you're just as well off riding the P2k compared to the p3c. But I'd love to see this done.


I would love to see the bike companies stop all the wind-tunnel marketing BS, and just do this much more straightforward experiment, and show us the data, even if it's negative (shows no measurable effect), or not.