Login required to started new threads

Login required to post replies

Prev Next
How Much Credibility do you Assign to Bike Brand Wind Tunnel Claims?
Quote | Reply
The marketing script in triathlon bikes has been consistent for over a decade. Every bike brand checks similar boxes:

1. Tease a new release.

2. Release with accompanying "data" published in "white paper" to establish an argument for technical merit: "Best in wind tunnel/real world/etc."

3. Sponsor pro athlete and hope for results.

4. If results happen, attribute them by association to the bike, but never directly.

Here is my inquiry:

Does this language still inspire you to make a new bike purchase?

If you answered "Yes", Stop here and set down your pencil.

If you answered "No", please explain your answer briefly and suggest an alternative to this marketing script.


Tom Demerly
The Tri Shop.com
Last edited by: Tom Demerly: Oct 15, 15 9:49
Quote Reply
Re: How Much Credibility do you Assign to Bike Brand Wind Tunnel Claims? [Tom Demerly] [ In reply to ]
Quote | Reply
Tom Demerly wrote:
The marketing script in triathlon bikes has been consistent for over a decade. Every bike brand checks similar boxes:

1. Tease a new release.

2. Release with accompanying "data" published in "white paper" to establish an argument for technical merit: "Best in wind tunnel/real world/etc."

3. Sponsor pro athlete and hope for results.

4. If results happen, attribute them by association to the bike, but never directly.

Here is my inquiry:

Does this language still inspire you to make a new bike purchase.

If you answered "Yes", Stop here and set down your pencil.

If you answered "No", please explain your answer briefly and suggest an alternative to this marketing script.

No.

I honestly don't really trust any "data", because all of the "data" and white papers are lacking in details. The only way I would trust it is if there is a standard methodology, properly validated so that it is agreed as representative of the real world, summary results are presented and complete datasets are available for download and independent analysis, with annotations for why any particular data point should be excluded from the analysis. Everyone follows that protocol, with independent observers confirming that the protocol has been followed.

Until then, I'll choose my bike by geometry, visual appeal, and colour.

____________________________________
Swimming Workout of the Day: http://forum.slowtwitch.com/...ost=5784860#5784860;
Favourite Swim Sets: http://forum.slowtwitch.com/...m.cgi?;post=5004659;
Winner of the 2017 50 fly east coast Smackdown. http://forum.slowtwitch.com/...ost=6294538#p6294538
Quote Reply
Re: How Much Credibility do you Assign to Bike Brand Wind Tunnel Claims? [JasoninHalifax] [ In reply to ]
Quote | Reply
Thank you Jason. Appreciated Sir.

I think your answer may speak for the other 63 people who looked at this thread but didn't answer.


Tom Demerly
The Tri Shop.com
Quote Reply
Re: How Much Credibility do you Assign to Bike Brand Wind Tunnel Claims? [Tom Demerly] [ In reply to ]
Quote | Reply
No.

There are too many variables the setup for me to trust the comparisons. I admit I haven't read all of the white papers (I've scanned a few), but the marketing materials I've seen don't include any error bars or provide the detail in setup or methodology. I'd be very interested to know how the claimed margin of difference compared to those error bars.

I'd like to see some independent third parties conduct wind tunnel tests with the exact same methodology and wind tunnel. I'd also like to see those tests done in various different setups (rider position, bottle setups, wheels). All with the level of uncertainty explicit.



-Andrew
Quote Reply
Re: How Much Credibility do you Assign to Bike Brand Wind Tunnel Claims? [Tom Demerly] [ In reply to ]
Quote | Reply
Nope.

For the same reason you don't trust the politician when they say something self-serving, you don't trust a bike company with their wind tunnel data. I don't entirely think they fudge the data, but they almost certainly pick wind tunnel settings and tests that will be most favourable to their frame and unfavourable to the competition's.

Similarly, you can't really trust most magazines' tests since they're usually accepting cash for ads, and the sometimes-unsaid proposition there is "we buy five digits worth of ads, you say nice things about our bike".

Actually-independent owners that aren't trying to sell ads or pageviews can be trustworthy since they don't have conflicts of interest (other than not wanting to tank the resale value of their frame), but it's going to be rare that you can get enough bike owners together at a windtunnel and put on a good test. And even if you do put on a good test, you may just inadvertently pick test parameters that favour one frame over another, so you may get a false ranking anyway.

Basically, I'm utterly nihilistic when it comes to comparing frames. Way too many parameters to fiddle with and far too many huge conflicts of interest behind any data release.

This isn't unique to cycling: compliant car reviewers get access to nice vehicles, positive game reviewers get pre-release access and betas, journalists that uncritically parrot "anonymous government sources" are granted more access to future scoops. Disseminating accurate, unbiased information is a hard problem to solve in any industry when there's so many people with lots of interest in muddying the waters.

My favourite source of data is the kona top 10. You can't say "top 10 at kona = fastest equipment" since most equipment in the kona top 10 will be chosen by the sponsors, not the athlete. But you can say "this stuff is good enough that it didn't sabotage their race". Anything ridden to a top-10 finish may or may not be the fastest, but it's definitely good enough.

STAC Zero Trainer - Zero noise, zero tire contact, zero moving parts. Suffer in Silence starting fall 2016
Last edited by: AHare: Oct 15, 15 10:12
Quote Reply
Re: How Much Credibility do you Assign to Bike Brand Wind Tunnel Claims? [Tom Demerly] [ In reply to ]
Quote | Reply
No. I also didn't think that Air Jordans would make me jump like Mike. Everytime I think about trading in my old QR and getting a new superbike, I just remind myself that Mark Allen and Dave Scott could beat me on a unicycle.
Quote Reply
Re: How Much Credibility do you Assign to Bike Brand Wind Tunnel Claims? [Tom Demerly] [ In reply to ]
Quote | Reply
Verify your sources. And corroborate the data from other sources. That's the key.

A bike maker releasing some wind tunnel data which makes their frame look better than competitor's should be viewed with skepticism of course, for two reasons: that's a single data point, and there's conflict of interest that may skew the data in their favor if for no other reason than the fact the test isn't blind.

If an independent entity were to test that frame, then the data should be viewed as more trustworthy (the single data point, again, is of concern as it may or may not be accurate). When several entities run frames through wind tunnels and come up with similar numbers for a given frame, then we can begin to trust both the credibility and accuracy of that data.

That's hard to do, lacking standardized testing methods as it's been pointed out.

Great example of good data is Shiv vs P5. Several bike makers have tested their bikes against those two and in their (independent) testing the P5 was a little faster at lower yaw angles and Shiv at higher yaw angles.
Quote Reply
Re: How Much Credibility do you Assign to Bike Brand Wind Tunnel Claims? [Tom Demerly] [ In reply to ]
Quote | Reply
No, because i'm not sure that the lab demonstrated gains are not outweighed by other variables on the day, and I'm even less convinced that there's as much separating brands as they'd have us believe in that frame style, design and geometry appear to be gravitating to a "mean"
Quote Reply
Re: How Much Credibility do you Assign to Bike Brand Wind Tunnel Claims? [Tom Demerly] [ In reply to ]
Quote | Reply
I'm pretty sure the large majority of ST will say no but as large as ST is it's a small fraction of the tri world where I think this really does work.

1. The tease is exciting. The amount of chatter over the Cervelo 140.6 days annoucement among friends is ridiculous.
2. The whitepaper could be completely fabricated (who knows it might even be) but the majority don't know or simply don't care.
3. Define results. If you simply mean fastest bike split and <insert race here> it is happening and,
4. When it happens you can be sure it is attributed partially to the bike. Case in point http://www.slowtwitch.com/...k_Twelsiek_5422.html with no mention of his pedestrian run.


Rodney
TrainingPeaks | Altra Running | RAD Roller
http://www.goinglong.ca
Quote Reply
Re: How Much Credibility do you Assign to Bike Brand Wind Tunnel Claims? [Tom Demerly] [ In reply to ]
Quote | Reply
Cervelo is 2nd in every other manufacturer's data, I'd say that it's a 100% stone cold credible lock that Cervelo bikes are fast.


Tom Demerly wrote:
The marketing script in triathlon bikes has been consistent for over a decade. Every bike brand checks similar boxes:

1. Tease a new release.

2. Release with accompanying "data" published in "white paper" to establish an argument for technical merit: "Best in wind tunnel/real world/etc."

3. Sponsor pro athlete and hope for results.

4. If results happen, attribute them by association to the bike, but never directly.

Here is my inquiry:

Does this language still inspire you to make a new bike purchase?

If you answered "Yes", Stop here and set down your pencil.

If you answered "No", please explain your answer briefly and suggest an alternative to this marketing script.

Eric Reid - AeroFit
- Aerodynamic Optimized Bike Fitting
- Retul Pre-Purchase Bike Fitting
- Triathlon Coaching and Nutrition
Ask me: Scody Optimized Speed Suits | CeramicSpeed Oversized Pulley Systems | HUUB Skinsuits and Wetsuits |
Quote Reply
Re: How Much Credibility do you Assign to Bike Brand Wind Tunnel Claims? [Tom Demerly] [ In reply to ]
Quote | Reply
For me the benchmark is the old carbon P3. Once you've got something more aero than that, which is pretty common nowadays, then training is where you ought to focus your efforts. If all these super bikes really make the claimed 10 minute IM difference we would see times well under 8 hours for Hawaii. Just read Hutchinson's 'Faster' book to see a full expose of the aero-myth. There is a lot more than just aero to concentrate on too. For a small number of people a custom fit will result in a more balanced ride, for others a more comfortable frame will enable them to ride in an aero position for longer (fit being equal), some like something different and are willing to pay to stand out of the crowd or subsidise a small manufacturer, for others the convenience of an S&S frame would be worth the expense considering they like to race in far flung places.

SteveMc
Quote Reply
Re: How Much Credibility do you Assign to Bike Brand Wind Tunnel Claims? [Tom Demerly] [ In reply to ]
Quote | Reply
No. It boils down to the engine. At the MOP level, that stuff is going to "buy" me a very small amount of time on a 140.6 and even smaller time on a 70.3, in which it will have a very poor cost/benefit for me to leverage it out on purchasing a big money (assuming that the new bikes are all big money) bike.

That, and the fact that I think the "data" is always framed to benefit the bike that they are trying to sell. Marketing 101 stuff....nothing new.
Quote Reply
Re: How Much Credibility do you Assign to Bike Brand Wind Tunnel Claims? [SteveMc] [ In reply to ]
Quote | Reply
I have read the book and totally agree. I am more of the issue than what I am sitting on.
Quote Reply
Re: How Much Credibility do you Assign to Bike Brand Wind Tunnel Claims? [Tom Demerly] [ In reply to ]
Quote | Reply
No. I consider it necessary but not sufficient.
Quote Reply
Re: How Much Credibility do you Assign to Bike Brand Wind Tunnel Claims? [Tom Demerly] [ In reply to ]
Quote | Reply
There are certain bike frames that can be eliminated from your choices based on tunnel data. In the past the Ordu, the Argon 18 for example were pretty bad. When you see the superbike charts the data is all close enough imo. If a person sheds a shiv for a P5 there is not way to measure the delta, if there is any. From a 2004 QR to an IA, that is measurable.

To answer your question, the decision to buy which bike is made by what the LBS says, what people can get deal on, and then just random bs that we will never know. No company has any advantage on marketing.

Dave Scott and Mark Allen would be doing Kona in 3:55 if you added up the incremental advantages claimed over the years :)
Quote Reply
Re: How Much Credibility do you Assign to Bike Brand Wind Tunnel Claims? [Tom Demerly] [ In reply to ]
Quote | Reply
Very little, but I give them the benefit of the doubt that they have found some particular yaw angle or set of circumstances where they have made gains. In fairness probably the most fantastic claim I saw was related to the new Specialized Venge where they claimed about a time savings of 5 minutes over 40km versus their previous model. I may be glossing over some facts with my memory here but CyclingTips did some field testing and found that it held up in practice. I never would have expected this, but I would like to see more affirmation of aerodynamic claims on the road.
Quote Reply
Re: How Much Credibility do you Assign to Bike Brand Wind Tunnel Claims? [Tom Demerly] [ In reply to ]
Quote | Reply
No.

A significant part of the problem is that some big manufacturers have "peed in the pool". There have been many of the "white papers" that would do things like compare a road bike to a tri-bike, but use riders in different positions, clothes, helmets, etc., across the tests of bikes. So was the result, the bike, or the other factors? That is just a simple example. All that type of paper tells me is that the vendor is trying to make me think the bike is bette than it is, and I should not trust them. At this point, there is only one vendor who I generally trust.
Quote Reply
Re: How Much Credibility do you Assign to Bike Brand Wind Tunnel Claims? [Tom Demerly] [ In reply to ]
Quote | Reply
No.
There have been enough discussions on this forum about how individual the aerodynamics are between person to person when testing in wind tunnels with regard to different helmets, positions, skinsuits, shaving of legs, etc, not even getting into the myriad options for hydration and spares locations, that once you put a rider on a bike all claims of "best" aerodynamics are just best guesses. If other pieces of equipment can test differently on different people because the interaction between moving air and a solid object is complex with minor variances having relatively large consequences then is it not also likely that different body shapes, pedaling form (knees in vs out), foot angle etc. also affect the overall aeroness of a particular bike and may interact with different frames in positive or negative ways. If this is true (I'm sure some smarter people will let me know if I'm way off on this) then with all the current frames being "relatively" close you may as well choose the one you like the best and makes you feel faster.

Anyway that's what I tell myself - it must be the engine that makes me go fast when I race my slow and outdated Shiv.
Quote Reply
Re: How Much Credibility do you Assign to Bike Brand Wind Tunnel Claims? [Tom Demerly] [ In reply to ]
Quote | Reply
I trust them about as much as I trust ads for penis enlargement pills.
Quote Reply
Re: How Much Credibility do you Assign to Bike Brand Wind Tunnel Claims? [Tom Demerly] [ In reply to ]
Quote | Reply
I tend to listen to the proclamations of companies such as Trek, Cervelo, and Specialized (in that order) because I feel they have earned it. Everyone else I tend to take with a huge grain of salt. I'll look at their whitepaper and their data but I'm constantly looking for holes in it. When it comes to pro endorsements, I tend to mostly disregard them except for someone like Gomez swimming in a Roka wetsuit. If he's wearing that suit I know, at the very least, it's not slowing him down.

Alternative marketing script? Sheesh, that's tough. All I can say is that if you're going to put out data make sure it's good data with a solid methodology behind it.

WTB: TriRig Omega SV (not x). PM me if you have one :)
Quote Reply
Re: How Much Credibility do you Assign to Bike Brand Wind Tunnel Claims? [dgran] [ In reply to ]
Quote | Reply
The five minutes claim from specialized was comparing a rider on the bias whilst wearing an evade and skinsuit etc vs a rider on regular roadie clothing and a non-aero helmet whilst riding a Tarmac

So there are a tonne of factors contributing to that 5:00 difference

The best time to plant a tree was 20 years ago. The second best time is now - Ancient Chinese proverb
Quote Reply
Re: How Much Credibility do you Assign to Bike Brand Wind Tunnel Claims? [JasoninHalifax] [ In reply to ]
Quote | Reply
The problem with standardised tests is that you get designs that perform well in standardised tests. Cough cough... VW.


In answer to the question: no, but I still love reading a 'white paper'. Bikes are having a lot of money spent on their development now, so it's nice to read a narrative on the process, even if it is easily distorted.

I've never found sponsorship to be a good indicator of how a particular product will work for me.

I can't think of an alternative to the white paper as noted above. If aero is the measure of performance then any test is very limited in scope.

Making aero easier at Red is Faster
Quote Reply
Re: How Much Credibility do you Assign to Bike Brand Wind Tunnel Claims? [SkippyKitten] [ In reply to ]
Quote | Reply
that's why i added that the standard protocol would be "properly validated as representative of the real world".

standardised tests, properly set up, allow for good, meaningful comparison across time and space. You can take a result from one day, in one tunnel, and compare it to a result from another day, in another tunnel. Right now, you can't. Not very well, in any case.

____________________________________
Swimming Workout of the Day: http://forum.slowtwitch.com/...ost=5784860#5784860;
Favourite Swim Sets: http://forum.slowtwitch.com/...m.cgi?;post=5004659;
Winner of the 2017 50 fly east coast Smackdown. http://forum.slowtwitch.com/...ost=6294538#p6294538
Quote Reply
Re: How Much Credibility do you Assign to Bike Brand Wind Tunnel Claims? [JasoninHalifax] [ In reply to ]
Quote | Reply
Good luck with designing a 'real world' test that doesn't lead to designs optimised to perform well in that test.

Making aero easier at Red is Faster
Quote Reply
Re: How Much Credibility do you Assign to Bike Brand Wind Tunnel Claims? [Tom Demerly] [ In reply to ]
Quote | Reply
tease is irrelevant, unless the timing is right when i am considering a purchase.
as i think someone else said, whitepapers/test data etc and pro rider performances are necessary but not sufficient.
i like to see that they have done some reasonable testing and found that their bike is at least comparable to known top performers (at least given some not completely unrealistic parameters). whether its the "fastest" or not is insignificant, it just has to be thereabouts. P5 and shiv are great comparisons as they have been well tested and show known patterns across yaw range which helps validate test results. any supposedly aero bike release that does not include some form of tunnel test data makes you wonder whether they didn't do any testing or do they just not want to tell anyone the results.
if you're not willing/able to front up with some cash and persuade a top pro to ride your bike then i have to question your commitment and/or product. i don't believe that a top pro becomes so because of what they ride

so that establishes a possible contender. from there the decision comes down to aspects like fit, looks, storage etc integration, local shop support, brand trust...

fwiw on this basis i recently bought a trek speed concept:
- nice white paper showing aero results with P5 and shiv displaying expected behaviour, SC results similar to P5
- surprisingly few top pros so i guess that doesn't really mean to much to me
- white paper also addresses fit, storage
- i like the looks
- a good local shop
- brand i trust

i don't really think its practical to expect everyone's tests to be exactly the same, there are just too many variables and any standard is subject to gaming. i hate setting of specific targets that encourage people to meet the test rather than the real goal. everyone design a bike to meet what you think is important and then feel free to test against that basis. as long as its not too far from reality (and accompanied by explanations to allow readers to judge that) all good. it would be nice if we could get closer to an agreed standard though and that would not mean testing at 50km/h!

in summary, no that language does not really inspire me. however it does help and i don't really see much in the way of alternatives
Quote Reply

Prev Next