Blindfolding a rider in the wind tunnel wouldn't be impossible would it? How about also not telling him anything about the gear he'll be sitting on prior to the test? The guys operating the tunnel and recording data shouldn't be told anything about the gear being tested either - it could be supplied unpainted without logos. No company reps at the tunnel.

Now, this isn't perfect, but in my opinion it would produce valid results. Of course I doubt any bike company will ever do this because it's a bit of a hassle and because, for the most part, the objective of their testing isn't to obtain valid results.

 Congratulations - you just won the Frank Day Award for a Complete Lack of Understanding of Physics!

Rik[/reply]
that's right, now you just let me know when the ratio of P3C wins starts to even remotely match the ratio of P3C sales and maybe i'll start to think you're on to something........

let me know too when athletic achievements always match perfectly with our perception of physics

I'm not saying "yes", I'm saying that research results are always subject to uncertainty and bias, and that large potential sources of bias are ignored by guys who do wind tunnel testing and the sort of field testing you describe. Yet these sources of bias are acknowledged to be potentially large in other fields of research, such as medicine and psychology. Why are they ignored by bike testers? Probably because they're inconvenient to deal with, and because dealing with them makes the data less malleable by people with a vested interest.[/reply]
So you're not saying "yes, you don't believe in wind tunnel testing for body position" you're saying:

"Because the rider wasn't blinded to the bike he was riding, there is a possibility that he would hold his head a touch lower, hunch his shoulders a little more, or otherwise make subtle improvements to position while on the "faster" frame. He may deny that he did it, or he may not even be aware that he did, but he may have done it and you have no way to control for it."

"The subtle changes wouldn't have to be that large to completely invalidate any estimate of time savings over 40k."

"While the lack of blinding may not explain all of the effect, it may explain some. If it does, then any estimate of the magnitude of difference [..] has to be taken with a pinch of salt."

"So how much of a pinch of salt do we need to take it with? Oh that's right, we can't accurately adjust for threats to validity - that's what makes them threats to validity. So we don't actually know with a huge degree of certainty that there was any difference"

"large potential sources of bias are ignored by guys who do wind tunnel testing"

So how is this different from saying, "yes, I don't believe in wind tunnel testing for body position?"[/reply] Because, as you can see in my post above, I believe that with a few changes in methodology and execution, wind tunnel testing for body position could be made a whole lot more valid.

In the tunnel, they do their best to ensure you're holding base position from run to run, i.e. with side and front live views vs. baseline. Sure it's possible to 'cheat' a little - but who's getting cheated then? Yourself!!!

Same goes for personal field testing - who is benefitting from any bias?

But I will never argue that bias isn't possible. In fact it was one of the 1st questions I asked Tom offline!

I honestly don't have any strong opionions on what bike/equipment companies do or not to test and/or promote their products.

But you've been saying that any potential uncertainty invalidates all measured difference. Since no wind tunnel blindfolds riders, you believe that no reported difference CdA due to body position changes via wind tunnel testing is valid. As I've said, I'm not judging your stand on this -- I'm just making sure I understand it.

Ummm...you do realize that there's a pretty big difference in what's being measured in those fields above and what's being measured here, right?

---

http://bikeblather.blogspot.com/

You can avoid this to a large extent by mounting the ibike on the bike head tube. By doing this, you also avoid some tilt issues. It's easily done (on archaic round-tube bikes) with a few 1/2" PVC fittings. Your friends will tell you it looks very uncool, especially if you lash it on with an old bootlace. But it works.

Another big question is how the ibike responds to yaw. The makers say they've never tested it. I keep meaning to test mine (out the car window with a protractor), but can't seem to get around to it. But even if you know the ibike transfer function for scalar measurement of a vector, there is unavoidable information loss involved. The makers of ibike should add a second DP sensor and ports. Then we could get yaw data and discuss it endlessly - yippee!

You have read Dr. Coggan's many criticisms of the Luttrell study haven't you? Let's see, it is invalid simply because he didn't disclose he was given a pair of cranks to study. Or, he didn't control for this or that or whatever. Or, the Dixon study? Or, Joaquin's testing data (not the power file, the testing data). And you are getting upset because some here are simply pointing out there are potential biases and errors in the data that could affect the interpretation.

You guys belittle anything that is outside your belief system and stand together for any findings that support your bias. Or, people you know and who think like you can be trusted, others can't be trusted.

---

--------------
Frank,
An original Ironman and the Inventor of PowerCranks

Two things:

1. Head tube is one of the areas that frame builders have spent some time cleaning up -- this is one of the reasons you might worry that attaching an iBike there could mess up the flow.

2. Yaw is an interesting issue. I don't have any experience with this, of course, but I was thinking that if the measured wind speed varied with ground speed then if you can assume the direction was constant you could back out the yaw.

Whoops, I'd overlooked the sentence you highlighted. That was the sentence I was trolling donm for. The key there is that I was trying to get him to acknowledge that the size of the potential bias (relative to the effect) matters. Donm has been saying any potential bias invalidates all findings. The reason why we worry about blinding in these fields is because the effects we're trying to tease out are often small and subtle. That's why I was trying to get him to look at the size of the difference in the estimated CdAs.

Quote:

you've been saying that any potential uncertainty invalidates all measured difference.

I don't think I have been saying that and, just to clarify, this is not what I believe. It's the nature of research that there is always uncertainty. Good research methods seek to minimise that uncertainty.

?????

But this is what you wrote:

"The subtle changes wouldn't have to be that large to completely invalidate any estimate of time savings over 40k."

"While the lack of blinding may not explain all of the effect, it may explain some. If it does, then any estimate of the magnitude of difference [.] has to be taken with a pinch of salt."

"So how much of a pinch of salt do we need to take it with? Oh that's right, we can't accurately adjust for threats to validity - that's what makes them threats to validity. So we don't actually know with a huge degree of certainty that there was any difference"

So you're saying lack of blinding explains some of the effect and some effect suffices to say we don't can't know that there was any difference.

I've deleted your incorrect interpretation of my words, but otherwise I stand behind everything that's quoted above.

I'm trying to help you understand the issue with observational, poorly controlled study methods - they produce evidence, but not particularly strong evidence. Evidence provided by scientific research is never definitive as there is ALWAYS a degree of uncertainty associated with it. You always have to look at study results and ask yourself, "have I actually measured what I set out to measure?" Everyone has their own threshold of what constitutes an acceptable level of evidence; yours appears to be lower than mine in this case.

Have I clarified my position, or do I need to read any more posts where you quote me and then put your own little convenient spin on them? Painting me as the, "doesn't understand the nature of uncertainty in science / has totally unrealistic and unreasonable standards of evidence / will never be satisfied by any research that could happen in the real world" straw man is becoming tiresome.

You're right, I didn't realise that. Thanks for filling me in. How much do you know about what's measured in psychological and medical research?

If you want to deny that bias introduced by the subject or observer could affect results of aerodynamic testing, be my guest.
The fact is, if humans are involved, they introduce bias. There are ways of controlling for this, but if you want to bury your head in the sand and ignore them, be my guest.

Are you saying that without the P3C there is no way you would have cracked the top 12 in the Cat 4 race at the Socal District TT?

I've deleted your incorrect interpretation of my words, but otherwise I stand behind everything that's quoted above.

I'm trying to help you understand the issue with observational, poorly controlled study methods - they produce evidence, but not particularly strong evidence. Evidence provided by scientific research is never definitive as there is ALWAYS a degree of uncertainty associated with it. You always have to look at study results and ask yourself, "have I actually measured what I set out to measure?" Everyone has their own threshold of what constitutes an acceptable level of evidence; yours appears to be lower than mine in this case.

Have I clarified my position, or do I need to read any more posts where you quote me and then put your own little convenient spin on them? Painting me as the, "doesn't understand the nature of uncertainty in science / has totally unrealistic and unreasonable standards of evidence / will never be satisfied by any research that could happen in the real world" straw man is becoming tiresome.[/reply]
Hmmm. Actually, I think your position has been switching as you've gotten more desperate so I'm all for pressing ahead. I think I'm pretty clear on the issue of poorly controlled study methods. I teach in this area and lecture on this kind of stuff all the time. I tell my students that when they evaluate research they shouldn't stop at pointing out the potential for error. Anyone can do that, and we have a name for it: the "devastating critique," in which actual data-driven research is contrasted with the Platonic Ideal of a study and then dismissed as wanting. (I tell my PhD students that's what MA students do). What makes them worthy of a degree and a future income high in the, um, four figures is the ability to estimate the size of the potential error and figure out a way to tell when the data have been tainted.

The reason why we use blinding is because we're often trying to tease out small effects and we're worried that the bias due to lack of blinding is going to be large relative to those small effects. I've been pointing out several facets of Tom's results that suggest that lack of blinding can't explain the magnitude of difference he observed. You've refused to address that. If you wish to continue this conversation (and I can understand if you do not) then please turn yourself away from the devastating critique and toward the evidence for bias.

Dr. Chung.

While you have been "pointing out several facets of Tom's results that suggest that lack of blinding can't explain the magnitude of difference he observed." Here are a couple issues that suggest the opposite.

1. n=1
2. The person who designed the study, was also the subject of the study, also collected the data, and then analyzed the data.

This does not mean that his results are necessarily wrong. But, it does mean that skepticism of the results (especially the interpretation of the results) is not the sign of a weak mind. From a scientific perspective, imho, skepticism is the stronger position.

---

--------------
Frank,
An original Ironman and the Inventor of PowerCranks

Actually, I'm thinking the P3C (instead of my P2K, all other things being equal) gave me ~1:15 in time savings...which would still have put me in the top 10.

Hey!...wait a minute...now your goofin' on me, aren't you? :-P

Yeah, as sad as that sounds, the P3C was good enough for a couple of places in my lowly Cat 4 race....hmmmphh... >:-(

---

http://bikeblather.blogspot.com/

Frank:

I don't think I've been saying that skepticism is the sign of a weak mind. As I said early on, I was agog at the results. What I'm saying is that it's time to move past the "devastating critique" phase and on to the "estimate the size and consequence of the putative bias" phase. One of the consequences of the bias that donm has proposed ought to be a distortion of the virtual elevation profiles. Where is that distortion?

Coming from you, even you have to be able to see the irony in that statement...

Anyway, I'm as skeptical as the next guy. While I haven't done any formal testing, in pouring over a couple of race files comparing a P3 to a P3C, I saw a very small difference, with a very slight advantage to the P3C.

The problem is, as I stated before, people aren't being skeptical, they're just being silly. As far as your points about hand position/head position/riding a straight line, let's look at them. Hand position is the easiest to keep consistent, particularly when looking at Tom's set up. Also, in looking at the results, Tom makes a pretty interesting point: his field tests of the P2K were consistent with the runs he did for this test. Lastly, I think it's kind of laughable to think that he's suddenly have been able to morph his body into a faster position with the bike set-up remaining the same, even while racing?? It appears that the field test results seem to correlate to his race results.

As far as riding a straighter line, how do the distances match up for each of the runs??

So, we're left with the following: Tom's ducking his head more because he WANTS to buy a more expensive bike (hmmm....), and, of course, 'what pro's do' and 'in the Tour de France....'. Boy, that one never gets old.

Again, if someone wants to pose some serious scrutiny of the methods, I'd like to see it.

"n=1"

There's n=1 for the precision in measurement, and there's n=1 for the accuracy in measurement.

I hope your misgivings are with the accuracy of the measurement. Because here, while there is only one test subject, on two different bikes (identically set up, for all intents and purposes). However, there are *thousands* if not **millions** of data points, which are all the measurements that the powertap takes. These lead to the precision of the results.

What we should be talking about is the accuracy of the results, i.e. any systematic errors.

I hope you're not confusing one with the other.

---

___________________________
Chewie
Slowtwitch Aeroweenie since '06

I know nothing of distortion of the virtual elevation profiles. Based upon what I have learned here, the actual CdA number Tom generated is probably pretty accurate. However, the accuracy of that number does not mean we can assign all that aerodynamic savings to the frame alone which, it seems, is trying to be done here. Even if we assume that Tom injected no bias into the data (which you seem to believe but goes against human nature), it would still only mean this was the frame savings for him, and not the frame savings that everyone might see since the frame might interact differently with different body types and set-ups. This would require many many more trials before one can make an assessment as to how reliable this data is and whether the frame savings hold across the board or what the average frame savings might be. Real world racing data, as pointed out here, suggests the frame is not as advantageous as Tom's data suggests.

---

--------------
Frank,
An original Ironman and the Inventor of PowerCranks

I really fail to see the logic of your arguments.

Isn't this exactly the same thing that goes on with most of the PowerCrank studies (Joaquin comes to mind...)?

---

"Non est ad astra mollis e terris via." - Seneca | rappstar.com | FB - Rappstar Racing | IG - @jordanrapp

Has anyone suggested otherwise? Tom posted this up as an interesting single case study for the rest of us to consider and mentioned the strong agreement with the improvement the Coggans experienced. He never touted it as "conclusive proof that the P3C is 2.1663s/km faster than the P2K" - it is faster for him and I rather suspect that is what he cares about.