Blind testing Comfort Cervelo S3 Vs R3

[jonnyo]

I always said with strong conviction that the Cervélo R3 was a much more comfortable ride than the aero Cervélo S3. I could not tell why but I always found the vibration on the road to be much harsher on the S3.

After discussing with many and a total lack of data about comfort and how to measure it…I decided to use the best option, a home made, not professional blind test of those bikes.

The first issue is…how do to a blind test, well, I didn’t think I could cover those frame in a proper way and it would be too much work on a brand new bike sitting at the store. Instead, we decided to build a helmet that would not allow me to see underneath me, so I could see forward…but no down. Impossible for me to know what bike I m riding!

Genius isn’t it?

The helmet:

The fit!!!

Both bike with same position, both bike with the same set of wheel…put on one bike…then on the other one
90psi
Same saddles
Different group(components) Because of this, I wasn’t allow to shift gear…I was wearing very big cloves and hoding the hood of the brake so could not feel much of the levers of difference in bars.

The mechanic would put the bike in a 53X17 and send me around a 1000m loop with 300m of rough pavement. I would repeat as long as we wish. I would not handle the bikes or changes…someone bring the bike to me, put me on the bike without me touching the bike. So…I had no clue what bike I was riding.

Results
1st run I was 80% sure to be on the S3…a little harsh ride. The true R3
2nd run I was convience I was on the R3…more comfy but difference was small The true S3
3rd run I was convience I was still on the same ride as run #2… The true S3
4th run I found that one to be the comfiest ride of all 4 runs…but I was in a state of comfusing as to what was what The true R3

So in the end, I was not able to identify the bike in the same way I would have predicted. I have realise that when blind…it become much harder to see a difference in comfort. Of course, there is a lot of mistake/issue with this test, but it show how wrong my perception was in the first place. Both bike are much closer than I realise and a lot more testing would be needed to see if a difference is reliably recognisable.

So…if you think one bike is much harsher than another one…perhaps try to blind test… it will be a eye opener… I was dead wrong in my hypotheses and learned something today…

[alexZA]

legendary
.

[tridork]

I’s been my contention for years, that with road bikes, it’s arguably impossible to tell any difference between bikes when they are set up identically.

I’m talking Frame A vs Frame B, both with same fit points, groupset, bars, wheels, tires etc. In a proper blind test you can’t tell one from the other.

What you can tell is tire pressure! Even just say 3-5 psi will make a considerable difference with perceived confort/difference between bikes. The difference that you perceive is from things like ‘seeing’ a straight fork, seeing gel inserts somewhere, frame size, set post length, etc etc. Your eyes quickly tell you if you should expects something.

I must say I love your fairly successful attempt at doing it truly blind. From what differences you perceived, I wonder how much you got from small differences in fit, or differences in bikes (groupset, wheels or whatever)

[jonnyo]

there was no difference in wheels… we reused the same set of wheel for each test…as for components…we lock both bike in a single gear so group should have very little influence. Perhaps not?

so I was down to simple frame. But what was very suprising to me was the state of comfusion I was in… I could hardly with any certitude see differences. And 5 days ago in a normal road test…I was in love with the R3 and complaining of the harshness of the S3…

Perception…not so accurate.

[Hoshie99]

Too funny.

Anyway, good luck deciding now dude…Small differences, lots of discussion- a typical analytic gear guy’s purchase dilemma.

My guess is you’d be happy with either if the fit coordinates are good.

j

[sausskross]

Article written by Damon Rinard … http://www.cervelo.com/en/engineering/ask-the-engineers/just-noticeable-difference.html

[BrianB]

what? no pics of you actually on the bike with that “helmet”?

you know what they say – pics or it didn’t happen

[tridork]

I remember years ago reading an article in a magazine (remember those?). It was about the famous Jackie Stewart doing some tire testing for Ford.

On the same day, he sat in the car and just drove. (he happened to be testing what tires they should supply on the Ford Thunderbird, ooooooo how the mighty have fallen). He would drive on a set of tires for a set number of laps of the test track then pull in. The crew would change wheels while he sat in the car. Next test, another test, mix in the ones from the previous test, and generally try and make sure it was randomized. Ford was apparently blown away by how accurate he was at determining which tires he’d tested before and how remarkably identical his times and impressions were of the various tires. He was able to perceive subtle changes in pressure, rubber compound etc, incredibly accurately and repeatably.

Most people can’t tell with that level of accuracy. It’s a lot like being a wine drinker, but not a semelier. They do it all the time, we do it once a week and by comparison, just chug it down. The same happens with bikes I think. I think a trained, full time bike tester could tell differences, but as much as we all think we are different and experts, I don’t think any of us can actually tell. And if we can tell a difference, we couldn’t determine with any accuracy what is causing the difference.

As a well timed example, I have ridden quite a bit ( but not fast) since I started cycle commuting 25km each way to school, when I was 14. Now I’m 53 and commute 13km each way to work every day. I’ve had about 20 bikes in that time and now I have 11 in my quiver. This past weekend I swapped out my saddle and wheels. My training wheels had finally died after 80,000km. The saddle was worn thin, so I swapped out that with the wheels. The wheels are almost brand new, as is the saddle. When riding, everything starts out fine, but then slowly a sound becomes apparent. It starts with a click, but finishes with a clunk. It happens at the same part of the pedal stroke. Nothing is hitting (like a wheel magnet). Even being a reasonable bike mechanic and mechanical engineer, and knowing that the noise only started when I changed the saddle and wheels. It is resonating through the frame and is clearly audible. I’m pretty sure it’s either wheel or saddle related (since that’s what I changed recently, and the noise started right after the change) I can’t determine where the noise is coming from. I’ve removed the wheels, even though the noise isn’t related to wheel rotation and no improvement. I’m now suspecting the saddle and will remove, inspect and re-install the saddle with lotsa lube on everything, this weekend and see.

Still, just like telling the difference between two frames, determining the source of the click/clunk is very very difficult.

[helo_guy]

Love it!

Also interesting how easy it was to get some interesting results.

[pk1]

brilliant!
it seems to me from your results that you are actually able to tell the difference and the r is indeed more comfortable than the s, it just took you a bit to get the calibration right - on the last run you picked the r as being the most comfortable. a few thousand more test runs required to be statistically meaningful though! does seem to indicate that the difference is small and not as obvious as you had thought and the s is not inherently a harsh ride

[RChung]

Nice experiment.

[Thom]

Someone needs to bump this thread every time someone starts talking about frames and ride quality. Nice work.

[jonnyo]

the mechanic where thinking of playing a prank on me by adding a 3rd bike. They had a S5 in the store…

But yes, I would like to re run the test with a bit more organisation. Do perhaps 10-15 runs. What this experiement show me is that…more testing is needed, yes, I might have been able to find the most comfy bike in those 2.

Another question came up…can you really honest feel stiffness of a bike and stiffness of the front end? does cornering ‘‘blindly’’ would show the R3 stiffer from end over the S3? I didn’t had the guts to push it today.

[MarkyV]

that helmet set up is so you.

[jrielley]

Just another reason you are my coach!

[Nick_B]

Really nice job
.

[The_Guardian]

This is excellent - both for taking the time to do this and for accepting the results. Well done.

[Former_User]

VERY cool…and that’s a very clever solution to the “blinding”!

You know, the mechanic could have really screwed with you if he’d wanted by playing with the tire pressures

[Former_User]

the mechanic where thinking of playing a prank on me by adding a 3rd bike. They had a S5 in the store…

But yes, I would like to re run the test with a bit more organisation. Do perhaps 10-15 runs. What this experiement show me is that…more testing is needed, yes, I might have been able to find the most comfy bike in those 2.

Another question came up…can you really honest feel stiffness of a bike and stiffness of the front end? does cornering ‘‘blindly’’ would show the R3 stiffer from end over the S3? I didn’t had the guts to push it today.

Based on the fact that the geometries are identical, I’d bet you’d have the same problems trying to discern any cornering differences.

[sausskross]

Spotted the gloves on the 3rd picture for *blinding *even the hands!