How can we determine if an article about some particular area of research accurately assesses the validity of the research on which any conclusions are based? For instance, how can we trust that wearing compressions socks improves running performance when we read an article in a magazine that makes this claim?
An author may attempt to increase the validity of a claim by stating that, “the research shows …”. However, this alone is a fairly weak argument to utilize the results discussed in a magazine as a basis for deciding whether we use a particular piece of equipment for racing. Assuming that the author provides a reference, it may be that the author is “cherry picking” the research to support their argument because it fits with their interests (social, political, financial, etc). In the scientific community, this is known as a “limitation” because of selection bias.
It is also important to not only identify the research but also to question whether the results from a study discussed are accurately described? For example, if a group of athletes were given compression socks prior to a 10 km running race, and subsequently completed the race with personal records, we cannot conclude that the improvement was a result of wearing compression socks. That is, we cannot say that there is a direct “cause and effect”. If an author makes such a claim, they are dismissing any confounding variables (other factors that may account for the difference) such as weather conditions, current training and nutritional status, etc. The only way to accurately determine a “cause and effect” relationship in a case such as this is to compare the results of athletes wearing the compression socks with a second group of athletes wearing a placebo (sham) compression socks.
How then can we make an informed decision about which training equipment to use or what training program will optimize sport performance? The best ‘level of evidence’ we currently have are systematic reviews of randomized controlled trials (RCT). This type of review incorporates systematic methods to select studies in order to limit selection bias and provide reliable conclusions on a given topic (2).
Finally, what is so important about using a randomized controlled trial? Randomization is a process of allocating study participants into an intervention group and a control group (1). By randomizing subjects we can account for confounding variables as well as limit selection bias (1). In addition, incorporating a control group allows us to compare the results of the actual intervention with a sham intervention. This way we can actually determine if there is a “cause and effect” relationship with a given intervention. If there is no statistically significant difference between groups following the intervention we can conclude that the intervention is no better than the placebo effect.
REFERENCES
1. Akobeng A. Understanding randomised controlled trials. Arch Dis Child 2005;90:840-844.
2. Liberati A, Altman D, Tetzlaff J, Mulrow C, Gøtzsche C, Ioannidis J, Clarke M, Devereaux P, Kleijnen J, Moher D. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. J Clin Epidemiol 2009;62:e1-e34.
Michael Rosenblat
http://www.evidencebasedcoaching.ca
Evidence-Based Coaching: Making science work for athletes.
An author may attempt to increase the validity of a claim by stating that, “the research shows …”. However, this alone is a fairly weak argument to utilize the results discussed in a magazine as a basis for deciding whether we use a particular piece of equipment for racing. Assuming that the author provides a reference, it may be that the author is “cherry picking” the research to support their argument because it fits with their interests (social, political, financial, etc). In the scientific community, this is known as a “limitation” because of selection bias.
It is also important to not only identify the research but also to question whether the results from a study discussed are accurately described? For example, if a group of athletes were given compression socks prior to a 10 km running race, and subsequently completed the race with personal records, we cannot conclude that the improvement was a result of wearing compression socks. That is, we cannot say that there is a direct “cause and effect”. If an author makes such a claim, they are dismissing any confounding variables (other factors that may account for the difference) such as weather conditions, current training and nutritional status, etc. The only way to accurately determine a “cause and effect” relationship in a case such as this is to compare the results of athletes wearing the compression socks with a second group of athletes wearing a placebo (sham) compression socks.
How then can we make an informed decision about which training equipment to use or what training program will optimize sport performance? The best ‘level of evidence’ we currently have are systematic reviews of randomized controlled trials (RCT). This type of review incorporates systematic methods to select studies in order to limit selection bias and provide reliable conclusions on a given topic (2).
Finally, what is so important about using a randomized controlled trial? Randomization is a process of allocating study participants into an intervention group and a control group (1). By randomizing subjects we can account for confounding variables as well as limit selection bias (1). In addition, incorporating a control group allows us to compare the results of the actual intervention with a sham intervention. This way we can actually determine if there is a “cause and effect” relationship with a given intervention. If there is no statistically significant difference between groups following the intervention we can conclude that the intervention is no better than the placebo effect.
REFERENCES
1. Akobeng A. Understanding randomised controlled trials. Arch Dis Child 2005;90:840-844.
2. Liberati A, Altman D, Tetzlaff J, Mulrow C, Gøtzsche C, Ioannidis J, Clarke M, Devereaux P, Kleijnen J, Moher D. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. J Clin Epidemiol 2009;62:e1-e34.
Michael Rosenblat
http://www.evidencebasedcoaching.ca
Evidence-Based Coaching: Making science work for athletes.
Last edited by:
mrosenblat: Jan 26, 15 11:48