Critical Appraisal Questions
More than the level of evidence is needed to determine if a study provides solid evidence for a clinical question. Not all studies are as carefully planned out or executed as others and may be limited by the type of participants or reason a variable was measured. This section describes what to look for in individual studies to decide if the results are valid and clinically useful. The following list of questions asks about characteristics that are present in studies that provide the best evidence. When a study falls short of one of the guidelines, it is said to have a limitation and that the reader should take that limitation into account before applying the results. These are basic appraisal questions for intervention studies and can be slightly modified with other types of research articles and clinical questions. By consistently asking these questions when reading an article, one will learn how to pick out key information that is likely to help answer the clinical question. The Critical Appraisal Worksheet at the end of the chapter provides a format to follow when evaluating articles. The CEBM5 has also created Critical Appraisal Worksheets that depend on the type of article one may be evaluating (www.cebm.net/critical-appraisal).
• Participants should be similar to the population they represent. If not, the study results may not be applicable to other members of that population. For example, were the older adults in a resistance training study healthier and more physically active to start with than typical older adults? Are the study participants similar in most ways to the current patient? Age, gender, socioeconomic and education level, comorbidities, and health behaviors are some of the many characteristics to consider.
• In experimental studies, participants are put into control or treatment group(s). If participants of one group are not similar to the other group, one of the dissimilar characteristics (confounding variable), rather than the treatment, may explain differences between the groups. Consider the older adults again. If the resistance training group was more physically active from the start than a group that is given tai chi classes, maybe the better balance
Low Impact Exercise For Weight Loss Photo Gallery
Click on Photos for Next Low Impact Exercise For Weight Loss Gallery Images
• The best technique for creating equal groups is through random assignment. There are several ways researchers will randomize their samples. Picking colored marbles from a bag, using a computerized number generator, or doing a coin toss could be used to assign a participant to a treatment or control group. Participants must have an equal chance of being assigned to either group. Using order of appointments or having participants make groups by counting off 1 or 2 is not random.
• Participants who leave the study early should be accounted for in the analysis. It is common for participants, especially those in the control group, to drop out of studies. Researchers should show whether the lost participants made the groups less similar.
2. Was participant or researcher bias controlled for?
• Sample bias is when the participants, because of who they are or what they know, can shift the results. Participants who know they are receiving the study treatment are more likely to report that they had improvements or will subconsciously work harder to show that the treatment is effective. The term blinded refers to hiding the group assignment from the participants so they cannot influence the outcome variables. In exercise and physical modality studies, it is difficult to blind participants to their treatments.
• Researchers and those collecting the study data also have knowledge about how the treatment may or may not work. They may unintentionally tip off or cue participants as to the desired outcomes of an experiment, known as demand characteristics. To prevent this bias from impacting the results, some studies are double-blinded. This means that the participants and the person taking the measurements do not know to which treatment the participants are assigned. This strong study design is what is used in drug studies.
3. Were the groups given comparable treatments or placebos?
• The only difference between the 2 groups should be the treatment being tested. All other aspects of the procedures should be identical for each participant. When giving a placebo or sham treatment, the procedure must be identical to the actual treatment. For example, if one is studying the effects of ultrasound on an injury, the sham ultrasound treatment should include all of the prep and clean-up steps of the real ultrasound treatment.