MAARIE Questions for Final Critical Analysis
NOTE: A wise student will review the feedback for all previous analysis assignments BEFORE answering these questions for the final analysis assignment.
Method- The purpose and population for the investigation
1. Evaluate for conflict of interest. Who conducted the study? A group of researchers based at a reputable university or research institution, or an individual without clear affiliations? Be particularly cautious about accepting claims about treatment effectiveness in articles by people who intend to sell the treatment. Who funded the study? If a researcher received funding from an independent granting agency that uses a peer-review process for evaluating and awarding funds, the findings might be more credible and less subject to bias than if the funding agency has a vested interest in the results. If a commercial entity sponsors a study, this should be disclosed. But although sponsorship may influence the authors’ conclusions in some instances, this is not always the case. There are many research partnerships among industry, university-based researchers, and institutes, so this can be a difficult issue to assess. Make sure you include an answer to the question and your evidence for that answer; for example “I believe a conflict of interest may exist because . . .” or “I believe a COI is unlikely because . . .”
2. Study hypothesis What is the study question being investigated?
3. Study population What specific population is being investigated? What are the inclusion and exclusion criteria for the subjects of the investigation? Some of this information may be in the supplementary material.
4. Sample size and statistical power
How many individuals are included in the study and the control groups? What is the stated study power and are the participant numbers adequate to sufficiently power the study? Your answer should include a statement about the study being powered sufficiently or not and your evidence.
Assignment- Selection of participants for the study and control groups
What specific method(s) is/are used to randomize participants to study and control groups (computer generated in 1 to 1 allocation or were blocks used, etc.? Are the numbers of participants in the treatment and groups similar?
6. Confounding variables
Look at the baseline characteristics, are there differences between the baseline characteristics for study and control groups that may affect the outcome of the investigation? Offer of at least one confounding variable that the authors hadn’t included in the baseline characteristics or the inclusion and exclusion criteria that might affect the measured outcome(s)?
7. Masking or blinding
Are the participants and the investigators aware of the participants’ assignment to a particular study or control group? If patients are not aware but investigators were the study is single blind. If the patients and the researchers don’t know, then the study is double-blinded. If an independent/impartial group was used to evaluate the data and this group didn’t know about the participant’s assignment then report this information.
Assessment- Measurement of outcomes or endpoints in the study and control groups
What primary (and secondary) outcomes did the authors choose to measure. Are the outcomes selected appropriate when you consider the hypothesis for the study?
9. Accurate and precise
How will the outcomes (primary and secondary) that were selected be accurately and precisely measured? Remember to answer the question as well as provide your evidence of this.
10. Complete and unaffected by observation
Is the follow-up of participants nearly 100% complete (did the same number of participants who started the study end the study—or did the authors tell you what happened to them) and is it affected by the participants’ or the investigators’ knowledge of the study or control group assignment?
Results—Comparison of outcomes in the study and control groups
11. Estimation and Inference—What statistical technique(s) are used to report the data (OR, RR, ARR, etc.) AND what is the correct interpretation of those results specific to this study? Don’t just interpret the results generically. See the feedback for the Week Five analysis assignment for specific examples of how this is done. Your answer should include an interpretation of the values AND confidence intervals AND p values.
12. Adjustment—Are any statistical techniques used to take into account or control for potential confounding variables? The authors will typically explain any adjustments in the results section of the study. The most commonly used adjustment is analysis of variance (ANOVA), but there are many others. The authors should list the adjustments, the method(s) used to conduct these adjustments, and the reason adjustments were used; for example, the authors may adjust for things such as a higher male population when compared to females or for smoking vs. non-smoking, etc.
Interpretation—Meaning of the results for those included in the investigation
13. What harms and interactions were identified—Are the risks (side effects/complications) worth the benefit? Authors of modern studies should always provide the reader with the possible harms of a treatment or intervention.
14. Subgroups—Are the outcomes observed in subgroups within the investigation different from outcomes observed in the overall investigation?
Extrapolation—Meaning for those not included in the investigation
15. To similar individuals, groups or populations—The authors should not discuss extending the conclusions to individuals, groups, or populations that are similar to those who participated in the investigation. Do they? Your answer should state something such as “the authors were careful not to extrapolate to other populations for example they wrote (you would provide an example of a phrase or sentence from the study). The most common place to evaluate for extrapolation is in the discussion and conclusion sections of the study.
16. Beyond the data—The investigators should not discuss extrapolating by extending the conclusions beyond the dose, duration, or other characteristics of the investigation. Do they? See my comments in number 15.
17. Cost—To the best of your ability, determine whether the study results justify the cost of the treatment/therapy.
18. What is the answer to the PICO question that corresponds with this study? Remember, I am NOT asking you to formulate a PICO question; rather, you are asked to ANSWER the PICO question that I provided you in the final analysis assignment document.
MAARIE Questions for Final Critical Analysis