Guide for Medical Professionals: Evaluating Journal Articles in Gastroenterology, Study notes of Medicine

Download Guide for Medical Professionals: Evaluating Journal Articles in Gastroenterology and more Study notes Medicine in PDF only on Docsity! Critiquing a Research Article Temple DOM Research Curriculum March 2015 Adam C. Ehrlich, MD, MPH Assistant Professor of Medicine Section of Gastroenterology Background • The amount of medical knowledge is rapidly expanding. • Even with the improvements in access that come with electronic indexing and the Internet, it is impossible to keep up with the medical literature. How do you get an article? • You subscribe to the journal • Someone gives you an article to read • You search for an article to answer a clinical question • You are required to review something for a journal club Background Table |—Ten reasons to read clinica! journals To impress others To keep abreast of professional news To understand pathobiology Ta find out how a seasoned clinician handles a particular prablem To find out whether to use a mew or existing diagnostic test on your patients* To learn the clinical features and course of a disorder” To determine etiology or causation" Ta distinguish useful from useless or even harmful therapy” To sort out claims concerning the need for and the use, quality and cost-effectiveness of clinical and other health caret 10. To be titillated by the letters to the editor 4B Oo 1 ch Ol me tu RD “Reasons covered in detail in this series of Clinical Epidemiology Rounds. TA later series of Clinical Epidemiology Rounds will attempt to demystify the articles read for this reason. Sackett DL. CMA Journal. 1981. Disclaimer • Remainder of the talk is my opinion • There are a variety of methods and algorithms for doing a critical analysis of a journal article. • Underlying theme is to be systematic. OK, so now you’re going to read the article… • What is your purpose? • Three major areas to assess: – Validity – Results – Applicability • What type of article is it? – Cohort – Diagnosis – Prognosis – Treatment – Meta-analysis PICO • Patients • Intervention • Comparator • Outcome PICO • In (P) IBD patients, does (I) a new adherence scale (C) compared to physician perception and pill counts (O) predict adherence to medications? Applicability • Does the study apply my patient population? Were the study patients similar to my patients? • Does the study apply to the particular patient I’m treating? • If not, is it too great of a leap to apply the findings? Types of Studies Te NEW ENGLAND JOURNAL of MEDICINE OCTOBER 4, 2007 Effectiveness of Influenza Vaccine in the Community-Dwelling Elderly Kristin L Nichol, M.D, M.P.H. M.B.A., James D. Nordin, M.D.,M.P.H., David B. Nelson, Ph.D., John P. Mullooly, Ph.D, and Eelko Hak, Ph.D. ABSTRACT BACEGROUND Retiable estimates of the effectiveness of influenza vaccine among persons 65 years of age and older are important for informed vaccinatton policies and programs. Short- term studies may provide misleading pictures of long-term benefits, and residual confounding may have biased past results. This study examined the effectiveness of influenza vaccine in seniors over the long term wh ile addressing potential bias and idual confounding in the results. METHODS ‘Data were pooled from 18 cohorts of community-dwelling elderly members of one U.S. uealth mainten ance organization (HMO) for 1990-1991 through 1999-2000 and of owo oder HMOs far 1996-1997 through 1990-2000. Logistic regression was used to estimate the effectiveness of the vaccine for the prevention of hospitalization for pneumonia or influenza and death after adjustment for important covariates. Ad- ditional analyses explored for evidence of hias and the potential effect of residual confounding. RESULTS ‘There were 713,872 person-seasons of observation. Most high-risk medical conditions thatwere measured were more prevalent among vaccinated than among unvaccinated persons. Vaccination was associated with a 274 reduction in the risk af hospitaliza- ‘ton for pneumonia or influenza (adjusted odds ratio, 0.73; 95% confidence interval ICH, 0.68 to-0.77) and a 48% reduction in the risk of death (adjusted odds ratio, 0.52; ‘9% Cl, 0.50 to 0.55). Estimates were generally stable across age and risk subgroups. In the sensitivity analyses, we modeled the effect of a hypothetical unmeasured con- founder thatwould have caused overestimation of vaccine effectiveness in the main analysis; vaccination was still assoated with statistically signifscant — though lower — reductions in the risks of both hospitalization and death. From tha Medicina Servica and Canter far ‘Chronic Disoase Outcomes Reeuarch Min- ‘wapolis Vetorans Affairs Medical Contar and Univarsty of Minnesota, Minnaapo fis PULN. DBLN: the HealthPortnars jet Permananto Northwest Portlnd. OR (LPM and tho Julius Centar for Haalth Sorvices and Primary Care. Univar- sty Medical Centos, Utrecht, tha Nothar- lands (EH), Adéress reprint caquests to Diagnostic ass Comparison of Magnetic Resonance and Balloon Enteroscopic Examination of the Small Intestine in Patients With Crohn’s Disease Kento Takenaka,’ Kazuo Ohtsuka,’ Yoshio Kitazume,* Masakazu Nagahori, ' Eiko Saito, Makoto Naganuma,’ Akihiro Araki, ori, Toshimitsu Fujii,’ and Mamoru Watanabe ‘Department of Gastroenterology and Hepatology, “Department of Radiology, Tokyo Medical and Dental University, Tokyo, Japan See Covering the Cover synopsis on page 257. BACKGROUND & AIMS: Magneti is a recommended imaging techn volvement in Crohn's disease (CD). H : raphy has not been compared directly pe jejunum and um. V evaluated the usefi ny (MREC) by comparing its findings with those f an-assisted entero: scopy. METHODS: In a prospecti ° study, MREC and enteroscopy on 100 patients mucosal lesions were evaluated. Phy- re blinded to results from other MREC were compared directly with those from enterascopy; the sensitivity and specificit MREC detected CD lesions were assessed, RESULTS: MREC detected ulcerative lesions and all mucosal lesions in the small (' CL 83.7% 90.6%) and 94.8 %). MREC detected major stenosis %) and 90.0% (95% Cl, BB.4%-91.5%) and all stenoses with 408% sensitivity ( Cl, 30.8%-49.4%) and 93.7% specificity (95% CL 91.1%, 95.9%). CONCLUSIONS: MREC. is useful for detecting active le. sions in the small intestine. However, MR imaging is less sensitive ntestinal damage, such as stenoses. Enterascopy is dentifying intestinal damage. Suitable imaging ap proaches should be selected to assess CD lesions in deep small intestine. Keywords: IBD; Diagnosis; Ulcer; Intestinal Damage. natalizumab, and adalimumab.* Conventionally, imaging of CD lesions has relied mainly on ileocolonoscopy (ICS) and small-bowel follow-through (SBFT). ICS is useful for the detection of inflammation in the colon and the distal end of ¢ ileum; however, it is impossible to access the deep small intestine using this procedure. Because small-bowel lesions » of CD patients,” convention: S has limitations in detecting CD lesions, SBFT is used to detect @ presence of fistulae or mucosal damage in CD. However, @ detection of small erosions or ay pends on the skill of the examiner. The range of diagnostic and therapeutic investigations for the small wireless capsule endoscopy computed tomograg (MR) enteroclysis or MR enterography (MRE). tivity values for the detection of extra-enteric complications y higher for CT and MRE than for SBFT.”” In addition, balloon-assisted enterascopy such as double-balloon endosco endoscopy ° techniques. An advantage of enteroscopy is that it enables concise assessment of the mucosa and acquisition of histopathologic specimens. In addition, endoscopic therapeutic procedures such as balloon dilatation of stenoses can be performed. Enterascopy is ex- pe become an integral procedure for CD assessment The European Crohn's and Colitis Organisation guideline recommends MR or CT enterography or enteroclysis as imaging techniques with the highest diagnostic accuracy for the detection of intestinal involvement in CD, including extramural complications.*? However, a recent study Diagnostic Test • Did clinicians face diagnostic uncertainty? – Consider: Who are the patients and do they reflect the general population? – Did the patients need a diagnostic test or was the diagnosis clear? • Was there a blind comparison between the test and an appropriate independent reference standard? – Blinding – Reference/Gold standard • Was the gold standard test performed on everyone? Diagnostic Test • What likelihood ratios were associated with the range of test results? – Describes the impact of the test result on the pre-test probability of the disease – LR+ = sensitivity/(1-specificity) – LR- = (1-sensitivity)/specificity – General guidelines: LR+ >10 essentially makes the diagnosis, LR- <0.1 essentially rules out the disease Prognosis • Was the sample of patients appropriate to the question at hand and representative of patients with this problem? – Consider: Different forms of identifying the cohort may carry different biases. Referrals to a tertiary care center? Automatically enrolled? • Were the patients sufficiently similar with respect to prognostic risk? – Are patients similar enough to analyze as a group? • Was follow-up sufficiently long and complete? • Was the primary outcome appropriate and clearly defined? Th NEW ENGLAND JOURNAL of MEDICINE ESTAMLIEWED IN 1812 DECEMBER 18, 2014 Cytisine versus Nicotine for Smoking Cessation Natalie Walker, Ph.D., Colin Howe, Ph.D, Marewa Glover, Ph.D, Hayden M Joanne Bames, Ph.D. Vili No: M.8,ChB., Ph.D, Ph.D. Varsha Parag, M.Sc, Bruce Bassett, B+ ABSTRACT BACEGROUND Placebo-control!ed trials indicate that cytisine, a partial agonist that binds the nicotinic acetylcholine receptor and és used for smoking cessation, almost doubles the chances of quitting at 6 months. We investigated whether cytisine was atleast fy as effective as nicotine-replacement therapy in helping smokers to quit. METHODS ‘We conducted a pragmatic, openabel, noninferiority trial in New Zealand in which 1310 adult daily smokers who were motivated to quit and called the na- tional quitline were randomly assigned in 2 Isl ratio to receive cytisine for 25 days or nicotine-reptacement therapy fir 8 weeks. Cytisine was provided by mail, free of charge, and nicotine-replacement therapy was provided through vouchers for low-cost patches along with gum or lozenges. Low-int telephone-delivered behavioral support was provided to both groups through the guitline. The pri- ‘mary outcome was selfreported continuous abstinence at 1 month. RESULTS At 1 month, continuous abstinence from smoking was reported for 40% of par- ‘ticipants receiving cytisine (264 of 655) and 31 of participants receiving nicotine- replacement therapy (203: of 655), for a difference of 93 percentage paints (95% confidence interval, 4.2 to 14.5). ‘The offactiveness of cytisine for continuous ab- stinence was superior to that of nicotine-replacement therapy at 1 week, 2 months, and 6 months. In a prespecified subgroup analysis of the primary outcome, cyt sine was superior to nicotine-replacement therapy among wome and noninferior among men. Self reported adverse events over 6 months occurred more frequently in the cytisine group (288 events among 204 participants) than in the grup receiv- ing nicotine-replacement therapy (I74 events among 134 participants}; adverse events were primarily nausea and vomiting and sleep disarders. CONCLUSIONS When combined with brief behavioral support, cytisine was found to be superior From ths National Irestitute far Heath In ovation (NW PCB) theCan- tre for Tobacco Control Research, De- Ff Social and Community H tion Health and the Schoo! of Phacrnacy LE), University of Auckland. Auckland and the Quit Group, Walington (BB) — New Zealand: and Quaen Mary Uni ‘versity of London. Wolfson Instituta of Modicine. and UK Contra for Tabatco and Alcohol Studies. Barts and thoLandian Stool ofledicine and D istry — all in Lendion (HM). uddress, reprint requests ta Dr. Walker st the Na- onal Institute for Heath Innovation, School of Population Haaith. University ef auckland. Provata Bag 97019, Avckiand ‘AZ Now Zoaland or at nove auridand ae me engl) Mod 2on4;37-2353-82, DOH 101053) NEIMoRIACr754 prighee 24 Mececham ite Socey. Treatment • Was the assignment of patients to treatments randomized? Was allocation concealed? • Were all patients who entered the trial accounted for at its conclusion? – How and why were patients lost to follow up? • Were patients analyzed in the groups to which they were randomized? – Intention to treat, modified intention to treat, per protocol • Were patients and clinicians kept blind to which treatment was being received? – Patients, doctors, outcome assessors • Were the groups similar at the start of the trial? – Table 1 Meta-Analysis • Did the review ask a clear and focused clinical question? • Did the review include the right type of article? – RCTs, other articles – Did the included studies address the question or is the data nested in another study? • Were all relevant studies identified? – Databases searched, search terms, reference tracking, unpublished studies, non-English • Was the validity of the included studies appraised? – Garbage in, garbage out • Were assessments of study quality reproducible? – Agreement between assessors (Kappa statistic) • Were results combined appropriately? – Test for heterogeneity – I2 < 50% is acceptable or using Random Effects Model Summary • Critiquing a journal article is an important skill both for academic and private practice physicians • After deciding to read an article, the key is to be systematic • Depending on study type, look for methodological validity, results, and applicability • I have “cheat sheets” for reviewing different types of articles if you are interested. Adam C. Ehrlich, MD, MPH Department of Medicine, Section of Gastroenterology Temple University School of Medicine Modified from Deborah Korenstein, MD at Icahn School of Medicine at Mount Sinai Critical Appraisal Form - Cohort studies This critical appraisal form should be used for cohort studies about prognosis or benefit or harm of a treatment or exposure. The following questions will help focus your attention on the important methodological issues related to cohort studies. They are divided into three sections: validity, results, applicability. VALIDITY: 1. Was the method for determining the exposure and control groups objective and accurate? How was the control group established? Consider: • Different forms of identifying cohorts or exposure within them may carry different degrees of bias. Patient report of exposures may have associated recall bias. • In general, look for objective determinations of exposure, and quantification of the amount of exposure. Yes □ Cannot tell □ No □ How was the cohort established? How were the exposure and control groups established? ________________________________________________________________ ________________________________________________________________ ________________________________________________________________ 2. Were there any serious covariate imbalances? In other words, were the two groups adequately similar at the start of the trial? Consider: • There will always be differences, but the differences should not be so fundamental that comparisons do not make sense • Think of the major variables impacting the outcome of interest, and look for patient characteristics regarding those variables. Yes □ Cannot tell □ No □ What were the important covariates (for the outcome of interest) and were they similar? ________________________________________________________________ ________________________________________________________________ Adam C. Ehrlich, MD, MPH Department of Medicine, Section of Gastroenterology Temple University School of Medicine Modified from Deborah Korenstein, MD at Icahn School of Medicine at Mount Sinai 3. Was the reference (“gold”) standard test performed on all patients regardless of the result of the test being evaluated? Think about • Whether patients with both positive and negative index tests actually received the reference standard test • If not all patients receive the reference standard, verification bias is said to be present Yes □ Cannot tell □ No □ Describe which patients received the reference standard: ________________________________________________________________ ________________________________________________________________ ________________________________________________________________ RESULTS What likelihood ratios were associated with the range of possible test results? Consider: • Likelihood ratios (LR) describe the impact of a test result on the pre-test probability of disease and can be calculated for a positive test, a negative test, or for a particular test result or range of results. • The LR is the ratio of likelihood of having disease with a given test result divided by the likelihood of not having disease with that same test result • The LR for a positive test is: sensitivity/1-specificity and for a negative test is 1-sensitivity/specificity • A LR of >10 for a positive test means that a positive result essentially “makes the diagnosis”; a negative LR of <.1 means a negative test can essentially rule out the disease. • See the end of this document for a figure to apply the likelihood ratio to the pre-test probability to establish a post-test probability of having the disease. What are the likelihood ratios? ___________________________________________________________ ___________________________________________________________ ___________________________________________________________ ___________________________________________________________ APPLICABILITY 1. Will the reproducibility of the test result and its interpretation be satisfactory in my clinical setting? Consider • Is the test utilized in the study available? • Are local providers capable of interpreting the test accurately? Adam C. Ehrlich, MD, MPH Department of Medicine, Section of Gastroenterology Temple University School of Medicine Modified from Deborah Korenstein, MD at Icahn School of Medicine at Mount Sinai Yes □ Cannot tell □ No □ Describe any issues related to using the test locally: ___________________________________________________________ ___________________________________________________________ 2. Are the results applicable to the patient in my practice? Consider if the patients in the study are adequately similar to my own patients, and in particular to patients in whom I would order this test. Yes □ Cannot tell □ No □ Why (or why not) are the results applicable to your patients? What aspects of patient spectrum contribute to applicability? ___________________________________________________________ ___________________________________________________________ 3. Will the results change my management strategy? Consider • By how much do the LRs associated with this test change my pre-test probability of disease? • Is the test accurate enough to impact my treatment plan? In what type of patients is it useful? Yes □ Cannot tell □ No □ How would this test change management strategy? ___________________________________________________________ ___________________________________________________________ ___________________________________________________________ 4. Will patients be better off as a result of the test? Consider • In what way will patients be better off? • Will this test spare patients from having other testing done • How do we really determine if patients are better off? Yes □ Cannot tell □ No □ Describe how patients will be better off as a result of this test. ___________________________________________________________ ___________________________________________________________ ___________________________________________________________ Adam C. Ehrlich, MD, MPH Department of Medicine, Section of Gastroenterology Temple University School of Medicine Modified from Deborah Korenstein, MD at Icahn School of Medicine at Mount Sinai Nomogram for Likelihood Ratio 1. Estimate the pre-test probability. This may be from previously established data or your estimate that is specific to your patient. 2. Calculate the likelihood ratio as described above in the results section. 3. Draw a straight line between the pre-test probability and the likelihood ratio. If you continue the line, you can connect it to the post-test probability. Adam C. Ehrlich, MD, MPH Department of Medicine, Section of Gastroenterology Temple University School of Medicine Modified from Deborah Korenstein, MD at Icahn School of Medicine at Mount Sinai 6. If results were combined, was it done appropriately? Consider the following issues: • Results should only be combined if they are similar from study to study. • Results of each study should be presented clearly, usually in a Forest plot showing the effect of the intervention in each study. Sometimes these are included in supplementary materials. • Were the results similar from study to study? Look for a test of heterogeneity (or homogeneity). The favored test for heterogeneity is the I2 test. The result ranges from 0-100%, with 0% indicating no heterogeneity (i.e. all differences in study results are due to chance alone) and 100% indicating heterogeneity (i.e. no differences in study results are due to chance alone). In general, I2 below 50% represents acceptable lack of heterogeneity, meaning that the studies are similar and pooling results is appropriate. • If there is no heterogeneity, authors will generally pool results using the Fixed Effects Model. If there is heterogeneity, look for them to use the Random Effects Model, which is more conservative and allows for pooling of somewhat disparate studies, with wider confidence intervals. Yes □ Cannot tell □ No □ Describe the degree of similarity among the studies and how pooling was done. ___________________________________________________________ ___________________________________________________________ ___________________________________________________________ RESULTS 1. How are the results presented and what are are the main results? Look for • How results are presented- usually RR or OR are combined for the main result Describe the main result and how it is presented. ___________________________________________________________ ___________________________________________________________ ___________________________________________________________ ___________________________________________________________ 2. How precise are the results? Look for • Confidence intervals around the main summary results • Does the confidence interval cross 1? Adam C. Ehrlich, MD, MPH Department of Medicine, Section of Gastroenterology Temple University School of Medicine Modified from Deborah Korenstein, MD at Icahn School of Medicine at Mount Sinai Is precision addressed, and if so, how? ___________________________________________________________ ___________________________________________________________ ___________________________________________________________ APPLICIBILITY 1. Were all clinically important outcomes considered? Yes □ Cannot tell □ No □ What are the most important outcomes? ___________________________________________________________ ___________________________________________________________ 2. Can the results be applied to my patient care? Consider: • How does the patient population compare to my own? • Were the interventions similar to what I can accomplish in my setting • The overall quality of the included studies. How does this review apply to patient care? ___________________________________________________________ ___________________________________________________________ ___________________________________________________________ Adam C. Ehrlich, MD, MPH Department of Medicine, Section of Gastroenterology Temple University School of Medicine Modified from Deborah Korenstein, MD at Icahn School of Medicine at Mount Sinai Modified JADAD score ≥4 considered good quality study Adam C. Ehrlich, MD, MPH Department of Medicine, Section of Gastroenterology Temple University School of Medicine Modified from Deborah Korenstein, MD at Icahn School of Medicine at Mount Sinai Think about • The properties of the included patients and whether those properties make it different from other populations • Was follow up sufficiently long? • Whether confounding factors were adequately adjusted for Yes □ Cannot tell □ No □ What are the important issues regarding the applicability of this study? ________________________________________________________________ ________________________________________________________________ Adam C. Ehrlich, MD, MPH Department of Medicine, Section of Gastroenterology Temple University School of Medicine Modified from Deborah Korenstein, MD at Icahn School of Medicine at Mount Sinai Critical Appraisal Worksheet - Treatment The following questions will help focus your attention on the important methodological issues related to articles on therapy. They are divided into three sections: validity, results, applicability. VALIDITY 1. Was the assignment of patients to treatments randomized? Was allocation concealed? Consider: • How was the randomization done? Is that method likely to be effective? • Allocation concealment means that investigators assessing patients for entry into the trial (PRIOR to randomization) would be unable to predict to which group the next patient will be randomized. Authors should describe the method for randomization (centrally done, opaque envelopes) in sufficient detail to ascertain allocation concealment. Yes □ Cannot tell □ No □ Describe the method of randomization and allocation concealment: ___________________________________________________________ ___________________________________________________________ ___________________________________________________________ 2. Were all patients who entered the trial accounted for at its conclusion? Consider: • How many patients were lost to follow up? • Why patients were lost to follow up? • How the study considered patients who were lost> Many studies will carry forward the last available data for patients who were lost to follow up. Yes □ Cannot tell □ No □ How complete was the follow up and how did the investigators manage the data from patients who dropped out? ______________________________________________________________ ______________________________________________________________ 3. Were patients analyzed in the groups to which they were randomized? Adam C. Ehrlich, MD, MPH Department of Medicine, Section of Gastroenterology Temple University School of Medicine Modified from Deborah Korenstein, MD at Icahn School of Medicine at Mount Sinai This is known as intention-to-treat analysis, in which patients are counted in their original group regardless of the treatment they ultimately receive. It is important in minimizing bias. Sometimes when many patients “crossed over’ to the other group investigators will also do a per protocol, or as treated, analysis, which looks at the treatment each patient actually received. Yes □ Cannot tell □ No □ How did the investigators manage cross overs? ________________________________________________________________ ________________________________________________________________ ________________________________________________________________ 4. Were patients and clinicians kept blind to which treatment was being received? Consider all the parties who should be blinded: • Patient receiving the treatment, especially if outcomes are more subjective • Doctors caring for the patients • Outcome assessors. This is probably the most important. In studies in which patients cannot be blinded for logistical reasons, the investigators determining which patients met study endpoints should still be blinded • Investigators writing the manuscript (until the last minute). Few studies do this but it probably represents the least biased method for interpreting the data. Yes □ Cannot tell □ No □ Who was blinded? If aspects of the study were not blinded, how much bias was introduced? ________________________________________________________________ ________________________________________________________________ ________________________________________________________________ 5. Were the groups similar at the start of the trial? Consider • This information is often presented in “Table 1” • Overall, were the groups similar with regard to most features? • Are all the pertinent features of the patient population presented? The paper should describe patient similarity with regard to all variables that might impact the outcome of interest. • If there are differences between the groups, are they clinically (as opposed to statistically) significant? In which direction do the differences bias the study (i.e. in favor of which group doing better)? Yes □ Cannot tell □ No □ In what ways did the groups differ? List any clinically important variables that were not described. ________________________________________________________________ ________________________________________________________________