Week 3: Lab

Week 3 Lab AssignmentName:________________________
Instructor Name: _______________
Please use this template to help answer the questions listed in the lab
instructions. The “parts” below refer to the parts listed in the lab instructions.
Type your answers and post your screenshots in the spaces given below.
Then, save this document with your name and submit it inside the course
room.
Part 1. Read the assigned article.
Please reach out to your instructor if you did not receive the assigned article
for the term by Monday of Week 3.
Part 2. Analyze the article.
Title: Review of [Type out name of Article]
Author(s): [Type out names of Author(s) of the Article]
Summarize the article in one paragraph:
Post a screenshot of a graph/chart from the article that you will analyze:
1
Analysis
(Answer the following questions thoroughly in complete sentences)
A. What type of study is used in the article (quantitative or qualitative)?
Explain how you came to that conclusion.
B. What type of graph or table did you choose for your lab (bar graph,
histogram, stem & leaf plot, etc.)? What characteristics make it this type
(you should bring in material that you learned in the course)?
C. Describe the data displayed in your frequency distribution or graph
(consider class size, class width, total frequency, list of frequencies, class
consistency, explanatory variables, response variables, shapes of
distributions, etc.)
2
D. Draw a conclusion about the data from the graph or frequency distribution
in context of the article.
E. How else might this data have been displayed (Pick two different graphs
that could have been used to display the same data as your selected
graph/table)?
Discuss pros and cons of 2 other presentation options, such as tables or
different graphical displays.
Explain how these graphs would be structured to display the data in the
article. Why don’t you think those two graphs were not used in this article?
F. Give the full APA reference of the article you are using for this lab.
Be sure your name is on the Word document, save it, and then submit it. In
the assignment module, click “start assignment” and then “upload file” and
“submit assignment”.
3
4
REVIEW
Annals of Internal Medicine
Evidence Relating Health Care Provider Burnout and Quality of Care
A Systematic Review and Meta-analysis
Daniel S. Tawfik, MD, MS; Annette Scheid, MD; Jochen Profit, MD, MPH; Tait Shanafelt, MD; Mickey Trockel, MD, PhD;
Kathryn C. Adair, PhD; J. Bryan Sexton, PhD; and John P.A. Ioannidis, MD, DSc
Background: Whether health care provider burnout contributes to lower quality of patient care is unclear.
Purpose: To estimate the overall relationship between burnout
and quality of care and to evaluate whether published studies
provide exaggerated estimates of this relationship.
Data Sources: MEDLINE, PsycINFO, Health and Psychosocial
Instruments (EBSCO), Mental Measurements Yearbook (EBSCO),
EMBASE (Elsevier), and Web of Science (Clarivate Analytics),
with no language restrictions, from inception through 28 May
2019.
Study Selection: Peer-reviewed publications, in any language,
quantifying health care provider burnout in relation to quality of
patient care.
Data Extraction: 2 reviewers independently selected studies,
extracted measures of association of burnout and quality of care,
and assessed potential bias by using the Ioannidis (excess significance) and Egger (small-study effect) tests.
Data Synthesis: A total of 11 703 citations were identified, from
which 123 publications with 142 study populations encompassing 241 553 health care providers were selected. Quality-of-care
outcomes were grouped into 5 categories: best practices (n =
14), communication (n = 5), medical errors (n = 32), patient out-
H
ealth care providers face a rapidly changing landscape of technology, care delivery methods, and
regulations that increase the risk for professional burnout. Studies suggest that nearly half of health care providers may have burnout symptoms at any given time
(1). Burnout has been linked to adverse effects, including suicidality, broken relationships, decreased productivity, unprofessional behavior, and employee turnover,
at both the provider and organizational levels (2– 6).
Recent attention has been focused on the relation
between health care provider burnout and reduced
quality of care, with a growing body of primary literature and systematic reviews reporting associations between burnout and adherence to practice guidelines,
communication, medical errors, patient outcomes, and
safety metrics (7–11). Most studies in this field use retrospective observational designs and apply a wide
range of burnout assessments and analytic tools to
evaluate myriad outcomes among diverse patient populations (12). This lack of a standardized approach to
measurement and analysis increases risk of bias, potentially undermining scientific progress in a rapidly expanding field of research by hampering the ability to
decipher which of the apparent clinically significant results represent true effects (13). The present analysis
sought to appraise this body of primary and review literature, developing an understanding of true effects
comes (n = 17), and quality and safety (n = 74). Relations between burnout and quality of care were highly heterogeneous
(I2 = 93.4% to 98.8%). Of 114 unique burnout– quality combinations, 58 indicated burnout related to poor-quality care, 6 indicated burnout related to high-quality care, and 50 showed no
significant effect. Excess significance was apparent (73% of studies observed vs. 62% predicted to have statistically significant
results; P = 0.011). This indicator of potential bias was most
prominent for the least-rigorous quality measures of best practices and quality and safety.
Limitation: Studies were primarily observational; neither causality nor directionality could be determined.
Conclusion: Burnout in health care professionals frequently is
associated with poor-quality care in the published literature. The
true effect size may be smaller than reported. Future studies
should prespecify outcomes to reduce the risk for exaggerated
effect size estimates.
Primary Funding Source: Stanford Maternal and Child Health
Research Institute.
Ann Intern Med. 2019;171:555-567. doi:10.7326/M19-1152
For author affiliations, see end of text.
This article was published at Annals.org on 8 October 2019.
Annals.org
within the field by using a detailed evaluation for reporting biases.
Reporting biases take many forms, each contributing to overrepresentation of “positive” findings in the
published literature. Publication bias occurs when studies with negative results are published less frequently
or less rapidly than those with positive results (14). Selective outcome reporting occurs when several outcomes of potential interest are evaluated, but only
those with positive results are presented or emphasized (13). Selective analysis reporting occurs when
several analytic strategies are used, but those that produce the largest effects are presented. Overall, these
biases result in an excess of statistically significant results in the published literature, threatening reproducibility of findings, promoting misappropriation of resources, and skewing the design of studies assessing
interventions to reduce burnout or improve quality (13).
See also:
Editorial comment . . . . . . . . . . . . . . . . . . . . . . . . . 589
Web-Only
Supplement
© 2019 American College of Physicians 555
REVIEW
METHODS
We conducted a systematic literature review and
meta-analysis to provide summary estimations of the
relation between provider burnout and quality of care,
estimate study heterogeneity, and explore the potential
of reporting bias in the field. We followed the PRISMA
(Preferred Reporting Items for Systematic reviews and
Meta-Analyses) and MOOSE (Meta-analysis of Observational Studies in Epidemiology) guidelines for methodology and reporting (15, 16).
Data Sources and Searches
We searched MEDLINE, PsycINFO, Health and Psychosocial Instruments (EBSCO), Mental Measurements
Yearbook (EBSCO), EMBASE (Elsevier), and Web of Science (Clarivate Analytics) from inception through 28
May 2019, with no language restrictions. We used
search terms for burnout and its subdomains (emotional exhaustion, depersonalization, and reduced personal accomplishment), health care providers, and
quality-of-care markers, as shown in Supplement Tables 1 to 3 (available at Annals.org).
Study Selection
We included all peer-reviewed publications reporting original investigations of health care provider burnout in relation to an assessment of patient care quality.
Providers included all paid professionals delivering
outpatient, prehospital, emergency, or inpatient care,
including medical, surgical, and psychiatric care, to patients of any age. We chose an inclusive method of
identifying burnout studies, considering assessments to
be related to burnout if the authors defined them as
such and used any inventory intended to identify burnout,
either in part or in full. Likewise, we chose an inclusive
approach to identify quality-of-care metrics, including any
assessment of processes or outcomes indicative of care
quality. We included objectively measured and subjectively reported quality metrics originating from the provider, other sources within the health care system, or patients and their surrogates. We considered medical
malpractice allegations a subjective patient-reported
quality metric. Although patient satisfaction is an important outcome, it is not consistently indicative of care quality or improved medical outcomes, suggesting that it may
be related to factors outside the provider’s immediate
control, such as facility amenities and access to care (17–
20). Thus, for the purposes of this review, we excluded
metrics solely indicative of patient satisfaction to reduce
bias from these non–provider-related factors that may affect satisfaction.
We included peer-reviewed, indexed abstracts if
they reported a study population not previously or subsequently reported in a full-length article. For study
populations described in more than 1 full-length article, we included the primary result from the paper with
the earliest publication date as the primary outcome,
with any unique outcomes from subsequent articles as
secondary outcomes. We supplemented the database
searches with manual bibliography reviews from included studies and related literature reviews (7–9, 21–
556 Annals of Internal Medicine • Vol. 171 No. 8 • 15 October 2019
Burnout and Quality of Care
24). In line with our aim to look for reporting bias, we
did not expand our search beyond peer-reviewed publications and did not contact authors for unpublished
data. If an article presented insufficient data to calculate
an effect size, we supplemented the information with
data from subsequent peer-reviewed publications
when available; however, we still attributed these effect
sizes to the initial report. We excluded any studies that
were purely qualitative.
All investigators contributed to the development of
study inclusion and exclusion criteria. The literature review and study selection were conducted by 2 independent reviewers in parallel (D.S.T. and either A.S. or
K.C.A.), with ambiguities and discrepancies resolved by
consensus.
Data Extraction and Quality Assessment
We extracted data into a standard template reflecting publication characteristics, methods of assessing
burnout and quality metrics, and strength of the reported relationship. Data were extracted by 2 independent reviewers (D.S.T. and A.S.), with discrepancies resolved by consensus. We estimated effect sizes and
precision using the Hedges g and SEs, respectively.
The Hedges g estimates effect size similarly to the Cohen d, but with a bias correction factor for small samples. In general, 0.2 indicates small effect; 0.5, medium
effect; and 0.8, large effect.
We classified each assessment of burnout as overall burnout, emotional exhaustion, depersonalization,
or low personal accomplishment. We also identified
burnout assessments as standard if defined as an emotional exhaustion score of 27 or greater or a depersonalization score of 10 or greater on the Maslach Burnout
Inventory, or as the midpoint and higher on validated
single-item scales. We categorized quality metrics within
5 groups— best practices, communication, medical errors,
patient outcomes, and quality and safety—and reverse
coded any “high-quality” metrics such that positive effect
sizes indicate burnout’s relation to poor-quality care.
For publications with several distinct (nonoverlapping) study populations reported separately, we considered each population separately for analytic purposes.
For publications with more than 1 outcome for the same
study population, we decided to perform analyses using
only 1 outcome per study, ideally the specified primary
outcome. If no primary outcome was clear, we chose the
first-listed outcome, consistent with reporting conventions
of presenting the primary outcome first. We considered
other outcomes secondary, excluding them from the primary analyses to avoid bias from intercorrelation but including them in selected descriptive statistics and stratified analyses when appropriate.
Data Synthesis and Analysis
We calculated the Hedges g from odds ratios (dichotomized data) by using the transformation
冑3
or from correlation coefficients (unscaled
log共OR兲*
␲
2*r
continuous data) by using the transformation
,
冑1 ⫺ r2
Annals.org
REVIEW
Burnout and Quality of Care
冑
N⫺2
conN
sistent with published norms (25, 26). Further details
are provided in the Supplement (available at Annals
.org).
Most studies reported burnout as a dichotomous
variable or with unscaled effect size estimates, facilitating the aforementioned transformations. We scaled effect sizes accordingly for the 6 studies reporting burnout only as a continuous variable in order to maintain
comparability, adapting our methods from published
guidelines (27, 28). On the basis of known distributions
of burnout scores among providers (29 –31), we calculated the difference between the mean scores of providers with and without burnout to average 47.6% of
the span of the particular burnout scale used. We thus
converted effect sizes from continuous scales to the
corresponding effect size reflecting a 47.6% change in
scale score when needed to extrapolate to dichotomized burnout. We also performed sensitivity analyses
excluding these few scaled effect sizes. Details of this
process are presented in the Supplement.
Initially, we intended to primarily perform a
random-effects meta-analysis including all primary (or
first-listed) effect sizes, with secondary meta-analyses
stratified by quality metric category and by each unique
burnout– quality metric combination. However, because
of high heterogeneity in the pooled meta-analyses, we
report only summary effects from the unique burnout–
quality metric combinations. We also performed sensitivity analyses limited to studies with standard burnout
assessments and those with independently observed or
objectively measured quality-of-care markers. We used
the empirical Bayes method with Knapp–Hartung modboth multiplied by a bias correction factor
ification to estimate the between-study variance ␶2 (32).
We evaluated study heterogeneity using I2. Details regarding this meta-analytic approach are presented in
the Supplement.
We performed the Ioannidis test to evaluate for excess significance (33) by identifying the study population with the highest precision (1/SE) among those with
the lowest risk of bias (studies using a fully validated
burnout inventory with an objective quality metric). We
then calculated the power of all studies to detect the
effect size of this study and compared the observed
versus expected number of studies with statistically significant results by using paired t tests. Next, we stratified excess significance testing by outcome category.
Because small studies may carry increased risk of
bias, we performed the Egger test to look for smallstudy effects (34). We regressed standard normal deviate (Hedges g/SE) on precision (1/SE) by using robust
SEs due to clustering of effect sizes at the study population level.
We used Stata 15.0 (StataCorp) for all analyses. All
tests were 2-sided. For summary effects, we considered
2 different thresholds of statistical significance, P < 0.050 and the newly proposed P < 0.005 (35, 36). We made no further corrections for multiple testing. This study was performed in accordance with the institutional review board requirements of Stanford University and was classified as research not involving human subjects. Role of the Funding Source The funders had no role in study design, data collection, analysis, interpretation, or writing of the report. Figure 1. Evidence search and selection. Articles identified in MEDLINE and PsyclNFO (n = 6715) Articles identified in EMBASE (n = 3871) Articles identified in Web of Science (n = 3116) Duplicate publications (n = 1999) Titles/abstracts screened (n = 11 703) Not relevant (n = 11 390) Selected for full-text review (n = 313) Excluded (n = 193) No burnout predictor: 123 No quality outcome: 46 Review/repeat population: 16 Not quantitative: 7 Not health care providers: 1 Bibliographic reviews (n = 3) Included in final analysis (n = 123) Annals.org Annals of Internal Medicine • Vol. 171 No. 8 • 15 October 2019 557 REVIEW Burnout and Quality of Care Figure 2. Summary of all included burnout– quality metric combinations, showing frequency of effect size reporting (count) and value of summary effect size (Hedges g). t en m pl is h cc om la Lo w pe rs o na er so al ep D ot io n Em ut rn o Bu na us ha ex na rs o pe liz at io n tio n pl is h cc om la liz at io n w Lo er so ep D ot io n Em Bu rn o ut al na ex ha us tio n m en t Burnout Metric Best practices Inappropriate laboratory tests Inappropriate timing of discharge Suboptimal patient care practices Inappropriate use of patient restraints Poor adherence to infection control Inappropriate antibiotic prescribing Lack of close monitoring Low best practice score Neglect of work Poor adherence to management guidelines Poor communication Low patient enablement score Forgetting to convey information Low attention to patient impact Low physcian empathy score Not fully discussing treatment options Poor handoff quality Short consultation length 20 Count 25 Communication 15 10 Errors Quality Metric 30 Self-reported medical errors Self-reported medication errors Self-reported treatment/medication errors Medical error score Observed medical errors Accident propensity Diagnosis delay Diagnostic errors Observed medication errors Self-reported impairment 7 5 3 1 Adverse events Health care–associated infections Patient falls Length of stay Urinary tract infections Mortality Poor pain control HIV viral load suppression Morbidity Posthospitalization recovery time 1.0 2.0 1.5 Outcomes 0 Hedges g 0.5 –0.5 Quality and safety Low quality of care Low patient safety score Low safety climate score Low quality during most recent shift Low work unit safety grade Poor patient care quality score Malpractice allegations Low individual safety grade Low safety perceptions Near-miss reporting Prolonged emergency department visit RESULTS The search identified 11 703 citations. Screening resulted in 313 potentially eligible publications retrieved in full text—120 of which were included—plus 3 additional publications identified by bibliography review (Figure 1). Overall, we included 123 publications from 1994 through 2019 (37–159), encompassing 142 distinct study populations, as detailed in Supplement Table 4 (available at Annals.org). The median sample size was 376 (interquartile range, 129 to 1417). The 142 558 Annals of Internal Medicine • Vol. 171 No. 8 • 15 October 2019 –1.0 –1.5 –2.0 study populations included physicians (n = 71 [50%]), nurses (n = 84 [59%]), and other providers (n = 18 [13%]) for a total of 241 553 health care providers evaluated. Quality metrics covered inpatients (n = 122 [86%]); outpatients (n = 62 [44%]); and adult (n = 134 [94%]), pediatric (n = 93 [65%]), medical (n = 135 [95%]), and surgical (n = 89 [63%]) patients. Only 4 studies explicitly specified a primary outcome. Six studies did not provide sufficient data to derive an effect size from the original publication but provided usable Annals.org REVIEW Burnout and Quality of Care data published in a subsequent review (39, 66, 69, 107, 115, 117). One research group reported results from a single study population in 2 publications; the first published effect was considered primary, with results from the later publication considered secondary effects (112, 160). Overall burnout, emotional exhaustion, and depersonalization were the primary predictors for 56, 75, and 11 study populations, respectively, from a variety of survey instruments, as outlined in Supplement Table 5 (available at Annals.org). The 50 distinct quality metrics included 10 best practices, 8 communication, 10 medical errors, 10 patient outcomes, and 12 quality and safety measures (26 measured provider perception of quality, 15 used independent or objective measures of quality, and 9 included both types of assessments). As illustrated in Figure 2, 38 (33%) of the 114 distinct burnout– quality combinations were reported 3 or more times. The most frequently reported effect related emotional exhaustion to low quality of care (n = 41), with most of the reported effect sizes in the quality and safety and medical errors categories. Although all 5 categories of outcomes had estimates more frequently relating burnout in the direction of poor quality of care (denoted in red in Figure 2), 7 of the 16 estimates pointing in the opposite direction were found in the communication category. Results were similar when limited to primary (or first-listed, when primary was not specified) effect sizes only (Supplement Figure 1, available at Annals.org). Meta-analyses combining burnout and quality metrics within quality categories revealed I2 values of 93.4% to 98.8%, indicating extremely high heterogeneity; therefore, summary effects are provided only at the level of the 114 distinct burnout– quality combinations, 46 of which included primary effect sizes. Metaanalyses of these 46 combinations revealed 24 (52%) with a statistically significant summary effect greater than 0 (burnout related to poor quality of care), 1 (2%) with statistically significant summary effects less than 0 (burnout related to high quality of care), and 21 (46%) with no difference at the P < 0.050 threshold. When the P < 0.005 threshold was used, the respective numbers were 18 (39%), 1 (2%), and 27 (59%). Results are summarized in Table 1, and primary effect sizes from all included studies are shown in Supplement Figure 2 (available at Annals.org). Results were similar when secondary effect sizes were included. Of the 114 distinct burnout– quality metric combinations, 58 (51%) had statistically significant summary effects greater than 0, 6 (5%) had statistically significant effects less than 0, and 50 (44%) showed no difference at the P < 0.050 threshold. When the P < 0.005 threshold was used, the respective numbers were 47 (41%), 6 (5%), and 61 (54%). Results from all burnout– quality metric combinations are shown in Supplement Figure 3 (available at Annals.org). Our findings were similar when limited to studies explicitly using standard burnout definitions, but the observed relationships were attenuated when limited to independent or objective quality metrics, as shown in Table 1. The most precise study with low risk of bias (143) reported a small effect size (Hedges g = 0.26, analogous to an odds ratio of 1.5 to 1.6). Using this estimate, the Ioannidis test found an excess of observed versus predicted statistically significant studies (73% observed vs. 62% predicted at the 0.050 significance threshold, P = 0.011) (Table 2). When stratified by quality metric category, an excess of statistically significant studies was seen in the categories of best practices and quality and safety. Results were similar for the P < 0.005 threshold. The Egger test did not show small-study effects (intercept, ⫺1.32 [95% CI, ⫺3.48 to 0.85]), indicating that smaller studies did not systematically overestimate effect sizes (Figure 3). A funnel plot relating effect size to SE is shown in Supplement Figure 4 (available at Annals.org). DISCUSSION This overview extends previous work in the field by including a comprehensive evaluation for reporting biases in the health care provider burnout literature, encompassing 145 published study populations that quantified the relation between burnout and quality of care over 25 years for 241 553 health care professionals. Most of the evidence suggests a relationship between provider burnout and impaired quality of care, consistent with recent reviews of various dimensions (7– 10, 22). Although the effect sizes in the published literature are modestly strong, our finding of excess significance implies that the true magnitude may be smaller than reported, and the studies that attempted to lower the risk of bias demonstrate fewer significant associations than the full evidence base. That only 4 studies Table 1. Number and Direction of Summary Effect Sizes for Each Combination of Burnout and Quality Metric* Criteria for Inclusion Burnout–Quality Combinations, n† P < 0.050 Threshold, n (%) P < 0.005 Threshold, n (%) Hedges g > 0‡ Hedges g < 0§ No Effect円円 Hedges g > 0‡ Hedges g < 0§ No Effect円円 Primary effects only 46 Primary and secondary effects 114 Standard burnout definitions 24 Independent/objective quality metrics 48 24 (52) 58 (51) 15 (62) 14 (29) 1 (2) 6 (5) 1 (4) 2 (4) 21 (46) 50 (44) 8 (33) 32 (67) 18 (39) 47 (41) 14 (58) 9 (19) 1 (2) 6 (5) 1 (4) 2 (4) 27 (59) 61 (54) 9 (38) 37 (77) * Summary effect sizes obtained via empirical Bayes meta-analysis. † Number of distinct burnout– quality combinations represented. ‡ Indicates burnout related to poor-quality care. § Indicates burnout related to high-quality care. 兩兩 Not significantly different from 0 at the specified P value threshold. Annals.org Annals of Internal Medicine • Vol. 171 No. 8 • 15 October 2019 559 REVIEW Burnout and Quality of Care Table 2. Predicted Versus Observed Significance for Primary* Effect Sizes, Among All Included Studies and Stratified by Quality Metric Category Category Full cohort Best practices Communication Medical errors Patient outcomes Quality and safety Studies, n 142 14 5 32 17 74 P < 0.050 Threshold P < 0.005 Threshold Predicted Significance, % Observed Significance, n (%) P Value Predicted Significance, % Observed Significance, n (%) P Value 62 12 43 50 64 65 104 (73) 9 (64) 3 (60) 20 (62) 9 (53) 62 (84) 0.011 0.001 0.67 0.169 NP