Assignment: Lab
Required Resources
Read/review the following resources for this activity:
Scenario/Summary
This week’s lab highlights the use of graphics, distributions, and tables to summarize and interpret data.
Deliverables
The deliverable is a Word document with your answers to the questions posed below based on the article you find.
Required Software
Steps to Complete Week 3 Lab
Part 1:
Part 2:
Post a screenshot of the article’s frequency table and/or graph. Example:Frequency Distribution -OR- GraphAnswer the following questions about your table or graph. What type of study is used in the article (quantitative or qualitative)?Explain how you came to that conclusion. What type of graph or table did you choose for your lab (bar graph, histogram, stem & leaf plot, etc.)?What characteristics make it this type (you should bring in material that you learned in the course)?Describe the data displayed in your frequency distribution or graph (consider class size, class width, total frequency, list of frequencies, class consistency, explanatory variables, response variables, shapes of distributions, etc.) Draw a conclusion about the data from the graph or frequency distribution in the context of the article. How else might this data have been displayed?Discuss the pros and cons of 2 other presentation options, such as tables or different graphical displays.Why do you think those two other presentation options (i.e., tables or different graphs) were not used in this article?
Put values in blue cells; output or answers in YELLOW cells
Data
1
1
2
2
4
5
6
9
11
Mean
Median
Mode
Sample Variance
Sample Standard Deviation
Population Variance
Population Standard Deviation
Range
Count (n)
Min
Quartile 1
Median
Quartile3
Max
Interquartile Range (IQR)
4.5556
4.0000
1.0000
12.7778
3.5746
11.3580
3.3702
10.0000
9.0000
1.0000
1.5000
4.0000
7.5000
11.0000
6.0000
2
#N/A
#N/A
#N/A
(Returns more than one mode)
Maslach Burnout Inventory and a Self-Defined, Single-Item Burnout
Measure Produce Different Clinician and Staff Burnout Estimates
Margae Knox, MPH, Rachel Willard-Grace, MPH, Beatrice Huang, BA, and Kevin Grumbach, MD
Center for Excellence in Primary Care, Department of Family and Community Medicine, University of California, San Francisco, CA, USA.
BACKGROUND: Clinicians and healthcare staff report
high levels of burnout. Two common burnout assessments are the Maslach Burnout Inventory (MBI) and a
single-item, self-defined burnout measure. Relatively little is known about how the measures compare.
OBJECTIVE: To identify the sensitivity, specificity, and
concurrent validity of the self-defined burnout measure
compared to the more established MBI measure.
DESIGN: Cross-sectional survey (November 2016–January 2017).
PARTICIPANTS: Four hundred forty-four primary care
clinicians and 606 staff from three San Francisco Aarea
healthcare systems.
MAIN MEASURES: The MBI measure, calculated from a
high score on either the emotional exhaustion or cynicism
subscale, and a single-item measure of self-defined burnout. Concurrent validity was assessed using a validated,
7-item team culture scale as reported by Willard-Grace
et al. (J Am Board Fam Med 27(2):229–38, 2014) and a
standard question about workplace atmosphere as
reported by Rassolian et al. (JAMA Intern Med
177(7):1036–8, 2017) and Linzer et al. (Ann Intern Med
151(1):28–36, 2009).
KEY RESULTS: Similar to other nationally representative
burnout estimates, 52% of clinicians (95% CI: 47–57%)
and 46% of staff (95% CI: 42–50%) reported high MBI
emotional exhaustion or high MBI cynicism. In contrast,
29% of clinicians (95% CI: 25–33%) and 31% of staff (95%
CI: 28–35%) reported Bdefinitely burning out^ or more
severe symptoms on the self-defined burnout measure.
The self-defined measure’s sensitivity to correctly identify
MBI-assessed burnout was 50.4% for clinicians and
58.6% for staff; specificity was 94.7% for clinicians and
92.3% for staff. Area under the receiver operator curve
was 0.82 for clinicians and 0.81 for staff. Team culture
and atmosphere were significantly associated with both
self-defined burnout and the MBI, confirming concurrent
validity.
CONCLUSIONS: Point estimates of burnout notably differ
between the self-defined and MBI measures. Compared to
the MBI, the self-defined burnout measure misses half of
high-burnout clinicians and more than 40% of highburnout staff. The self-defined burnout measure has a
Electronic supplementary material The online version of this article
(https://doi.org/10.1007/s11606-018-4507-6) contains supplementary
material,
which is available to authorized users.
?
Received November 10, 2017
Revised March 26, 2018
Accepted May 16, 2018
Published online June 4, 2018
1344
low response burden, is free to administer, and yields
similar associations across two burnout predictors from
prior studies. However, the self-defined burnout and MBI
measures are not interchangeable.
KEY WORDS: burnout; measurement; health services research.
J Gen Intern Med 33(8):1344–51
DOI: 10.1007/s11606-018-4507-6
© Society of General Internal Medicine 2018
BACKGROUND
1
The National Academy of Medicine and Agency for Health2
care Quality and Research , 3 have spotlighted concerning
levels of burnout among clinicians and healthcare staff, particularly in primary care. High levels of burnout are
concerning not only for clinician and staff well-being. A
burntout workforce may adversely affect clinical quality, pa1
tient experience, and costs of care.
Several burnout-related initiatives aim to support the
Bquadruple aim^ of a sustainable clinician and staff work
experience in addition to improved patient experience, quality,
4
and lower costs. The American Medical Association’s
BSTEPS Forward^ modules offer guidance on practice trans5
formation and clinician and trainee well-being. The Society
for General Internal Medicine made burnout a theme of its
6
2017 Annual Meeting. The American Board of Family Medicine added work experience questions to its 2016 recertifica7
tion registration to better understand and track burnout, and
CEOs of leading healthcare organizations have issued a call to
action that argues for regular measurement of physician well8
being.
Within this context of heightened attention to clinician and
staff well-being, greater understanding of the instruments used
to measure burnout is essential. Two common measures are
the Maslach Burnout Inventory (MBI) and a five-choice,
single item based on self-defined burnout. The MBI, considered an industry standard, has been fielded across large samples of diverse occupations in multiple countries. It is composed of three dimensions: emotional exhaustion, cynicism (or
depersonalization), and personal accomplishment (or professional efficacy). The 16-item General MBI survey uses the
terms cynicism and personal accomplishment while the 22item Health Services Personnel survey uses the analogous
terms depersonalization and professional efficacy, with results
JGIM
1345
Knox et al.: Clinician and Staff Burnout Measures
9
consistent across survey versions. MBI instruments require a
license fee to administer.
The self-defined burnout measure is one of the ten survey
questions on the BMini-Z^ work experience instrument devel10
oped by Linzer and colleagues. The item originated from a
11
study of burnout in HMOs and has subsequently been in7
cluded in several major studies. , 12–14 It is free to use.
Estimates of Clinicians and Staff Burnout
The most recent national estimate of burnout from MBI subscales identified high emotional exhaustion among 46.9% of
physicians and high cynicism among 34.6% of physicians,
with 54.4% of physicians reporting one or both symptoms.
Primary care physicians from that study reported even higher
burnout levels; more than 60% expressed high emotional
15
exhaustion and/or cynicism. Other studies indicate burnout
is also high among healthcare staff, with high emotional
16
exhaustion ranging from 30 to 40%. , 17 However, studies
using the self-defined burnout measure have found lower point
estimates: about one in four family physicians and general
internists report Bburning out,^ Bpersistent burnout
7
symptoms,^ or Bcomplete burnout.^ , 12, 18 These divergent
estimates of burnout prevalence raise questions as to whether
study populations differ in their well-being, or how MBI and
self-defined measures perform differently.
Published Comparisons of Burnout Measures
Few studies directly compare how the MBI and selfdefined burnout measures perform in the same sample of
physicians. One physician survey found the self-defined
burnout measure strongly correlated with the MBI emotional exhaustion subscale (r = 0.64, p < 0.0001); the cynicism subscale was less strongly correlated (r = 0.324, p
18
value not reported). A second study of Australian oncology workers also found the self-defined burnout measure
and MBI emotional exhaustion subscale strongly correlated
19
(r = 0.68, p < 0.0001). Only one study used an analytic
approach other than correlations. That study, a survey of
rural physicians and advance practice clinicians, examined
whether the self-defined burnout measure predicted high
and low MBI subscale burnout categories using multivariate linear mixed models. The self-defined burnout measure
significantly predicted high emotional exhaustion but did
not predict low emotional exhaustion or any category of
20
cynicism.
Prior studies have also analyzed sensitivity and specificity for MBI single-item measures (one item from each
MBI subscale’s five items). A correlation of 0.76–0.83
was found for the MBI emotional exhaustion single-item
vs. subscale. A correlation of 0.61–0.72 was found for the
21
MBI cynicism single-item vs. subscale. Compared to the
self-defined burnout single-item, the MBI emotional exhaustion single-item had a high correlation (r = 0.79) and
17
sensitivity and specificity over 80%.
These results suggest that the single-item self-defined burnout measure and MBI subscales have strong agreement. However, no comparison to our knowledge has described sensitivity and specificity for the self-defined burnout measure and
MBI subscales. Moreover, we are not aware of any previous
study examining concurrent validity of the self-defined and
MBI responses with related, validated work environment
measures. Given many national surveillance efforts and program evaluations using the self-defined burnout measure,
there is need for greater understanding among policy makers,
researchers, and healthcare leaders on how results compare
with the more established MBI burnout subscales.
OBJECTIVE
We compared the self-defined burnout measure and MBI in
the same sample of primary care clinicians and staff. Our aims
were to (1) compare the prevalence of burnout from the
different measures, (2) test the sensitivity and specificity of
the self-defined burnout measure to identify individuals experiencing high burnout compared to standard MBI benchmarks,
and (3) determine if the self-defined burnout measure and MBI
have similar associations with a clinic team culture survey
measure previously found to be significantly associated with
22
MBI scores and a workplace atmosphere survey measure
previously found to be significantly associated with the self23
defined burnout measure.
METHODS
Design
This study was a cross-sectional survey, approved by the
Institutional Review Board of the University of California,
San Francisco (protocol numbers 11-08048 and 17-23324).
Participants
We surveyed clinicians and staff working in primary care
clinics in three San Francisco area health systems: a
university-run clinic network, a network of neighborhood
and hospital-based clinics administered by a county health
department, and a large private medical group. All clinicians
and staff at the university and county clinics and all clinicians
at the private group were eligible to participate. Clinicians
consist of physicians of family and internal medicine, physician assistants, and nurse practitioners. Staff members include
registered nurses, medical assistants, and administrative support. The survey was fielded between November 2016 and
January 2017, and was primarily administered electronically.
Each person received an e-mail invitation to complete the
survey, with up to five reminder e-mails to non-respondents.
Paper surveys were administered during staff meetings at
some county health network sites based on leadership request.
Respondents at two systems were entered into a $25 gift card
1346
JGIM
Knox et al.: Clinician and Staff Burnout Measures
raffle; the third system elected to give each respondent $50 for
participation.
calculated using Pearson’s correlation coefficients to measure
the association between the self-defined measure with the
17
MBI emotional exhaustion and MBI cynicism subscales. ,
Measures
19, 27
The survey included the 16-item MBI General Survey subscales for emotional exhaustion and cynicism as well as a
single-item, self-defined burnout measure. MBI subscales
were each composed of five burnout symptoms. Respondents
rated how often they experience each symptom from 0 (never)
to 6 (every day), and responses were summed for each subscale (composite score of 0–30 points). High, medium, and
low MBI burnout cut points were based on a distribution of the
24
composite score into terciles from a reference population.
High emotional exhaustion was defined as a composite score
greater than or equal to 16; high cynicism was a composite
9
score greater than or equal to 11. We primarily analyzed MBI
scores based on the presence of high emotional exhaustion or
high cynicism, as done in commonly cited national prevalence
15
estimates. , 25
Self-defined burnout was a single question that assessed
burnout on a scale from 1 to 5. Most studies using this measure
define high burnout as answering positively to option 3, 4, or
7
5. , 12 Response options were as follows: (1) BI enjoy my work.
I have no symptoms of burnout^; (2) BOccasionally I am under
stress, and I don’t always have as much energy as I once did,
but I don’t feel burned out^; (3) BI am definitely burning out
and have one or more symptoms of burnout, such as physical
and emotional exhaustion^; (4) BThe symptoms of burnout
that I’m experiencing won’t go away. I think about work
frustrations a lot^; and (5) BI feel completely burned out and
often wonder if I can go on. I am at the point where I may need
some changes or may need to seek some sort of help.^
The survey also included a validated 7-item measure of
team culture previously found to be associated with the
MBI. Team culture included agreement with statements such
as, BThe group of staff and providers I work with most
regularly work well together as a team^ and BI can rely on
other people at my clinic to do their jobs well.^ Respondents
rated each item from 1 (strongly disagree) to 10 (strongly
agree). A composite score was calculated as an average across
22
the seven items. Workplace atmosphere was assessed on a
scale from 1 to 5 in response to: BWhich number best describes
the atmosphere in your primary work area?^ Response
anchors included 1 (calm), 3 (busy but reasonable), and 5
10
(hectic, chaotic). , 23
Data Analysis
26
All analyses were conducted using Stata 13 and stratified by
clinician or staff respondent. We stratified clinician and staff
14
analyses to be consistent with other reportings and based on
prior findings of differences in burnout between clinicians and
22
staff. We also conducted sub-analyses by gender and parttime work status to confirm whether results were consistent.
Correlations for comparing our results with other studies were
We used a self-defined burnout cut point of 3 (definitely
burning out) or above to test sensitivity and specificity for
detecting respondents with high burnout compared to standard
MBI classification. We also explored cut points of 2 (under
stress) and 4 (persistent symptoms) to assess sensitivity and
specificity trade-offs and produced area under receiver operator curves (AUC). An AUC of 1.0 indicates a perfect diagnostic test; above 0.9 indicates excellent discrimination; 0.8–0.9 is
good; 0.7–0.8 is fair; and 0.5–0.7 is non-discriminating to poor
28
discrimination. , 29
Table 1 Characteristics of Survey Respondents and Burnout Levels
Clinicians
Staff
n
n
All respondents
444
Respondent characteristics
System
System 1 (university
114
operated)
System 2 (county
175
administered)
System 3 (private medical 155
group) a
Gender b
Male
73
Female
216
Transgender/other
0
Tenure with health system
< 1 year
32
1–5 years
106
6–10 years
91
11–15 years
76
> 15 years
135
Clinic sessions per week
1–2 half-days (clinicians)
98
3–5 half-days (clinicians)
226
6 or more half-days
117
(clinicians)
Less than 20 h per week
n/a
(staff)
More than 20 h per week
n/a
(staff)
Burnout levels
MBI (exhaustion)
Low (0–10)
141
Medium (11–15)
100
High (16+)
197
MBI (cynicism)
Low (0–10)
202
Medium (6–10)
89
High (11+)
147
MBI (high exhaustion or
231
cynicism)
Self-defined burnout measure
1—no symptoms
90
2
217
3—burning out
99
4
21
5—burned out, seeking
5
help
Column
%
606
26
181
30
39
425
70
35
n/a
n/a
26
75
0
101
497
5
17
82
1
7
24
21
17
31
68
133
113
107
165
12
23
19
18
28
22
51
27
n/a
n/a
n/a
n/a
n/a
n/a
n/a
57
10
n/a
535
90
32
23
45
252
121
226
42
20
38
46
20
34
52
277
123
198
277
46
21
33
46
21
50
23
5
1
161
237
110
55
16
28
41
19
9
3
MBI Maslach Burnout Inventory, n/a not applicable
Staff in system 3 were not surveyed
b
Gender was not asked in system 3
a
Column
%
JGIM
Knox et al.: Clinician and Staff Burnout Measures
1347
Figure 1 High burnout based on MBI subscales and the self-defined burnout measure.
We assessed concurrent validity by calculating the proportion of burned out and not burned out respondents for both the
self-defined and MBI measures in connection with (1) strong
team culture and (2) hectic or chaotic work atmosphere. Unadjusted odds ratios were calculated to compare associations
across the two burnout measures.
RESULTS
The response rate was 74%. Four hundred forty-four of 592
clinicians and 606 of 826 staff responded. Respondents were
predominantly female (Table 1). About half had worked with
their health system more than 10 years. Almost all staff (90%)
worked more than 20 h a week while fewer clinicians (27%)
worked six or more half-days in clinic.
High burnout based on the MBI—high emotional
exhaustion or high cynicism—was reported by 52% of
clinicians (95% CI: 47–57%) and 46% of staff (95% CI:
42–50%). Burnout levels for emotional exhaustion and
cynicism subscales individually are reported in Table 1.
High self-defined burnout based on a score of 3
(Bdefinitely burning out^) or greater was reported by
29% of clinicians (95% CI: 25–33%) and 31% of staff
(95% CI: 28–35%). The lower proportion of self-defined
burnout was statistically significant among both clinicians and staff compared to the overall MBI measure
and MBI emotional exhaustion subscale (McNemar’s
test, p < 0.001) but not compared to the MBI cynicism
subscale (Fig. 1).
The correlation between the self-defined burnout measure
and MBI exhaustion subscale was 0.63 for both clinicians and
staff (p value < 0.001). The correlation between the selfdefined burnout measure and MBI cynicism subscale was
0.57 for clinicians and 0.48 for staff (p value < 0.001).
Using the common cut point of 3 or greater for self-defined
burnout, sensitivity was 50.4% among clinicians and 58.6%
among staff—i.e., the proportion of respondents with MBIassessed burnout whose self-defined response also identifies
burnout. Specificity—the proportion of respondents without
MBI-assessed burnout who also did not report self-defined
burnout—was 94.7% for clinicians and 92.3% for staff. A
higher cut point of 4 on the self-defined burnout measure
dropped sensitivity to 11.5% among clinicians and 24.3%
1348
JGIM
Knox et al.: Clinician and Staff Burnout Measures
Table 2 Sensitivity and Specificity of Self-Defined Single-Item Burnout Measure Cut Points for Detecting High Burnout as Measured by MBI
Subscales
Clinicians
Staff
Sensitivity
Specificity
Sensitivity
Specificity
Self-defined single-item
%
95% CI
%
95% CI
%
95% CI
%
95% CI
Cut point = 4+
Cut point = 3+ (standard)
Cut point = 2+
11.5
50.4
98.2
7.7–16.4
43.7–57.1
95.5–99.5
100.0
94.7
41.8
98.2–100.0
90.7–97.3
34.9–48.8
24.3
58.6
91.4
19.3–29.8
52.4–64.5
87.4–94.5
98.1
92.3
44.4
95.9–99.3
88.7–95.0
38.8–50.1
among staff and increased specificity to 100% for clinicians
and 98.1% for staff. A lower cut point of 2 greatly increased
sensitivity to 98.2% among clinicians and 91.4% among staff,
however decreased specificity to 41.8% for clinicians and
44.4% for staff (Table 2). The AUC was 0.82 for clinicians
and 0.81 for staff (Fig. 2). Additional sensitivity, specificity,
and AUC estimates for the individual MBI emotional exhaustion and cynicism subscales are provided as an online appendix. Sub-analyses stratified by gender and clinician half-days
per week yielded similar results.
In an assessment of concurrent validity, strong team culture
was significantly associated with lower burnout for both the
MBI (clinician OR 0.34, 95% CI 0.23–0.51) and self-defined
burnout (clinician OR 0.33, 95% CI 0.22–0.51). A hectic or
chaotic environment was significantly associated with greater
burnout for both the MBI (clinician OR 3.56, 95% CI 2.36–
5.36) and self-defined burnout measure (clinician OR 3.07,
95% CI 1.99–4.72) (Table 3).
DISCUSSION
The most important finding from our study is that the prevalence of burnout among primary care clinicians and staff
differed considerably depending on survey instrument. For
example, burnout prevalence among clinicians was more than
50% higher using the MBI compared to the self-defined
measure’s cut point of 3 (52 vs. 29%). Similarly, the selfdefined measure’s sensitivity to detect individuals with burnout in our sample missed about 50% of clinicians and 41% of
staff that MBI symptoms had classified as experiencing burnout. A lower self-defined cut point of 2 increased sensitivity to
above 90%, but at the expense of substantially decreased
specificity. One explanation for the lower self-defined burnout
point estimate may be reluctance to self-identify as burned out
given an allusion to depression or ineffectiveness. In contrast,
the MBI allows individuals to identify with burnout symptoms
without directly identifying as burned out.
Consistent with other studies, we found a strong, significant
correlation between self-defined burnout and the MBI exhaustion subscale and a modest, significant correlation between
self-defined burnout and the MBI cynicism subscale. Our
analyses add to prior studies by reporting an AUC of 0.81–
0.82, indicating moderate to good discrimination between the
29
self-defined burnout measure and MBI. , 30
Our results are also the first to demonstrate concurrent
validity for both burnout measures in association with team
22
culture, which had only been examined with the MBI, and
workplace atmosphere, which had only been examined with
10
the self-defined measure. , 23 Strong team culture was significantly associated with about one-third lower burnout for both
burnout measures. A chaotic workplace atmosphere was significantly associated with about three times higher burnout for
both burnout measures. The similar magnitude and variance of
Figure 2 Area under the receiver operator curve (AUC) for the MBI (high exhaustion or high cynicism) vs. self-defined burnout measure.
JGIM
1349
Knox et al.: Clinician and Staff Burnout Measures
Table 3 Concurrent Validity: Associations with Team Culture and Workplace Atmosphere for MBI and Self-Defined Burnout Measures
Clinicians
MBI burnout
Team culture
(score of 7 or greater
on a 10-point scale)
Atmosphere in your
primary work area
(4 or 5 on a 5-point
scale, 5 being Bhectic,
chaotic^)
Self-defined burnout
Team culture
(score of 7 or greater
on a 10-point scale)
Atmosphere in your
primary work
area (4 or 5 on a 5-point
scale, 5 being Bhectic,
chaotic^)
All clinicians
(N = 444)
Staff
Burned out
(N = 231)
Not
burned
out
(N = 213)
117
(50.7%)
168 (37.8%)
Unadjusted
odds ratio
All staff
(N = 606)
160
(75.1%)
OR (95% CI)
0.34
(0.23–0.51)
N (column %)
330
105
(54.5%)
(37.9%)
225
(68.4%)
OR (95% CI)
0.28
(0.20–0.39)
119
(51.5%)
49
(23.0%)
3.56
(2.36–5.36)
301
(49.7%)
181
(65.3%)
120
(36.5%)
3.29
(2.35–4.59)
All clinicians
(N = 432)
Burned out
(N = 125)
Unadjusted
odds ratio
All staff
(N = 579)
Burned out
(N = 181)
55 (44.0%)
0.33
(0.22–0.51)
317
(54.8%)
62 (34.3%)
Not
burned
out
(N = 398)
255
(64.1%)
Unadjusted
odds ratio
271 (62.7%)
Not
burned
out
(N = 307)
216
(70.4%)
0.29
(0.20–0.42)
163 (37.7%)
71 (56.8%)
92
(30.0%)
3.07
(1.99–4.72)
288
(49.7%)
133
(73.5%)
155
(38.9%)
4.34
(2.95–6.39)
N (column %)
277 (62.4%)
associations for both burnout measures suggests that the two
burnout measures would perform similarly when exploring
other burnout predictors.
Our findings have implications for using and interpreting the self-defined and MBI burnout measures. The
self-defined burnout measure appears to be an acceptable alternative to the MBI if primary aims are to track
burnout trends within a single population or measure
work environment factors that predict burnout. However,
our results indicate it would be inappropriate to directly
contrast high burnout estimates from the self-defined
measures and MBI subscale measures.
Researchers and health system leaders should use
caution when comparing burnout prevalence across different populations or studies. For example, a recently
published study using the self-defined measure concluded that physician burnout may be decreasing in the
7
USA. Differences may actually be due to the measurement instrument used and the self-defined measure’s low
sensitivity relative to the MBI.
The National Academy of Medicine notes, B[Burnout]
terminology and measurement tools used vary substantially across studies…hampering efforts to quantitatively
summarize outcomes (for example through meta-analy1
ses), and slowing the rate of advancement in the field.^
Similar to other researchers who have identified need
for greater consistency in defining and reporting burn31
out, , 32 our study underlines these challenges to compare and pool findings across studies when different
burnout measures are used.
The self-defined burnout measure has several attractive qualities. It does not require a license fee, has low
Burned out
(N = 277)
Not
burned
out
(N = 329)
Unadjusted
odds ratio
response burden, and may have more face validity to
24
healthcare workers than a multi-item scale score. One
drawback of the self-defined burnout measure is limited
validity testing in contrast to MBI measures, which have
been associated with outcomes such as clinical diagnosis
33
of depression. Our study also demonstrates the limitations due to the ordinal nature of the self-defined measure. Forty to fifty percent of respondents in our sample
selected the level 2 category, a skewed response that
creates a large step-off effect. Consequently, the selfdefined cut point cannot be smoothly titrated to achieve
an optimal balance of sensitivity and specificity relative
to the MBI (Fig. 2). This aspect of the self-defined
measure may also reduce its predictive and discriminant
utility when analyzing burnout gradients rather than yes/
no classifications.
Our study has several limitations. First, we studied
clinicians and staff in three large health systems in a
single region, which may limit generalizability. Yet
burnout prevalence in our sample was similar to national
7
samples of family physicians and general internists, , 15
suggesting that respondents’ work experience resembles
that of other settings. Second, our sample had a high
proportion of women and part-time clinicians.
Sensitivity/specificity sub-analyses stratified by gender
and sessions per week did not meaningfully differ from
the full sample. Third, as with any survey, response bias
may influence the validity of the results. Our response
rate of 74% is much higher than for most surveys of
healthcare workers, mitigating potential non-response bias. Last, our study relied on survey measures of burnout
1350
Knox et al.: Clinician and Staff Burnout Measures
and did not assess well-being with direct observation or
qualitative methods.
The self-defined burnout measure and MBI each have
advantages and disadvantages. We do not conclude from
our study that there is necessarily a preferred burnout
survey instrument. However, researchers and health system leaders addressing burnout must be aware of the
measures’ different properties and lack of equivalency
for assessing burnout prevalence. We recommend that
organizations such as the National Academy of Medicine
and Agency for Healthcare Research and Quality take the
lead in developing and promoting national guidelines that
establish greater consistency across burnout survey
efforts. The commitment by the American Academy of
Family Physician to offer members free MBI survey
access is one example of constructive action by a nation34
al organization. Greater consistency and clarity in
reporting how burnout is defined is essential to support
meaningful comparisons across health systems and to
enhance understanding of burnout consequences and
interventions.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
Acknowledgements: The authors thank Dr. Mark Linzer, MD, for his
helpful review and comments on a manuscript draft.
16.
Corresponding Author: Margae Knox, MPH; Center for Excellence in
Primary Care, Department of Family and Community Medicine
University of California, San Francisco, CA, USA (e-mail: Margae.
Knox@ucsf.edu).
17.
18.
Funding Participating health systems funded data collection as part
of ongoing quality improvement efforts.
19.
Compliance with Ethical Standards
20.
This study was a cross-sectional survey, approved by the Institutional
Review Board of the University of California, San Francisco (protocol
numbers 11-08048 and 17-23324).
21.
Prior Presentations: None.
22.
Conflict of Interest: The authors declare that they do not have a
conflict of interest.
23.
REFERENCES
1.
2.
3.
4.
Dyrbye LN, Shanafelt TD, Sinsky C, et al. Burnout among health care
professionals: a call to explore and address this underrecognized threat to
safe, high-quality care. National Academy of Medicine Perspectives:
Expert Voices in Healthcare. Washington DC; 2017 [accessed 2018
Apr 25]. Available from: https://nam.edu/burnout-among-health-careprofessionals-a-call-to-explore-and-address-this-underrecognizedthreat-to-safe-high-quality-care/.
Agency for Healthcare Quality and Research. AHRQ Announces Interest
in Innovative Primary Care Research. Notice Number: NOT-HS-16-011.
2016.
Agency for Healthcare Quality and Research. Physician burnout. Rockville, MD; 2017 [updated 2017 July; accessed 2018 April 25]. Available
from: https://www.ahrq.gov/professionals/clinicians-providers/ahrqworks/burnout/index.html.
Bodenheimer T, Sinsky C. From triple to quadruple aim: care of the
patient requires care of the provider. Ann Fam Med. 2014;12(6):573–6.
PMC4226781.
24.
25.
26.
27.
28.
29.
30.
JGIM
American Medical Association. STEPS Forward; 2017 [accessed 2018
Apr 25]. Available from: https://www.stepsforward.org/.
Society for General Internal Medicine, editor Resilience and Grit:
Pursuing Organizational Change and Preventing Burnout in GIM. SGIM
Annual Meeting; 2017 April 19–22; Washington, DC.
Puffer JC, Knight HC, O'neill TR, et al. Prevalence of Burnout in Board
Certified Family Physicians. The Journal of the American Board of Family
Medicine. 2017;30(2):125–6.
Noseworthy J, Madara J, Cosgrove D, et al. Physician burnout is a
public health crisis: a message to our fellow health care CEOs. Health
Affairs Blog. 2017.
Maslach C, Jackson S, Leiter M. MBI: Maslach Burnout Inventory
Manual. Third ed: Consulting Psychologists Press; 1996.
Linzer M, Poplau S, Babbott S, et al. Worklife and wellness in academic
general internal medicine: results from a national survey. J Gen Int Med.
2016:1–7.
Schmoldt R, Freeborn D, Klevit H. Physician burnout: recommendations for HMO managers. HMO practice. 1994;8(2):58–63.
Linzer M, Manwell LB, Williams ES, et al. Working conditions in primary
care: physician reactions and care quality. Annals of Internal Medicine.
2009;151(1):28–36.
Williams ES, Konrad TR, Linzer M, et al. Refining the measurement of
physician job satisfaction: results from the physician worklife survey.
Med Care. 1999:1140–54.
Lewis SE, Nocon RS, Tang H, et al. Patient-centered medical home
characteristics and staff morale in safety net clinics. Archives of Internal
Medicine. 2012;172(1):23–31.
Shanafelt TD, Hasan O, Dyrbye LN, et al., editors. Changes in burnout
and satisfaction with work-life balance in physicians and the general US
working population between 2011 and 2014. Mayo Clin Proc; 2015:
Elsevier.
Helfrich CD, Dolan ED, Simonetti J, et al. Elements of Team-Based
Care in a Patient-Centered Medical Home Are Associated with Lower
Burnout Among VA Primary Care Employees. Journal of General Internal
Medicine. 2014;29(2):659–66.
Dolan ED, Mohr D, Lempa M, et al. Using a single item to measure
burnout in primary care staff: a psychometric evaluation. Journal of
General Internal Medicine. 2015;30(5):582–7.
Rohland BM, Kruse GR, Rohrer JE. Validation of a single-item measure
of burnout against the Maslach Burnout Inventory among physicians.
Stress and Health. 2004;20(2):75–9.
Hansen V, Girgis A. Can a single question effectively screen for burnout
in Australian cancer care workers? BMC health services research.
2010;10(1):341.
Waddimba AC, Scribani M, Nieves MA, et al. Validation of Single-Item
Screening Measures for Provider Burnout in a Rural Health Care
Network. Evaluation & the Health Professions. 2016;39(2):215–25.
West CP, Dyrbye LN, Sloan JA, et al. Single item measures of emotional
exhaustion and depersonalization are useful for assessing burnout in
medical professionals. Journal of general internal medicine.
2009;24(12):1318–21.
Willard-Grace R, Hessler D, Rogers E, et al. Team structure and culture
are associated with lower burnout in primary care. The Journal of the
American Board of Family Medicine. 2014;27(2):229–38.
Rassolian M, Peterson LE, Fang B, et al. Workplace factors associated
with burnout of family physicians. JAMA internal medicine.
2017;177(7):1036–8.
Schaufeli WB, Dierendonck DV. A Cautionary Note about the CrossNational and Clinical Validity of Cut-off Points for the Maslach Burnout
Inventory. Psychological Reports. 1995;76(3_suppl):1083–90.
Shanafelt TD, Boone S, Tan L, et al. Burnout and satisfaction with
work-life balance among US physicians relative to the general US
population. Archives of Internal Medicine. 2012;172(18):1377–85.
StataCorp. Stata: Release 13. Statistical Software. College Station, TX:
StataCorp LP; 2013.
West CP, Dyrbye LN, Satele DV, et al. Concurrent validity of single-item
measures of emotional exhaustion and depersonalization in burnout
assessment. Journal of General Internal Medicine. 2012;27(11):1445–52.
Simundic A-M. Measures of Diagnostic Accuracy: Basic Definitions.
EJIFCC (Journal of the International Federation of Clinical Chemistry
and Laboratory Medicine). 2009;19(4):203–11.
Tape TG. Interpreting diagnostic tests: the area under an ROC curve.
University of Nebraska Medical Center; 2017 [accessed 2018 Apr 25].
Available from: http://gim.unmc.edu/dxtests/roc3.htm.
Rice ME, Harris GT. Comparing effect sizes in follow-up studies: ROC
Area, Cohen’s d, and r. Law and human behavior. 2005;29(5):615.
JGIM
31.
32.
33.
Knox et al.: Clinician and Staff Burnout Measures
Dyrbye LN, West CP, Shanafelt TD. Defining burnout as a dichotomous
variable. Journal of General Internal Medicine. 2009;24(3):440.
Eckleberry-Hunt J, Kirkpatrick H, Barbera T. The problems with
burnout research. Acad Med. 2017.
Schaufeli WB, Bakker AB, Hoogduin K, et al. On the clinical validity of
the Maslach Burnout Inventory and the Burnout Measure. Psychology &
health. 2001;16(5):565–82.
34.
1351
American Academy of Family Physicians. 2017 Leadership conference:
speakers chart paths from physician burnout to well-being. Kansas City,
Missouri; 2017 [updated 2017 May 8; accessed 2018 Apr 25]. Available
from: http://www.aafp.org/news/inside-aafp/20170508physicianwellness.html.
JGIM: Journal of General Internal Medicine is a copyright of Springer, 2018. All Rights
Reserved.
Essay Writing Service Features
Our Experience
No matter how complex your assignment is, we can find the right professional for your specific task. Achiever Papers is an essay writing company that hires only the smartest minds to help you with your projects. Our expertise allows us to provide students with high-quality academic writing, editing & proofreading services.
Free Features
Free revision policy
$10Free bibliography & reference
$8Free title page
$8Free formatting
$8How Our Dissertation Writing Service Works
First, you will need to complete an order form. It's not difficult but, if anything is unclear, you may always chat with us so that we can guide you through it. On the order form, you will need to include some basic information concerning your order: subject, topic, number of pages, etc. We also encourage our clients to upload any relevant information or sources that will help.
Complete the order form
Once we have all the information and instructions that we need, we select the most suitable writer for your assignment. While everything seems to be clear, the writer, who has complete knowledge of the subject, may need clarification from you. It is at that point that you would receive a call or email from us.
Writer’s assignment
As soon as the writer has finished, it will be delivered both to the website and to your email address so that you will not miss it. If your deadline is close at hand, we will place a call to you to make sure that you receive the paper on time.
Completing the order and download