You are here

  • Home
  • Archive
  • Volume 12, Issue 4
  • US emergency care patterns among nurse practitioners and physician assistants compared with physicians: a cross-sectional analysis

Article Text

Article menu
Emergency medicine
Original research
US emergency care patterns among nurse practitioners and physician assistants compared with physicians: a cross-sectional analysis
  1. John N Mafi1,2,
  2. Alexander Chen3,
  3. Rong Guo1,
  4. Kristen Choi1,4,
  5. Peter Smulowitz5,
  6. Chi-Hong Tseng1,
  7. Joseph A Ladapo1,
  8. Bruce E Landon6,7
  1. 1Division of General Internal Medicine and Health Services Research, University of California Los Angeles David Geffen School of Medicine, Los Angeles, California, USA
  2. 2RAND Health, RAND Corporation, Santa Monica, California, USA
  3. 3David Geffen School of Medicine, University of California Los Angeles, Los Angeles, California, USA
  4. 4School of Nursing, University of California Los Angeles, Los Angeles, California, USA
  5. 5Department of Emergency Medicine, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, Massachusetts, USA
  6. 6Department of Health Care Policy, Harvard Medical School, Boston, Massachusetts, USA
  7. 7Division of General Internal Medicine and Primary Care, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, Massachusetts, USA
  1. Correspondence to Dr John N Mafi; jmafi{at}mednet.ucla.edu

Abstract

Objectives Nurse practitioners and physician assistants (NPs/PAs) increasingly practice in emergency departments (EDs), yet limited research has compared their practice patterns with those of physicians.

Design, setting and participants Using nationally representative data from the National Hospital Ambulatory Medical Care Survey (NHAMCS), we analysed ED visits among NPs/PAs and physicians between 1 January 2009 and 31 December 2017. To compare NP/PA and physician utilisation, we estimated propensity score-weighted multivariable regressions adjusted for clinical/sociodemographic variables, including triage acuity score (1=sickest/5=healthiest). Because NPs/PAs may preferentially consult physicians for more complex patients, we performed sensitivity analyses restricting to EDs with >95% of visits including the NP/PA–physician combination.

Exposures NPs/PAs.

Main outcome measures Use of hospitalisations, diagnostic tests, medications, procedures and six low-value services, for example, CT/MRI for uncomplicated headache, based on Choosing Wisely and other practice guidelines.

Results Before propensity weighting, we studied visits to 12 410 NPs/PAs-alone, 21 560 to the NP/PA–physician combination and 143 687 to physicians-alone who saw patients with increasing age (41, 45 and 47 years, p<0.001) and worsening triage acuity scores (3.03, 2.85 and 2.67, p<0.001), respectively. After weighting, NPs/PAs-alone used fewer medications (2.62 vs 2.80, p=0.002), diagnostic tests (3.77 vs 4.66, p<0.001), procedures (0.67 vs 0.77, p<0.001), hospitalisations (OR 0.35 (95% CI 0.26 to 0.46)) and low-value CT/MRI studies (OR 0.65 (95% CI 0.53 to 0.80)) than physicians. Contrastingly, the NP/PA–physician combination used more medications (3.08 vs 2.80, p<0.001), diagnostic tests (5.07 vs 4.66, p<0.001), procedures (0.86 vs 0.77, p<0.001), hospitalisations OR 1.33 (95% CI 1.17 to 1.51) and low-value CT/MRI studies (OR 1.23 (95% CI 1.07 to 1.43)) than physicians—results were similar among EDs with >95% of NP/PA visits including the NP/PA–physician combination.

Conclusions and relevance While U.S. NPs/PAs-alone used less care and low-value advanced diagnostic imaging, the NP/PA–physician combination used more care and low-value advanced diagnostic imaging than physicians alone. Findings were reproduced among EDs where nearly all NP/PA visits were collaborative with physicians, suggesting that NPs/PAs seeing more complex patients used more services than physicians alone, but the converse might be true for more straightforward patients.

  • Health policy
  • ACCIDENT & EMERGENCY MEDICINE
  • Quality in health care

Data availability statement

Data are available in a public, open access repository. All data are publicly available on the Centers for Disease Control and Prevention, National Center for Health Statistics (CDC/NCHS) website: https://www.cdc.gov/nchs/ahcd/datasets_documentation_related.htm%23data

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

https://doi.org/10.1136/bmjopen-2021-055138

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Strengths and Limitations of this Study

    • This is the first study, known to these authors, to use nationally representative data to compare both overall and low-value care patterns among nurse practitioners and physician assistants (NPs/PAs) and physicians practising in US emergency departments.

    • Because patients are non-randomly assigned to providers, it is impossible to eliminate selection bias.

    • The low-value care measures included in the analysis reflect a narrow aspect of quality of care; therefore, differences in utilisation do not necessarily imply differences in overall quality of care among NPs/PAs and physicians in the emergency department setting.

    Introduction

    Each year, US emergency departments (EDs) face growing pressure to manage more and increasingly complex patients, with total annual ED visits rising from 117 million to 139 million over the past decade.1–5 As the US population ages, ED visits have grown in illness severity and resource utilisation.6 7 Approximately 9 million hospital admissions originate from EDs for individuals aged 65 years and older each year, representing over 70% of hospital admissions among older adults, and hospitalisations among older adults in the ED are increasing over time.7 8 Even as visit volume and complexity increase, US EDs face a growing shortage of emergency medicine physicians, raising concerns of ED overcrowding and threatening access to emergency care.9

    In response to these trends, commentators have increasingly advocated for expanding the role of nurse practitioners and physician assistants (NPs/PAs) to increase access to emergency care at a lower cost, in part because NPs/PAs have lower salaries than physicians.10 Whether working independently or seeing patients collaboratively with a physician, over 27 000 NPs/PAs currently work in US EDs, and NPs/PAs have seen a rapidly growing number of patients in the ED setting, from 15 million visits in 2007 to over 40 million visits in 2016.11–18 Nevertheless, controversy exists on how to best incorporate NPs/PAs into emergency care. For example, according to national and international surveys, many ED physicians believe that NPs should not practice autonomously and work best in collaborative teams alongside physicians, whereas NPs generally advocate for greater autonomy.19–24

    Despite strong opinions, limited empirical research has directly compared NPs/PAs and physicians regarding overall and low-value care patterns in the ED, which has important implications for the efficiency of US emergency care. Low-value care is particularly salient for efficiency because it reflects patient care that offers no net benefit in specific clinical scenarios—both raising costs and potentially harming patients.25 26 Prior work comparing ED NPs/PAs and physicians caring for ED patients has typically focused on single clinical conditions or single medical centres, or relied on older data, and largely has found conflicting results.27–29 One US study suggested that EDs employing NPs/PAs used more services than EDs that did not; however, this ecological work was at the ED level and could not directly compare individual NPs/PAs with physicians.30 Another study using data from the National Hospital Ambulatory Medical Care Survey (NHAMCS) suggested that PAs used less services than physicians after stratifying by triage acuity, but this work did not assess low-value care or account for other potentially confounding factors such as age, comorbidities or diagnoses.18 In this context, we used nationally representative data on ED visits to compare overall and low-value care patterns among NPs/PAs and physicians.

    Methods

    Data sources

    We performed a cross-sectional analysis using nationally representative data from NHAMCS in order to compare emergency care patterns among NPs/PAs and physicians. The dataset includes data on visits to non-federal, hospital-based EDs. We restricted our study to ED visits for adults aged 18 years and older with any presenting complaint, between 1 January 2009 and 31 December 2017, which included 177 657 ED visits in the sample, representing an estimated 918 071 431 US ED visits.

    Data collection procedures

    Data were collected using a standardised survey form completed by providers or staff after each visit in each year. The survey includes items identifying the patient’s primary visit complaints; two secondary visit complaints; up to three diagnoses derived from the International Classification of Diseases, Ninth and Tenth Revisions, Clinical Modification (ICD-9-CM and ICD-10-CM); four chronic comorbidities (cerebrovascular disease (CVD), congestive heart failure (CHF), diabetes mellitus, and human immunodeficiency virus (HIV); payer for the visit (eg, Medicare); whether the ED visit resulted in hospitalisation; clinician and patient demographic information; tests and treatments ordered; and medications given during the visit or prescribed at discharge.4 The data span the entire ED visit from arrival to discharge.

    Main exposure: provider type

    We compared visits where the clinician was an NP/PA or physician (attending physician). NP/PA visits were further subdivided by NP/PA-alone visits where the NP/PA saw the patient without a physician (NP/PA-alone) versus a collaborative visit that included an NP/PA and physician both evaluating the patient (the NP/PA–physician combination). Importantly, aside from a few exceptions, NPs and PAs were generally indistinguishable across study outcomes measures (see online appendix tables 1 and 2 for details); therefore, we combined NPs and PAs into a single exposure variable: NP/PA visits.

    Outcome measures

    Our primary outcome measures focused on overall utilisation of services, including use of diagnostic tests, procedures (eg, suturing), medications either given during the visit or prescribed at discharge, the length of time-per-visit (calculated from date and time of ED arrival and discharge in minutes) and whether the visit resulted in hospitalisation. As secondary outcome measures, we assessed utilisation of six health services widely considered to be low-value in specific clinical scenarios, based on prior work using NHAMCS data, Choosing Wisely recommendations, and other broadly accepted clinical practice guidelines.25 31–40 Because NHAMCS routinely collects data on symptoms, vital signs, visit diagnoses and comorbid conditions directly from patients’ medical records, it allowed for the exclusion of important alarm features that help to define low-value care. These six low-value care measures included: (1) antibiotics (numerator) for upper respiratory infections without alarm features (denominator) such as chronic obstructive pulmonary disease (COPD), and this measure was combined with another measure of antibiotics (numerator) for skin abscesses without alarm features (denominator) such as cellulitis or signs of sepsis; (2) plain X-rays (numerator) for back pain without alarm features (denominator) such as cancer, trauma or neurological deficit; (3) advanced diagnostic imaging, including both MRI and CT (numerator) for back pain or headache without alarm features (denominator) such as trauma, cancer and neurological deficit; (4) opioids (numerator) for back pain or headache without alarm features (denominator) such as cancer and trauma; (5) inappropriate medications (numerator) for adults aged 65 years and older (denominator) such as benzodiazepines according to Beers Criteria recommendations; and (6) guideline-discordant antibiotic choice (numerator) for uncomplicated urinary tract infections (denominator).

    Analyses

    Patient selection bias

    A fundamental threat to validity of any comparison of physician and NP/PA utilisation is that ED patients are non-randomly assigned to NPs/PAs. NPs/PAs seeing patients in the same ED with physicians typically are assigned lower acuity/complexity patients than physicians, which has been illustrated in multiple prior analyses.13 14 Additionally, in cases where the decision to involve a physician is at the discretion of the NP/PA, they would more likely involve a physician for more complex cases, even for patients with relatively low triage acuity. This phenomenon also has been demonstrated in previous work.13 Each scenario introduces a potential bias where NPs/PAs-alone would tend to see relatively lower complexity patients (even among those visits that were judged to be lower acuity), typically leading to relatively less utilisation, even after controlling for diagnosis and other clinical/sociodemographic variables. In contrast, the NP/PA–physician combination visits where the physician is consulted at the discretion of the NP/PA would tend to be for higher complexity or less straightforward patients, even after controlling for acuity level and diagnosis, typically entailing relatively more utilisation.

    We took several steps to account for these potential selection biases. First, in our primary approach, we estimated inverse probability propensity score weighting based on our prior work.41 Our propensity score weights accounted for the NHAMCS’ complex survey design and nationally representative population weights and were based on patient age, sex, race, nurse-rated standard triage acuity scores (1=immediate, 2=emergent, 3=urgent, 4=semi-urgent, 5=non-urgent), primary diagnosis categories (eg, cardiovascular), chronic comorbidities (eg, HIV), residence (eg, nursing home residence, an important marker of frailty), insurance type, rural vs urban, four US regions and year (years combined into two year groupings per National Center for Health Statistics (NCHS) guidelines) to create comparison groups that were balanced on clinically relevant and observable information included in the dataset. This approach balances visits based on observable characteristics but cannot do so on unmeasured characteristics (eg, severity within a diagnosis or cases with non-straightforward presentations). Second, we performed sensitivity analyses among EDs where nearly all NP/PA visits included the NP/PA–physician combination. This approach addresses the concern that NP/PAs may choose to involve physicians in more complex or serious cases wherein the observed increase in utilisation might be explained by unmeasured severity. Evaluating practice patterns in this subgroup of EDs effectively minimises the selection effect of collaborative visits as essentially all visits involving an NP or PA are the NP/PA–physician combination visits. Thus, the decision to invite a physician to see a patient alongside the NP/PA was no longer a function of the patient’s illness severity as it was the default for virtually all patients. We also assessed the degree to which this was occurring by measuring the unadjusted, population-weighted rate of physician consultations stratified by triage severity (0–3 vs 4–5) among EDs where ≤95% of NP/PA visits were collaborative (eg, in EDs where there was some discretion in whether NPs/PAs collaborated with a physician). Conversely, we were unable to perform analyses limiting to EDs where nearly all of NP/PA visits were NP/PA-alone visits, because we could not identify a sufficient number of EDs where NPs/PAs mostly practised alone.

    Statistical analysis

    We estimated weighted multivariable logistic and linear regression models to compare NPs/PAs and physicians, adjusting for the same variables outlined in the propensity score in a double-robust fashion. Our analyses used data from the entire NAMCS and NHAMCS samples in order to account for the complex survey design and sampling weights to produce national estimates.42 Moreover, because the 2014–2017 NHAMCS surveys inadvertently underestimated population variance, we used a statistical significance threshold of p<0.01 in accordance with the NCHS guidelines, and conservatively used this threshold across all data years.

    We also assessed whether our propensity weighting model violated the positivity assumption, which assumes that our propensity score is bounded between 0 and 1.43 When comparing the distribution of scores between any NPs/PAs and physicians, we found excellent overlap and no violation of the positivity assumption. However, we did find subtle positivity assumption violations regarding the propensity scores of NPs/PAs-alone versus physicians and the NP/PA–physician combination versus physicians alone, with a relatively small number of outlier probabilities (n=53 and n=343 respectively) potentially biasing our results. While these outliers were small in number, we performed sensitivity analyses using propensity overlapping weights, and results remained consistent with our primary findings.43 Finally, some of the utilisation measures (eg, number of medications) had missing data leading to differing sample sizes; however, for measures other than length of visit, these missing data comprised ~3% of the sample and were included in our analyses.

    We performed all analyses using SAS (V.9.4). Because the NHAMCS is a deidentified, publicly available national database, the UCLA Institutional Review Board deemed this analysis to be exempt from review as human subject research. This study followed the Strengthening the Reporting of Observational Studies in Epidemiology guidelines for reporting cross-sectional studies (see online supplemental appendix).

    Results

    We identified visits to 12 410 NPs/PAs-alone, 21 560 including the NP/PA–physician combination and 143 687 physicians alone, reflecting an estimated 918 071 431 US adult ED visits between 2009 and 2017. NPs/PAs-alone, the NP/PA–physician combination and physicians alone saw patients with increasing mean age (41, 45 and 47 years, p<0.001), increasing prevalence of comorbidities such as congestive heart failure (1.5%, 3.5% and 4.1%, p<0.001) and worsening mean triage acuity scores (3.03, 2.85, and 2.67, p<0.001), respectively (table 1) (note, triage acuity scores are categorised as follows: (1=immediate, 2=emergent, 3=urgent, 4=semi-urgent and 5=non-urgent). Patient seeing any NP/PA were more likely than patients seeing a physician to be black (25.4% vs 22.0%, p=0.001) and more likely to be insured by Medicaid (25.3% vs 21.6%, p<0.001), respectively. After propensity score weighting, there were no clinically or statistically significant differences in these demographic and clinical characteristics.

    View this table:
    Table 1

    Patientcharacteristics during ED visits to NPs/PAs, NPs/PAs-alone, the NP/PA–physician combination and physician alone* with and without propensity score weighting†

    Overall and low-value health service utilisation comparisons

    Because bivariable results were similar to multivariable results, we present multivariable results in the manuscript and bivariable results in the online appendix tables 3 and 4. Overall, visits to any NP/PAs (including both NPs/PAs-alone and the NP/PA–physician combination) resulted in similar rates of diagnostic tests, procedures, time per visit and hospitalisations; however, these visits resulted in more medications used (2.91 vs 2.76 per visit, p=0.001) than visits with physicians (table 2). Low-value care utilisation was similar across all six measured services among NPs/PAs and physicians (table 3).

    View this table:
    Table 2

    Adjusted* comparisons of overall utilisation during ED visits to any NPs/PAs, NPs/PAs alone the NP/PA–physician combination and physicians alone

    View this table:
    Table 3

    Adjusted* ORs comparing use of low-value health services during ED visits to any NPs/PAs, NPs/PAs alone, the NP/PA-physician combination and physicians alone

    Stratified and sensitivity analyses

    NPs/PAs-alone used fewer medications (2.62 vs 2.80, p=0.002), diagnostic tests (3.77 vs 4.66, p<0.001), procedures (0.67 vs 0.77, p<0.001), hospitalisations (adjusted OR (aOR) 0.35 (95% CI 0.26 to 0.46)) and low-value CT/MRI studies (0.65 (95% CI 0.53 to 0.80)) than physicians (tables 2 and 3). Time per visit was 245.9 min for NPs/PAs-alone and 267.6 min for physicians, although this difference was non-significant (p=0.02) using our prespecified significance threshold of p<0.01. In contrast, the NP/PA–physician combination used more medications (3.08 vs 2.80, p<0.001), diagnostic tests (5.07 vs 4.66, p<0.001), procedures (0.86 vs 0.77, p<0.001), hospitalisations aOR 1.33 (95% CI 1.17 to 1.51), low-value CT/MRI studies (aOR 1.23 (95% CI 1.07 to 1.43)) and spent more time (289.8 vs 267.6 min, p<0.001) than physicians during ED visits. After restricting the sample to EDs where all or almost all NP/PA visits (>95%) included the NP/PA–physician combination, we found similar results to our main findings (tables 4 and 5).

    View this table:
    Table 4

    Adjusted* overall utilisation within EDs where over 95% of NP/PA visits included the NP/PA–physician combination

    View this table:
    Table 5

    Adjusted* low-value health service utilisation within EDs where over 95% of NP/PA visits included the NP/PA–physician combination

    Prior to propensity weighting, when restricting to EDs where ≤95% of NP/PA visits included the NP/PA–physician combination, 64.3% of 22 486 NP/PA visits versus 48.1% of 19 162 NP/PA visits resulted in consulting a physician for lower (ie, sicker) versus higher (ie, healthier) triage acuity scoring patients, respectively. After propensity weighting, consultation rates equalised to 49.7% of NP/PA visits among lower triage acuity scoring patients and 49.7% of NP/PA visits among higher triage acuity scoring patients.

    Discussion

    In this large, nationally representative analysis of US emergency care, visits involving NPs/PAs used similar amounts of services and resulted in similar rates of hospitalisations when compared with visits with physicians. These overall findings, however, obscure important differences revealed in our stratified analyses. NP/PAs-alone used less care and low-value advanced diagnostic imaging whereas the NP/PA–physician combination used more care, low-value advanced diagnostic imaging, and time per visit than physicians alone. Because NPs/PAs-alone tend to see less complex patients than physicians, it is possible that they deliver fewer services than physicians for simpler, more algorithmic cases. In contrast, for more complex or less straightforward cases that might require input of a physician, NPs/PAs used more services than physicians alone, and these findings were reproduced in EDs where nearly all NP/PA visits were collaborative with physicians. With NPs/PAs playing an increasingly growing role in emergency care, these findings have important implications for policymakers, health system leaders and clinicians who all share a stake in improving the value and efficiency of emergency care.

    The major challenge to studying the care of NP/PAs outside of experimental settings is overcoming patient selection bias. In this case, selection bias might manifest in two different ways. First, as our unadjusted data suggest, in EDs that use both NPs/PAs and physicians, cases allocated to NP/PAs generally are of lower complexity and, potentially, more straightforward.13 14 Though we attempt to control for this imbalance using propensity weighting methods, it is possible that there are additional unmeasured variables (eg, severity/complexity conditional on diagnosis or triage score) that our approach fails to incorporate. Nevertheless, while we cannot fully exclude unmeasured patient selection factors, our findings are consistent with NPs/PAs-alone delivering fewer services than physicians for simpler and more algorithmic cases. Our sensitivity analyses of low triage acuity visits, where fewer resulted in visits including the NP/PA–physician combination, is consistent with this hypothesis. Such findings are also consistent with the broader primary care literature (including our own work), where primary care NPs/PAs appear to practice at similar or better efficiency of care than physicians for less emergent/complex conditions.25 44–50

    The second source of selection bias of concern relates to collaborative care, wherein NP/PAs choose to involve physicians in the care of a specific patients. When such consultations are initiated at the discretion of NPs/PAs, we would expect that cases where collaborative care was sought would have higher acuity/complexity, even after controlling for diagnosis and other clinical/sociodemographic variables. Thus, one would expect to see higher use of services for visits including the NP/PA–physician combination when compared with visits including physicians alone. In this case, by restricting the analyses to EDs where virtually all visits were staffed collaboratively, we effectively mitigated this potential source of bias. This exercise revealed that visits including the NP/PA–physician combination were associated with a higher use of services, including medications, testing and low-value imaging when compared with physician only cases. To be sure, it is possible the physician may be the driving force behind the NP/PA combination ordering more services than physicians alone. However, our sensitivity analyses among EDs where there effectively was no discretion occurring to create visits including the NP/PA-physician combination suggests this was not the case. Moreover, prior research suggests that NPs/PAs most frequently see the patient prior to involving an attending physicians in ED settings, often for lower acuity cases, and not the converse.51–55 Consistent with these data, prior physician and NP survey data suggest that physicians may manage undifferentiated and more complex ED patients more efficiently than NPs21 56 and our sensitivity analyses are consistent with this perspective.

    Taken together, our results provide new insight into currently unresolved questions related to NP/PA scope-of-practice policies within ED settings. According to international survey data, many emergency physicians believe that NPs should not practice autonomously and work best in collaborative teams alongside physicians, whereas NPs generally advocate for more autonomy, and NPs/PAs are increasingly practicing independently within US EDs.19–21 As the debate continues over whether NPs/PAs scope of ED practice should continue to expand, our analysis adds important data to the field, suggesting that NPs/PAs-alone may provide less services and low-value diagnostic imaging for lower acuity, less complex patients than physicians. Our data also suggest, contrastingly, that NPs/PAs may use more services and low-value diagnostic imaging than physicians in managing more complex cases. Such findings can inform current ongoing debates among US policymakers, health system leaders and clinicians on how to best incorporate NPs/PAs into ED care. We encourage further rigorous study of these questions, specifically randomised controlled trials evaluating the impact of various collaborative arrangements among NPs/PAs and physicians on the quality of emergency care.

    Our study has several limitations. First is the inability to fully minimise selection bias as patients are non-randomly assigned to providers, which we attempted to address in detail above. In particular, we could not perform analyses limiting to EDs where virtually all of NP/PA visits were NP/PA-alone visits, because we could not find a sufficient number of EDs where NPs/PAs mostly practiced alone. Moreover, although our analyses adjusted for social determinants of health, such as race, insurance status, and rural and urban location, we could not fully separate the difference between patient selection to NPs/PAs and physicians within EDs versus patient section between EDs as a function of the geographic location of the ED.57–59 Nonetheless, there is a rich literature that NPs/PAs are more likely to serve socioeconomically disadvantaged communities.60 61 Second, differences in utilisation do not necessarily imply differences in quality of care as the six low-value health service measures only comprise a narrow aspect of quality, and we did not assess other factors such as patient-oriented outcomes or diagnostic accuracy. Third, we could not exclude the possibility of physicians being first to examine the patient prior to involving the NP/PA or more generally influencing medical decision making during NP/PA-alone visits; however, prior literature suggests that this is not the usual model of collaborative care.51–55 Fourth, we could not identify with complete certainty whether a service was low- value with the data we had. However, NHAMCS routinely collects data on symptoms, vital signs, visit diagnoses and comorbid conditions directly from patients’ medical records, which can help exclude important alarm features. Moreover, we have no reason to suspect that misclassification of low-value care would differ by NPs/PAs versus physicians. Fifth, we could not account for variations in organisational practices and state-level scope-of-practice laws across the USA, which may limit NPs/PAs’ ability to directly order diagnostic tests or medications.62 63 However, adjusting for larger US census regions did not alter our findings, and a previous rigorous analysis of the impact of state scope-of-practice laws on NP/PA practice patterns only found a modest association.62

    Conclusions

    Compared with US ED physicians, NPs/PAs-alone used less care and low-value advanced diagnostic imaging, while the NP/PA-physician combination used more care and low-value advanced diagnostic imaging. Because NPs/PAs-alone tend to see fewer complex patients than physicians, it is possible that they used fewer services than physicians for simpler, more algorithmic cases. In contrast, NPs/PAs used more services than physicians for more complex cases—and these findings were reproduced in EDs where nearly all NPs/PAs saw patients collaboratively with physicians. With NPs/PAs playing an increasingly growing role in US EDs, these findings have important implications for policymakers and clinicians who all have a stake in improving the efficiency of emergency care.

    Data availability statement

    Data are available in a public, open access repository. All data are publicly available on the Centers for Disease Control and Prevention, National Center for Health Statistics (CDC/NCHS) website: https://www.cdc.gov/nchs/ahcd/datasets_documentation_related.htm%23data

    Ethics statements

    Patient consent for publication

    Ethics approval

    Not applicable.

    Acknowledgments

    We would like to thank Sara Delgado (UCLA) and Julia Cave Arbanas (UCLA) for their administrative support related to this work.

    References

    1. ↵
      1. Hsia RY,
      2. Sabbagh SH,
      3. Guo J, et al
      . Trends in the utilisation of emergency departments in California, 2005-2015: a retrospective analysis. BMJ Open 2018;8:e021392doi:10.1136/bmjopen-2017-021392pmid:http://www.ncbi.nlm.nih.gov/pubmed/30037870
      OpenUrlAbstract/FREE Full Text
    2. ↵
      1. Tang N,
      2. Stein J,
      3. Hsia RY, et al
      . Trends and characteristics of US emergency department visits, 1997-2007. JAMA 2010;304:664–70.doi:10.1001/jama.2010.1112pmid:http://www.ncbi.nlm.nih.gov/pubmed/20699458
      OpenUrlCrossRefPubMedWeb of Science
    3. ↵
      1. Greenwood-Ericksen MB,
      2. Kocher K
      . Trends in emergency department use by rural and urban populations in the United States. JAMA Netw Open 2019;2:e191919.doi:10.1001/jamanetworkopen.2019.1919pmid:http://www.ncbi.nlm.nih.gov/pubmed/30977849
      OpenUrlPubMed
    4. ↵
      1. CDC
      . National center for health statistics. data from: the National Hospital ambulatory medical care survey, 2009-2017 http://www.cdc.gov/nchs/ahcd/ahcd_questionnaires.htm
    5. ↵
      1. Niska R,
      2. Bhuiya F,
      3. Xu J
      . National Hospital ambulatory medical care survey: 2007 emergency department summary. Natl Health Stat Report 2010;26:1–31.pmid:http://www.ncbi.nlm.nih.gov/pubmed/20726217
      OpenUrlPubMed
    6. ↵
      1. Chou SC,
      2. Baker O,
      3. Schuur JD
      . Changes in emergency department care intensity from 2007-16: analysis of the National Hospital ambulatory medical care survey. West J Emerg Med 2020;21:209–16.
      OpenUrlCrossRefPubMed
    7. ↵
      1. Pines JM,
      2. Mullins PM,
      3. Cooper JK, et al
      . National trends in emergency department use, care patterns, and quality of care of older adults in the United States. J Am Geriatr Soc 2013;61:12–17.doi:10.1111/jgs.12072pmid:http://www.ncbi.nlm.nih.gov/pubmed/23311549
      OpenUrlCrossRefPubMed
    8. ↵
      1. Crowley C,
      2. Stuck AR,
      3. Martinez T, et al
      . Survey and chart review to estimate Medicare cost savings for home health as an alternative to hospital admission following emergency department treatment. J Emerg Med 2016;51:643–7.doi:10.1016/j.jemermed.2016.07.096pmid:http://www.ncbi.nlm.nih.gov/pubmed/27692839
      OpenUrlPubMed
    9. ↵
      1. Reiter M,
      2. Wen LS,
      3. Allen BW
      . The emergency medicine workforce: profile and projections. J Emerg Med 2016;50:690–3.doi:10.1016/j.jemermed.2015.09.022pmid:http://www.ncbi.nlm.nih.gov/pubmed/26823136
      OpenUrlPubMed
    10. ↵
      1. Tintinalli JE
      . Mid-level providers and emergency care: let's not lose the force. Emerg Med Australas 2014;26:403–7.doi:10.1111/1742-6723.12263pmid:http://www.ncbi.nlm.nih.gov/pubmed/25065770
      OpenUrlPubMed
    11. ↵
      1. Brown DFM,
      2. Sullivan AF,
      3. Espinola JA, et al
      . Continued rise in the use of mid-level providers in US emergency departments, 1993-2009. Int J Emerg Med 2012;5:21.doi:10.1186/1865-1380-5-21pmid:http://www.ncbi.nlm.nih.gov/pubmed/22621709
      OpenUrlCrossRefPubMed
    12. ↵
      1. Menchine MD,
      2. Wiechmann W,
      3. Rudkin S
      . Trends in midlevel provider utilization in emergency departments from 1997 to 2006. Acad Emerg Med 2009;16:963–9.doi:10.1111/j.1553-2712.2009.00521.xpmid:http://www.ncbi.nlm.nih.gov/pubmed/19799572
      OpenUrlCrossRefPubMedWeb of Science
    13. ↵
      1. Wiler JL,
      2. Rooks SP,
      3. Ginde AA
      . Update on midlevel provider utilization in U.S. emergency departments, 2006 to 2009. Acad Emerg Med 2012;19:986–9.doi:10.1111/j.1553-2712.2012.01409.xpmid:http://www.ncbi.nlm.nih.gov/pubmed/22905963
      OpenUrlPubMed
    14. ↵
      1. Ginde AA,
      2. Espinola JA,
      3. Sullivan AF, et al
      . Use of midlevel providers in US EDS, 1993 to 2005: implications for the workforce. Am J Emerg Med 2010;28:90–4.doi:10.1016/j.ajem.2008.09.028pmid:http://www.ncbi.nlm.nih.gov/pubmed/20006209
      OpenUrlPubMed
    15. ↵
      1. Hoyt KS,
      2. Proehl JA
      . Emergency nurse practitioners: frequently asked questions. Adv Emerg Nurs J 2014;36:291–3.doi:10.1097/TME.0000000000000040pmid:http://www.ncbi.nlm.nih.gov/pubmed/25356887
      OpenUrlPubMed
    16. ↵
      Statistical profile of certified physician assistants, 2019. Available: https://prodcmsstoragesa.blob.core.windows.net/uploads/files/2019StatisticalProfileofCertifiedPhysicianAssistants.pdf [Accessed 09-14-2020].
    17. ↵
      1. Chou S-C,
      2. Baker O,
      3. Schuur JD
      . Changes in emergency department care intensity from 2007-16: analysis of the National Hospital ambulatory medical care survey. West J Emerg Med 2020;21:209–16.doi:10.5811/westjem.2019.10.43497pmid:http://www.ncbi.nlm.nih.gov/pubmed/32191178
      OpenUrlCrossRefPubMed
    18. ↵
      1. Wu F,
      2. Darracq MA
      . Physician assistant utilization in U.S. emergency departments; 2010 to 2017. Am J Emerg Med 2020.
    19. ↵
      1. Larkin GL,
      2. Hooker RS
      . Patient willingness to be seen by physician assistants, nurse practitioners, and residents in the emergency department: does the presumption of assent have an empirical basis? Am J Bioeth 2010;10:1–10.doi:10.1080/15265161.2010.494216pmid:http://www.ncbi.nlm.nih.gov/pubmed/20694894
      OpenUrlPubMed
    20. ↵
      1. Weiland TJ,
      2. Mackinlay C,
      3. Jelinek GA
      . Perceptions of nurse practitioners by emergency department doctors in Australia. Int J Emerg Med 2010;3:271–8.doi:10.1007/s12245-010-0214-8pmid:http://www.ncbi.nlm.nih.gov/pubmed/21373292
      OpenUrlPubMed
    21. ↵
      1. Li J,
      2. Westbrook J,
      3. Callen J, et al
      . The impact of nurse practitioners on care delivery in the emergency department: a multiple perspectives qualitative study. BMC Health Serv Res 2013;13:356.doi:10.1186/1472-6963-13-356pmid:http://www.ncbi.nlm.nih.gov/pubmed/24053508
      OpenUrlPubMed
    22. ↵
      Ama successfully fights scope of practice expansions that threaten patient safety. American Medical Association (AMA) scope of practice. Available: https://www.ama-assn.org/practice-management/scope-practice/ama-successfully-fights-scope-practice-expansions-threaten [Accessed 23 Dec 2021].
    23. ↵
      American Academy of physician assistants (AAPA) scope of practice. Available: https://www.aapa.org/download/61319/ [Accessed 12-23-2021].
    24. ↵
      American association of nurse practitioners (AANP) position statements. Available: https://www.aanp.org/advocacy/advocacy-resource/position-statements [Accessed 12-23-2021].
    25. ↵
      1. Mafi JN,
      2. Wee CC,
      3. Davis RB, et al
      . Comparing use of low-value health care services among U. S. Advanced Practice Clinicians and Physicians. Ann Intern Med 2016;165:237–44.
      OpenUrl
    26. ↵
      1. Mafi JN,
      2. Parchman M
      . Low-Value care: an intractable global problem with no quick fix. BMJ Qual Saf 2018;27:333–6.doi:10.1136/bmjqs-2017-007477pmid:http://www.ncbi.nlm.nih.gov/pubmed/29331955
      OpenUrlFREE Full Text
    27. ↵
      1. Tsai C-L,
      2. Sullivan AF,
      3. Ginde AA, et al
      . Quality of emergency care provided by physician assistants and nurse practitioners in acute asthma. Am J Emerg Med 2010;28:485–91.doi:10.1016/j.ajem.2009.01.041pmid:http://www.ncbi.nlm.nih.gov/pubmed/20466230
      OpenUrlCrossRefPubMed
    28. ↵
      1. Hoyt KS,
      2. Ramirez E,
      3. Topp R, et al
      . Comparing nurse practitioners/physician assistants and physicians in diagnosing adult abdominal pain in the emergency department. J Am Assoc Nurse Pract 2018;30:655–61.doi:10.1097/JXX.0000000000000083pmid:http://www.ncbi.nlm.nih.gov/pubmed/30095670
      OpenUrlPubMed
    29. ↵
      1. Ducharme J,
      2. Alder RJ,
      3. Pelletier C, et al
      . The impact on patient flow after the integration of nurse practitioners and physician assistants in 6 Ontario emergency departments. CJEM 2009;11:455–61.doi:10.1017/S1481803500011659pmid:http://www.ncbi.nlm.nih.gov/pubmed/19788790
      OpenUrlPubMed
    30. ↵
      1. Aledhaim A,
      2. Walker A,
      3. Vesselinov R, et al
      . Resource utilization in Non-Academic emergency departments with advanced practice providers. West J Emerg Med 2019;20:541–8.doi:10.5811/westjem.2019.5.42465pmid:http://www.ncbi.nlm.nih.gov/pubmed/31316691
      OpenUrlPubMed
    31. ↵
      1. Mafi JN,
      2. McCarthy EP,
      3. Davis RB, et al
      . Worsening trends in the management and treatment of back pain. JAMA Intern Med 2013;173:1573–81.doi:10.1001/jamainternmed.2013.8992pmid:http://www.ncbi.nlm.nih.gov/pubmed/23896698
      OpenUrlPubMed
    32. ↵
      1. Barnett ML,
      2. Linder JA,
      3. Clark CR, et al
      . Low-Value medical services in the safety-net population. JAMA Intern Med 2017;177:829–37.doi:10.1001/jamainternmed.2017.0401pmid:http://www.ncbi.nlm.nih.gov/pubmed/28395014
      OpenUrlPubMed
    33. ↵
      1. Cassel CK,
      2. Guest JA
      . Choosing wisely: helping physicians and patients make smart decisions about their care. JAMA 2012;307:1801–2.doi:10.1001/jama.2012.476pmid:http://www.ncbi.nlm.nih.gov/pubmed/22492759
      OpenUrlCrossRefPubMedWeb of Science
    34. ↵
      1. Chou R,
      2. Qaseem A,
      3. Owens DK, et al
      . Diagnostic imaging for low back pain: advice for high-value health care from the American College of physicians. Ann Intern Med 2011;154:181–9.doi:10.7326/0003-4819-154-3-201102010-00008pmid:http://www.ncbi.nlm.nih.gov/pubmed/21282698
      OpenUrlCrossRefPubMedWeb of Science
    35. ↵
      1. By the American Geriatrics Society 2015 Beers Criteria Update Expert Panel
      . American geriatrics Society 2015 updated beers criteria for potentially inappropriate medication use in older adults. J Am Geriatr Soc 2015;63:2227–46.doi:10.1111/jgs.13702pmid:http://www.ncbi.nlm.nih.gov/pubmed/26446832
      OpenUrlCrossRefPubMed
    36. ↵
      1. Mafi JN,
      2. Edwards ST,
      3. Pedersen NP, et al
      . Trends in the ambulatory management of headache: analysis of NAMCS and NHAMCS data 1999-2010. J Gen Intern Med 2015;30:548–55.doi:10.1007/s11606-014-3107-3pmid:http://www.ncbi.nlm.nih.gov/pubmed/25567755
      OpenUrlCrossRefPubMed
    37. ↵
      1. Edwards ST,
      2. Mafi JN,
      3. Landon BE
      . Trends and quality of care in outpatient visits to generalist and specialist physicians delivering primary care in the United States, 1997-2010. J Gen Intern Med 2014;29:947–55.doi:10.1007/s11606-014-2808-ypmid:http://www.ncbi.nlm.nih.gov/pubmed/24567201
      OpenUrlPubMed
    38. ↵
      1. Mafi JN,
      2. Russell K,
      3. Bortz BA, et al
      . Low-Cost, high-volume health services contribute the most to unnecessary health spending. Health Aff 2017;36:1701–4.doi:10.1377/hlthaff.2017.0385pmid:http://www.ncbi.nlm.nih.gov/pubmed/28971913
      OpenUrlAbstract/FREE Full Text
    39. ↵
      1. Mafi JN,
      2. Reid RO,
      3. Baseman LH, et al
      . Trends in low-value health service use and spending in the US Medicare fee-for-service program, 2014-2018. JAMA Netw Open 2021;4:e2037328.doi:10.1001/jamanetworkopen.2020.37328pmid:http://www.ncbi.nlm.nih.gov/pubmed/33591365
      OpenUrlPubMed
    40. ↵
      1. Reid RO,
      2. Mafi JN,
      3. Baseman LH, et al
      . Waste in the Medicare program: a national cross-sectional analysis of 2017 low-value service use and spending. J Gen Intern Med 2021;36:2478-2482.doi:10.1007/s11606-020-06061-0pmid:http://www.ncbi.nlm.nih.gov/pubmed/32728953
      OpenUrlPubMed
    41. ↵
      1. Ladapo JA,
      2. Richards AK,
      3. DeWitt CM, et al
      . Disparities in the quality of cardiovascular care between HIV-infected versus HIV-uninfected adults in the United States: a cross-sectional study. J Am Heart Assoc 2017;6. doi:doi:10.1161/JAHA.117.007107. [Epub ahead of print: 14 Nov 2017].pmid:http://www.ncbi.nlm.nih.gov/pubmed/29138182
    42. ↵
      1. Statistics NCfH
      . Using ultimate cluster models with NAMCS and NHAMCS public use files, 2004. Available: https://www.cdc.gov/nchs/data/ahcd/ultimatecluster.pdf [Accessed 08 Dec 2016].
    43. ↵
      1. Zhou Y,
      2. Matsouaka RA,
      3. Thomas L
      . Propensity score weighting under limited overlap and model misspecification. Stat Methods Med Res 2020;29:3721–56.doi:10.1177/0962280220940334pmid:http://www.ncbi.nlm.nih.gov/pubmed/32693715
      OpenUrlCrossRefPubMed
    44. ↵
      1. Mundinger MO,
      2. Kane RL,
      3. Lenz ER, et al
      . Primary care outcomes in patients treated by nurse practitioners or physicians: a randomized trial. JAMA 2000;283:59–68.doi:10.1001/jama.283.1.59pmid:http://www.ncbi.nlm.nih.gov/pubmed/10632281
      OpenUrlCrossRefPubMedWeb of Science
    45. ↵
      1. Mehrotra A,
      2. Liu H,
      3. Adams JL, et al
      . Comparing costs and quality of care at retail clinics with that of other medical settings for 3 common illnesses. Ann Intern Med 2009;151:321.doi:10.7326/0003-4819-151-5-200909010-00006pmid:http://www.ncbi.nlm.nih.gov/pubmed/19721020
      OpenUrlCrossRefPubMedWeb of Science
    46. ↵
      1. Horrocks S,
      2. Anderson E,
      3. Salisbury C
      . Systematic review of whether nurse practitioners working in primary care can provide equivalent care to doctors. BMJ 2002;324:819–23.doi:10.1136/bmj.324.7341.819pmid:http://www.ncbi.nlm.nih.gov/pubmed/11934775
      OpenUrlAbstract/FREE Full Text
    47. ↵
      1. Martínez-González NA,
      2. Djalali S,
      3. Tandjung R, et al
      . Substitution of physicians by nurses in primary care: a systematic review and meta-analysis. BMC Health Serv Res 2014;14:214.doi:10.1186/1472-6963-14-214pmid:http://www.ncbi.nlm.nih.gov/pubmed/24884763
      OpenUrlCrossRefPubMed
    48. ↵
      1. Kinnersley P,
      2. Anderson E,
      3. Parry K
      . Randomised controlled trial of nurse practitioner versus general practitioner care for patients requesting "same day" consultations in primary care. BMJ (Clinical research ed) 2000;320(7241:1043–8.
    49. ↵
      1. Martin-Misener R,
      2. Harbman P,
      3. Donald F, et al
      . Cost-Effectiveness of nurse practitioners in primary and specialised ambulatory care: systematic review. BMJ Open 2015;5:e007167.doi:10.1136/bmjopen-2014-007167
    50. ↵
      1. Maier CB,
      2. Barnes H,
      3. Aiken LH, et al
      . Descriptive, cross-country analysis of the nurse practitioner workforce in six countries: size, growth, physician substitution potential. BMJ Open 2016;6:e011901.doi:10.1136/bmjopen-2016-011901pmid:http://www.ncbi.nlm.nih.gov/pubmed/27601498
      OpenUrlAbstract/FREE Full Text
    51. ↵
      1. McMullen M,
      2. Alexander MK,
      3. Bourgeois A, et al
      . Evaluating a nurse practitioner service. Dimens Crit Care Nurs 2001;20:30–4.doi:10.1097/00003465-200109000-00008pmid:http://www.ncbi.nlm.nih.gov/pubmed/22076518
      OpenUrlPubMed
    52. ↵
      1. Sanchez M,
      2. Smally AJ,
      3. Grant RJ, et al
      . Effects of a fast-track area on emergency department performance. J Emerg Med 2006;31:117–20.doi:10.1016/j.jemermed.2005.08.019pmid:http://www.ncbi.nlm.nih.gov/pubmed/16798173
      OpenUrlCrossRefPubMedWeb of Science
    53. ↵
      1. Oredsson S,
      2. Jonsson H,
      3. Rognes J, et al
      . A systematic review of triage-related interventions to improve patient flow in emergency departments. Scand J Trauma Resusc Emerg Med 2011;19:43.doi:10.1186/1757-7241-19-43pmid:http://www.ncbi.nlm.nih.gov/pubmed/21771339
      OpenUrlCrossRefPubMed
    54. ↵
      1. Sakr M,
      2. Kendall R,
      3. Angus J, et al
      . Emergency nurse practitioners: a three part study in clinical and cost effectiveness. Emerg Med J 2003;20:158–63.doi:10.1136/emj.20.2.158pmid:http://www.ncbi.nlm.nih.gov/pubmed/12642530
      OpenUrlAbstract/FREE Full Text
    55. ↵
      1. Cooke MW,
      2. Wilson S,
      3. Pearson S
      . The effect of a separate stream for minor injuries on accident and emergency department waiting times. Emerg Med J 2002;19:28–30.doi:10.1136/emj.19.1.28pmid:http://www.ncbi.nlm.nih.gov/pubmed/11777867
      OpenUrlAbstract/FREE Full Text
    56. ↵
      1. Donelan K,
      2. DesRoches CM,
      3. Dittus RS, et al
      . Perspectives of physicians and nurse practitioners on primary care practice. N Engl J Med 2013;368:1898–906.doi:10.1056/NEJMsa1212938pmid:http://www.ncbi.nlm.nih.gov/pubmed/23675658
      OpenUrlCrossRefPubMedWeb of Science
    57. ↵
      1. Goyal MK,
      2. Kuppermann N,
      3. Cleary SD, et al
      . Racial disparities in pain management of children with appendicitis in emergency departments. JAMA Pediatr 2015;169:996–1002.doi:10.1001/jamapediatrics.2015.1915pmid:http://www.ncbi.nlm.nih.gov/pubmed/26366984
      OpenUrlPubMed
    58. ↵
      1. Singhal A,
      2. Tien Y-Y,
      3. Hsia RY
      . Racial-Ethnic disparities in opioid prescriptions at emergency department visits for conditions commonly associated with prescription drug abuse. PLoS One 2016;11:e0159224.doi:10.1371/journal.pone.0159224pmid:http://www.ncbi.nlm.nih.gov/pubmed/27501459
      OpenUrlCrossRefPubMed
    59. ↵
      1. Mills AM,
      2. Shofer FS,
      3. Boulis AK, et al
      . Racial disparity in analgesic treatment for ED patients with abdominal or back pain. Am J Emerg Med 2011;29:752–6.doi:10.1016/j.ajem.2010.02.023pmid:http://www.ncbi.nlm.nih.gov/pubmed/20825892
      OpenUrlCrossRefPubMed
    60. ↵
      1. Grumbach K,
      2. Hart LG,
      3. Mertz E, et al
      . Who is caring for the underserved? A comparison of primary care physicians and nonphysician clinicians in California and Washington. Ann Fam Med 2003;1:97–104.doi:10.1370/afm.49pmid:http://www.ncbi.nlm.nih.gov/pubmed/15040439
      OpenUrlAbstract/FREE Full Text
    61. ↵
      1. Morgan PA,
      2. Smith VA,
      3. Berkowitz TSZ, et al
      . Impact of physicians, nurse practitioners, and physician assistants on utilization and costs for complex patients. Health Aff 2019;38:1028–36.doi:10.1377/hlthaff.2019.00014pmid:http://www.ncbi.nlm.nih.gov/pubmed/31158006
      OpenUrlCrossRefPubMed
    62. ↵
      1. Gadbois EA,
      2. Miller EA,
      3. Tyler D, et al
      . Trends in state regulation of nurse practitioners and physician assistants, 2001 to 2010. Med Care Res Rev 2015;72:200–19.doi:10.1177/1077558714563763pmid:http://www.ncbi.nlm.nih.gov/pubmed/25542195
      OpenUrlCrossRefPubMed
    63. ↵
      1. Pittman P,
      2. Leach B,
      3. Everett C, et al
      . Np and PA Privileging in acute care settings: do scope of practice laws matter? Med Care Res Rev 2020;77:112–20.doi:10.1177/1077558718760333pmid:http://www.ncbi.nlm.nih.gov/pubmed/29482454
      OpenUrlPubMed

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    Footnotes

    • Contributors JM, BL, JL, CH-T and RG designed the study. Data analysis was carried out by RG and JM. The manuscript was drafted by all authors. All authors read and approved the final manuscript. JM is the corresponding author and guarantor. The guarantor accepts full responsibility for the work and the conduct of the study, had access to the data, and controlled the decision to publish.

    • Funding JM was supported by the National Institute of Health, National Institute on Aging, grant no. 1K76AG064392-01A1, and the National Institutes of Health (NIH) National Center for Advancing Translational Sciences, grant no. KL2TR001882.

    • Competing interests JM and his team received non-financial support from Milliman MedInsight to use its Health Waste Calculator free of charge to study low-value care among US Medicare beneficiaries in 2020. JM has also presented his research on low-value care to industry groups, Humana, GoodRx, AHIP, the VBID Low-Value Care Taskforce and Milliman MedInsight. He received no honoraria for these presentations. No other relationships or activities that could appear to have influenced the submitted work.

    • Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.