Elsevier

Journal of Affective Disorders

Volume 278, 1 January 2021, Pages 122-130
Journal of Affective Disorders

Research paper
Sexuality and gender invariance of the PHQ-9 and GAD-7: Implications for 16 identity groups

https://doi.org/10.1016/j.jad.2020.09.069Get rights and content

Highlights

  • Individual CFA's were conducted on the PHQ-9 and GAD-7 across 16 identity groups.

  • Gay men, questioning women, and queer men had unacceptable fit indices for the PHQ-9.

  • Only cisgender men and women evidenced residual invariance for the PHQ-9.

  • All non-cisgender groups had significantly higher item 9 factor loadings than cisgenders.

  • The GAD-7 had better fit across groups, though no groups had residual invariance.

Abstract

Background: The Patient Health Questionnaire-9 (PHQ-9) and the Generalized Anxiety Disorder-7 (GAD-7) are two commonly used screening tools for depression and anxiety, respectively. Despite the widespread use of these instruments, researchers have yet to examine whether scores may differ as a function of gender identity or sexuality orientation. Method: Using data from the 2018 and 2019 National Healthy minds study (N = 46,672), the present study tested each instrument for measurement invariance across 16 gender and sexual minority groups. Results: Multigroup structural equation modeling revealed that several sexual minority groups evidenced questionable fit indices for both measures. Gay men, questioning women, and queer men had unacceptable fit indices for the PHQ-9. Only cisgender heterosexual men and women evidenced residual invariance on the PHQ-9. All gender minority groups evidenced significantly higher factor loadings for item 9 (the self-harm indicator) for the PHQ-9. Most identity groups evidenced scalar or partial scalar invariance for the GAD-7; however, no groups evidenced residual invariance on the GAD-7. Limitations: Findings may not generalize to non-college student populations. Conclusions: Researchers should weight means when conducting between group comparisons for groups that failed scalar invariance. Gender and sexual minorities may have inflated scores using the PHQ-9 and GAD-7 cut-offs.

Introduction

A strong body of research indicates sexual and gender minority (LGBTQ+) individuals disproportionately suffer from psychopathology compared to their cisgender and heterosexual counter parts (Borgogna et al., 2018; Ross et al., 2018; Sutter and Perrin, 2016). Despite improvements in mental health awareness, prevention, and interventions for LGBTQ+ individuals (Holman and Oswald, 2016; Martin et al., 2018; Pryor et al., 2017), continued work is necessary. Particularly, psychometric comparisons of popular mental health measures represent an area in need of further research. Most mental health outcome measures have been validated using primarily cisgender/heterosexual samples. Even if LGBTQ+ individuals participate in validation studies, they are often removed or are combined with other LGBTQ+ participants when examined. This practice, though at times necessary for statistical modeling, promulgates an empirical assumption that all LGBTQ+ people respond to mental health measures in the same manner as cisgender/heterosexual participants. In turn, any potential measurement bias from this practice could lead to inaccurate representations of the true mental health discrepancies across LGBTQ+ populations. The Patient Health Questionnaire-9 (PHQ-9; Kroenke et al., 2001) and the Generalized Anxiety Disorder-7 (GAD–7; Spitzer et al., 2006) are arguably the most popular mental health measures in both research and practice. By extension, psychometric validations of the factor structures of the PHQ-9 and GAD-7 are necessary across LGBTQ+ populations to ensure comparisons using these measures are not distorted. However, researchers have yet to conduct large scale measurement invariance examinations on the PHQ-9 and GAD-7 across specific LGBTQ+ groups. We addressed this problem by conducting a large-scale measurement invariance study of the PHQ-9 and GAD-7 across 16 sexual and gender identity samples.

Meyer's minority stress model (2003, 2015) is the premier theoretical framework for understanding LGBTQ+ mental health disparities. Meyer's model extends social stress theory (c.f., Burke, 1991) to LGBTQ+ individuals. Specifically, sexual and gender minority groups experience additive stress from having a marginalized identity. Discrimination experiences (including perceived discrimination; Borgogna and McDermott, 2019) such as microaggressions and discriminatory policies function as an external stressors in this framework. Internalized heterosexism and homonegativity function as internalized stressors. Empirical findings align with the minority stress model in predicting mental and physical health problems for LGBTQ+ individuals (Lefevor et al., 2019; Meyer and Frost, 2013; Parra et al., 2016).

Sexual and gender minority individuals also experience nuanced forms of minority stress unique to specific LGBTQ+ groups. For instance, transgender individuals experience gender-related forms of minority stress (e.g., misgendering) that are not necessarily experienced by sexual minority individuals (McLemore, 2018; Velez et al., 2017). Similarly, gender and sexual minority stressors differ within gender minority groups. Indeed, genderqueer individuals experience unique stressors associated with identifying outside of the gender binary (Hyde et al., 2019; Matsuno and Budge, 2017; Richards et al., 2016). Similarly, non-monosexual individuals (e.g., bisexual, pansexual) experience minority stressors associated with being a minority within the sexual minority community (e.g., biphobia; Alarie and Gaudet, 2013; Ghabrial and Ross, 2018; La Roi et al., 2019; Lambe et al., 2017).

Given the unique nature of LGBTQ+ identities (Scheer et al., 2019), it is possible that the mental health presentations are also unique. That is, the depression associated with LGBTQ+ minority stress is categorically different from the depression associated with a divorce or poor diet. Researchers have highlighted this problem in conjunction with the classification of psychological disorders more broadly (e.g., Hayes et al., 1996; Jablensky, 2016; Westen, 2012). Though a solution to the syndromal classification of mental illness is beyond the scope of the current report, researchers have developed sophisticated techniques to examine whether measures that are designed to assess latent constructs, such as depression, are equivalent between groups in terms of their meaning, scale, and degree of precision. By extension, researchers can test whether commonly used instruments that were validated in primarily cisgender/heterosexual samples measure the same constructs in LGBTQ+ samples. While many researchers have conducted such psychometric evaluations on the PHQ-9 and GAD-7 (e.g., Teymoori et al., 2019), there is a notably lack of comparisons across sexual/gender diverse samples.

Measurement invariance testing is the primary statistical technique for such explorations (Meade et al., 2008; Vandenberg and Lance, 2000). Measurement invariance is generally tested in four forms/steps.

  • 1

    Configural Invariance: At the most basic level, responses to an instrument will demonstrate the same factor structure. For example, a unidimensional depression measure in cisgender/heterosexuals should evidence a unidimensional pattern in LGBTQ+ individuals.

  • 2

    Metric Invariance: Assuming configural invariance is achieved, researchers can test whether the instrument items have the same relationship to the underlying latent construct across groups (i.e., equivalent factor loadings; Xu and Tracey, 2017). For example, metric invariance would suggest that the depression construct, as measured by the PHQ–9, is equivalent for cisgender/heterosexuals and LGBTQ+ individuals.

  • 3

    Scalar Invariance: Assuming metric invariance is met, researchers can then test the equivalence of intercepts between groups. Support for scalar invariance suggests equivalent means between groups. In other words, a mean score of 15 on the PHQ-9 represents the same level of depression for cisgender/heterosexuals as LGBTQ+ individuals.

  • 4

    Residual Invariance: Finally, if scalar invariance is supported, researchers can test the equivalence of errors. This is the most rigorous form of invariance. Support for residual invariance would suggest that the instrument measures the same underlying construct between groups with an equivalent degree of precision. If residual invariance is achieved, an instrument may be assumed to be entirely equivalent between groups in terms of its measurement function.

Understanding the measurement invariance of the PHQ-9 and GAD-7 between LGBTQ+ and cisgender/heterosexual individuals is critical for researchers and clinicians, as these instruments were validated using primarily cisgender/heterosexual samples. Furthermore, it is important to understand how these instruments function across differing LGBTQ+ groups (i.e., Is the PHQ-9 measuring depression equally between gay men and bisexual women?). Preliminary research on highly specified scales suggests some degree of equivalence among these groups. For instance, all eight subscales of the Lesbian, Gay, and Bisexual Identity Scale (Mohr and Kendra, 2011) have demonstrated scalar invariance between gay men and lesbian women (Niepel et al., 2019). Similarly, the Spiritual Values/Religion subscale of Self-Description Questionnaire III have demonstrated scalar invariance across heterosexual and non-heterosexual groups (Ong et al., 2019). Conversely, the Everyday Discrimination Scale (Williams et al., 1997) has demonstrated partial metric invariance between cisgender and non-cisgender participants (as well as between specific non-cisgender groups), whereas the Discrimination-Related Vigilance Scale (Clark et al., 2006) has demonstrated metric invariance between cisgender and non-cisgender groups broadly, but only partial metric invariance across specific gender identities (Bauerband et al., 2019). These findings suggest there may be important similarities or differences across sexual/gender minority groups, which could influence the way they respond to specific instruments.

To date, the PHQ-9 and GAD-7 lack measurement invariance examinations across sexual and gender minority groups. This is a substantial gap in the literature given the popularity of the PHQ-9 and GAD-7. To address this limitation, we conducted a large measurement invariance study across a sexual orientation and gender diverse dataset of college students. Given there is limited LGBTQ+ measurement invariance research across these instruments, our analyses were highly exploratory. However, consistent with the minority stress model (Meyer, 2003) and findings suggesting LGBTQ+ mental health problems are associated with factors that are not present in traditional mental health problems for cisgender/heterosexual individuals (e.g., Bauerband et al., 2019), we broadly hypothesized that some degree of measurement non-invariance would be present for both measures across every LGBTQ+ group compared to cisgender/heterosexual participants.

Section snippets

Participants/Procedure

We analyzed data from the 2017–2018 and 2018–2019 Healthy Minds Study's (HMS) to explore the measurement invariance of multiple sexual and gender identities on the PHQ-9 and GAD-7. The HMS is an annual survey examining various mental health variables across college students from 60 different universities in the United States and Canada. We initially sampled N = 62,025 participants from the 2018–2019 dataset; however, because substantially unequal sample sizes between groups can bias measurement

Participant selection

PHQ-9. After data cleaning, most groups had n’s > 200, with exception to queer men (n = 180), transmen (n = 156), transwomen (n = 61), and genderqueer-M (n = 149) participants; thus, we supplemented these groups with responses from the 2017–2018 HMS. Specifically, we randomly selected 46 queer men, 254 transmen, 96 transwomen, and 91 genderqueer-M individuals from the 2017–2018 dataset to meet the minimum baseline criteria for each group. Upon including these additional participants, the

Discussion

Our results suggest the measurement accuracy of the PHQ-9 and GAD-7 in capturing depression and anxiety across LGBTQ+ groups is complex. Results were consistent with our broad hypothesis that some degree of measurement non-invariance would be present for both instruments across groups. Only cisgender/heterosexual men and women evidenced full residual invariance on the PHQ-9. Notably, we had to exclude gay men, questioning women, queer men, and genderqueer-M individuals from measurement

Author contributions

Nicholas C. Borgogna is a Ph.D. candidate in the Combined-Integrated Clinical and Counseling Psychology program at the University of South Alabama. Ryon C. McDermott is an associate professor in the Counseling and Instructional Sciences Department at the University of South Alabama. Rachel E. Brenner is now an Assistant Professor in the Psychology Department at Colorado State University. Nicholas C. Borgogna is the corresponding author: [email protected]. The authors would also like

CRedit authorship contribution statement

Nicholas C. Borgogna: Conceived and designed the study; Analyzed the data; Wrote the paper. Rachel E. Brenner: Wrote the paper. Ryon C. McDermott: Conceived and designed the study; Wrote the paper.

Declarations of Competing Interest

All authors declare that they have no financial, personal, or professional competing interests.

References (50)

  • R. Clark et al.

    Large arterial elasticity varies as a function of gender and racism-related vigilance in black youth

    J. Adolesc. Heal.

    (2006)
  • M. Alarie et al.

    “I don't know if she is bisexual or if she just wants to get attention”: analyzing the various mechanisms through which emerging adults invisibilize bisexuality

    J. Bisex.

    (2013)
  • L.A. Bauerband et al.

    Measuring minority stress: invariance of a discrimination and vigilance scale across transgender and cisgender LGBQ individuals

    Psychol. Sex

    (2019)
  • N.C. Borgogna et al.

    Perceived discrimination is disproportionally associated with sexual minority mental health: implications for non-monosexual sexual minorities

    J. Gay Lesbian Ment. Health

    (2019)
  • N.C. Borgogna et al.

    Anxiety and depression across gender and sexual minorities: implications for transgender, gender nonconforming, pansexual, demisexual, asexual, queer, and questioning individuals

    Psychol. Sex. Orientat. Gend. Divers.

    (2018)
  • P.J. Burke

    Identity processes and social stress

    Am. Sociol. Rev.

    (1991)
  • F.F. Chen

    Sensitivity of goodness of fit indexes to lack of measurement invariance

    Struct. Equ. Model.

    (2007)
  • G.W. Cheung et al.

    A direct comparison approach for testing measurement invariance

    Organ. Res. Methods

    (2012)
  • G.W. Cheung et al.

    Evaluating goodness-of-fit indexes for testing measurement invariance

    Struct. Equ. Model.

    (2002)
  • G.W. Cheung et al.

    Assessing extreme and acquiescence response sets in cross-cultural research using structural equations modeling

    J. Cross. Cult. Psychol.

    (2000)
  • M.A. Ghabrial et al.

    Representation and erasure of bisexual people of color: a content analysis of quantitative bisexual mental health research

    Psychol. Sex Orientat. Gend. Divers.

    (2018)
  • M. Green et al.

    The lesbian, gay, bisexual and transgender community online: discussions of bullying and self-disclosure in YouTube videos

    Behav. Inf. Technol.

    (2015)
  • S.C. Hayes et al.

    Experiential avoidance and behavioral disorders: a functional dimensional approach to diagnosis and treatment

    J. Consult. Clin. Psychol.

    (1996)
  • E.G. Holman et al.

    A decade of changes: within-group analysis of LGBTQ individuals’ perceptions of their community context and the relevance for social service providers

    J. Gay Lesbian Soc. Serv.

    (2016)
  • J.S. Hyde et al.

    The future of sex and gender in psychology: five challenges to the gender binary

    Am. Psychol.

    (2019)
  • A. Jablensky

    Psychiatric classifications: validity and utility

    World Psychiatry

    (2016)
  • R.B. Kline

    Principles and Practice of Structural Equation Modeling, 4th ed.

    (2016)
  • L.E. Kuper et al.

    Exploring cross-sectional predictors of suicide ideation, attempt, and risk in a large online sample of transgender and gender nonconforming youth and young adults

    LGBT Heal

    (2018)
  • T.A. Kyriazos

    Applied psychometrics: sample size and sample power considerations in factor analysis (EFA, CFA) and SEM in general

    Psychology

    (2018)
  • C. La Roi et al.

    Differences in sexual identity dimensions between bisexual and other sexual minority individuals: implications for minority stress and mental health

    Am. J. Orthopsychiatry

    (2019)
  • Lambe, J., Cerezo, A., O'Shaughnessy, T., 2017. Minority stress, community involvement, and mental health among...
  • G.T. Lefevor et al.

    Health disparities between genderqueer, transgender, and cisgender individuals: an extension of minority stress theory

    J. Couns. Psychol.

    (2019)
  • G. Martin et al.

    “Students at the Margins”: student affairs administrators creating inclusive campuses for LGBTQ students in the South

    J. Stud. Aff. Res. Pract.

    (2018)
  • E. Matsuno et al.

    Non-binary/genderqueer identities: a critical review of the literature

    Curr. Sex. Heal. Rep.

    (2017)
  • E. McDermott et al.

    Norms and normalisation: understanding lesbian, gay, bisexual, transgender and queer youth, suicidality and help-seeking

    Cult. Health Sex.

    (2018)
  • Cited by (0)

    View full text