Research paperSexuality and gender invariance of the PHQ-9 and GAD-7: Implications for 16 identity groups
Introduction
A strong body of research indicates sexual and gender minority (LGBTQ+) individuals disproportionately suffer from psychopathology compared to their cisgender and heterosexual counter parts (Borgogna et al., 2018; Ross et al., 2018; Sutter and Perrin, 2016). Despite improvements in mental health awareness, prevention, and interventions for LGBTQ+ individuals (Holman and Oswald, 2016; Martin et al., 2018; Pryor et al., 2017), continued work is necessary. Particularly, psychometric comparisons of popular mental health measures represent an area in need of further research. Most mental health outcome measures have been validated using primarily cisgender/heterosexual samples. Even if LGBTQ+ individuals participate in validation studies, they are often removed or are combined with other LGBTQ+ participants when examined. This practice, though at times necessary for statistical modeling, promulgates an empirical assumption that all LGBTQ+ people respond to mental health measures in the same manner as cisgender/heterosexual participants. In turn, any potential measurement bias from this practice could lead to inaccurate representations of the true mental health discrepancies across LGBTQ+ populations. The Patient Health Questionnaire-9 (PHQ-9; Kroenke et al., 2001) and the Generalized Anxiety Disorder-7 (GAD–7; Spitzer et al., 2006) are arguably the most popular mental health measures in both research and practice. By extension, psychometric validations of the factor structures of the PHQ-9 and GAD-7 are necessary across LGBTQ+ populations to ensure comparisons using these measures are not distorted. However, researchers have yet to conduct large scale measurement invariance examinations on the PHQ-9 and GAD-7 across specific LGBTQ+ groups. We addressed this problem by conducting a large-scale measurement invariance study of the PHQ-9 and GAD-7 across 16 sexual and gender identity samples.
Meyer's minority stress model (2003, 2015) is the premier theoretical framework for understanding LGBTQ+ mental health disparities. Meyer's model extends social stress theory (c.f., Burke, 1991) to LGBTQ+ individuals. Specifically, sexual and gender minority groups experience additive stress from having a marginalized identity. Discrimination experiences (including perceived discrimination; Borgogna and McDermott, 2019) such as microaggressions and discriminatory policies function as an external stressors in this framework. Internalized heterosexism and homonegativity function as internalized stressors. Empirical findings align with the minority stress model in predicting mental and physical health problems for LGBTQ+ individuals (Lefevor et al., 2019; Meyer and Frost, 2013; Parra et al., 2016).
Sexual and gender minority individuals also experience nuanced forms of minority stress unique to specific LGBTQ+ groups. For instance, transgender individuals experience gender-related forms of minority stress (e.g., misgendering) that are not necessarily experienced by sexual minority individuals (McLemore, 2018; Velez et al., 2017). Similarly, gender and sexual minority stressors differ within gender minority groups. Indeed, genderqueer individuals experience unique stressors associated with identifying outside of the gender binary (Hyde et al., 2019; Matsuno and Budge, 2017; Richards et al., 2016). Similarly, non-monosexual individuals (e.g., bisexual, pansexual) experience minority stressors associated with being a minority within the sexual minority community (e.g., biphobia; Alarie and Gaudet, 2013; Ghabrial and Ross, 2018; La Roi et al., 2019; Lambe et al., 2017).
Given the unique nature of LGBTQ+ identities (Scheer et al., 2019), it is possible that the mental health presentations are also unique. That is, the depression associated with LGBTQ+ minority stress is categorically different from the depression associated with a divorce or poor diet. Researchers have highlighted this problem in conjunction with the classification of psychological disorders more broadly (e.g., Hayes et al., 1996; Jablensky, 2016; Westen, 2012). Though a solution to the syndromal classification of mental illness is beyond the scope of the current report, researchers have developed sophisticated techniques to examine whether measures that are designed to assess latent constructs, such as depression, are equivalent between groups in terms of their meaning, scale, and degree of precision. By extension, researchers can test whether commonly used instruments that were validated in primarily cisgender/heterosexual samples measure the same constructs in LGBTQ+ samples. While many researchers have conducted such psychometric evaluations on the PHQ-9 and GAD-7 (e.g., Teymoori et al., 2019), there is a notably lack of comparisons across sexual/gender diverse samples.
Measurement invariance testing is the primary statistical technique for such explorations (Meade et al., 2008; Vandenberg and Lance, 2000). Measurement invariance is generally tested in four forms/steps.
- 1
Configural Invariance: At the most basic level, responses to an instrument will demonstrate the same factor structure. For example, a unidimensional depression measure in cisgender/heterosexuals should evidence a unidimensional pattern in LGBTQ+ individuals.
- 2
Metric Invariance: Assuming configural invariance is achieved, researchers can test whether the instrument items have the same relationship to the underlying latent construct across groups (i.e., equivalent factor loadings; Xu and Tracey, 2017). For example, metric invariance would suggest that the depression construct, as measured by the PHQ–9, is equivalent for cisgender/heterosexuals and LGBTQ+ individuals.
- 3
Scalar Invariance: Assuming metric invariance is met, researchers can then test the equivalence of intercepts between groups. Support for scalar invariance suggests equivalent means between groups. In other words, a mean score of 15 on the PHQ-9 represents the same level of depression for cisgender/heterosexuals as LGBTQ+ individuals.
- 4
Residual Invariance: Finally, if scalar invariance is supported, researchers can test the equivalence of errors. This is the most rigorous form of invariance. Support for residual invariance would suggest that the instrument measures the same underlying construct between groups with an equivalent degree of precision. If residual invariance is achieved, an instrument may be assumed to be entirely equivalent between groups in terms of its measurement function.
Understanding the measurement invariance of the PHQ-9 and GAD-7 between LGBTQ+ and cisgender/heterosexual individuals is critical for researchers and clinicians, as these instruments were validated using primarily cisgender/heterosexual samples. Furthermore, it is important to understand how these instruments function across differing LGBTQ+ groups (i.e., Is the PHQ-9 measuring depression equally between gay men and bisexual women?). Preliminary research on highly specified scales suggests some degree of equivalence among these groups. For instance, all eight subscales of the Lesbian, Gay, and Bisexual Identity Scale (Mohr and Kendra, 2011) have demonstrated scalar invariance between gay men and lesbian women (Niepel et al., 2019). Similarly, the Spiritual Values/Religion subscale of Self-Description Questionnaire III have demonstrated scalar invariance across heterosexual and non-heterosexual groups (Ong et al., 2019). Conversely, the Everyday Discrimination Scale (Williams et al., 1997) has demonstrated partial metric invariance between cisgender and non-cisgender participants (as well as between specific non-cisgender groups), whereas the Discrimination-Related Vigilance Scale (Clark et al., 2006) has demonstrated metric invariance between cisgender and non-cisgender groups broadly, but only partial metric invariance across specific gender identities (Bauerband et al., 2019). These findings suggest there may be important similarities or differences across sexual/gender minority groups, which could influence the way they respond to specific instruments.
To date, the PHQ-9 and GAD-7 lack measurement invariance examinations across sexual and gender minority groups. This is a substantial gap in the literature given the popularity of the PHQ-9 and GAD-7. To address this limitation, we conducted a large measurement invariance study across a sexual orientation and gender diverse dataset of college students. Given there is limited LGBTQ+ measurement invariance research across these instruments, our analyses were highly exploratory. However, consistent with the minority stress model (Meyer, 2003) and findings suggesting LGBTQ+ mental health problems are associated with factors that are not present in traditional mental health problems for cisgender/heterosexual individuals (e.g., Bauerband et al., 2019), we broadly hypothesized that some degree of measurement non-invariance would be present for both measures across every LGBTQ+ group compared to cisgender/heterosexual participants.
Section snippets
Participants/Procedure
We analyzed data from the 2017–2018 and 2018–2019 Healthy Minds Study's (HMS) to explore the measurement invariance of multiple sexual and gender identities on the PHQ-9 and GAD-7. The HMS is an annual survey examining various mental health variables across college students from 60 different universities in the United States and Canada. We initially sampled N = 62,025 participants from the 2018–2019 dataset; however, because substantially unequal sample sizes between groups can bias measurement
Participant selection
PHQ-9. After data cleaning, most groups had n’s > 200, with exception to queer men (n = 180), transmen (n = 156), transwomen (n = 61), and genderqueer-M (n = 149) participants; thus, we supplemented these groups with responses from the 2017–2018 HMS. Specifically, we randomly selected 46 queer men, 254 transmen, 96 transwomen, and 91 genderqueer-M individuals from the 2017–2018 dataset to meet the minimum baseline criteria for each group. Upon including these additional participants, the
Discussion
Our results suggest the measurement accuracy of the PHQ-9 and GAD-7 in capturing depression and anxiety across LGBTQ+ groups is complex. Results were consistent with our broad hypothesis that some degree of measurement non-invariance would be present for both instruments across groups. Only cisgender/heterosexual men and women evidenced full residual invariance on the PHQ-9. Notably, we had to exclude gay men, questioning women, queer men, and genderqueer-M individuals from measurement
Author contributions
Nicholas C. Borgogna is a Ph.D. candidate in the Combined-Integrated Clinical and Counseling Psychology program at the University of South Alabama. Ryon C. McDermott is an associate professor in the Counseling and Instructional Sciences Department at the University of South Alabama. Rachel E. Brenner is now an Assistant Professor in the Psychology Department at Colorado State University. Nicholas C. Borgogna is the corresponding author: [email protected]. The authors would also like
CRedit authorship contribution statement
Nicholas C. Borgogna: Conceived and designed the study; Analyzed the data; Wrote the paper. Rachel E. Brenner: Wrote the paper. Ryon C. McDermott: Conceived and designed the study; Wrote the paper.
Declarations of Competing Interest
All authors declare that they have no financial, personal, or professional competing interests.
References (50)
- et al.
Large arterial elasticity varies as a function of gender and racism-related vigilance in black youth
J. Adolesc. Heal.
(2006) - et al.
“I don't know if she is bisexual or if she just wants to get attention”: analyzing the various mechanisms through which emerging adults invisibilize bisexuality
J. Bisex.
(2013) - et al.
Measuring minority stress: invariance of a discrimination and vigilance scale across transgender and cisgender LGBQ individuals
Psychol. Sex
(2019) - et al.
Perceived discrimination is disproportionally associated with sexual minority mental health: implications for non-monosexual sexual minorities
J. Gay Lesbian Ment. Health
(2019) - et al.
Anxiety and depression across gender and sexual minorities: implications for transgender, gender nonconforming, pansexual, demisexual, asexual, queer, and questioning individuals
Psychol. Sex. Orientat. Gend. Divers.
(2018) Identity processes and social stress
Am. Sociol. Rev.
(1991)Sensitivity of goodness of fit indexes to lack of measurement invariance
Struct. Equ. Model.
(2007)- et al.
A direct comparison approach for testing measurement invariance
Organ. Res. Methods
(2012) - et al.
Evaluating goodness-of-fit indexes for testing measurement invariance
Struct. Equ. Model.
(2002) - et al.
Assessing extreme and acquiescence response sets in cross-cultural research using structural equations modeling
J. Cross. Cult. Psychol.
(2000)