Skip to main content

Advertisement

Log in

Demographic and functional differences among social security disability claimants

  • Published:
Quality of Life Research Aims and scope Submit manuscript

Abstract

Purpose

Sociodemographic characteristics may influence responses on self-reported measures. Differential item functioning (DIF) is when individuals expected to have the same ability level on a construct of interest have a different probability of endorsing an item on an item response theory (IRT) scale due to population characteristics. The goal of this study was to identify DIF for items in an outcome instrument by sociodemographic factors and, one controlling for DIF, assess true differences in function by those same factors.

Methods

The Work Disability Functional Assessment Battery 2.0 (WD-FAB 2.0) is an IRT-based self-reported measure of activity limitations relevant to work. Two samples from WD-FAB developed were used: 3793 SSA disability claimants randomly drawn from a pool of 16,500 claimants and a general sample if 2100 working age adults. We used a two-step IRT-based DIF method for three pairs of respondent characteristics: age, gender, and race/ethnicity, and calculated the weighted absolute difference between item characteristic curves. Independent two-group T-tests assessed differences in scores across groups.

Results

Seventeen items displayed DIF. Men had higher scores than women on two physical and two mental function scales. Older respondents had lower physical and higher mental function scores. The lower education group had lower mental function scores.

Conclusion

DIF impacts function measurement and is important when assessing psychometric characteristics of instruments. Self-report measures should include diverse samples to conduct similar analyses. WD-FAB 2.0 scores are now reflections of function with reduced bias related to gender, race/ethnicity, or age.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Social Security Administration. (2017). Annual statistical report on the social security disability insurance program.

  2. Brandt, D. E., Houtenville, A. J., Huynh, M. T., Chan, L., & Rasch, E. K. (2011). Connecting contemporary paradigms to the social security administration’s disability evaluation process. Journal of Disability Policy Studies. https://doi.org/10.1177/1044207310396509.

    Article  Google Scholar 

  3. Lord, F. M. (1980). Applications of item response theory to practical testing problems. Abingdon: Routledge.

    Google Scholar 

  4. Rivers D. (2006). Sample matching: Representative sampling from internet panels. Polimetrix White Paper Series.

  5. Reeve, B. B., Hays, R. D., Bjorner, J. B., et al. (2007). Psychometric evaluation and calibration of health-related quality of life item banks: Plans for the patient-reported outcomes measurement information system (PROMIS). Medical Care, 45(5 Suppl 1), S22-31. https://doi.org/10.1097/01.mlr.0000250483.85507.04.

    Article  PubMed  Google Scholar 

  6. Norris, J. M. (2001). Computer-adaptive testing: A primer. Language Learning & Technology., 5(2), 23–27.

    Google Scholar 

  7. Warm, T. A. (1989). Weighted likelihood estimation of ability in item response theory. Psychometrika., 54(3), 427–450.

    Article  Google Scholar 

  8. Wang, S., & Wang, T. (2001). Precision of warm’s weighted likelihood estimates for a polytomous model in computerized adaptive testing. Applied Psychological Measurement, 25(4), 317–331.

    Article  Google Scholar 

  9. Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika monograph supplement.

  10. Hambleton, R., & Pitoniak, M. (2002). Testing and measurement: Advances in item response theory and selected testing practices.

  11. Meterko, M., Marfeo, E. E., McDonough, C. M., et al. (2014). The work disability functional assessment battery (WD-FAB): Feasibility and psychometric properties. Archives of Physical Medicine and Rehabilitation. https://doi.org/10.1016/j.apmr.2014.11.025.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Ni, P., McDonough, C. M., Jette, A. M., et al. (2013). Development of a computer-adaptive physical function instrument for social security administration disability determination. Arch Phys Med Rehabil., 94(9), 1661–1669.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Marfeo, E. E., Ni, P., Haley, S. M., et al. (2013). Development of an instrument to measure behavioral health function for work disability: Item pool construction and factor analysis. Archives of Physical Medicine and Rehabilitation, 94(9), 1670–1678.

    Article  PubMed  PubMed Central  Google Scholar 

  14. McDonough, C. M., Ni, P., Peterik, K., Marfeo, E. E., Marino, M. E., Meterko, M., … & Chan, L. (2017). Improving measures of work-related physical functioning. Quality of life research26(3), 789–798.

  15. Marfeo, E. E., Ni, P., McDonough, C., Peterik, K., Marino, M., Meterko, M., … & Jette, A. M. (2018). Improving assessment of work related mental health function using the work disability functional assessment battery (WD-FAB). Journal of Occupational Rehabilitation28(1), 190–199.

  16. Meterko, M., Marino, M., Ni, P., Marfeo, E., McDonough, C. M., Jette, A., … & Chan, L. (2019). Psychometric evaluation of the improved work-disability functional assessment battery. Archives of Physical Medicine and Rehabilitation100(8), 1442–1449.

  17. Jette, A. M., Ni, P., Rasch, E., Marfeo, E., McDonough, C., Brandt, D., … & Chan, L. (2019). The Work Disability Functional Assessment Battery (WD-FAB). Physical Medicine and Rehabilitation Clinics.

  18. Coe, N. B., Haverstick, K., Munnell, A. H., & Webb, A. (2011). What explains state variation in SSDI application rates? Center for Retirement Research at Boston College Working Paper. (2011-23).

  19. Coe, N. B., Haverstick, K., Munnell, A. H., & Webb, A. (2012). Why do state disability application rates vary over time? Center for Retirement Research at Boston College Working Paper. 2012(12-2).

  20. Von Wachter, T., Song, J., & Manchester, J. (2011). Trends in employment and earnings of allowed and rejected applicants to the social security disability insurance program. American Economic Review, 107, 3308–3329.

    Article  Google Scholar 

  21. Wilson, K. B. (2000). Predicting vocational rehabilitation acceptance based on race, education, work status, and source of support at application. Rehabilitation Counseling Bulletin, 43(2), 97–105.

    Article  Google Scholar 

  22. Langer, M. M. (2008). A reexamination of Lord's Wald test for differential item functioning using item response theory and modern error estimation.

  23. Woods, C. M., Cai, L., & Wang, M. (2013). The langer-improved wald test for DIF testing with multiple groups evaluation and comparison to two-group IRT. Educational and Psychological Measurement, 73(3), 532–547.

    Article  Google Scholar 

  24. Edelen, M. O., Stucky, B. D., & Chandra, A. (2013). Quantifying ‘problematic’ DIF within an IRT framework: Application to a cancer stigma index. Quality of Life Research, 24, 1–9.

    Google Scholar 

  25. Oshima, T., Kushubar, S., Scott, J., & Raju, N. (2009). DFIT8 for window user’s manual: Differential functioning of items and tests. St. Paul, MN: Assessment Systems Corporation.

    Google Scholar 

  26. Rice, M. E., & Harris, G. T. (2005). Comparing effect sizes in follow-up studies: ROC area, Cohen’s d, and r. Law and Human Behavior, 29(5), 615.

    Article  PubMed  Google Scholar 

  27. Abel, J. R., Gabe, T. M., & Stolarick, K. (2012). Workforce skills across the urban-rural hierarchy. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.2010646.

    Article  Google Scholar 

  28. Autor, D. (2010). The polarization of job opportunities in the US labor market: Implications for employment and earnings. Center for American Progress and The Hamilton Project.

  29. Lin, S., Beck, A. N., Finch, B. K., Hummer, R. A., & Master, R. K. (2012). Trends in US older adult disability: Exploring age, period, and cohort effects. American Journal of Public Health, 102(11), 2157–2163.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Loprest, P., & Maag, E. (2007). The relationship between early disability onset and education and employment. Journal of Vocational Rehabilitation, 26(1), 49–62.

    Google Scholar 

  31. Hopman, W., Harrison, M., Coo, H., Friedberg, E., Buchanan, M., & Van Den Kerkhof, E. (2009). Associations between chronic disease, age and physical and mental health status. Chronic Diseases in Canada, 29(3), 108–116.

    Article  CAS  PubMed  Google Scholar 

  32. Ward, B. W. (2013). Prevalence of multiple chronic conditions among US adults: Estimates from the national health interview survey, 2010. Preventing Chronic Disease. https://doi.org/10.5888/pcd10.120203.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Haveman, R., & Wolfe, B. (2000). The economics of disability and disability policy. Handbook of Health Economics, 1, 995–1051.

    Article  Google Scholar 

  34. Duggan, M, & Imberman, S. A. (2009). Why are the disability rolls skyrocketing? The contribution of population characteristics, economic conditions, and program generosity. In: Health at older ages: The causes and consequences of declining disability among the elderly. Chicago: University of Chicago Press, pp. 337–379.

  35. Hendley, A. A., & Bilimoria, N. F. (1999). Minorities and social security: An analysis of ethnic differences in the current program. Social Security Bulletin, 62, 59.

    CAS  PubMed  Google Scholar 

  36. Kington, R. S., & Smith, J. P. (1997). Socioeconomic status and racial and ethnic differences in functional status associated with chronic diseases. American Journal of Public Health, 87(5), 805–810.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  37. Williams, D. R. (1999). Race, socioeconomic status, and health the added effects of racism and discrimination. Annals of the New York Academy Sciences, 896(1), 173–188.

    Article  CAS  Google Scholar 

  38. Nichols, A. & Simms, M. (2012). Racial and ethnic differences in receipt of unemployment insurance benefits during the great recession. Unemployment and Recovery Project Brief.

  39. Kao, G., & Thompson, J. S. (2003). Racial and ethnic stratification in educational achievement and attainment. Annual Review of Sociology, 29(1), 417–442.

    Article  Google Scholar 

  40. DeBose, C. E. (1992). Codeswitching: Black English and standard English in the African-American linguistic repertoire. Journal of Multilingual & Multicultural Development, 13(1–2), 157–167.

    Article  Google Scholar 

  41. Fleishman, J. A., Spector, W. D., & Altman, B. M. (2002). Impact of differential item functioning on age and gender differences in functional disability. The Journals of Gerontology, Series B: Psychological Sciences and Social Sciences, 57(5), S275-84.

    Article  Google Scholar 

  42. Sullivan, O. (1997). Time waits for no (wo) man: An investigation of the gendered experience of domestic time. Sociology, 31(2), 221–239.

    Article  Google Scholar 

  43. Teresi, J. A., & Fleishman, J. A. (2007). Differential item functioning and health assessment. Quality of Life Research, 16(1), 33–42.

    Article  PubMed  Google Scholar 

  44. Teresi, J. A., Ocepek-Welikson, K., Kleinman, M., et al. (2009). Analysis of differential item functioning in the depression item bank from the patient reported outcome measurement information system (PROMIS): An item response theory approach. Psychological Science Quarterly, 51(2), 148–180.

    Google Scholar 

  45. Rose, M., Bjorner, J. B., Becker, J., Fries, J., & Ware, J. (2008). Evaluation of a preliminary physical function item bank supported the expected advantages of the patient-reported outcomes measurement information system (PROMIS). Journal of Clinical Epidemiology, 61(1), 17–33.

    Article  CAS  PubMed  Google Scholar 

  46. Iwata, N., Turner, R. J., & Lloyd, D. A. (2002). Race/ethnicity and depressive symptoms in community-dwelling young adults: A differential item functioning analysis. Psychiatry Research, 110(3), 281–289.

    Article  PubMed  Google Scholar 

  47. Gao, Y., & Zhu, W. (2011). Differential item functioning analysis of the 2003–04 NHANES physical activity questionnaire. Research Quarterly for Exercise and Sport, 82(3), 381–390.

    Article  PubMed  Google Scholar 

  48. Hafner-Eaton, C. (1993). Physician utilization disparities between the uninsured and insured: Comparisons of the chronically III, acutely III, and well nonelderly populations. JAMA, 269(6), 787–792.

    Article  CAS  PubMed  Google Scholar 

  49. Williams, D. R., & Collins, C. (1995). US socioeconomic and racial differences in health: Patterns and explanations. Annual Review of Sociology, 21, 349–386.

    Article  Google Scholar 

  50. Willimas, D., Yu, Y., & Jackson, J. (1997). Racial differences in physical and mental health. Journal of Health Psychology, 2, 335–351.

    Article  Google Scholar 

  51. Yee, S. (2011). Health and health care disparities among people with disabilities. Washington, DC: Disability Rights Education & Defense Fund.

    Google Scholar 

  52. Fremstad, S. (2009). Half in ten. Why taking disability into account is essential to reducing income poverty and expanding economic inclusion. Washington, DC: Center for Economic and Policy Research.

    Google Scholar 

  53. Hanmer, J., Lawrence, W. F., Anderson, J. P., Kaplan, R. M., & Fryback, D. G. (2006). Report of nationally representative values for the noninstitutionalized US adult population for 7 health-related quality-of-life scores. Medical Decision Making, 26(4), 391–400.

    Article  PubMed  Google Scholar 

  54. Milner, A., LaMontagne, A., Aitken, Z., Bentley, R., & Kavanagh, A. (2014). Employment status and mental health among persons with and without a disability: Evidence from an Australian cohort study. Journal of Epidemiology and Community Health. https://doi.org/10.1136/jech-2014-204147.

    Article  PubMed  Google Scholar 

  55. McFarland, M. J., & Wagner, B. G. (2015). Does a college education reduce depressive symptoms in American young adults? Social Science & Medicine, 146, 75–84.

    Article  Google Scholar 

  56. Kessler, R. C., Amminger, G. P., Aguilar-Gaxiola, S., Alonso, J., Lee, S., & Ustun, T. B. (2007). Age of onset of mental disorders: A review of recent literature. Current Opinion in Psychiatry, 20(4), 359–364. https://doi.org/10.1097/YCO.0b013e32816ebc8c.

    Article  PubMed  PubMed Central  Google Scholar 

  57. Lee, M., & Mather, M. (2008) US labor force trends. Vol 63. Population Reference Bureau.

  58. Bauer, D. J., Belzak, W. C. M., & Cole, V. T. (2020). Simplifying the assessment of measurement invariance over multiple background variables: using regularized moderated nonlinear factor analysis to detect differential item functioning. Structural Equation Modeling, 27(1), 43–55. https://doi.org/10.1080/10705511.2019.1642754.

    Article  PubMed  Google Scholar 

  59. Bauer, D. J. (2017). A more general model for testing measurement invariance and differential item functioning. Psychological Methods, 22(3), 507–526. https://doi.org/10.1037/met0000077.

    Article  PubMed  Google Scholar 

  60. Gottfredson, N. C., Cole, V. T., Giordano, M. L., Bauer, D. J., Hussong, A. M., & Ennetta, S. T. (2019). Simplifying the implementation of modern scale scoring methods with an automated R package: Automated moderated nonlinear factor analysis (aMNLFA). Addictive Behaviors, 94, 65–73. https://doi.org/10.1016/j.addbeh.2018.10.031.

    Article  PubMed  Google Scholar 

  61. Wu, X., Sawatzky, R., Hopman, W., Mayo, N., Sajobi, T. T., Liu, J., et al. (2017). Latent variable mixture models to test for differential item functioning: a population-based analysis. Health and Quality of Life Outcomes, 15, 102. https://doi.org/10.1186/s12955-017-0674-0.

    Article  PubMed  PubMed Central  Google Scholar 

  62. DeMars, C. E., & Lau, A. (2011). Differential item functioning detection with latent classes: How accurately can we detect who is responding differentially? Educational and Psychological Measurement, 71(4), 597–616. https://doi.org/10.1177/0013164411404221.

    Article  Google Scholar 

  63. Ma, S., Huang, J., Zhang, Z., & Liu, M. (2019). Exploration of heterogeneous treatment effects via concave fusion. The International Journal of Biostatistics, 16(1), 20180026. https://doi.org/10.1515/ijb-2018-0026.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Molly Marino.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Marino, M., Ni, P., Kazis, L. et al. Demographic and functional differences among social security disability claimants. Qual Life Res 30, 1757–1768 (2021). https://doi.org/10.1007/s11136-021-02765-w

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11136-021-02765-w

Keywords

Navigation