Gender and culture bias in letters of recommendation for computer science and data science masters programs.
Journal
Scientific reports
ISSN: 2045-2322
Titre abrégé: Sci Rep
Pays: England
ID NLM: 101563288
Informations de publication
Date de publication:
01 09 2023
01 09 2023
Historique:
received:
07
05
2023
accepted:
28
08
2023
medline:
4
9
2023
pubmed:
2
9
2023
entrez:
1
9
2023
Statut:
epublish
Résumé
Letters of Recommendation (LORs) are widely utilized for admission to both undergraduate and graduate programs, and are becoming even more important with the decreasing role that standardized tests play in the admissions process. However, LORs are highly subjective and thus can inject recommender bias into the process, leading to an inequitable evaluation of the candidates' competitiveness and competence. Our study utilizes natural language processing methods and manually determined ratings to investigate gender and cultural differences and biases in LORs written for STEM Master's program applicants. We generate features to measure important characteristics of the LORs and then compare these characteristics across groups based on recommender gender, applicant gender, and applicant country of origin. One set of features, which measure the underlying sentiment, tone, and emotions associated with each LOR, is automatically generated using IBM Watson's Natural Language Understanding (NLU) service. The second set of features is measured manually by our research team and quantifies the relevance, specificity, and positivity of each LOR. We identify and discuss features that exhibit statistically significant differences across gender and culture study groups. Our analysis is based on approximately 4000 applications for the MS in Data Science and MS in Computer Science programs at Fordham University. To our knowledge, no similar study has been performed on these graduate programs.
Identifiants
pubmed: 37658207
doi: 10.1038/s41598-023-41564-w
pii: 10.1038/s41598-023-41564-w
pmc: PMC10474141
doi:
Substances chimiques
Caffeine
3G6A5W338E
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
14367Informations de copyright
© 2023. Springer Nature Limited.
Références
Dutt, K., Pfaff, D. L., Bernstein, A. F., Dillard, J. S. & Block, C. J. Gender differences in recommendation letters for postdoctoral fellowships in geoscience. Nat. Geosci. 9, 805–808 (2016).
doi: 10.1038/ngeo2819
Filippou, P. et al. The presence of gender bias in letters of recommendations written for urology residency applicants. Urology 134, 56–61 (2019).
doi: 10.1016/j.urology.2019.05.065
pubmed: 31491451
Grimm, L. J., Redmond, R. A., Campbell, J. C. & Rosette, A. S. Gender and racial bias in radiology residency letters of recommendation. J. Am. Coll. Radiol. 17, 64–71 (2020).
doi: 10.1016/j.jacr.2019.08.008
pubmed: 31494103
Madera, J. M., Hebl, M. R. & Martin, R. C. Gender and letters of recommendation for academia: Agentic and communal differences. J. Appl. Psychol. 94, 1591 (2009).
doi: 10.1037/a0016539
pubmed: 19916666
Polanco-Santana, J. C., Storino, A., Souza-Mota, L., Gangadharan, S. P. & Kent, T. S. Ethnic/racial bias in medical school performance evaluation of general surgery residency applicants. J. Surg. Educ. 78, 1524–1534 (2021).
doi: 10.1016/j.jsurg.2021.02.005
pubmed: 33637477
Schmader, T., Whitehead, J. & Wysocki, V. H. A linguistic comparison of letters of recommendation for male and female chemistry and biochemistry job applicants. Sex Roles 57, 509–514 (2007).
doi: 10.1007/s11199-007-9291-4
pubmed: 18953419
pmcid: 2572075
Turrentine, F. E., Dreisbach, C. N., St Ivany, A. R., Hanks, J. B. & Schroen, A. T. Influence of gender on surgical residency applicants’ recommendation letters. J. Am. Coll. Surgeons 228, 356–365 (2019).
doi: 10.1016/j.jamcollsurg.2018.12.020
Trix, F. & Psenka, C. Exploring the color of glass: Letters of recommendation for female and male medical faculty. Discourse Soc. 14, 191–220 (2003).
doi: 10.1177/0957926503014002277
Bouton, L. F. A cross-cultural analysis of the structure and content of letters of reference. Stud. Second. Lang. Acquis. 17, 211–244 (1995).
doi: 10.1017/S0272263100014169
de Brey, C., Snyder, T. D., Zhang, A. & Dillow, S. A. Digest of education statistics 2019. NCES 2021-009. National Center for Education Statistics (2021).
Eagly, A. H., Wood, W. & Diekman, A. B. Social role theory of sex differences and similarities: A current appraisal. Dev. Soc. Psychol. Gender 12, 174 (2000).
Eagly, A. H. Sex differences in social behavior: A social-role interpretation (Psychology Press, 2013).
Trapnell, P. D. & Paulhus, D. L. Agentic and communal values: Their scope and measurement. J. Pers. Assess. 94, 39–52 (2012).
doi: 10.1080/00223891.2011.627968
pubmed: 22176265
Rucker, D. D., Galinsky, A. D. & Magee, J. C. The agentic-communal model of advantage and disadvantage: How inequality produces similarities in the psychology of power, social class, gender, and race. Adv. Exp. Soc. Psychol. 58, 71–125 (2018).
doi: 10.1016/bs.aesp.2018.04.001
Bernstein, R. H. et al. Assessing gender bias in particle physics and social science recommendations for academic jobs. Soc. Sci. 11, 74 (2022).
doi: 10.3390/socsci11020074
Grova, M. M. et al. Gender bias in surgical oncology fellowship recommendation letters: Gaining progress. J. Surg. Educ. 78, 866–874 (2021).
doi: 10.1016/j.jsurg.2020.08.049
pubmed: 33317986
Pennebaker, J., Booth, R., Boyd, R. & Francis, M. Linguistic inquiry and word count: Liwc2015. 2015. Austin, TX: Pennebaker Conglomerates (www. LIWC. net) (2015).
Steffensen, M. S. Register, cohesion, and cross-cultural reading comprehension. Center for the Study of Reading Technical Report; no. 220 (1981).
Bouton, L. F. A cross-cultural study of ability to interpret implicatures in English. World Englishes 7, 183–196 (1988).
doi: 10.1111/j.1467-971X.1988.tb00230.x
Schaumberg, R. L. & Flynn, F. J. Self-reliance: A gender perspective on its relationship to communality and leadership evaluations. Acad. Manag. J. 60, 1859–1881 (2017).
doi: 10.5465/amj.2015.0018
United States Census Bureau. Sexual orientation and gender identity in the household pulse survey. https://www.census.gov/library/visualizations/interactive/sexual-orientation-and-gender-identity.html (2021).
IBM. Watson natural language understanding. https://www.ibm.com/cloud/watson-natural-language-understanding (2022).
Langerhuizen, D. W. et al. Analysis of online reviews of orthopaedic surgeons and orthopaedic practices using natural language processing. JAAOS-J. Am. Acad. Orthop. Surgeons 29, 337–344 (2021).
doi: 10.5435/JAAOS-D-20-00288
Sarraf, D., Vasiliu, V., Imberman, B. & Lindeman, B. Use of artificial intelligence for gender bias analysis in letters of recommendation for general surgery residency candidates. Am. J. Surg. 222, 1051–1059 (2021).
doi: 10.1016/j.amjsurg.2021.09.034
pubmed: 34674847
Baral, S. et al. Investigating patterns of tone and sentiment in teacher written feedback messages. In International Conference on Artificial Intelligence in Education, 341–346 (Springer, 2023).
Ravichandiran, S. Getting Started with Google BERT: Build and train state-of-the-art natural language processing models using BERT (Packt Publishing Ltd, 2021).
Dale, R. Gpt-3: What’s it good for?. Nat. Lang. Eng. 27, 113–118 (2021).
doi: 10.1017/S1351324920000601
Microsoft. Natural language processing technology. https://docs.microsoft.com/en-us/azure/architecture/data-guide/technology-choices/natural-language-processing (2022).
Judge, T. A. & Higgins, C. A. Affective disposition and the letter of reference. Org. Behav. Hum. Decision Process. 75, 207–221 (1998).
doi: 10.1006/obhd.1998.2789
Wiens, A. N., Jackson, R. H., Manaugh, T. S. & Matarazzo, J. D. Communication length as an index of communicator attitude: A replication. J. Appl. Psychol. 53, 264 (1969).
doi: 10.1037/h0027796
Akos, P. & Kretchmar, J. Gender and ethnic bias in letters of recommendation: considerations for school counselors. Prof. Sch. Couns. 20, 1096–2409 (2016).
doi: 10.5330/1096-2409-20.1.102
Vik, P. Regression, ANOVA, and the general linear model: A statistics primer (SAGE Publications, 2013).
Arsham, H. & Lovric, M. Bartlett’s test. (2011).
Abdi, H. & Williams, L. J. Tukey’s honestly significant difference (HSD) test. Encyclopedia Res. Design 3, 1–5 (2010).
Lee, H.-Y. Linguistic politeness in the Chinese language and culture. Theory Pract. Lang. Stud. 10, 1–9 (2020).
doi: 10.17507/tpls.1001.01
Council on Foreign Relations. Women’s workplace equity index. https://www.cfr.org/legal-barriers/country-rankings/ (2018).
Zhang, N., Blissett, S., Anderson, D., O’Sullivan, P. & Qasim, A. Race and gender bias in internal medicine program director letters of recommendation. J. Grad. Med. Educ. 13, 335–344 (2021).
doi: 10.4300/JGME-D-20-00929.1
pubmed: 34178258
pmcid: 8207902
Bloodhart, B., Balgopal, M. M., Casper, A. M. A., Sample McMeeking, L. B. & Fischer, E. V. Outperforming yet undervalued: Undergraduate women in stem. Plos one 15, e0234685 (2020).
doi: 10.1371/journal.pone.0234685
pubmed: 32584838
pmcid: 7316242
Zhao, Y., Xu, Q., Chen, M. & Weiss, G. M. Predicting student performance in a master of data science program using admissions data. International Educational Data Mining Society (2020).
Latu, I. M., Mast, M. S., Lammers, J. & Bombari, D. Successful female leaders empower women’s behavior in leadership tasks. J. Exp. Soc. Psychol. 49, 444–448 (2013).
doi: 10.1016/j.jesp.2013.01.003
Blake-Beard, S., Bayne, M. L., Crosby, F. J. & Muller, C. B. Matching by race and gender in mentoring relationships: Keeping our eyes on the prize. J. Soc. Issues 67, 622–643 (2011).
doi: 10.1111/j.1540-4560.2011.01717.x
Lockwood, P. “someone like me can be successful’’: Do college students need same-gender role models?. Psychol. Women Q. 30, 36–46 (2006).
doi: 10.1111/j.1471-6402.2006.00260.x