The consequences of generative AI for online knowledge communities.
Journal
Scientific reports
ISSN: 2045-2322
Titre abrégé: Sci Rep
Pays: England
ID NLM: 101563288
Informations de publication
Date de publication:
06 May 2024
06 May 2024
Historique:
received:
23
10
2023
accepted:
02
05
2024
medline:
7
5
2024
pubmed:
7
5
2024
entrez:
6
5
2024
Statut:
epublish
Résumé
Generative artificial intelligence technologies, especially large language models (LLMs) like ChatGPT, are revolutionizing information acquisition and content production across a variety of domains. These technologies have a significant potential to impact participation and content production in online knowledge communities. We provide initial evidence of this, analyzing data from Stack Overflow and Reddit developer communities between October 2021 and March 2023, documenting ChatGPT's influence on user activity in the former. We observe significant declines in both website visits and question volumes at Stack Overflow, particularly around topics where ChatGPT excels. By contrast, activity in Reddit communities shows no evidence of decline, suggesting the importance of social fabric as a buffer against the community-degrading effects of LLMs. Finally, the decline in participation on Stack Overflow is found to be concentrated among newer users, indicating that more junior, less socially embedded users are particularly likely to exit.
Identifiants
pubmed: 38710885
doi: 10.1038/s41598-024-61221-0
pii: 10.1038/s41598-024-61221-0
doi:
Types de publication
Journal Article
Research Support, Non-U.S. Gov't
Langues
eng
Sous-ensembles de citation
IM
Pagination
10413Informations de copyright
© 2024. The Author(s).
Références
Noy, S. & Zhang, W. Experimental evidence on the productivity effects of generative artificial intelligence. Science https://doi.org/10.2139/ssrn.4375283 (2023).
doi: 10.2139/ssrn.4375283
pubmed: 37440646
Peng, S., Kalliamvakou, E., Cihon, P., Demirer, M. The impact of AI on developer productivity: Evidence from Github copilot. Preprint at https://arXiv.org/2302.06590 (2023).
Dell-Acqua, F. et al. Navigating the jagged technological frontier: Field experimental evidence of the effects of AI on knowledge worker productivity and quality. Harvard Business School Working Paper, no. 24-013(2023).
Hwang, E. H., Singh, P. V. & Argote, L. Knowledge sharing in online communities: Learning to cross geographic and hierarchical boundaries. Organ. Sci. 26(6), 1593–1611 (2015).
doi: 10.1287/orsc.2015.1009
Hwang, E. H. & Krackhardt, D. Online knowledge communities: Breaking or sustaining knowledge silos?. Prod. Oper. Manag. 29(1), 138–155 (2020).
doi: 10.1111/poms.13098
Bang, Y. et al. A multitask, multilingual, multimodal evaluation of ChatGPT on reasoning, hallucination, and interactivity. In Proc. of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 675–718 (2023).
Saxenian, A. Regional Advantage: Culture and Competition in Silicon Valley and Route 128 (Harvard University Press, 1996). https://doi.org/10.4159/9780674418042 .
doi: 10.4159/9780674418042
Atkin, D., Chen, M. K., Popov, A. The returns to face-to-face interactions: Knowledge spillovers in Silicon Valley. National Bureau of Economic Research, no. w30147(2022).
Roche, M. P., Oettl, A., & Catalini, C. (Co-)working in close proximity: Knowledge spillovers and social interactions. National Bureau of Economic Research, no. w30120 (2022).
Tubiana, M., Miguelez, E. & Moreno, R. In knowledge we trust: Learning-by-interacting and the productivity of inventors. Res. Policy 51(1), 104388 (2022).
doi: 10.1016/j.respol.2021.104388
Hooijberg, R. & Watkins, M. When do we really need face-to-face interactions? https://hbr.org/2021/01/when-do-we-really-need-face-to-face-interactions (Harvard Business Publishing, 2021).
Allen, T. J. Managing the Flow of Technology: Technology Transfer and the Dissemination of Technological Information within the R&D Organization (MIT Press Books, 1984).
Abadie, A. Using synthetic controls: Feasibility, data requirements, and methodological aspects. J. Econ. Lit. 59(2), 391–425 (2021).
doi: 10.1257/jel.20191450
Hollingsworth, A., Wing, C. Tactics for design and inference in synthetic control studies: An applied example using high-dimensional data. Available at SSRN, Paper no. 3592088 (2020).
Tibshirani, R. Regression shrinkage and selection via the lasso. J. R. Stat. Soc. Ser. B Stat. Methodol. 58(1), 267–288 (1996).
doi: 10.1111/j.2517-6161.1996.tb02080.x
Goldberg, S., Johnson, G. & Shriver, S. Regulating privacy online: An economic evaluation of the GDPR. Am. Econ. J. Econ. Policy 16(1), 325–358 (2024).
doi: 10.1257/pol.20210309
Eichenbaum, M., Godinho de Matos, M., Lima, F., Rebelo, S. & Trabandt, M. Expectations, infections, and economic activity. J. Polit. Econ. https://doi.org/10.1086/729449 (2023).
doi: 10.1086/729449
Angrist, J. D. & Pischke, J. S. Mostly Harmless Econometrics: An Empiricist’s Companion (Princeton University Press, 2009).
doi: 10.1515/9781400829828
Peters, J. Reddit thinks AI chatbots will ‘complement’ human connection, not replace it. The Verge. https://www.theverge.com/2023/2/10/23594786/reddit-bing-chatgpt-ai-google-search-bard (Accessed 17 September 2023) (2023).
Antelmi, A., Cordasco, G., De Vinco, D., Spagnuolo, C.The age of snippet programming: Toward understanding developer communities in stack overflow and reddit. In Companion Proceedings of the ACM Web Conference, pp. 1218–1224 (2023).
Sengupta, S. ‘Learning to code in a virtual world’ A preliminary comparative analysis of discourse and learning in two online programming communities. In Conference Companion Publication of the 2020 on Computer Supported Cooperative Work and Social Computing, pp. 389–394 (2020).
Athey, S., Bayati, M., Doudchenko, N., Imbens, G. & Khosravi, K. Matrix completion methods for causal panel data models. J. Am. Stat. Assoc. 116(536), 1716–1730 (2021).
doi: 10.1080/01621459.2021.1891924
Wu, L. & Kane, G. C. Network-biased technical change: How modern digital collaboration tools overcome some biases but exacerbate others. Organ. Sci. 32(2), 273–292 (2021).
doi: 10.1287/orsc.2020.1368
Kabir, S., Udo-Imeh, D. N., Kou, B., Zhang, T. Who answers it better? An in-depth analysis of ChatGPT and stack overflow answers to software engineering questions. Preprint at https://arXiv.org/2308.02312 (2023).
Villalobos, P., Sevilla, J., Heim, L., Besiroglu, T., Hobbhahn, M., Ho, A. Will we run out of data? An analysis of the limits of scaling datasets in machine learning. Preprint at https://arXiv.org/2211.04325 (2022).