A simulation-based analysis of the impact of rhetorical citations in science.

Journal

Nature communications

ISSN: 2041-1723

Titre abrégé: Nat Commun

Pays: England

ID NLM: 101528555

Informations de publication

Date de publication:
10 Jan 2024

Historique:

received: 24 04 2023

accepted: 05 12 2023

medline: 11 1 2024

pubmed: 11 1 2024

entrez: 10 1 2024

Statut: epublish

Résumé

Authors of scientific papers are usually encouraged to cite works that meaningfully influenced their research (substantive citations) and avoid citing works that had no meaningful influence (rhetorical citations). Rhetorical citations are assumed to degrade incentives for good work and benefit prominent papers and researchers. Here, we explore if rhetorical citations have some plausibly positive effects for science and disproportionately benefit the less prominent papers and researchers. We developed a set of agent-based models where agents can cite substantively and rhetorically. Agents first choose papers to read based on their expected quality, become influenced by those that are sufficiently good, and substantively cite them. Next, agents fill any remaining slots in their reference lists with rhetorical citations that support their narrative, regardless of whether they were actually influential. We then turned agents' ability to cite rhetorically on-and-off to measure its effects. Enabling rhetorical citing increased the correlation between paper quality and citations, increased citation churn, and reduced citation inequality. This occurred because rhetorical citing redistributed some citations from a stable set of elite-quality papers to a more dynamic set with high-to-moderate quality and high rhetorical value. Increasing the size of reference lists, often seen as an undesirable trend, amplified the effects. Overall, rhetorical citing may help deconcentrate attention and make it easier to displace established ideas.

Identifiants

DOI: 10.1038/s41467-023-44249-0 PMID: 38200080

pubmed: 38200080

doi: 10.1038/s41467-023-44249-0

pii: 10.1038/s41467-023-44249-0

doi:

Types de publication

Journal Article

Langues

eng

Sous-ensembles de citation

Pagination

431

Informations de copyright

Références

Baldi, S. Normative versus social constructivist processes in the allocation of citations: a network-analytic model. Am. Sociol. Rev. 63, 829–846 (1998).

Zuckerman, H. Citation analysis and the complex problem of intellectual influence. Scientometrics 12, 329–338 (1987).

doi: 10.1007/BF02016675

Leahey, E., Lee, J. & Funk, R. J. What types of novelty are most disruptive? Am. Sociol. Rev. 88, 562–597 (2023).

doi: 10.1177/00031224231168074

Bornmann, L. & Daniel, H. D. What do citation counts measure? A review of studies on citing behavior. J. Doc. 64, 45–80 (2008).

Gilbert, G. N. Referencing as persuasion. Soc. Stud. Sci. 7, 113–122 (1977).

doi: 10.1177/030631277700700112

Cozzens, S. What do citations count? The rhetoric-first model. Scientometrics 15, 437–447 (1989).

doi: 10.1007/BF02017064

Latour, B. & Woolgar, S. Laboratory Life: The Construction of Scientific Facts (Princeton University Press, 2013).

Nicolaisen, J. Citation analysis. Annu. Rev. Inf. Sci. Technol. 41, 609–641 (2007).

doi: 10.1002/aris.2007.1440410120

Teplitskiy, M., Duede, E., Menietti, M. & Lakhani, K. R. How status of research papers affects the way they are read and cited. Res. Policy 51, 104484 (2022).

doi: 10.1016/j.respol.2022.104484

Allen, B. Referring to schools of thought: an example of symbolic citations. Soc. Stud. Sci. 27, 937–949 (1997).

doi: 10.1177/030631297027006004

Catalini, C., Lacetera, N. & Oettl, A. The incidence and role of negative citations in science. Proc. Natl Acad. Sci. USA 112, 13823–13826 (2015).

pubmed: 26504239 pmcid: 4653214 doi: 10.1073/pnas.1502280112

Wilhite, A. W. & Fong, E. A. Coercive citation in academic publishing. Science 335, 542–543 (2012).

pubmed: 22301307 doi: 10.1126/science.1212540

Cobb, C. L., Crumly, B., Montero-Zamora, P., Schwartz, S. J., & Martínez Jr, C. R. The problem of miscitation in psychological science: righting the ship. Am. Psychol. (2023).

Rubin, A. & Rubin, E. Systematic bias in the progress of research. J. Political Econ. 129, 2666–2719 (2021).

doi: 10.1086/715021

White, H. Reward, persuasion, and the Sokal Hoax: a study in citation identities. Scientometrics 60, 93–120 (2004).

doi: 10.1023/B:SCIE.0000027313.91401.9b

Frandsen, T. F. & Nicolaisen, J. Citation behavior: a large-scale test of the persuasion by name-dropping hypothesis. J. Assoc. Inf. Sci. Technol. 68, 1278–1284 (2017).

doi: 10.1002/asi.23746

Penders, B. Ten simple rules for responsible referencing. PLoS Comput. Biol. 14, e1006036 (2018).

pubmed: 29649210 pmcid: 5896885 doi: 10.1371/journal.pcbi.1006036

Nature Genetics Editorials. Neutral citation is poor scholarship. Nat. Genet. 49, 1559 (2017).

doi: 10.1038/ng.3989

Petrić, B. Rhetorical functions of citations in high-and low-rated master’s theses. J. Engl. Acad. Purp. 6, 238–253 (2007).

doi: 10.1016/j.jeap.2007.09.002

Rose, R. What’s love got to do with it? Dev. Pract. 40, 3–5 (2014).

Hoppe, T. A., Arabi, S. & Hutchins, B. I. Predicting substantive biomedical citations without full text. Proc. Natl Acad. Sci. USA 120, e2213697120 (2023).

pubmed: 37463199 pmcid: 10372685 doi: 10.1073/pnas.2213697120

Kozlowski, D., Andersen, J. P. & Larivière, V. Uncited articles and their effect on the concentration of citations. arXiv preprint arXiv:2306.09911 (2023).

Cronin, B. The need for a theory of citing. J. Doc. 37, 16–24 (1981).

Tahamtan, I. & Bornmann, L. What do citation counts measure? An updated review of studies on citations in scientific documents published between 2006 and 2018. Scientometrics 121, 1635–1684 (2019).

doi: 10.1007/s11192-019-03243-4

Tahamtan, I. & Bornmann, L. The social systems citation theory (SSCT): a proposal to use the social systems theory for conceptualizing publications and their citations links. Prof. Inf. 31, e310411 (2022).

Jurgens, D., Kumar, S., Hoover, R., McFarland, D. & Jurafsky, D. Measuring the evolution of a scientific field through citation frames. Trans. Assoc. Comput. Linguist. 6, 391–406 (2018).

doi: 10.1162/tacl_a_00028

Nicholson, J. M. et al. SCITE: a smart citation index that displays the context of citations and classifies their intent using deep learning. Quant. Sci. Stud. 2, 882–898 (2021).

doi: 10.1162/qss_a_00146

Tenopir, C., King, D. W., Christian, L. & Volentine, R. Scholarly article seeking, reading, and use: a continuing evolution from print to electronic in the sciences and social sciences. Learn. Publ. 28, 93–105 (2015).

doi: 10.1087/20150203

Renear, A. H. & Palmer, C. L. Strategic reading, ontologies, and the future of scientific publishing. Science 325, 828–832 (2009).

pubmed: 19679805 doi: 10.1126/science.1157784

Wang, D., Song, C. & Barabási, A. L. Quantifying long-term scientific impact. Science 342, 127–132 (2013).

pubmed: 24092745 doi: 10.1126/science.1237825

Wang, P. & Domas White, M. A cognitive model of document use during a research project. Study II. Decisions at the reading and citing stages. J. Am. Soc. Inf. Sci. 50, 98–114 (1999).

doi: 10.1002/(SICI)1097-4571(1999)50:2<98::AID-ASI2>3.0.CO;2-L

Lerman, K., Hodas, N., & Wu, H. (2017). Bounded rationality in scholarly knowledge discovery. arXiv preprint arXiv:1710.00269.

Petersen, A. M. et al. Reputation and impact in academic careers. Proc. Natl Acad. Sci. USA 111, 15316–15321 (2014).

Simcoe, T. S. & Waguespack, D. M. Status, quality, and attention: what’s in a (missing) name? Manag. Sci. 57, 274–290 (2010).

doi: 10.1287/mnsc.1100.1270

Wang, P. & Soergel, D. A cognitive model of document use during a research project. Study I. Document selection. J. Am. Soc. Inf. Sci. 49, 115–133 (1998).

doi: 10.1002/(SICI)1097-4571(199802)49:2<115::AID-ASI3>3.0.CO;2-T

Azoulay, P., Stuart, T. & Wang, Y. Matthew: effect or fable? Manag. Sci. 60, 92–109 (2013).

doi: 10.1287/mnsc.2013.1755

Milard, B. & Tanguy, L. Citations in scientific texts: do social relations matter? J. Assoc. Inf. Sci. Technol. 69, 1380–1395 (2018).

doi: 10.1002/asi.24061

Murray, S. O. & Poolman, R. C. Strong ties and scientific literature. Soc. Netw. 4, 225–232 (1982).

doi: 10.1016/0378-8733(82)90023-5

Merton, R. K. The Sociology of Science: Theoretical and Empirical Investigations (University of Chicago Press, 1973)

Horbach, S., Aagaard, K. & Schneider, J. W. Meta-research: how problematic citing practices distort science. MetaArXiv https://doi.org/10.31222/osf.io/aqyhg (2021).

Leigh Star, S This is not a boundary object: reflections on the origin of a concept. Sci. Technol. Hum. Values 35, 601–617 (2010).

doi: 10.1177/0162243910377624

Mizruchi, M. S. & Fein, L. C. The social construction of organizational knowledge: a study of the uses of coercive, mimetic, and normative isomorphism. Adm. Sci. Q. 44, 653–683 (1999).

doi: 10.2307/2667051

Rekdal, O. B. Academic urban legends. Soc. Stud. Sci. 44, 638–654 (2014).

pubmed: 25272616 pmcid: 4232290 doi: 10.1177/0306312714535679

Hargens, L. L. Using the literature: reference networks, reference contexts, and the social structure of scholarship. Am. Sociol. Rev. 65, 846–865 (2000).

doi: 10.1177/000312240006500603

Shwed, U. & Bearman, P. S. The temporal structure of scientific consensus formation. Am. Sociol. Rev. 75, 817–840 (2010).

pubmed: 21886269 pmcid: 3163460 doi: 10.1177/0003122410388488

Beel, J. & Gipp, B. Google Scholar’s ranking algorithm: the impact of citation counts (an empirical study). In Flory, A. & Collard, M. (Eds.), 2009 Third International Conference on Research Challenges in Information Science 439–446 (IEEE, 2009).

Latour, B. Science in Action: How to Follow Scientists and Engineers through Society (Harvard University Press, 1987).

Lin, Y., Evans, J. A. & Wu, L. New directions in science emerge from disconnection and discord. J. Informetr. 16, 101234 (2022).

doi: 10.1016/j.joi.2021.101234

Wu, L., Wang, D. & Evans, J. A. Large teams develop and small teams disrupt science and technology. Nature 566, 378–382 (2019).

pubmed: 30760923 doi: 10.1038/s41586-019-0941-9

Chu, J. S. & Evans, J. A. Slowed canonical progress in large fields of science. Proc. Natl Acad. Sci. USA 118, e2021636118 (2021).

Evans, J. A. Electronic publication and the narrowing of science and scholarship. Science 321, 395–399 (2008).

pubmed: 18635800 doi: 10.1126/science.1150473

Park, M., Leahey, E. & Funk, R. J. Papers and patents are becoming less disruptive over time. Nature 613, 138–144 (2023).

pubmed: 36600070 doi: 10.1038/s41586-022-05543-x

Parolo, P. D. B. et al. Attention decay in science. J. Informetr. 9, 734–745 (2015).

doi: 10.1016/j.joi.2015.07.006

Price, D. J. Little Science, Big Science (Columbia University Press, 1963).

Nielsen, M. W. & Andersen, J. P. Global citation inequality is on the rise. Proc. Natl Acad Sci. USA 118, e2012208118 (2021).

Gomez, C. J., Herman, A. C. & Parigi, P. Leading countries in global science increasingly receive more citations than other countries doing similar research. Nat. Hum. Behav. 6, 919–929 (2022).

Feenberg, D., Ganguli, I., Gaule, P. & Gruber, J. It’s good to be first: Order bias in reading and citing NBER working papers. Rev. Econ. Stat. 99, 32–39 (2017).

doi: 10.1162/REST_a_00607

Andersen, J. P. Field-level differences in paper and author characteristics across all fields of science in Web of Science, 2000–2020. Quant. Sci. Stud. 4, 394–422 (2023).

Granovetter, M. Threshold models of collective behavior. Am. J. Sociol. 83, 1420–1443 (1978).

doi: 10.1086/226707

Centola, D. An experimental study of homophily in the adoption of health behavior. Science 334, 1269–1272 (2011).

pubmed: 22144624 doi: 10.1126/science.1207055

Greenberg, S. A. How citation distortions create unfounded authority: analysis of a citation network. BMJ 339, b2680 (2009).

Letrud, K. & Hernes, S. Affirmative citation bias in scientific myth debunking: A three-in-one case study. PLoS ONE 14, e0222213 (2019).

pubmed: 31498834 pmcid: 6733478 doi: 10.1371/journal.pone.0222213

De Vries, Y. A. et al. The cumulative effect of reporting and citation biases on the apparent efficacy of treatments: the case of depression. Psychol. Med. 48, 2453–2455 (2018).

pubmed: 30070192 pmcid: 6190062 doi: 10.1017/S0033291718001873

West, J. D. & Bergstrom, C. T. Misinformation in and about science. Proc. Natl Acad. Sci. USA 118, e1912444117 (2021).

pubmed: 33837146 pmcid: 8054004 doi: 10.1073/pnas.1912444117

Guetzkow, J., Lamont, M. & Mallard, G. What is originality in the humanities and the social sciences? Am. Sociol. Rev. 67, 190–212 (2004).

doi: 10.1177/000312240406900203

Bradley, K. J. & Aguinis, H. Team performance: nature and antecedents of nonnormal distributions. Organ. Sci. 34, 1266–1286 (2022).

Lancho-Barrantes, B. S., Guerrero-Bote, V. P. & Moya-Anegón, F. What lies behind the averages and significance of citation indicators in different disciplines? J. Inf. Sci. 36, 371–382 (2010).

doi: 10.1177/0165551510366077

Denrell, J. & Liu, C. When reinforcing processes generate an outcome-quality dip. Organ. Sci. 32, 1079–1099 (2021).

doi: 10.1287/orsc.2020.1414

van de Rijt, A. Self-correcting dynamics in social influence processes. Am. J. Sociol. 124, 1468–1495 (2019).

Zuckerman, E. W. Construction, concentration, and (dis) continuities in social valuations. Annu. Rev. Sociol. 38, 223–245 (2012).

doi: 10.1146/annurev-soc-070210-075241