A new tool for comprehensive reference list analysis in scholarly journalsA new tool for comprehensive reference list analysis in scholar journals
https://doi.org/10.24069/sep.2026.11.48
Abstract
Due to the rapid growth in the number of scientific publications and increasingly stringent quality requirements, journal editorial offices are in greater need of effective tools to verify reference lists. Existing solutions are either commercially unavailable for regional journals or provide fragmented information without the possibility of comprehensive analysis. This paper presents an open-source software tool, Comprehensive Reference List Analysis (CRLA), available at https://litanalysis.streamlit. app and developed in Python using the Streamlit framework. The aim of this article is to demonstrate the functional capabilities of CRLA as a ready-to-use solution for automated reference list analysis and to illustrate its practical applicability using several articles as examples. The tool integrates the Crossref and OpenAlex APIs and enables automatic identifier extraction, metadata enrichment, duplicate detection, suspicious DOI identification, retracted article detection, repository source identification, as well as the calculation of bibliometric indicators and visualization of results. To demonstrate the functionality of the tool, the reference lists of three articles (published in the journal Scientific Editor and Publisher in 2025 and 2026) were analyzed. The obtained results show that CRLA can reveal structural features of reference lists depending on article subject matter: ranging from a predominance of web sources in reviews of publishing practices to nearly complete DOI coverage in bibliometric studies. The tool also identifies self-citations, international collaborations, and potentially problematic sources (duplicates, incorrect DOIs). Thus, CRLA offers a convenient solution, especially for journals with limited resources, enabling the automation of routine checks and improving the quality of peer review.
About the Author
Dmitry A. Medvedev
Ural Federal University, Yekaterinburg, Russian Federation
Russian Federation
Russian Federation
Dr. Sci. (Chem.), Editor-in-Chief of the Chimica Techno Acta journal, Professor of the Chemical Technology Institute, Ural Federal University, Ekaterinburg, Russian Federation
References
1. Hanson M.A., Barreiro P.G., Crosetto P, Brockington D The strain on scientific publishing. Quantitative Science Studies. 2024;5(4):823-843. https://doi.org/10.1162/qss_a_00327
2. Marginson S What drives global science? The four competing narratives. Studies in Higher Education. 2022;47(8):1566-1584. https://doi.org/10.1080/03075079.2021.1942822
3. Gelling L, East L, Hayter M, Heaslip V, Mccormack B, Montayre J, Neville S, Pandian V, Salamonson Y, Uchmanowicz I The Peer Reviewer Crisis: Sustaining Scholarly Community or Reimagining the Future of Peer Review?. Journal of Clinical Nursing. 2026; In press. https://doi.org/10.1111/jocn.70374
4. Adam D The peer-review crisis: how to fix an overloaded system. Nature. 2025;644(8075):24-27. https://doi.org/10.1038/d41586-025-02457-2
5. Vitak J, Bruckman A, Lampe C, Page X, Wong-Villacrés M Beyond "Reviewer 2" Problems: Responding to the Peer Review Crisis in Computing Research. Companion Publication of the 2024 Conference on Computer-Supported Cooperative Work and Social Computing. 2024;110-113. https://doi.org/10.1145/3678884.3689131
6. Teixeira da silva J.A. Does the disconnect between the peer-reviewed label and reality explain the peer review crisis, and can open peer review or preprints resolve it? A narrative review. Naunyn-Schmiedeberg's Archives of Pharmacology. 2026;399(2):1549-1563. https://doi.org/10.1007/s00210-025-04486-0
7. Pecegueiro C.M., Garcia J.C. Repositórios de Preprints nas Ciências Sociais. AtoZ: novas práticas em informação e conhecimento. 2024;13:1-11. https://doi.org/10.5380/atoz.v13i0.89627
8. Colavizza G, Cadwallader L, Laflamme M, Dozot G, Lecorney S, Rappo D, Hrynaszkiewicz I An analysis of the effects of sharing research data, code, and preprints on citations. Plos One. 2024;19(10):e0311493. https://doi.org/10.1371/journal.pone.0311493
9. Ni R, Waltman L To preprint or not to preprint: A global researcher survey. Journal of the Association for Information Science and Technology. 2024;75(6):749-766. https://doi.org/10.1002/asi.24880
10. Lozada-Martinez I.D., Lozada-Martinez L.M., Fiorillo-Moreno O Leiden manifesto and evidence-based research: Are the appropriate standards being used for the correct evaluation of pluralism, gaps and relevance in medical research?. Journal of the Royal College of Physicians of Edinburgh. 2024;54(1):4-6. https://doi.org/10.1177/14782715241227991
11. Rushforth A, De Rijcke S Practicing responsible research assessment: Qualitative study of faculty hiring, promotion, and tenure assessments in the United States. Research Evaluation. 2024;33rvae007. https://doi.org/10.1093/reseval/rvae007
12. Vignau B, Clemente P, Berthomé P Systematic literature review: References extraction helper and automatic analysis. Software Impacts. 2024;21100669. https://doi.org/10.1016/j.simpa.2024.100669
13. Marx W, Bornmann L Change of perspective: bibliometrics from the point of view of cited references—a literature overview on approaches to the evaluation of cited references in bibliometrics. Scientometrics. 2016;109(2):1397-1415. https://doi.org/10.1007/s11192-016-2111-2
14. Kumar A Quality of references reveal the merit of the research. Journal of Indian Society of Periodontology. 2022;26(6):519-520. https://doi.org/10.4103/jisp.jisp_410_22
15. Szomszor M, Pendlebury D.A., Adams J How much is too much? The difference between research influence and self-citation excess. Scientometrics. 2020;123(2):1119-1147. https://doi.org/10.1007/s11192-020-03417-5
16. Teixeira da silva J.A., Bornemann-Cimenti H Why do some retracted papers continue to be cited?. Scientometrics. 2017;110(1):365-370. https://doi.org/10.1007/s11192-016-2178-9
17. Walters W.H., Wilder E.I. Fabrication and errors in the bibliographic citations generated by ChatGPT. Scientific Reports. 2023;13(1):14045. https://doi.org/10.1038/s41598-023-41032-5
18. Topaz M, Roguin N, Gupta P, Zhang Z, Peltonen L Fabricated citations: an audit across 2·5 million biomedical papers. The Lancet. 2026;407(10541):1779-1781. https://doi.org/10.1016/S0140-6736(26)00603-3
19. Alexander C IMF eLibrary. Reference Reviews. 2014;28(8):19-20. https://doi.org/10.1108/rr-06-2014-0156
20. Velez-Estevez A, Perez I, García-Sánchez P, Moral-Munoz J, Cobo M New trends in bibliometric APIs: A comparative analysis. Information Processing & Management. 2023;60(4):103385. https://doi.org/10.1016/j.ipm.2023.103385
21. Bornmann L, Daniel H What do citation counts measure? A review of studies on citing behavior. Journal of Documentation. 2008;64(1):45-80. https://doi.org/10.1108/00220410810844150
22. Bramer W.M. Reference checking for systematic reviews using Endnote. Journal of the Medical Library Association. 2018;106(4):542–546. https://doi.org/10.5195/jmla.2018.489
23. Глиснер Р.М., Суд А Специальные выпуски и их роль в научной коммуникации в условиях трансформации издательской деятельности. Научный редактор и издатель. 2025;9(2):152-167. https://doi.org/10.24069/SEP-24-21
24. Gustina A, Putera P.B., Tikhonova E.V. Bibliometric analysis as a catalyst for research quality and strategic science policy. Science Editor and Publisher. 2025;10(1):114-136. https://doi.org/10.24069/SEP-25-38
25. Gureyev V.N., Mazov N.A., Kurmysheva L.K. Thematic coverage of Russian universities academic journals at the national and international levels. Science Editor and Publisher. 2026;10(2):199-222. https://doi.org/10.24069/SEP-25-40
26. Cobo M, López-Herrera A, Herrera-Viedma E, Herrera F Science mapping software tools: Review, analysis, and cooperative study among tools. Journal of the American Society for Information Science and Technology. 2011;62(7):1382-1402. https://doi.org/10.1002/asi.21525
27. Van eck N.J., Waltman L Citation-based clustering of publications using CitNetExplorer and VOSviewer. Scientometrics. 2017;111(2):1053-1070. https://doi.org/10.1007/s11192-017-2300-7
28. Pranckutė R Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World. Publications. 2021;9(1):12. https://doi.org/10.3390/publications9010012
29. Van raan A The use of bibliometric analysis in research performance assessment and monitoring of interdisciplinary scientific developments. TATuP - Zeitschrift für Technikfolgenabschätzung in Theorie und Praxis. 2003;12(1):20-29. https://doi.org/10.14512/tatup.12.1.20
30. Alonso-Alvarez P., van Eck N. J. Coverage and metadata completeness and accuracy of African research publications in OpenAlex: A comparative analysis. arXiv (2024). https://doi.org/10.48550/arXiv.2409.01120
31. Visser M, Van eck N.J., Waltman L Large-scale comparison of bibliographic data sources: Scopus, Web of Science, Dimensions, Crossref, and Microsoft Academic. Quantitative Science Studies. 2021;2(1):20-41. https://doi.org/10.1162/qss_a_00112
32. Delgado-Quirós L, Ortega J.L. Completeness degree of publication metadata in eight free-access scholarly databases. Quantitative Science Studies. 2024;5(1):31-49. https://doi.org/10.1162/qss_a_00286
33. Hauschke C, Nazarovets S (Non-)retracted academic papers in OpenAlex. Journal of Information Science. 2025;01655515251322478. https://doi.org/10.1177/01655515251322478
Review
For citations:
Medvedev D.A. A new tool for comprehensive reference list analysis in scholarly journalsA new tool for comprehensive reference list analysis in scholar journals. Science Editor and Publisher. (In Russ.) https://doi.org/10.24069/sep.2026.11.48
Views:
25
JATS XML
Email this article 
