Bibliometric analysis as a catalyst for research quality and strategic science policy

Introduction. Bibliometric practices have become central to contemporary research evaluation, influencing academic careers, institutional rankings, and national science policy. However, their non-selective application has provoked serious concerns regarding fairness, epistemic bias, and the erosion of academic values. While reform initiatives such as the Leiden Manifesto and DORA have gained international visibility, the integration of their principles into institutional and national frameworks remains inconsistent. This article addresses the unresolved tension between the global discourse on responsible metrics and its uneven operationalization across contexts.
Purpose. This study offers a conceptual examination of bibliometric governance, advancing the argument that research metrics must be reconfigured as embedded instruments of multi-level science governance. To support this theoretical perspective, the article integrates illustrative bibliometric mapping and interpretive policy synthesis.
Method. Using a curated dataset of the 500 most cited publications on research evaluation (2015–2024) retrieved from Scopus, a co-occurrence analysis of keywords and abstract terms was conducted using VOSviewer. The resulting thematic clusters were used not as empirical evidence, but as heuristic anchors for the conceptual discussion. These were interpreted alongside key policy documents and reform agendas through a conceptually driven analytical framework.
Results. The bibliometric visualization revealed a fragmented discourse, where normative frameworks for responsible metrics are conceptually visible but poorly integrated into performance-driven evaluation literature. Illustrative national cases (e.g., China, Italy, Indonesia) further demonstrate the divergence between policy rhetoric and implementation. The article proposes a multi-level conceptual model of metric responsibility that captures individual, institutional, and national dimensions of reform.
Conclusion. The findings support the need for a governance-centered approach to metrics, one that balances quantitative indicators with contextual judgment, ethical reflexivity, and policy alignment. By combining conceptual reasoning with empirical illustration, the article contributes to the theoretical grounding of responsible metrics and provides a reflexive framework for science policy and institutional reform in the digital age.

1. Archambault É., Larivière V. The limits of bibliometrics for the analysis of the social sciences and humanities literature. In: World Social Science Report, 2010: Knowledge Divides. Paris: UNESCO Publishing; 2010, pp. 251–254.

2. Glänzel W., Thijs B., Debackere K. The application of citation-based performance classes to the disciplinary and multidisciplinary assessment in national comparison and institutional research assessment. Scientometrics. 2014;101:939–952. https://doi.org/10.1007/s11192-014-1247-1

3. El-Ouahi J. Scientometric rules as a guide to transform science systems in the Middle East and North Africa. Scientometrics. 2024;129:869–888. https://doi.org/10.1007/s11192-023-04916-x

4. Cobo M. J., López-Herrera A. G., Herrera-Viedma E., Herrera F. Science mapping software tools: Review, analysis, and cooperative study among tools. Journal of the American Society for Information Science and Technology. 2011;62(7):1382–1402. https://doi.org/10.1002/asi.21525

5. Thelwall M. The pros and cons of the use of altmetrics in research assessment. Scholarly Assessment Reports. 2020;2(1):1–9. https://doi.org/10.29024/sar.10

6. Putera P. B., Iriany I. S., Gustina A., Pasciana R., Wismayanti Y. F., Qatrunnada N. Weather, climate, and society: A retrospective between 2009 and 2022 using bibliometric and content analysis. Weather, Climate, and Society. 2024;16(4):651–672. https://doi.org/10.1175/WCAS-D-23-0047.1

7. Bornmann L. Alternative metrics in scientometrics: A meta-analysis of research into three altmetrics. Scientometrics. 2015;103(3):1123–1144. https://doi.org/10.1007/s11192-015-1565-y

8. Lendvai G. F. Valuing diversity, from afar: A scientometric analysis of the Global North countries’ overrepresentation in top communication journals. Communication Studies Review. 2025;4(1):82–108. https://doi.org/10.1515/omgc-2024-0056

9. Albanna B., Handl J., Heeks R. Publication outperformance among Global South researchers: An analysis of individual-level and publication-level predictors of positive deviance. Scientometrics. 2021;126(10):8375–8431. https://doi.org/10.1007/s11192-021-04128-1

10. Turki H., Fraumann G., Hadj Taieb M. A., Ben Aouicha M. Global visibility of publications through digital object identifiers. Frontiers in Research Metrics and Analytics. 2023;8:1207980. https://doi.org/10.3389/frma.2023.1207980

11. Кириллова О. В., Тихонова Е. В. Критерии качества научного журнала: измерение и значимость. Научный редактор и издатель. 2022;7(1):12-27. https://doi.org/10.24069/SEP-22-39

12. Ismail S., Nason E., Marjanovic S., Grant J. Bibliometrics as a tool for supporting prospective R&D decision-making in the health sciences: Strengths, weaknesses and options for future development. Santa Monica, CA: RAND Corporation; 2009. Available from: https://www.rand.org/pubs/technical_reports/TR685.html (accessed: 20.05.2025).

13. Bornmann L. Do altmetrics point to the broader impact of research? An overview of benefits and disadvantages of altmetrics. Journal of Informetrics. 2014;8(4):895–903. https://doi.org/10.1016/j.joi.2014.09.005

14. Senthil R., Anand T., Somala C. S., Saravanan, K. M. Bibliometric analysis of artificial intelligence in healthcare research: Trends and future directions. Future Healthcare Journal. 2024;11(3):100182. https://doi.org/10.1016/j.fhj.2024.100182

15. Sahar R., Munawaroh M. Artificial intelligence in higher education with bibliometric and content analysis for future research agenda. Discover Sustainability. 2025;6:401. https://doi.org/10.1007/s43621-025-01086-z

16. Ellegaard O. The application of bibliometric analysis: disciplinary and user aspects. Scientometrics. 2018;116(1):181–202. https://doi.org/10.1007/s11192-018-2765-z

17. Mallapaty S. China bans cash rewards for publishing papers. Nature. 2020;579(7797):18. https://doi.org/10.1038/d41586-020-00574-8

18. Zhu H., Shuhuai L. Instant prediction of scientific paper cited potential based on semantic and metadata features: Taking artificial intelligence field as an example. PLoS ONE. 2024;19(12):1–20. https://doi.org/10.1371/journal.pone.0312945

19. Hicks D., Wouters P., Waltman L., de Rijcke S., Rafols I. Bibliometrics: The Leiden Manifesto for research metrics. Nature. 2015;520:429–431. https://doi.org/10.1038/520429a

20. Wilsdon J. The metric tide: Report of the independent review of the role of metrics in research assessment and management. 2015. SAGE Publications. https://doi.org/10.4135/9781473978782

21. Blockmans W., Engwall L., Weaire D. Bibliometrics: Use and Abuse in the Review of Research Performance. London: Portland Press; 2015.

22. Asubiaro T., Onaolapo S., Mills D. Regional disparities in Web of Science and Scopus journal coverage. Scientometrics. 2024;129(3):1469–1491. https://doi.org/10.1007/s11192-024-04948-x

23. Sugimoto C. R., Robinson-Garcia N., Murray D. S., Yegros-Yegros A., Costas R., Larivière V. Scientists have most impact when they’re free to move. Nature. 2017;550:29–31. https://doi.org/10.1038/550029a

24. Põder E. What is wrong with the current evaluative bibliometrics? Frontiers in Research Metrics and Analytics. 2022;6:824518. https://doi.org/10.3389/frma.2021.824518

25. Ioannidis J. P. A., Maniadis Z. Quantitative research assessment: Using metrics against gamed metrics. Internal and Emergency Medicine. 2024;19(1):39–47. https://doi.org/10.1007/s11739-023-03447-w

26. Falagas M. E., Alexiou V. G. Editors may inappropriately influence authors’ decisions regarding selection of references in scientific articles. International Journal of Impotence Research. 2007;19(5):443–445. https://doi.org/10.1038/sj.ijir.3901583

27. Ioannidis J. P. A., Thombs B. D. A user’s guide to inflated and manipulated impact factors. European Journal of Clinical Investigation. 2019;49(9):e13151. https://doi.org/10.1111/eci.13151

28. Norris S. Bibliometrics and research evaluation: uses and abuses (Book review). Journal of Librarianship and Scholarly Communication. 2019;7(1):eP2286. https://doi.org/10.7710/2162-3309.2286

29. Ioannidis J. P. A., Maniadis Z. In defense of quantitative metrics in researcher assessments. PLoS Biology. 2023;21(12):e3002408. https://doi.org/10.1371/journal.pbio.3002408

30. Tahamtan I., Bornmann L. Altmetrics and societal impact measurements: Match or mismatch? A literature review. Profesional de la Información. 2020;29(1):1–29. https://doi.org/10.3145/epi.2020.ene.02

31. Szomszor M., Adie E. Overton: A bibliometric database of policy document citations. Quantitative Science Studies. 2022;3(3):624–650. https://doi.org/10.1162/qss_a_00204

32. Morgan-Thomas A., Tsoukas S., Dudau A., Gąska P. Beyond declarations: Metrics, rankings and responsible assessment. Research Policy. 2024;53(10):105093. https://doi.org/10.1016/j.respol.2024.105093

33. Donthu N., Kumar S., Mukherjee D., Pandey N., Lim W. M. How to conduct a bibliometric analysis: An overview and guidelines. Journal of Business Research. 2021;133:285–296. https://doi.org/10.1016/j.jbusres.2021.04.070

34. Ma L. Metrics and epistemic injustice. Journal of Documentation. 2022;78(7):392–404. https://doi.org/10.1108/JD-12-2021-0240

35. Mondal H., Deepak K. K., Gupta M., Kumar R. The h-index: Understanding its predictors, significance, and criticism. Journal of Family Medicine and Primary Care. 2023;12(11):2531–2537. https://doi.org/10.4103/jfmpc.jfmpc_1613_23

36. Vivek N., Clark E., Gao L., Xu S., Baskauf S., Nguyen K., Goldin M., Prasad K., Miller A., Zhang P., Yang S., Rohde S., Topf M., Gelbard A. Social network analysis as a new tool to measure academic impact of physicians. Laryngoscope Investigative Otolaryngology. 2025;10(1):e70060. https://doi.org/10.1002/lio2.70060

37. O’Connor S. The San Francisco Declaration on Research Assessment (DORA) in nursing science. Nursing Science Quarterly. Available from: http://hdl.handle.net/10147/634330 (accessed: 20.05.2025).

38. Montazerian M., Shaghaei N., Drachen T. M., Dorch B. F. Editorial: Quality and quantity in research assessment: examining the merits of metrics. volume II. Frontiers in Research Metrics and Analytics. 2024;9:1400009. https://doi.org/10.3389/frma.2024.1400009

39. Saroyan A. Fostering creativity and critical thinking in university teaching and learning: Considerations for academics and their professional learning. OECD Education Working Papers. 2022;(280). https://doi.org/10.1787/09b1cb3b-en

40. Genderjahn S., Bertelmann R., Ferguson L.M., zu Castell W., Dransch D., Juckeland G. et al. Helmholtz Open Science Briefing: Helmholtz Open Science Forum “Research Evaluation, Reputation Systems, and Openness”. Potsdam: Helmholtz Open Science Office; 2023. https://doi.org/10.48440/os.helmholtz.065

41. Irawan D. E., Abraham J., Tennant J. P., Pourret O. The need for a new set of measures to assess the impact of research in earth sciences in Indonesia. European Science Editing. 2021;47:e59032. https://doi.org/10.3897/ese.2021.e59032

42. Bornmann L., Leydesdorff L. Scientometrics in a changing research landscape. EMBO Reports. 2014;15(12):1228–1232. https://doi.org/10.15252/embr.201439608

43. Ioannidis J. P. A., Pezzullo A. M., Cristiano A., Boccia S., Baas J. Linking citation and retraction data reveals the demographics of scientific retractions among highly cited authors. PLoS Biology. 2025;23(1):e3002999. https://doi.org/10.1371/journal.pbio.3002999

44. Safder I., Hassan S. U. Bibliometric-enhanced information retrieval: A novel deep feature engineering approach for algorithm searching from full-text publications. Scientometrics. 2019;119(1):257–277. https://doi.org/10.1007/s11192-019-03025-y

45. Kirchik O., Gingras Y., Larivière V. Changes in publication languages and citation practices and their effect on the scientific impact of Russian science (1993–2010). Journal of the American Society for Information Science and Technology. 2012;63(7):1411–1419. https://doi.org/10.1002/asi.22642

46. Pislyakov V., Shukshina E. Measuring excellence in Russia: Highly cited papers, leading institutions, patterns of national and international collaboration. Journal of the Association for Information Science and Technology. 2014;65(11):2321–2330. https://doi.org/10.1002/asi.23093

47. Raitskaya L., Tikhonova E. Pressure to publish internationally: Scholarly writing coming to the fore. Journal of Language and Education. 2020;6(1):4–7. https://doi.org/10.17323/jle.2020.10631

48. van Eck N. J., Waltman L. Citation-based clustering of publications using CitNetExplorer and VOSviewer. Scientometrics. 2017;111(2):1053–1070. https://doi.org/10.1007/s11192-017-2300-7