Preview

Science Editor and Publisher

Advanced search

Journal quality criteria: Measurement and significance

https://doi.org/10.24069/SEP-22-39

Abstract

The landscape of scientific communication imposes obvious requirements on its actors, which are not always unambiguously interpreted by the latter. The basic criterion of scientific communication is its quality. Since scientific journals serve as the main channel for rational debates based on facts, their quality level seems to be the most significant for the development of scientific knowledge. Today, scientific journals are undergoing a massive transformation: from changing the format of content presentation to becoming a body of collective communication. These changes cannot but affect the quality characteristics of journals. Since the changes are multilateral in nature, the scientific, editorial, and publishing communities have not formed an unambiguous opinion about which criteria of the journal characterize its quality aspects, and which can be attributed to “formal”, “technical”. In this article, the authors, based on Russian and international expert opinion, analyze the quality criteria for journals. Based on the data of the questionnaire developed by the authors, which considers the categories and criteria of the Scopus expert system, a survey of editors (n = 130) of Russian scientific journals was conducted. The questionnaire was completed anonymously, and the study participants were made aware of its objectives. The data obtained made it possible to analyze the editors’ ideas about the qualitative characteristics of a scientific journal and their influence on its promotion. A sufficiently deep understanding by the editors of the essence and specifics of the analyzed characteristics was recorded. At the same time, certain aspects require targeted work on their optimization and development. There is also an obvious need for further research on the topic, considering the subject field of journals traditions.

About the Authors

O. V. Kirillova
Association of Science Editors and Publishers
Russian Federation

Olga  V.  Kirillova - Cand.  Sci.  (Eng.),  President  of the Association of Science Editors and Publishers (ASEP), Chair of the Scopus ECSAC-RF, Editor-in-Chief of the journal “Science Editor and Publisher”, member of the Editorial Boards of the international journals «European Science Editing» (EASE, United Kingdom), “Science Editing” (KCSE, South Korea).

Moscow.



E. V. Tikhonova
MGIMO University; RUND University
Russian Federation

Elena V. Tikhonova - Cand. Sci. (Hist.), Associated Professor, RUND University, MGIMO University, Deputy Editor-in-Chief and Editorial Manager of the “Journal of Language and Education” (HSE), Chair of the EASE Russian Chapter.

Moscow.



References

1. Lambovska M., Todorova D. ‘Publish and Flourish’ instead of ‘Publish or Perish’: A motivation model for top-quality publications. Journal of Language and Education. 2021;7(1):41–155. https://doi.org/10.17323/jle.2021.1152

2. Rowley J., Sbaffi L., Sugden M., Gilbert A. Factors influencing researchers’ journal selection decisions. Journal of Information Science. 2022;48(3):321–335. https://doi.org/10.1177/0165551520958591

3. Nygaard L. P. Publishing and perishing: an academic literacies framework for investigating research productivity. Studies in Higher Education. 2017;42(3):519–532. https://doi.org/10.1080/03075079.2015.1058351

4. Schoenwolf G. C. Getting published well requires fulfilling editors’ and reviewers’ needs and desires. Development. Growth. Differentiation. 2013;55:735–743. https://doi.org/10.1111/dgd.12092

5. Рашби Н. Д. Контроль качества редакторской работы в научных журналах. Научный редактор и издатель. 2018;3(1–2):6–12. https://doi.org/10.24069/2542-0267-2018-1-2-6-12

6. Mcculloch S. Hobson’s choice: The effects of research evaluation on academics’ writing practices in England. Aslib Journal of Information Management. 2017;69(5):503–515. https://doi.org/10.1108/AJIM-12-2016-0216

7. Gaston T.E., Ounsworth F., Senders T., Ritchie S., Jones M. Factors affecting journal submission numbers: Impact factor and peer review reputation. Learned Publishing. 2020;33(2):154–162. https://doi.org/10.1002/leap.1285

8. Tennant J. P., Crane H., Crick T., Davila J., Enkhbayar A., Havemann J. et al. Ten hot topics around scholarly publishing. Publications. 2019;7(2):34. https://doi.org/10.3390/publications7020034

9. Curry S. Let’s move beyond the rhetoric: It’s time to change how we judge research. Nature. 2018;554:147. https://doi.org/10.1038/d41586-018-01642-w

10. McKiernan E.C., Schimansk L. A., Nieves C. M., Matthias L., Niles M. T., Alperin J. P. Meta-Research: Use of the journal impact factor in academic review, promotion, and tenure evaluations. eLife. 2019;8:e47338. https://doi.org/10.7554/eLife.47338

11. Hicks D., Wouters P., Waltman L., de Rijcke S., Rafols I. Bibliometrics: The Leiden Manifesto for research metrics. Nature. 2015;520(7548):429–431. https://doi.org/10.1038/520429a

12. Kim K., Chung Y. Overview of journal metrics. Science Editing. 2018;5(1):16–20. https://doi.org/10.6087/kcse.112

13. Fong E. A., Wilhite A. W. Authorship and citation manipulation in academic research. PLoS ONE. 2017;12(12):e0187394. https://doi.org/10.1371/journal.pone.0187394

14. Tennant J. P., Crane H., Crick T., Davila J., Enkhbayar A., Havemann J. et al. Ten hot topics around scholarly publishing. Publications. 2019;7(2):34. https://doi.org/10.3390/publications7020034

15. James C., Colledge L., Meester W., Azoulay N., Plume A. CiteScore metrics: Creating journal metrics from the Scopus citation index. Learned Publishing. 2019;32(4):367–374. https://doi.org/10.1002/leap.1246

16. Colledge L., De Moya-Anegón F., Guerrero-Bote V., López-Illescas C., El Aisati M., Moed H. F. SJR and SNIP: Two new journal metrics in Elsevier’s Scopus. Serials. 2010;23(3):215–221. https://doi.org/10.1629/23126

17. Колледж Л., Джеймс К. «Корзина метрик» – лучшее средство для оценки авторитета журнала. Научный редактор и издатель. 2016;1(1-4):25–31. https://doi.org/10.24069/2542-0267-2016-1-4-25-31

18. Gorraiz J., Gumpenberger C. PlumX metrics (Plum Analytics) in practice. Handbook Bibliometrics. Berlin: De Gruyter Saur. 2020. https://doi.org/10.1515/9783110646610-023

19. Джоши Я. Государственная публикационная политика и развитие научных журналов в Индии: достижения и проблемы = Joshi Ya. State publication policy and development of academic journals in India: achievements and concerns: доклад на 10-й Международной научно-практической конференции «Научное издание международного уровня – 2022: от настоящего к будущему», пленарное заседание, г. Москва, 26 апр. 2022 г. URL: https://rassep.ru/academy/biblioteka/111859/ (дата обращения: 10.07.2022). (На англ. яз.).

20. Vessuri H., Guédon J.-C., Cetto A. M. Excellence or quality? Impact of the current competition regime on science and scientific publishing in Latin America and its implications for development. Current Sociology. 2014;62:647–665. https://doi.org/10.1177%2F0011392113512839

21. Moore S., Neylon C., Paul Eve M., Paul O’Donnell D., Pattinson D. Excellence R Us”: University research and the fetishisation of excellence. Palgrave Communications. 2017;3:16105. https://doi.org/10.1057/palcomms.2016.105

22. Truman S. E. Undisciplined: Research-creation and what it may offer (traditional) qualitative research methods. Qualitative Inquiry. 2022. https://doi.org/10.1177/10778004221098380

23. Creswell J. W. Qualitative inquiry and research design: Choosing among five traditions. 2nd ed. Thousand Oaks, CA: Sage; 2007. 488 р.

24. Denzin N. K., Lincoln Y.S. (eds). The SAGE Handbook of Qualitative Research. 3rd ed. Sage Publications Ltd.; 2005. 1232 p.

25. Tracy S. J. Qualitative quality: Eight a “big-tent” criteria for excellent qualitative research. Qualitative Inquary. 2010;16(10):837–51. https://doi.org/10.1177%2F1077800410383121

26. Coghlan D., Shani A. B. Creating action research quality in organization development: rigorous, reflective and relevant. Systemic Practice and Action Research. 2014;27:523–536. https://doi.org/10.1007/s11213-013-9311-y

27. Fraser H., Parker T., Nakagawa S., Barnett A., Fidler F. Questionable research practices in ecology and evolution. PLoS ONE. 2018;13(7):e0200303. https://doi.org/10.1371/journal.pone.0200303.t001

28. VanderKaay S., Moll S. E., Gewurtz R. E., Jindal P., Loyola-Sanchez A., Packham T.L., Lim C. Y. Qualitative research in rehabilitation science: Opportunities, challenges, and future directions. Disability and Rehabilitation. 2018;40(6):705–713. https://doi.org/10.1080/09638288.2016.1261414

29. Navalta J.W., Stone W.J., Lyons T.S. Ethical issues relating to scientific discovery in exercise science. Discovery in Exercise Science. 2019;12(1):1–8. URL: https://digitalcommons.wku.edu/ijes/vol12/iss1/1 (accessed: 10.07.2022).

30. Magoon R., Jose J. Safeguarding anaesthesia research from spin. British Journal of Anaesthesia. 2020;125(6):E460–E462. https://doi.org/10.1016/j.bja.2020.08.042

31. Mack A. How to write a good scientific paper. Bellingham, Washington: SPIE; 2018. 108 p. https://doi.org/10.1117/3.2317707.sup

32. Whitfield R., Peters J. Quality in scholarly publishing. Managing Service Quality. 2000;10(3):151–155. https://doi.org/10.1108/09604520010336678

33. Grimes D.R., Bauch C. T., Ioannidis J. Modelling science trustworthiness under publish or perish pressure. Royal Society Open Science. 2018;5(1):171511. https://doi.org/10.1098/rsos.171511

34. Peters M.A., Besley T., Jandrić P., Zhu X. (eds). Knowledge socialism. The rise of peer production: collegiality, collaboration, and collective intelligence. Singapore: Springer; 2020. 325 p. https://doi.org/10.1007/978-981-13-8126-3

35. Peters M. A., Tesar M., Jackson L., Besley T., Jandrić P., Arndt S., Sturm S. (eds). The methodology and philosophy of collective writing. London: Routledge; 2021. 232 p. https://doi.org/10.4324/9781003171959

36. Peters M.A., Jandrić P., Irwin R., Locke K., Devine N., Heraud R. et al. Towards a philosophy of academic publishing. Educational Philosophy and Theory. 2016;48(14):1401–1425. https://doi.org/10.1080/00131857.2016.1240987

37. Besley T. Postdigital scholarly publishing. Postdigital Science and Education. 2021;3(3):654–659. https://doi.org/10.1007/s42438-021-00248-2

38. Baas J., Schotten M., Plume A. M., Cote G., Karimi R. Scopus as a curated, high-quality bibliometric data source for academic research in quantitative science studies. The MIT Press Journals. 2020;1(1):377–386. https://doi.org/10.1162/qss_a_00019

39. Богоров В. Г. Web of Science: критерии качества научного журнала: доклад на Международной научно-практической конференции «Обнаружение заимствований – 2020», 22 окт. 2021 г. URL: https://www.youtube.com/watch?v=nyEeX_7vnT8 (дата обращения: 10.07.2022).

40. Филиппов Ю.И. Индексация российских биомедицинских журналов в базе данных MEDLINE и на платформе PubMed: анализ позитивного и негативного опыта. Научный редактор и издатель. 2021;6(1):28–47. https://doi.org/10.24069/2542-0267-2021-1-28-47

41. Кириллова О.В., Андронова М. Б., Дивильковская Т.Ю., Хачко О.А. Новые подходы и результаты оценки информационным центром российского потока научных журналов: критерии и представление ранжированных данных. Образовательные технологии и общество. 2006;9(3);321–334. URL: https://readera.org/14062098 (дата обращения: 10.07.2022).

42. Алексеев В. М., Домнина Т.Н., Кириллова О.В., Солошенко Н. С., Хачко О.А. Зарубежные и российские научные электронные издания по приоритетным направлениям и критическим технологиям. Научно-техническая информация. Серия 1: Организация и методика информационной работы. 2008;(9):29–34.

43. Кириллова О.В. Экспертная оценка российских журналов по критериям Scopus: первый опыт РЭС. В сб.: Научное издание международного уровня – 2016: решение проблем издательской этики, рецензирования и подготовки публикаций: материалы 5-й Междунар. науч.-практ. конф., г. Москва, 17–20 мая 2016 г. Екатеринбург: УрГУ им. А.М. Горького; 2016. С. 148–154.

44. Кириллова О.В. Первый опыт организации работы журналов в направлении их развития при господдержке: достижения и трудности. В сб.: Научное издание международного уровня – 2015: современные тенденции в мировой практике редактирования, издания и оценки научных публикаций: материалы 4-й Междунар. науч.-практ. конф., г. Санкт-Петербург 26–29 мая 2015 г. СПб.: Сев.-Зап. ин-т упр. – фил. РАНХиГС; 2015. С. 67–69.

45. Кириллова О. В., Кузнецов А. Ю., Диментов А. В., Лебедев В. В., Шварцман М. Е. Категории и критерии оценки российских журналов и программы их развития. Научная периодика: проблемы и решения. 2014;(5):20–34. https://nppir.ru/PDF/03NP514.pdf (дата обращения: 10.07.2022).

46. Кириллова О.В. Редакционная подготовка научных журналов по международным стандартам: Рекомендации эксперта БД Scopus. М.; 2013. Ч. 1. 90 с. URL: https://rassep.ru/academy/biblioteka/106592/ (дата обращения: 10.07.2022).

47. Примеры типичных комментариев, используемых экспертами при обосновании их решения об удовлетворении / отклонении заявки на включение журнала в МНБД Scopus. Научный редактор и издатель. 2018;3(1-2):73–79. https://doi.org/10.24069/2542-0267-2018-1-2-73-79

48. Harvey L., Green P. Defining quality. Assessment and Evaluation in Higher Education. 1993;18(1):9–34. https://doi.org/10.1080/0260293930180102

49. Ibrahim A. M., Lillemoe K. D., Klingensmith M. E., Dimick J. B. Visual abstracts to disseminate research on social media: A prospective. Case-control crossover study. Annals of Surgery. 2017;266(6):e46–8. https://doi.org/10.1097/sla.0000000000002277

50. Barlow B., Barlow A., Webb A., Cain J. “Capturing your audience”: Analysis of Twitter engagements between tweets linked with an educational infographic or a peer-reviewed journal article. Journal of Visual Communication in Medicine. 2020;43(4):177–183. https://doi.org/10.1080/17453054.2020.1809358

51. Ferreira G. E., Elkins M. R., Jones C., O’Keeffe M., Cashin A. G. et al. Reporting characteristics of journal infographics: a cross-sectional study. BMC Medical Education. 2022;22:326. https://doi.org/10.1186/s12909-022-03404-9

52. Björk B. C. Acceptance rates of scholarly peer-reviewed journals: A literature survey. Profesional de la información. 2019;28(4):e280407. https://doi.org/10.3145/epi.2019.jul.07

53. Subramanyam R. Art of reading a journal article: Methodically and effectively. Journal of oral and maxillofacial pathology. 2013;17(1):65–70. https://doi.org/10.4103/0973-029X.110733

54. Carey M. A., Steiner K. L., Petri W.A. Jr. Ten simple rules for reading a scientific paper. PLoS Computational Biology. 2020;16(7):e1008032. https://doi.org/10.1371/journal.pcbi.1008032

55. Jamali H. R., Nicholas D., Watkinson A., Herman E., Tenopir C., Levine K. et al. How scholars implement trust in their reading, citing and publishing activities: geographical differences. Library & Information Science Research. 2014;36(3–4):192–202. https://doi.org/10.1016/j.lisr.2014.08.002

56. Mabe M. A. Scholarly communication: A long view. New Review of Academic Librarianship. 2010;16(Suppl 1):132–144. https://doi.org/10.1080/13614533.2010.512242

57. Tenopir C., Dalton E., Fish A. et al. What motivates authors of scholarly articles? The importance of journal attributes and potential audience on publication choice. Publications. 2016;4(3):22. https://doi.org/10.3390/publications4030022

58. Dyachenko E. L. Internationalization of academic journals: Is there still a gap between social and natural sciences? Scientometrics. 2014;101(1):241–255. https://doi.org/10.1007/s11192-014-1357-9


Review

For citations:


Kirillova O.V., Tikhonova E.V. Journal quality criteria: Measurement and significance. Science Editor and Publisher. 2022;7(1):12-27. (In Russ.) https://doi.org/10.24069/SEP-22-39

Views: 1439


ISSN 2542-0267 (Print)
ISSN 2541-8122 (Online)