Data and reproducibility. Interpretation of international guidelines and journals’ best policies and practices

Relevance. The 2022 Update of the COPE, DOAJ, OASPA, and WAME joint guidelines on The Principles of Transparency and Best Practice in Scholarly Publishing encourages journals to establish their own policy in “data sharing and reproducibility” (DS&R). However, this document neither provides detailed recommendations / templates nor explains the reproducibility crisis phenomenon.

Objectives. To analyze and interpret the international guidelines, the best practices of global publishers and journals, as well as typical mistakes and experience of selected Russian journals, to help a journal develop its own DS&R policy and its implementation.

Materials and methods. The analysis of various sources (literature, reporting guidelines, data repositories), policies of 83 Russian university journals, as well as policies of the top 5 international publishers and their journals. Interviews with 6 editors-in-chief of Russian journals regarding DS&R.

Results. All the top 5 global publishers in their DS&R policy adapt the TOP Guidelines and offer their own data sharing statement templates. Discussion and interpretations. The author suggests Russian translation of the TOP Guidelines and the main templates (e.g., data sharing statement). He also discusses 9 best journal policies and practices (including pre-registration studies).

Conclusions. Numerous international sources, as well as the experience of selected Russian journals, demonstrate that the implementation of the DS&R policy increases articles citation (averagely by 25.3%), the growth of journal’s bibliometric and altmetric indicators, and also contributes to the trust of the target audience. As a result, it strengthes the journal portfolio to enable publishing articles well ahead of schedule. However, only the declarative statement of DS&R policies by journals without proper implementation does not bring tangible benefits to the journals.

1. Hrynaszkiewicz I. Publishers’ responsibilities in promoting data quality and reproducibility. In: Bespalov A., Michel M., Steckler T. (eds) Good research practice in non-clinical pharmacology and biomedicine. Handbook of experimental pharmacology. Springer, Cham; 2019. Vol. 257, pp. 319–348. https://doi.org/10.1007/164_2019_290

2. Baker M. 1,500 scientists lift the lid on reproducibility. Nature. 2016;533:452–454. https://doi.org/10.1038/533452a

3. Lane T. Creating a culture of publication integrity together [Presentation]. In: COPE Forum on good publication practices, University of Macau, Dec. 2, 2022. URL: https://publicationethics.org/sites/default/files/culturepublication-integrity-macau.pdf (accessed 10.12.2022).

4. John L.K., Loewenstein G., Prelec D. Measuring the prevalence of questionable research practices with incentives for truth telling. Psychological Science. 2012;23(5):524–532. https://doi.org/10.1177/0956797611430953

5. Fanelli D. How many scientists fabricate and falsify research? A systematic review and meta-analysis of survey data. PLoS ONE. 2009;4(5):e5738. https://doi.org/10.1371/journal.pone.0005738

6. Berman F., Crosas M. The research data alliance: Benefits and challenges of building a community organization. Harvard Data Science Review. 2020;2(1). https://doi.org/10.1162/99608f92.5e126552

7. Hofman J.M., Goldstein D.G., Sen S., Poursabzi-Sangdeh F., Allen J., Dong L.L. et al. Expanding the scope of reproducibility research through data analysis replications. Organizational Behavior and Human Decision Processes. 2021;164:192–202. https://doi.org/10.1016/j.obhdp.2020.11.003

8. Targowski P., Iwanicka M., Sylwestrzak M., Frosinini C., Striova J., Fontana R. Using optical coherence tomography to reveal the hidden history of The Landsdowne Virgin of the Yarnwinder by Leonardo da Vinci and studio. Angewandte Chemie International Edition. 2018;57(25):7396–7400. https://doi.org/10.1002/anie.201713356

9. Taichman D.B., Sahni P., Pinborg A., Peiperl L., Laine C., James A. et al. Data sharing statements for clinical trials: A requirement of the international committee of medical journal editors. JAMA. 2017;317(24):2491–2492. https://doi.org/10.1001/jama.2017.6514

10. Bergeat D., Lombard N., Gasmi A., Le Floch B., Naudet F. Data sharing and reanalyses among randomized clinical trials published in surgical journals before and after adoption of a data availability and reproducibility policy. JAMA Netw Open. 2022;5(6):e2215209. https://doi.org/10.1001/jamanetworkopen.2022.15209

11. Fraggetta F., Yagi Y., Garcia-Rojo M., Evans A.J., Tuthill J.M., Baidoshvili A. et al. The importance of eSlide Macro images for primary diagnosis with whole slide imaging. Journal of Pathology Informatics. 2018;9(1):46. https://doi.org/10.4103/jpi.jpi_70_18

12. Stuart D., Baynes G., Hrynaszkiewicz I., Allin K., Penny D., Lucraft M., et al. Whitepaper: Practical challenges for researchers in data sharing figshare. Journal Contribution. 2018; 21 Mar. https://doi.org/10.6084/m9.figshare.5975011.v1

13. Hansson K., Dahlgren A. Open research data repositories: Practices, norms, and metadata for sharing images. Journal of the Association for Information Science and Technology. 2021;73(2):303–316. https://doi.org/10.1002/asi.24571

14. Fennell C., Cousijn H. Supporting data openness, transparency & sharing: Elsevier signs up to TOP guidelines & develops new data-sharing guidelines for journals. Elsevier Connect, Sept. 4, 2017. URL: https://www.elsevier.com/connect/editors-update/supporting-openness,-transparency-and-sharing (accessed 10.12.2022).

15. Witt M., Weisweiler N., Ulrich R. Happy 10th Anniversary, re3data! DataCite Blog, Aug. 30, 2022. https://doi.org/10.5438/mqw0-yt07

16. Soderberg C.K. Using OSF to share data: A step-by-step guide. Practices in Psychological Science. 2018;1(1):115–120. https://doi.org/10.1177/2515245918757689

17. Christensen G., Dafoe A., Miguel E., Moore D.A., Rose A. K. A study of the impact of data sharing on article citations using journal policies as a natural experiment. PLoS ONE. 2019;14(12):e0225883. https://doi.org/10.1371/journal.pone.0225883

18. McGillivray B., Marongiu P., Pedrazzini N., Ribary M., Wigdorowitz M., Zordan E. Deep impact: A study on the impact of data papers and datasets in the humanities and social sciences. Publications. 2022;10(4):39. https://doi.org/10.3390/publications10040039

19. Colavizza G., Hrynaszkiewicz I., Staden I., Whitaker K., McGillivray B. The citation advantage of linking publications to research data. PLoS ONE. 2020;15(4): e0230416. https://doi.org/10.1371/journal.pone.0230416

20. Dorch S. B. F., Drachen T.M., Ellegaard O. The data sharing advantage in astrophysics. arXiv:1511.02512. https://doi.org/10.48550/arXiv.1511.02512

21. Henneken E.A., Accomazzi A. Linking to Data – Effect on Citation Rates in Astronomy. arXiv:1111.3618. https://doi.org/10.48550/arXiv.1111.3618

22. Sears J. Data sharing effect on article citation rate in paleoceanography. Figshare. Dataset. Oct. 31, 2014. https://doi.org/10.6084/m9.figshare.1222998.v1

23. Piwowar H.A., Vision T.J. Data reuse and the open data citation advantage. PeerJ. 2013;1(3):e175. https://doi.org/10.7717/peerj.175

24. Vasilevsky N.A., Minnier J., Haendel M.A., Champieux R.E. Reproducible and reusable research: Are journal data sharing policies meeting the mark? PeerJ. 2017;5:e3208. https://doi.org/10.7717/peerj.3208

25. Smith G. How sharing your data could increase your citations. Springer Nature, June 8, 2021. URL: https://researchdata.springernature.com/posts/how-sharing-your-data-could-increase-your-citations (accessed 10.12.2022).

26. Мжельский А.А., Москалева О.В. Научная кооперация как фактор, влияющий на цитируемость статьи. Аналитический обзор. Управление наукой: теория и практика. 2020;2(3):138–164. https://doi.org/10.19181/smtp.2020.2.3.8

27. Nosek B.A., Alter G., Banks G.C., Borsboom D., Bowman S.D., Breckler S. J. et al. Promoting an open research culture: Author guidelines for journals could help to promote transparency, openness, and reproducibility. Science. 2015;348(6242):1422–1425. https://doi.org/10.1126/science.aab2374

28. Nundy S., Kakar A., Bhutta Z. A. How to write the material (patients) and methods section. In: How to practice academic medicine and publish from developing countries? Singapore: Springer; 2022, pp. 213–217. https://doi.org/10.1007/978-981-16-5248-6_19

29. Announcement: Where are the data? Nature. 2016;537:138. https://doi.org/10.1038/537138a

30. Chen Y., Yang K., Marušić A., Qaseem A., Meerpohl J. J., Flottorp S. et al. A reporting tool for practice guidelines in health care: The RIGHT statement. Annals of Internal Medicine. 2017;166:128–132. https://doi.org/10.7326/M16-1565

31. Belletti A., Todaro G., Valsecchi G., Losiggio R., Palumbo D., Landoni G., Zangrillo A. Barotrauma in coronavirus disease 2019 patients undergoing invasive mechanical ventilation: A systematic literature review. Critical Care Medicine. 2022;50(3):491–500. https://doi.org/10.1097/CCM.0000000000005283

32. Huang J., Deng Y., Tin M. S., Lok V., Ngai C. H., Zhang L. et al. Distribution, risk factors, and temporal trends for lung cancer incidence and mortality: A global analysis. Thoracic Oncology. 2022;161(4):1101–1111. https://doi.org/10.1016/j.chest.2021.12.655

33. Gerrits R.G., Jansen T., Mulyanto J., van den Berg M.J., Klazinga N.S., Kringos D.S. Occurrence and nature of questionable research practices in the reporting of messages and conclusions in international scientific Health Services Research publications: a structured assessment of publications authored by researchers in the Netherlands. BMJ Open. 2019;9:e027903. https://doi.org/10.1136/bmjopen-2018-027903

34. Vedantam S. Journals insist drug manufacturers register all trials. Washington Post, Sept. 9, 2004. URL: https://www.washingtonpost.com/archive/politics/2004/09/09/journals-insist-drug-manufacturers-register-alltrials/37be9ce1-23fd-4b07-943c-73336c5375a1/ (accessed 10.12.2022).

35. Smulders Y.M. A two-step manuscript submission process can reduce publication bias. Journal of Clinical Epidemiology. 2013;66(9):946–947. https://doi.org/10.1016/j.jclinepi.2013.03.023

36. Soderberg C.K., Errington T.M., Schiavone S.R., Bottesini J., Thorn F.S., Vazire S. et al. Initial evidence of research quality of registered reports compared with the standard publishing model. Nature Human Behaviour. 2021;5:990–997. https://doi.org/10.1038/s41562-021-01142-4

37. McNutt M.K., Bradford M., Drazen J.M., Hanson B., Howard B., Jamieson K.H., et al. Transparency in authors’ contributions and responsibilities to promote integrity in scientific publication. PNAS. 2018;115(11):2557–2560. https://doi.org/10.1073/pnas.1715374115

38. Adams J., Pendlebury D., Potter R., Szomszor M. Global Research Report: Multi-authorship and research analytics. Clarivate Analytics, ISI, Web of Science Group; 2019. 20 p. URL: https://clarivate.com/wp-content/uploads/2021/02/ISI_-Multiauthorship_Global_Research_Report.pdf (accessed 10.12.2022).

39. Заболотских И.Б., Трембач Н.В., Мусаева Т.С., Дунц П.В., Голубцов В.В., Григорьев Е.В. и др. Национальное многоцентровое проспективное обсервационное исследование «Роль сопутствующих заболеваний в стратификации риска послеоперационных осложнений» – STOPRISK: протокол исследования. Вестник интенсивной терапии имени А. И. Салтанова. 2022;(4):24–35. https://doi.org/10.21320/1818-474X-2022-4-24-35

40. Заболотских И.Б., Мжельский А.А. Что стоит за различными показателями роста [видео]. Пленарное заседание Всероссийской научно-образовательной конференции, посвященной 30-летнему юбилею журнала «Вестник интенсивной терапии имена А.И. Салтанова, 18–19 нояб. 2022 г. URL: https://www.youtube.com/watch?v=O26-VyOiaZI (дата обращения: 12.12.2022).

41. Bird A. Understanding the Replication Crisis as a Base Rate Fallacy. The British Journal for the Philosophy of Science. 2021;72(4):965–993. https://doi.org/10.1093/bjps/axy051