Scientific Research Funding Criteria: An Empirical Study of Peer Review and Scientometrics

Authors

Grigoriev O. Devyatkin D.

Annotation

In this paper we investigated the problem of scientific research funding from the perspective of data-mining. The object was to conduct versatile retrospective analysis of decisions made by the Russian Foundation for Basic Research regarding scientific research funding. The central task of the analysis was to compare the impact of various items of information on final decision making. In other words, we tried to answer two questions: (a) what does an evaluation committee mainly look at when it selects projects for funding; (b) are scientometric indicators (or science metrics) useful in decision analysis? To achieve this, we built predictive models (classifiers), performed introspection (extracted feature importance) and compared them. The input data was a set of review forms (questionnaires) from the Russian Foundation for Basic Research completed in by peer reviewers. Final decision is made by the foundation board (an evaluation committee). Finally, we concluded that the available input (project proposals, expert assessments and scientometric data) was not enough to explain all the decisions. We showed that scientometric data does not have any significant influence on project proposals assessment. It also means that h-index, mean impact factor, publication and citation number cannot supersede the peer review procedure.

External links

DOI: http://dx.doi.org/10.1007/978-3-319-78437-3_12

Article in the Studies in Systems, Decision and Control journal at SpringerLink (PDF): https://link.springer.com/content/pdf/10.1007%2F978-3-319-78437-3.pdf

ResearchGate: https://www.researchgate.net/publication/326217388_Scientific_Research_Funding_Criteria_An_Empirical_Study_of_Peer_Review_and_Scientometrics

Semantic Scholar: https://api.semanticscholar.org/CorpusID:65688935

Reference link

Devyatkin, D., Suvorov, R., Tikhomirov, I., Grigoriev, O. Scientific Research Funding Criteria: An Empirical Study of Peer Review and Scientometrics (2018) Studies in Systems, Decision and Control, 140, pp. 277-292.