In this paper we investigated the problem of scientific research funding from the perspective of data-mining. The object was to conduct versatile retrospective analysis of decisions made by the Russian Foundation for Basic Research regarding scientific research funding. The central task of the analysis was to compare the impact of various items of information on final decision making. In other words, we tried to answer two questions: (a) what does an evaluation committee mainly look at when it selects projects for funding; (b) are scientometric indicators (or science metrics) useful in decision analysis? To achieve this, we built predictive models (classifiers), performed introspection (extracted feature importance) and compared them. The input data was a set of review forms (questionnaires) from the Russian Foundation for Basic Research completed in by peer reviewers. Final decision is made by the foundation board (an evaluation committee). Finally, we concluded that the available input (project proposals, expert assessments and scientometric data) was not enough to explain all the decisions. We showed that scientometric data does not have any significant influence on project proposals assessment. It also means that h-index, mean impact factor, publication and citation number cannot supersede the peer review procedure.
DOI: http://dx.doi.org/10.1007/978-3-319-78437-3_12
Статья в журнале Studies in Systems, Decision and Control (PDF) на SpringerLink (на англ.): https://link.springer.com/content/pdf/10.1007%2F978-3-319-78437-3.pdf
Devyatkin, D., Suvorov, R., Tikhomirov, I., Grigoriev, O. Scientific Research Funding Criteria: An Empirical Study of Peer Review and Scientometrics (2018) Studies in Systems, Decision and Control, 140, pp. 277-292.