From Metrics to Trust: Rethinking SSH Assessment in the Czech Republic

ERIH PLUS has been with me for the 12 years I have been professionally interested in research evaluation. Given my disciplinary background, I pay particular attention to the Social Sciences and Humanities. Therefore, it was a great honour for me when erih+ recruited me as a National Expert. In this role, I would like to rehabilitate erih+ in the Czech Republic, where it has a somewhat complicated position due to the evolution of research evaluation and funding.

Before 2017, the Czech government assessed and funded research predominantly by scoring eligible outputs. Articles in WoS journals brought the most "profit", but in SSH fields, erih+ journals and journals in the positive list of national peer-reviewed outlets were also valued to a small extent. The quantification of quality and the absence of real research evaluation undermined the importance of strategic management at universities. It was easy to calculate the monetary value of individual publications and thus the profit delivered to the university. The financial implications also changed the very nature of scientific work and distorted the research culture in some fields. Both effects have particularly affected universities, which have typically used publication points to distribute the funds down to the lowest levels. This system was widely criticised and was abandoned in 2017.

Since 2017, a national methodology based predominantly on peer review has been gradually implemented, including not only outputs but also environment, inputs and future strategies in a system of five modules. Two modules focus on outputs: 1) peer review of selected outputs assessing scientific quality or societal impact and 2) performance analysis, based on the distribution of articles in WoS journals of different quartile rank derived from AIS. The full assessment then considers a wide range of issues, and discipline specificities and its overall design corresponds with developed assessment cultures and best practices.

There is however an interesting paradox: erih+ is now perceived as part of a discredited old assessment system and there is no greater appetite for its use at present. Concurrently, the mentally preferred signal of publication performance is predominantly a journal-level metric even though the interpretation of performance analysis is subject to ex-post evaluation in the six disciplinary panels (Frascati FORDs) and is, understandably, declared uninformative in the humanities.

Thus, in the eyes of current performance indicators, SSH are seen as low-performing. Although erih+ seems to be an ideal solution for SSH publishing patterns (it is developed by the SSH community, reforming the process of evaluating journals and responding to open science trends), in the Czech Republic positive lists are perceived as too subjective, susceptible to lobbying and gaming and thus not representing the wanted quality. Perhaps because of the shortcomings of the previous evaluation system. However, paradoxically, also other benefits of different erih+ roles have been reduced by this historical mistrust. The relevance of ERIH PLUS has diminished also for institutional evaluations, which often replicate the rules and expectations of national assessment.

I would like to see research assessment in the Czech Republic abandon questionable ideals of finding objective criteria and measuring productivity and rather include more confidence, and diversity and perhaps redefine its role. I wish that the Czech SSH community would recognise and use erih+'s full potential, which goes beyond research assessment, and I wish to contribute to this awareness.