| Retzer, V; Jurasinski, G: Towards objectivity in research evaluation using bibliometric indicators - A protocol for incorporating complexity, Basic and Applied Ecology, 10(5), 393-400 (2009) doi:10.1016/j.baae.2008.09.001 | |
| Abstract: Publications are thought to be an integrative indicator best suited to measure the multifaceted nature of scientific performance. Therefore, indicators based on the publication record (citation analysis) are the primary tool for rapid evaluation of scientific performance. Nevertheless, it has to be questioned whether the indicators really do measure what they are intended to measure because people adjust to the indicator value system by optimizing their indicator rather than their performance. Thus, no matter how sophisticated an indicator may be, it will never be proof against manipulation. A literature review identifies the most critical problems of citation analysis: database-related problems, inflated citation records, bias in citation rates and crediting of multi-author papers. We present a step-by-step protocol to address these problems. By applying this protocol, reviewers can avoid most of the pitfalls associated with the pure numbers of indicators and achieve a fast but fair evaluation of a scientist’s performance. We as ecologists should accept complexity not only in our research but also in our research evaluation and should encourage scientists other disciplines to do so as well. |
