14-Feb-2013 - Max-Planck-Gesellschaft zur Förderung der Wissenschaften e.V.

New citation indicators needed to measure research performance

How do you compare the impact of a researcher in chemistry or physics with a molecular biologist who may be working on similar projects? Two experts support the use of citation indicators that are based on percentiles, a statistical parameter that allows for comparisons with a carefully defined group of reference data. Journal impact factors and h-index alone do not make the grade.

“We argue that new citation impact indicators are needed and that these indicators should allow the comparison of the observed impact for a given publication set with a reference set of similar publications,” said Lutz Bornmann, a sociologist of science at the Division for Science and Innovation Studies at the Max Planck Society in Munich. “This is a better way to make meaningful assessments of scientific work.” He added: “Indicators must also take into account that the distribution of citations across papers is often skewed. The use of percentiles described in our paper provides a solution.”

A scientific paper and its citation in other papers reveal how science makes progress and can help to establish the credit for new discoveries. Citation indicators are used extensively to evaluate the impact of the research of individual scientists, institutes and university departments, and sometimes even countries. The outcomes of such analyses can have a profound influence on the careers of scientists or, for example, the future funding of research institutes.

The authors of the study emphasize that journal impact factors and h-index, two frequently used parameters to evaluate the quality and quantity of science, do not readily allow for comparisons between the work of different scientists or publications in journals from unrelated research fields. The problem is similar to comparing the performance of individuals who score goals in sports as different as football and handball. The number of goals in football is often around 2 but in handball can reach 20 or more. Is someone who scores an average of five goals in handball a better athlete than a football player who scores one goal a game?

“Experts of bibliometrics avoid using journal impact factors and the h-index because these indicators do not provide normalized values. For many years, they have been using reference sets to normalize the number of citations which is indispensable for fair assessment,” commented Werner Marx, co-author of the paper and head of the Information Retrieval Services at the Max Planck Institute for Solid State Research in Stuttgart. “The use of this type of advanced analysis is growing. However, we have an opportunity to do more and this could have a big impact on the way we answer important questions such as how good research really is.”

Max-Planck-Gesellschaft zur Förderung der Wissenschaften e.V.

Request information now

Recommend news PDF version / Print

Share on

Facts, background information, dossiers
  • EMBO
  • Max-Planck-Gesellschaft
  • MPI für Festkörperforschung
More about Max-Planck-Gesellschaft
More about European Molecular Biology Organization
More about MPI für Festkörperforschung
  • News

    The Higgs boson and superconductivity

    Without the Higgs mechanism, particles would have no mass. The Higgs boson, which was discovered in 2012, is therefore also referred to as the “God particle”. It arises as an oscillating excitation of the Higgs field, which penetrates the world. Superconductivity displays similar properties ... more

    An ultrafast microscope for the quantum world

    Processes taking place inside tiny electronic components or in molecules can now be filmed at a resolution of a few hundred attoseconds and down to the individual atom. The operation of components for future computers can now be filmed in HD quality, so to speak. Manish Garg and Klaus Kern, ... more

    Sensor the size of a nitrogen atom investigates hard drives

    Integrated circuitry is becoming increasingly complex. These days a Pentium processor contains some 30 million transistors. And the magnetic structures found in hard drives measure just 10 to 20 nanometers across – less than a flu virus at 80 to 120 nanometers in diameter. Dimensions are ra ... more