Can we quantify research performance and rank scientists?

The greatest danger of promoting the H-index as the sole criterion of one’s academic competence is that it discourages undertaking time consuming and challenging investigations, taking years and decades to yield publishable results. One groundbreaking discovery after a decade of work might produce only one paper! Today, in universities and research institutions, the H-index is often considered a factor in deciding recruitments, promotions and base evaluations. This procedure compels the staff of academic and research institutions to take up less stimulating projects having the potential to obtain immediate results.

By Prof. Kirthi Tennakone

Scientists toil day and night and infrequently obtain original results worth dissemination and scrutiny by peers. For those reasons, they publish their findings in scholarly journals. Most journals print the submissions after strict review – a reasonably good filtering process not entirely foolproof. Sometimes top-quality papers get rejected because the assessment is naturally subjective. Around ten reports related to the work that subsequently won the Nobel Prize have been rejected in the first instance. Similarly, flawed and marginal papers get printed in widely circulated journals

Today, the quantity and more importantly the quality of the work of a researcher or a university teacher is determined by his or her publications.

How would you assess the quality of a published article? The indication that your article is widely read and cited by other authors reflects its quality. Thus, the number of citations in some way measures the quality of a publication. To represent both the quantity and quality of someone’s research in terms of one number, a quantity termed the H-Index (named after its inventor J. E. Hirsch, a physicist at the University of California, San Diago) has been designed and widely adopted. It is the maximum number of papers an author has to his credit cited at least the same number of times.

If 20 is the maximum number of articles you have published that are cited at least 20 times each, then your H-index is 20. Giving one number to assess your research allows people to rank you as a scientist. Can we quantify such a complex attribute, the capacity to pursue original investigation in terms of one parameter? Just as the beauty of a woman cannot be measured and expressed as a single number.

There are other pitfalls in assigning H-index as a unique measure of scientific performance. Most papers today are multi-authored, calculation of the H-index does not take this into account and the weightage of contribution by each author. Those who establish collaborations with major foreign research groups have the advantage of getting co-authored papers more frequently cited. In some instances, financial contributions instead of merit enable researchers to join major collaborative efforts and a minor input suffices to get co-authorship. Publication in open access journals by paying fee also helps to boost H-Index, because the contents of these journals free available in the web.

The greatest danger of promoting the H-index as the sole criterion of one’s academic competence is that it discourages undertaking time consuming and challenging investigations, taking years and decades to yield publishable results. One groundbreaking discovery after a decade of work might produce only one paper!

Today, in universities and research institutions, the H-index is often considered a factor in deciding recruitments, promotions and base evaluations. This procedure compels the staff of academic and research institutions to take up less stimulating projects having the potential to obtain immediate results.

A few science publication data analysis systems regularly disclose their ranking of scientists worldwide based on their H-indices. The result of the so-called Alper -Doger Scientific Indexing system was released about a week ago. Some sectors of the scientific community seem to have reacted to it as if the results of competition have been released, sometimes distorting and misinterpreting the report.

The public should be made aware that H-index is not necessarily a measure of the caliber of a researcher. Those who do challenging and time-consuming work also deserve credit. The purpose of research is not building the H-Index, it is incidental and may or may not reflect continuation good science.

Author can be reached via email: ktenna@yahoo.co.uk