|Year : 2019 | Volume
| Issue : 11 | Page : 3475-3479
Impact factor: Mutation, manipulation, and distortion
Deepak Juyal1, Vijay Thawani2, Ashutosh Sayana3, Shekhar Pal1
1 Department of Microbiology, Govt. Doon Medical College, Dehrakhas, Patelnagar, Dehradun, Uttarakhand, India
2 Department of Pharmacology, People's College of Medical Sciences and Research Centre, Bhapur, Bhopal, Madhya Pradesh, India
3 Principal, Govt. Doon Medical College and, Professor and Head, Department of Surgery, Govt. Doon Medical College, Dehrakhas, Patelnagar, Dehradun, Uttarakhand, India
|Date of Submission||01-Jul-2019|
|Date of Decision||22-Aug-2019|
|Date of Acceptance||09-Oct-2019|
|Date of Web Publication||15-Nov-2019|
Mr. Deepak Juyal
Department of Microbiology, Govt. Doon Medical College, Dehrakhas, Patelnagar, Dehradun - 248 001, Uttarakhand
Source of Support: None, Conflict of Interest: None
Currently, there is an increased dependency on the impact factor in scientific research publications. Sometimes the overzealousness in implementing the impact factor value to a publication is detrimental to the growth of the scientific authors, especially the junior lot, for no fault of theirs. The inept and myopic application of the impact factor defeats the purpose of making the value judgment and hence has been criticized by many learned authors. The scientist Eugene Garfield, who formulated the impact factor, feels wrong that it is being not judiciously used. A corollary is the invention dynamite by Alfred Nobel which instead of helping, aiding, and reducing human effort and endeavor it was more misused for human annihilation and pained the scientist who gave it to the world. The authors reexamine the application of impact factor to scientific manuscripts for rightful application of the value judgment.
Keywords: Predatory journals, Research integrity, Science Citation Index, Thomson Reuters
|How to cite this article:|
Juyal D, Thawani V, Sayana A, Pal S. Impact factor: Mutation, manipulation, and distortion. J Family Med Prim Care 2019;8:3475-9
“In 1955, it did not occur to me that the impact would one day become so controversial. Like nuclear energy, the impact factor is a mixed blessing. I expected it to be used constructively while recognizing that in the wrong hands it might be abused.” –
| Background|| |
The journal impact factor (JIF) has become an important indicator of the quality of research publication. Hence, while considering the research impact, most consider the JIF as a barometer of research. Although it was never intended to be used to evaluate individual scientists, but rather as a measure of the quality of academic journals, JIF has been increasingly misused in this way. Researchers are often ranked on the basis of their publication in the journals with high IF, and in some countries, publication in a journal with an impact factor <5.0 is officially of no value. Science ministries in certain countries offer cash rewards to scientists publishing in high IF journals viz Nature, Science, Cell etc., Thus, it has become imperative for scientists to publish their work in journals with high IF. However, serious concerns have been raised about the use of JIF as a surrogate marker for the quality of research, individual articles, or a researcher itself.,
Impact factor calculation
The JIF was devised by Eugene Garfield in 1955 to help research libraries differentiate between journals when deciding which one to subscribe to. The term, IF was first used in 1961, after publication in Science Citation Index (SCI) in 1963, and the first ranking of the journals on the basis of IF was published in 1972. As a part of SCI and the social sciences citation index, Thomson Reuters began publishing Journal Citation Report (JCR) annually in 1975. The JIF is actually a measure of how frequently the articles published in that journal are cited. The IF of a journal for any specific year is calculated by dividing the total number of citations received by the articles published in the journal during the preceding 2 years (numerator), by the total number of articles published in the same 2 years (denominator). For example, if the IF of a journal is 3.0 in 2015, it reflects that on average the articles published in 2013 and 2014 were cited thrice in the collection of all Thomson Reuters indexed journals published in 2015.
Impact factor distortions
Scientific publications contribute to the dissemination of the research findings and knowledge to improve life. However, these fundamental tenets of research publications have probably been forgotten in IF mania, and misuse of JIF is extensive. Based on the notion that a journal is representative of its articles, publishing in high-IF journal is considered to evaluate the author's scientific achievement. This, in turn, has led to IF-based assessment for the appointment, allocation of research grants, and academic advancement of the researchers., Researchers also tend to publish their manuscripts in journals with high IF and are more concerned about “where they publish, rather than what they publish.” The contradiction is evident; on one hand we want our journals to attain international standards but when it comes to publishing our exemplary findings, we prefer international journals rather than the Indian ones. This takes even longer for our journals to get recognition and improve IF. The 2014 edition of JCR contained 8474 science and technology journals from all over the world, including 98 from India. Among 98 of these Indian journals only two had IF >2.000.
Since the JIF is calculated over a period of 2 years after being indexed in Thomson Reuters, a recently launched journal or a journal not indexed in Thomson Reuters cannot have an IF. Moreover, there are many peer-reviewed journals that are not indexed in Thomson Reuters and, therefore, do not have an IF. Researchers dislike publishing their findings in journals with no or low IF. Taking advantage of such existing IF craze, many agencies have started allocating fake IFs to the journals on payment basis, which may resemble the original IF., These bogus IF agenciesseem to be hand in glove with “predatory journals” displaying fake IFs predominantly on their websites, and we all have our mailboxes filled with their e-mails with a soliciting manuscripts. The sole aim of these dubious journals is to earn from publishing fees. Due to this demand and supply culture, budding researchers and even academic institutions fall prey to them. However, some researchers knowingly use sham publications and fake ” scientometrics” for their academic advancement based on the poor quality articles posing serious threat to the academic standard and integrity.
Impact factor manipulation
Over the years, critics have argued that the JIF, per se, may not reflect anything informative about the quality of empirical research. It is not an appropriate metric to measure the scientific content of individual articles or a scientist's credibility, and if applied to individual researchers, publications, or grants, it exerts an increasingly detrimental influence on the scientific enterprise. There is a great degree of mutation and manipulation in the evaluation of IF, and we enumerate some of these in the following text.
Eugene Garfield, the inventor of the IF never predicted that it would be used in the scientific community as a criterion for judging the quality of a scientist and determining the provision for research grants. Unfortunately, the IF of a journal is not statistically representative of its individual articles and Garfield himself has reported a poor correlation between the IF of a journal and the actual citation rates of its articles. It has been seen that citations of many articles may not peak until after the second year of publication and is beyond the brief period of time considered for calculating IF. In fact, Lariviere and Sugimoto, in their six-point critique of a JIF, explain that a 2-year period for citations could accidentally favor certain disciplines over others. Moreover, JIF can be skewed by publication of more reviews (which tend to be cited more frequently) or by self-citation of the articles. Recent example is of a journal Acta Crystallographica Section A: Foundations of Crystallography (pISSN 0108—7673) which had an IF of 2.051 in 2008, which changed to 49.926 in 2009 and then increased to 54.333 in 2010 and the IF in 2014 was 2.3074, reason being a single review article receiving a large number of citations.
Journals are also under continuous pressure to raise their IF, which can lead to editorial misconduct. Sometimes a journal may request authors to include references from its own previous publications in order to inflate its IF. Such practice was recently brought to light, where three Brazilian journals conspired to cite each other's published papers in a mutual effort to increase their JIF. Moreover, some papers are cited multiple times for negative reasons and yet these negative citations contribute to improving JIF. An article published in Science showed that many studies that have been proven to be fraudulent are not even retracted and continue to be cited.
While calculating the JIF, only original papers and review articles are counted in denominator while all the published materials (editorials, letters to editor, news, book reviews, etc.) including original papers and review articles are accepted in the numerator. This significantly boosts the JIF. Interestingly, even some of the reputed journals such as Nature and Science have been found to do so in order to boost their JIF., Also, the continuous pressure for publication in high-IF journals leads to “performance anxiety” among researchers and they indulge in unethical publication practices (data falsification and fabrication)., Such cases are mostly reported from countries where regulatory bodies demand academic faculty to regularly publish in high-IF journals. The pressure to publish creates a bias that discourages high-risk research and reduces the likelihood of unexpected breakthrough discoveries. Vannevar Bush had commented nearly 70 years ago that “Basic research is performed without thought of practical ends…. Many of the most important discoveries have come as a result of experiments undertaken with very different purposes in mind.”
Suggestions for reforms
Scientist's unhealthy obsession with JIF has been widely criticized, yet many are trapped into this value system when submitting their own work or judging the work of others. A recent study by Madhan et al. pointed out that cumulative JIFs were still being utilized as a criterion for prestigious awards such as the Tata Innovation Fellowship, Innovative Young Biotechnologist Award, National Bioscience Awards for Career Development, and so on. Similarly, the Indian Council of Medical Research routinely uses average JIF as a measure of the performance of its various laboratories.
In order to stop JIF misuse, the researchers should put a halt on the relentless chase for IF and rather focus on the originality and quality of their research work. In this regard, American Society of Microbiology (ASM), on 11th July 2016 announced to remove the IF from its journals and website, as well as from marketing and advertising, a move which was appreciated by many. Of note are prestigious journals such as Nature, Science, The New England Journal of Medicine, The Lancet, etc., which have existed and prospered for long, some even for centuries, before the advent of IF.
The misuse of the JIF as a metric of an individual scientist's or article's importance has been decried in a consensus statement from the San Francisco Declaration of Research Assessment (DORA). The aim of DORA was to put an end to the practice of using JIF as a valuation metric of individual researchers. The declaration states that “the impact factor must not be used as a surrogate measure of the quality of individual research article, to assess an individual scientist's contributions, or in hiring, promotion or funding decisions.” The DORA has made one general and 17 specific recommendations, which are enumerated in [Table 1].
|Table 1: Recommendations made by San Francisco Declaration of Research Assessment (DORA)|
Click here to view
In order to boost the growth of quality research in our country, a policy statement was released by the Indian National Science Academy on Dissemination and Evaluation of Research Output in India. This document elaborately discusses basic policy parameters such as promoting preprint repositories and incorporating quality peer review, minimizing interference caused by predatory journals as well as predatory conferences, policies for categorizing and evaluating research effort and rationalizing payment policies in the Indian scenario. However, such recommendations are yet to be executed, in actuality.
Over the years, several approaches have evolved to address the limitations posed by JIF in the valuation of researchers and research publications. Some of them are enumerated in [Table 2]. However, due to the inherent lacunae, there is no one size that fits all, set of metrics that can assess the credibility of the researchers or their publications. Research organizations should be consistent about which valuation metrics to be used while maintaining the ethos and values of scholarly scientific publishing over the mere accumulation of publications in prestigious journals.
|Table 2: Alternative and diverse evaluation metrics to measure the scientific impact|
Click here to view
| Conclusion|| |
Despite having widespread recognition that the IF is being misused, the misuse continues and is likely to continue because of the diverse confluence of forces within the scientific community that encourage, promote, and perpetuate it. We submit that the JIF remains a relatively crude index for evaluating the quality of a journal, its scientific content or the credibility of a researcher. Doing so will not only affect the research scientists involved but may even discourage ethical research and hamper the overall scientific progress. A comprehensive scientific evaluation of an article requires a multidimensional approach and is beyond the scope of a single metric such as IF. While evaluating the performance of a researcher, academic administrators should focus on contribution and content rather than on publication venue. However, changing the existing culture will be slow since the researchers are so deeply entrenched with JIF mania that it may take time to wean off from its influence. The removal of JIF from websites as done by ASM is a bold step in this direction and should be followed by others with right earnestly. To err is human but to correct is divine.
One must note that the traditional method of evaluation continues to be peer review, and there is no substitute for reading the article for assessing the research worthiness of authors, rather than reading the title of the paper or the title of the journal or its IF.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
Juyal D, Dhawan B, Thawani V, Thaledi S. Falling prey to an impact factor craze. Educ Health 2018;31:189-90.
] [Full text]
Alberts B. Impact factor distortions. Science 2013;340:787.
Al-Awqati Q. Impact factors and prestige. Kidney Int 2007;71:183-5.
Franzoni C, Scellato G, Stephan P. Science policy. Changing incentives to publish. Science 2011;333:702-3.
Ha TC, Tan SB, Soo KC. The journal impact factor: Too much of an impact? Ann Acad Med Singapore 2006;35:911-6.
Garfield E. The history and meaning of the journal impact factor. JAMA 2006;295:90-3.
Kapil A, Jain NC. Impact factor: Is it the ultimate parameter for the quality of publication? Indian J Med Microbiol 2016;34:1-2.
] [Full text]
Stephan P. Research efficiency: Perverse incentives. Nature 2012;484:29-31.
Lahiry S, Sinha R, Thakur S. Impact factor: Does it really have an impact? Indian J Dermatol Venereol Leprol 2019;85:541-5.
] [Full text]
Oh HC, Lim JF. Is the journal impact factor a valid indicator of scientific value? Singapore Med J 2009;50:749-51.
Ghosh S, Sinha JK. The need for rejuvenation of Indian biomedical journals. Indian J Med Res 2010;132:736-7.
] [Full text]
Journal Citation Reports 2014, Science edition (web-based) if it is web based- give web ID. Philadelphia: Thomson Reuters; 2014.
Beall J. Predatory publishers are corrupting open access. Nature 2012;489:179.
Jain NC. Predatory journals. Indian J Med Microbiol 2015;33:426.
] [Full text]
Adler KB. Impact factor and its role in academic promotion. Am J Respir Cell Mol Biol 2009;41:127.
Larivière V, Sugimoto CR. The journal impact factor: A brief history, critique, and discussion of adverse effects. Springer Handbook of Science and Technology Indicators; 2018. Cham (Switzerland): Springer International Publishing p. 1-33.
Feetham L. Can you measure the impact of your research? Vet Rec 2015;176:542-3.
Van Noorden R. Brazilian citation scheme outed. Nature 2013;500:510-1.
Smith R. Commentary: The power of the unrelenting impact factor-is it a force for good or harm? Int J Epidemiol 2006;35:1129-30.
Van Noorden R. The science that's never been cited. Nature 2017;552:162-4.
Juyal D, Thawani V, Thaledi S. Rise of academic plagiarism in India: Reasons, solutions and resolution. Lung India 2015;32:542-3.
] [Full text]
Juyal D, Thawani V, Thaledi S, Prakash A. The fruits of authorship. Educ Health 2014;27:217-20.
] [Full text]
Bush V. Science the Endless Frontier. Washington DC; United States Government Printing Office; 1945.
Madhan M, Gunasekaran S, Arunachalam S. Evaluation of research in India – Are we doing it right? Indian J Med Ethics 2018;3:221-9.
Casadevall A, Bertuzzi S, Buchmeier MJ, Davis RJ, Drake H, Fang FC, et al.
ASM journals eliminate impact factor information from journal websites. J Clin Microbiol 2016;54:2216-7.
DORA: San Francisco Declaration on Research Assessment. (2013) American Society for Cell Biology. Available at http://www.am.ascb.org/dora/
[Last accessed on 2019 April 21].
Chaddah P, Lakhotia SC. A policy statement on “dissemination and evaluation of research output in India” by the Indian national science academy (New Delhi). Proc Indian Natn Sci Acad 2018;84:319-29.
Saxena A, Thawani V, Chakrabarty M, Gharpure K. Scientific evaluation of the scholarly publications. J Pharmacol Pharmacother 2013;4:125-9.
] [Full text]
Hirsch JE. An index to quantify an individual's scientific research output. Proc Natl Acad Sci USA 2005;102:16569-72.
Dellavalle RP, Schilling LM, Rodriguez MA, Van de Sompel H, Bollen J. Refining dermatology journal impact factors using PageRank. J Am Acad Dermatol 2007;57:116-9.
West JD, Bergstrom TC, Bergstrom CT. The Eigen factor metrics: A network approach to assessing scholarly journals. College and Research Libraries 2010;71:236-44.
Costas R, Zahedi Z, Wouters P. Do “altmetrics” correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective. J Assoc Inf Sci Tech 2015;66:2003-19.
Gordon G, Lin J, Cave R, Dandrea R. The question of data integrity in article-level metrics. PLoS Biology 2015;13:e1002161.
[Table 1], [Table 2]