The Science Citation Index (SCI) is a citation index that has served the world's scientific community for decades. It is also a core indicator in judging scientific research achievements in China and is held in high respect. The common evaluation indicators used in China include SCI, EI (Engineering Index) and ISTP (Index to Scientific & Technical Proceedings), all three which are the world's leading scientific and technological literature retrieval systems. In fact, they are internationally recognized search tool for scientific statistics and scientific evaluation as well. Among the three, SCI is the most important and is highly regarded within the Chinese scientific community.
Tens of thousands of Chinese scientific researchers, university teachers and professionals have become "slaves" to SCI's impact factors, exhausting themselves in an attempt to pursue their dreams of publishing more articles in renowned journals. Due to the Covid-19 outbreak earlier this year, Chinese academicians have scrambled to publish papers overseas, and in doing so, once again triggering discussions and suspicions in the community concerning the pursuit of SCI by professionals. In fact, Shanghai Jiaotong University academicians Jiang Xiaoyuan and Mu Yunqiu have conducted research that reveals the underlying problems in SCI and its impact factors.
SCI was coined in 1957 by the Institute for Scientific Information (ISI) in Philadelphia and its founder Eugene Garfield (1925-2017), was an expert on the subject matter. To the public, the ISI may sound like a government scientific institution, but in reality, it is a commercial company. Prior to being renamed as the ISI, it was referred to as the "Eugene Garfield Institute". It adopted its current name in 1960 and was acquired by Thomas Reuters in 1992, with Garfield admitting this was his intention all along.
Yet, the SCI and its impact factor are not without problems.
Specifically, the impact factor’s calculation formula has long been misunderstood in China. The algorithm of the impact factor is as follows: "The impact factor of the N annual journal = the total number of citations of all source texts in the publication of the journal [(N-1) + (N-2)] in year N / the total number of citations published in the publication year of the journal [(N-1) + (N-2)]". However, the formula is widely applied incorrectly in China, with the total number of citations replaced with the number of published papers. Journals in other countries have adopted various ways to "bump up" its impact factors via two methods– (i) by reducing the denominator and (ii) expanding the numerator. And the ISI has not amended the act of "data manipulation" on a systematic level.
Manipulating the denominator and reducing the number of reference items is a way to enhance the impact factor and it can be achieved in two ways. The first is to dilute the content, and the second through direct negotiations. Diluting content involves adding a large number of non-academic texts to change "academic" journals to amphibious journals. Take the Nature, there are 18 columns in each journal article, but only 3 of them are academic. According to SCI’s statistics database, its academic content only accounts for 1/3 of the whole journal, as does The Lancet, where its academic content only accounts for 1/4 to 1/5. A study published in 2007 in the Journal of the Royal Society of Medicine discovered that between 1994 to 2005, the number of academic texts in well-known medical journals such as the Annals of Internal Medicine, the Journal of the American Medical Association, and the New England Journal of Medicine have been declining year by year, while the impact factor rose steadily.
Direct negotiation is another less transparent way of accomplishing things. The Lancet originally included 821 citations in 2000. Following its “communication” with the ISI, the citations were “corrected” to 684. PloS Medicine, another well-known medical journal, disclosed in 2006 that when the magazine was first included in SCI in 2005, it tried to persuade the ISI to reduce the denominator. If the denominator of PLoS Medicine only includes "original papers", the impact factor is 11; If all texts were included, the impact factor would be 3. Based on its impact factor of 8 in 2005, the negotiation has certainly produced some effects. Although JCR publishes the impact factor formula every year, the ISI and Thomson Reuters have never published details on the formula concerning denominator value.
Even if the denominator does not change, there are still many ways one can increase the numerator. Compared to the denominator, the act of manipulating numerator is more diverse.
The first is to publish the general review. The current international academic writing framework consist of a "literature review" section strictly requires that one cites review articles. This in turn increases the frequency of citation in said section. It is also through these reviews that many journals have obtained ultra-high impact factors. For example, the CA: A Cancer Journal for Clinicians, first published in 1950, has an impact factor of 187 and was ranked first for 12 consecutive years, which is twice the times any journals were ranked second, and four times of Nature. Its large amount of citations is mainly attributed to two reports: the Annual Report to the Nation on the Status of Cancer and the Global Cancer Statistics published from time to time. According to statistics, the contribution of both reports to the journal's impact factor always exceeds 70%.
The second is self and mutual citation, that is journal articles citing articles previously published in the same journal, or several journals citing each other on a large scale. The effect is greater when it is superimposed on review articles. The 2008 issue of Acta Crystallographica - Section A published the article “A Short History of SHELX”, its then-original impact factor which was only 2.0 suddenly soared to 49.9 in 2009. In 2010, it rose to 54.3. Two years later in 2011, the impact factor of the publication fell back to 2.0. Additionally, 73.6% of the citations in “A Short History of SHELX” come from four sister journals of Acta Crystallographica - Section A, that is to say, the mutual citations between journals have created the myth of the impact factor of Acta Crystallographica - Section A.
The third is the inclusion of non-academic content in journals. Non-academic content will have a positive impact on the citation of academic articles in magazines. According to related comparative studies, the New York Times reprinted the full text of the New England Journal of Medicine, which garnered the paper more attention and boosted the number of SCI citations. This means that the New York Times created clear effect on expanding the dissemination of academic papers. Nature’s had a similar branding strategy. Since its virgin publication in 1869, it has never stopped publishing popular scientific articles. It is also precisely these articles that have largely made it popular, enabling authors to have higher number of citations when publishing papers in Nature.
It is worth noting behind the impact factor lies a huge business that has long been formed since its birth. Although the current scale of the ISI's profits is a trade secret, hence making it difficult to obtain public data, using a Project 985 university in China as an example, the school currently subscribes to ISI's 7 information products and pays more than RMB 2 million per year. If the same goes to colleges and universities all over China and the world, the scale of business can be huge. What more, ISI had 300,000 customers in 1992, hence it is not difficult to imagine the scale of its profits. According to another article published by Scientific American in 1995, a Mexican journal was required to spend USD 100,000 to subscribe to ISI products to be included in the SCI.
China’s government has intervened in the "SCI-only" phenomena and impact factor in the country’s scientific community. On February 20, 2020, the Chinese Ministry of Education and the Ministry of Science and Technology issued a document on regulating the use of SCI in colleges and universities, which made SCI and academic evaluation a focus in the academic community. The Chinese government hopes to establish a new academic evaluation system to replace the “SCI-only” academic orientation and impact factors.
Final analysis conclusion:
The SCI and impact factors that have long influenced the Chinese scientific community have caused the community to subscribe to it religiously, yet the SCI-impact factor myth is one that is full of problems, and it is high time that the myth is debunked.
