SouthernWorldwide.com – Artificial intelligence is generating citations for medical research that do not exist, according to a recent study.
A comprehensive audit of millions of biomedical papers revealed that over 4,000 contained references to non-existent research. This finding was published in an article in The Lancet.
These fabricated citations pose a significant risk because they can influence clinical guidelines. Healthcare professionals rely on these guidelines, which are based on publicly available research, to make informed decisions about patient care.
Maxim Topaz, an associate professor at the Columbia School of Nursing and the lead author of the study, explained that when these fake references infiltrate the scientific literature, they can end up in clinical guidelines. This means doctors might base treatment decisions on studies that were never conducted.
Read more : Vance Faces Questions on 2028 Goals Amidst Trump's 'Dream Team' Talk
The situation is further exacerbated by the fact that, according to Topaz, none of the erroneous citations identified by his team have been corrected or retracted. This means they could still be impacting patient care.
Topaz noted that the incidence of fake references appearing in published medical literature is on the rise. He stated that the number of such incorrect citations has increased twelvefold over the past three years, affecting nearly 3,000 academic papers.
Topaz’s personal experience was a catalyst for this investigation. He shared that an AI application he was using to refine one of his scientific papers included a fabricated citation. This erroneous reference managed to pass through multiple layers of peer review before a vigilant editor detected it.
He expressed his astonishment, stating that after studying AI for 15 years, he realized that if such a mistake could happen to him, it could happen to anyone.
Topaz elaborated that these errors occur when an author makes a factual assertion and then requests a citation from the AI. In some instances, the AI might insert these citations inadvertently. While one might assume the facts are accurate, the presence of fabricated citations casts doubt on their validity.
In certain cases, AI tools have been observed citing a real author while inventing research and attributing it to them. Other times, the citations were entirely made up, Topaz added.
“This is just the tip of the iceberg,” he warned, suggesting that research in other fields could face similar issues. He emphasized that AI-generated scientific citations can appear entirely legitimate.
Topaz underscored the critical importance of researchers conducting rigorous fact-checking of their work to mitigate these risks.
