27 January 2006

Journals under fire

The recent spate of high profile scientific shams has left many wondering whether what appears in the world's most prestigious medical journals can be believed.

The recent spate of high profile scientific shams has left many wondering whether what appears in the world's most prestigious medical journals can be believed.

But despite the considerable embarrassment these deceptions have caused, leaders in the field say they do not believe the frauds signal a need for a revolution in how research is judged fit for publication.

They may, however, signal a need for reform.

Avoiding scandals
Editors at many of the leading scientific and medical journals are starting to tinker with their process in an effort to refurbish their image and avoid scandals in the future.

"Publish and hold your breath is a hell of a way of handling public health issues," said Adil Shamoo, a professor of biochemistry and bioethics at the University of Maryland in Baltimore.

Short history of recent fraud
The downward tailspin gathered force at the turn of the year, when it was revealed that two studies detailing South Korean researcher Hwang Woo-suk's breakthroughs with stem cell cloning were actually fabricated. The fraudulent findings had been published in two different papers in the journal Science.

That was preceded by the revelation in December that Merck employees had withheld critical data about heart attacks in a landmark Vioxx trial, published in the New England Journal of Medicine in November 2000.

And, most recently, a paper that appeared last October in The Lancet showing that non-steroidal anti-inflammatory drugs (NSAIDs) reduced the risk of oral cancer has turned out to be completely false. The database of 908 study participants itself was fabricated, with 250 of the people sharing the same birth date. The author of that study, Dr Jon Sudbo of the Norwegian Radium Hospital in Oslo, Norway, has now also confessed to faking data for mouth cancer studies published in the New England Journal of Medicine (April 2004) and the Journal of Clinical Oncology (March 2005).

Before they were revealed, however, the deceptions and their perpetrators had already been hailed by those in the scientific community. Hwang was widely touted to be a candidate for a Nobel Prize, while Sudbo's work was cited widely, including on the American Cancer Society's Web site.

Fallout has been considerable
Not surprisingly, the fallout from each of these episodes has been considerable.

Science has retracted both of Hwang's papers. The Lancet issued an "Expression of Concern" regarding Sudbo's research, as has the New England Journal of Medicine. The New England Journal of Medicine also published an "Expression of Concern" regarding the Vioxx data.

But what could have been done before the allegedly groundbreaking research was published? Is it inevitable that certain fabrications are going to slip past otherwise sound censors? Or does the breakneck speed at which scientific research now travels render existing systems obsolete?

"Science has grown by bounds and leaps," Shamoo acknowledged. "The world is demanding more health care and better care, and our lifespan is getting longer. There is a legitimate need for the growth. In the US alone, there are three million research personnel. It's a huge enterprise."

But experts stopped short of saying that such growth has led to an epidemic of scientific fraud.

"I don't think we've gotten more evil and devious," said Tony Delamothe, deputy editor of the British Medical Journal.

Rather, fraud may have increased proportionately to the amount of research done or, said Donald Kennedy, editor-in-chief of Science, "people are looking harder for problems, and it's likelier that problems will emerge."

But that doesn't mean editors are not engaging in some soul-searching.

Possible changes to review system
Some insiders have raised the possibility of making the existing peer-review system non-anonymous. This is something the BMJ did in 1999. But while it gives reviewers credit and makes them accountable, it doesn't necessarily guard against fraud, Delamothe noted.

"Fraud is about authors doing naughty things," he said. "Doing peer review isn't going to make a lot of difference... One to four individuals in a field reading an article are not set up to find fraud."

And other editors do not seem headed down this path. "The process benefits enormously from the kind of candour that results when referees know that their identity is held in confidence," Kennedy said.

Individual accountability suggested
A suggestion that has garnered more serious attention is holding individual authors accountable.

Science is considering requiring each author in a multi-author paper to submit a declaration of their precise role in the research. Such a system might have broken the Hwang case earlier as editors still don't know the precise roles of the myriad different players and institutions, Kennedy explained.

"Identifying the roles of different contributors is important," added David Magnus, director of the Stanford University Centre for Biomedical Ethics. "What does it mean that someone is listed as an author?"

In a similar vein, Science is considering asking each author to make a claim that they have read the paper, evaluated its conclusions and have confidence in them.

According to Kennedy, Science is also implementing a system to detect doctored photos, something that would have turned up problems with images submitted by Hwang. Sudbo also apparently manipulated images in his research.

Data audits suggested
In a 1987 letter-to-the-editor to Nature, Shamoo suggested that journals have independent organizations undertake random data audits. This would involve looking at the raw data, and making sure it corresponded to the final version.

"There's nothing magical or mysterious about it, but you don't do it to everyone," said Shamoo, who is also founder and editor-in-chief of the journal Accountability in Research. "You use statistical means, less than 1 percent will be audited."

"Will it decrease fraud to zero? No," Shamoo said. "But if we reduce it to 50 percent, it could save $10 billion to $15 billion."

But the problem may go even deeper.

"It's not that journals need to be more careful," Magnus said. "Institutions need to be set up so that we don't produce this problem."

"The goal is not better policing mechanisms," he said. "We have to set up structures and institutions and cultures so that people are trustworthy, so that people don't do this. The fact that so many different prominent institutional players have been implicated in one way or another in misconduct in South Korea, that raises real questions about overhauling the way we do science." – (HealthDayNews)

Read more: More studies fudged

January 2006


Read Health24’s Comments Policy

Comment on this story
Comments have been closed for this article.

Live healthier

Exercise benefits for seniors »

Working out in the concrete jungle Even a little exercise may help prevent dementia Here’s an unexpected way to boost your memory: running

Seniors who exercise recover more quickly from injury or illness

When sedentary older adults got into an exercise routine, it curbed their risk of suffering a disabling injury or illness and helped them recover if anything did happen to them.