Bias in medical research

Allen L. Barker alb at datafilter.com
Sun Jun 13 01:25:25 EST 2004

Concern mounts that medical research does not give the full story

Be wary next time you read about a medical breakthrough or a miracle
drug. New research has found that scientists are disturbingly
selective when reporting the results of clinical trials.

Many scientists cherry-pick favourable results. Others change tack
when unexpected or interesting results emerge, breaching research

A research team led by Oxford University academic An-Wen Chan analysed
102 trials and found that researchers failed to fully report almost
two-thirds of the results relating to potentially harmful outcomes.

Half of the results relating to the effectiveness of a treatment were
not made fully public.

Many researchers also failed to adhere to basic research
protocol. Scientists are supposed to specify their main objectives -
or "primary outcomes" - before beginning a trial to protect against
selective reporting.

However, in 62 per cent of cases, researchers changed the primary
outcomes in their published reports.

In one published trial, the assessment of pain intensity shifted from
being a primary outcome to being unspecified. In another, researchers
introduced new primary outcomes that were not mentioned in their
initial outline.

"The reporting of trial outcomes is not only frequently incomplete,
but also biased and inconsistent with protocols," the team reported in
the Journal of the American Medical Association. "Published articles,
as well as reviews that incorporate them, may therefore be unreliable
and overestimate the benefits of intervention."

It's a disturbing prospect. Ultimately, clinical trials influence
which drugs your doctor prescribes - and which drugs Pharmac funds.

"The worst possible situation for patients, healthcare professionals
and policy-makers occurs when ineffective or harmful interventions are
promoted," the team says. "But it is also a problem when expensive
therapies, which are thought to be better than cheaper alternatives,
are not truly superior."

Many researchers seem oblivious to the potential bias in their
published reports. Eighty-six per cent of respondents to the Oxford
study denied the existence of unreported outcomes despite clear
evidence to the contrary.

Auckland University Clinical Trials Research Unit (CTRU) director
Anthony Rodgers describes the team's findings as a "wake-up call", but
not surprising.

This study is not the first to cast suspicion upon potential bias in
medical research, he says.

In an effort to counter the problem, pressure is mounting for
compulsory registration of trials.

In New Zealand, there is no definitive register of clinical
trials. Some researchers sign up with off-shore registers but it is
not compulsory to do so.

Rodgers is among a group of academics agitating for a compulsory
register to be set up in New Zealand. "[It will] basically keep
researchers honest and make sure trials get published in the way they
were planned and inconvenient things don't get pushed under the

CTRU research fellow Andrew Jull says registers also help to stop bias
from creeping into common medical practice.

When clinicians or government funding agencies want to assess the
merits of a drug, they turn to reviews of published trials. Those
reviews give an overview of all research into a particular treatment
or condition.

But there is a growing concern that non-publication of some trials is
skewing results. Australian research has found that trials with a
positive outcome are three times more likely to be published than
those with a negative outcome, so the public frequently doesn't see
unfavourable or inconvenient findings.

"So if your trial finds that something doesn't work or, alternatively,
it doesn't find a result full stop, then it is apparently more
difficult to get it published," Jull says.

Failure to publish can have dangerous - even lethal - consequences.

When a recent review of antidepressant trials looked at published and
unpublished research it found that several drugs that seemed
beneficial for children actually appeared to cause more harm than good
when all published and unpublished research was taken into account.

An even more alarming example occurred in the 1980s and early 1990s
when thousands died while using an approved heart attack
medication. It was not until 1993 that a trial conducted 13 years
earlier was published, along with its evidence of a higher death rate.

Today, failure to publish is increasingly viewed as a form of
scientific misconduct. But overcoming journal editors' preference for
positive results can be difficult.

Such is the extent of the problem that a journal has been established
to cater specifically for negative findings - the Journal of Negative
Results in Biomedicine.

Jull says that when journal editors' bias toward positive results is
combined with the potentially biased reporting of researchers, it
creates a very real danger we will develop a skewed perception of how
effective certain drugs are.

Efforts have already been made to conquer the problem. The major
medical journals now subscribe to the Consort statement, which
provides strict guidelines for unbiased trial reporting. The journal
editors have also agreed to try to counter their bias toward positive

More people within the health profession are recognising that they
should take into account the results of unpublished trials in their
treatment reviews. The Cochrane Collaboration, which was established
in 1993 to provide systematic reviews of health research, now uses
both published and unpublished data.

Jull believes making it compulsory to register clinical trials would
greatly improve public access to results.

"Before you are allowed to proceed, you must register [your trial]
with a database that's accessible to all," he says. "So when people
reviewing evidence make a decision whether a treatment works or not,
they have access to all the evidence and not just the published

More than 500 applications for clinical trials have gone to research
ethics committees in New Zealand in the past five years. It is not
known how many were later withdrawn and how many have languished

Jull says ethics committees have a duty to ensure trial results become
public. "If it is unpublished, there is the potential that, even with
the best will, prescribers can be misled, and that can have disastrous

At the moment, there is little official interest in setting up a New
Zealand register, but Jull believes it would be a simple, cheap
exercise to set one up online. Researchers could enter their own
details over the internet.

"It's probably easier than in a lot of other countries because we have
ethics committees that work to a single operational standard. No
clinical trial can proceed without ethics committee approval. It would
be relatively simple for the ethics committees to say: Is this

As well as combating potential bias, a New Zealand register of trials
would help scientists to keep tabs on local research, making
collaboration easier and preventing duplication, Jull says. Patients
and clinicians could also use the register to track down trials in
which they could participate.

New Zealand punches above its weight in performing clinical
trials. Jull says it is time we started contributing to an
international register and ensuring that the results of our research
are recognised.

Mind Control: TT&P ==> http://www.datafilter.com/mc
Home page: http://www.datafilter.com/alb
Allen Barker

More information about the Neur-sci mailing list

Send comments to us at biosci-help [At] net.bio.net