Medical Publishing Bias and the Practice of Medicine

Publication bias is the well-established finding that scientific journals are more likely to publish research with a positive outcome than with a negative or equivocal outcome. Positive outcome means the study findings support the primary end point, such as a study showing that patients taking the investigated drug will get the desired benefit.

The Relevance

But publication bias is nothing new. So why has it become a hot topic in the past few years? "It takes a long time to get people's attention," said Drummond Rennie, MD, FACP, deputy editor of JAMA and codirector of the US Cochrane Center, San Francisco.

Not publishing the whole story can lead to inappropriate clinical conclusions. Researchers "are trying to find the truth about a particular intervention, but you won't find the truth if not all the results are published," states Kay Dickersin, PhD, director, Center for Clinical Trials, De?part?ment of Epidemiology, Johns Hopkins Medical School, Baltimore, and director of the US Cochrane Center. Dr Dickersin notes that there are 2 main types of publication bias?not publishing the results, likely because the results were negative or equivocal, and selective outcome reporting in published trials.

Publication bias is widespread, Dr Dickersin says, based on an analysis she led (JAMA. 2002;287:2825-2828) that showed that clinical trials with positive results are about 3 times more likely to get published than those with negative results. The pharmaceutical industry "would like to keep outcomes under lock and key," she said. She also noted that studies funded by industry are far more likely to end up in a file drawer than government-funded studies.

Most scholars who research publication bias believe that journal editors are not especially biased, and given the opportunity, they would publish trials with negative or equivocal results. The problem appears to lie with the investigators, who are not likely to submit negative or equivocal trials for publication.

Research has shown that newspapers and other popular media are more likely to cover positive clinical trial results than negative results, which might influence the decision about whether to submit results. Greater interest in positive trials by the popular press was reported by Gideon Koren, MD, FRCPC, director of the Motherisk Program, the Hospital for Sick Children, Toronto, Ontario, Canada, in a 1991 study (JAMA. 1991; 266:1824-1826). He and his colleagues evaluated the response of the popular press to 2 trials, 1 positive and 1 negative, which appeared in the same issue (of JAMA). The trial with the positive results received more press coverage than the trial with the negative results. Dr Koren said this finding illustrated "a simple fact about life: no news is no news."

Other leading researchers in the field have provided examples of harm from undisclosed results. In a recent editorial (JAMA. 2004;292:1359-1362), Dr Rennie explained that un?published data on selective serotonin reuptake inhibitors (SSRIs) in children might have falsely reassured child psychiatrists and psychiatric associations that these drugs were safer than they actually are when used to treat children with major depressive disorder. The controversy linking SSRIs to an increased risk of suicide in children and the subsequent lawsuit against the manufacturer of one of the SSRIs, paroxetine (Paxil; GlaxoSmith?Kline), highlights the importance of disclosing all the trial information about a particular drug.

New Initiatives

Despite the compelling relevance of full disclosure for medical practice, publication decisions go largely unmonitored. But several institutions and governing bodies have stepped up recently to try to create trial registries to facilitate the full reporting of trial results. Pharm?a?ceutical companies are following suit.

In 2004 the International Committee of Medical Journal Editors (ICMJE), a consortium of 11 major peer-reviewed journals (eg, N Engl J Med, JAMA, Ann Intern Med) announced a new policy on trial registration. The ICMJE member journals announced that registration in a public trials registry is now a requirement for any article that is to be considered for publication. These journals do not advocate any particular registry, and they set forth requirements for an acceptable public trials registry that includes open access and a method to validate data. They also stated that only www.clinicaltrials.gov, a trial registry sponsored by the US National Library of Medicine, currently meets these requirements.

When asked if the establishment of clinical trial registries would help address the problem of publication bias, Dr Rennie said that "it can't but make a difference." But he added with skepticism that it would be difficult to achieve full disclosure. "It is certain that the drug industry will do everything to not make [the clinical trials registry] happen."

In 2005, the World Health Organ?ization (WHO) announced that it had started a project to improve ?clinical trial registration. An-Wen Chan, MD, DPhil, scientific officer with the Inter?national Clini?cal Trials Registry Platform (ICTRP), Geneva, Switzer?land, explained that while the WHO is not creating its own registry, it has taken other steps. "The ICTRP was established in re?sponse to a resolution pas?sed by the World Health As?sembly, calling on the WHO to establish a voluntary plat?form to link clinical trials registers to ensure a single point of access and the unambiguous identification of trials with a view to enhancing access to information by patients, families, patient groups, and others. The ICTRP is developing in?ternational standards for trial registration to ensure that information about all trials is publicly disclosed," Dr Chan said.

To enable clinical trial registration online, the ICTRP "will also be launching a search portal to retrieve information about trials across multiple ?registers," Dr Chan added, noting that disclosing results for all existing trials, even those that remain un?pub?lished, will provide public access to the complete body of research know?ledge, and help prevent publication bias.

The pharmaceutical industry has also begun supporting the use of clinical trial registries. In 2005, several international pharmaceutical organizations, including the European Federation of Pharm?a?ceutical Industries and Associations, the Pharmaceutical Research and Manu?facturers of America (PhRMA), the International Federation of Pharm?a?ceutical Manufacturers Association, and the Japanese Pharmaceutical Manu?facturers Association, added their support for universal clinical trial registration, primarily of phase 3 trials.

Clinical trials information

www.clinicaltrials.gov

Database of the National Institutes of

Health, National Library of Medicine

www.who.int/ictrp/en

The World Health Organization's

internal clinical trials registry

www.clinicalstudyresults.org

Database established by PhRMA for

voluntary trial results registration

www.centerwatch.com

Commercial clinical trials listing service

www.lillytrials.com

Information about Eli Lilly-sponsored trials

www.roche-trials.com

Information about Roche-sponsored trials

Alan Goldhammer, MD, associate vice president for regulatory affairs at PhRMA?the US trade organization representing the pharmaceutical industry?explained why the organization is focusing on registration of phase 3 trials rather than phase 1 and 2 trials. "We do not support the registration of all trials. They don't help us shape medical practice. We don't see any reason to register [phase 1 and phase 2] trials," he said.

Jeff Trewitt, a spokesperson for PhRMA, supported the industry's push to register primarily phase 3 trials. "Our belief is that phase 3 is the most comprehensive testing in which everything is thoroughly evaluated." He pointed out that member pharmaceutical companies have registered trials of >230 drugs at www.ClinicalStudyResults.org?the site established by PhRMA for voluntary clinical trials registration.

"You Can't Hide Later On"

Dennis Dixon, PhD, president of the Society for Clinical Trials (SCT), suggests that voluntary clinical trials registration does not go far enough. Dr Dixon believes that all trials should be registered, to make research results available to everyone. He emphasized that federal legislation requiring the registration of all clinical trials is the most effective way to accomplish this goal. The SCT is currently advocating for such federal legislation.

As the effort to register more clinical trials moves forward, Dr Koren expressed his belief that widespread clinical trials registration will substantially decrease publication bias in the future. With most major journals beginning to require trial registration for publication, "it will be the first time you cannot hide later on." This transparency could be good for all of us?physicians, re?sear?ch?ers, and patients?who want to know the truth about scientific research and to allow physicians to prescribe medications with complete knowledge of trial results, both positive and negative.