Study Says Many Studies Are Misleading

ADVERTISEMENT

NEW YORK (MainStreet) — Studies about medicine can be misleading, says a new study that studied studies on medicine.

Researchers at the University of California Los Angles and Harvard analyzed medical studies published in six leading general medicine journals and discovered that many of them used tactics that made it hard for readers to properly understand and evaluate the results.

For instance, many medical trials focus on:

• Surrogate outcomes (37%), which refer to intermediate markers (like a heart medication's ability to lower blood pressure), instead of addressing the medication's impact on more important clinical outcomes (like, say, a heart attack).

• Composite outcomes (34%), which lump multiple individual outcomes of unequal importance together, such as hospitalizations and mortality, and make it difficult to understand the effects on outcomes individually.

• Disease-specific mortality (27%), which measures deaths from a specific cause rather than from any cause. UCLA and Harvard researchers say this can be misleading since treatments that reduce one type of death may increase the risk of dying from other causes.

Researchers reached these conclusions after looking at all studies published between June 1, 2008, and Sept. 30, 2010, in The New England Journal of Medicine, the Journal of the American Medical Association, The Lancet, the Annals of Internal Medicine, the British Medical Journal and the Archives of Internal Medicine.

They also reviewed each study's abstract to determine the percentage that reported results using relative rather than absolute numbers, a practice they also dubbed “misleading.”

"The way in which study results are presented is critical," Dr. Danny McCormick, the study's senior author, said in a  press release. "It's one thing to say a medication lowers your risk of heart attacks from two-in-a-million to one-in-a-million, and something completely different to say a medication lowers your risk of heart attacks by 50%. Both ways of presenting the data are technically correct, but the second way, using relative numbers, could be misleading."

Dr. Michael Hochman, another author of the study, said that their findings also suggests that commercial sponsors of research, such as pharmaceutical companies,  may promote the focus on outcomes that are most likely to indicate favorable results for their products.

For instance, the findings illustrate that while 45% of exclusively commercially funded trials used surrogate endpoints, while only 29% of trials receiving non-commercial funding did. 

Additionally, 39% of exclusively commercially funded trials used disease-specific mortality, while only 16% of trials receiving non-commercial funding did.

Both authors said that the use of the aforementioned data sets may be appropriate in some cases, such as early-phase studies being used to quickly determine whether a new treatment is effective.

However, they encourage other authors to report results in absolute numbers and suggest that committees who oversee the research scrutinize outcomes to ensure these lower-quality data points, like surrogate makers, are only used in appropriate circumstances.

“Medical journals should ensure that authors clearly indicate the limitations of lower-quality endpoints when they are used , something that does not always occur," McCormick said.

UCLA and Harvard researchers were careful to point out that they did not receive any internal or external funding for this research. Their findings, perhaps unsurprisingly, will be published online in the Journal of General Internal Medicine.

Show Comments

Back to Top