Detective Work
Reading Fine Print,
Insurers Question
Studies of Drugs

Kaiser's Veteran Sleuth Scours
Medical-Journal Articles
And Sees Marketing Spin

Doctors Fear Loss of Choices

By ANNA WILDE MATHEWS
Staff Reporter of THE WALL STREET JOURNAL

DOWNEY, Calif. -- When Eli Lilly & Co. wanted to get the big California health-maintenance organization Kaiser Permanente to use its new anti- depressant, it ran straight into Debbie Kubota.

Two studies published in psychiatry journals described the new drug, called Cymbalta, as "superior" to the older antidepressants Prozac and Paxil. But Dr. Kubota, a Kaiser pharmacist here, observed that in both studies the patients who did the best on Cymbalta received a higher dose of it than recommended on the label. Although each article mentioned the superiority of Cymbalta in its abstract, or summary, both acknowledged lower down that the studies weren't designed to show whether one drug was better than another.

Dr. Kubota's conclusion: The claim of superiority was "speculative." She issued a negative report to Kaiser's physician committees, which agreed and ruled that Kaiser would bar Cymbalta from its list of favored drugs for its biggest regions, Northern and Southern California.

As the cost of drugs in the U.S. approaches $250 billion a year, pharmaceutical companies are running up against a growing breed of detective trained to see through marketing spin. Working for insurers, state Medicaid programs and nonprofit bodies, these detectives cast a wary eye on published studies in medical journals, once considered an unimpeachable source. They search for subtle aspects of clinical-trial design that might show the drugs are not all they're cracked up to be.

"You could be duped," says Siri Childs, who oversees pharmacy policy for the Washington state Medicaid program. "We know now that just because it's published in a medical journal, that doesn't assure its quality."

The Cochrane Collaboration, a nonprofit that analyzes the quality of studies and collects the ones it considers good into broader analyses, has a volunteer corps of about 7,500 reviewers, mostly doctors and academics. That's up from about 2,800 five years ago. Another player is the Drug Effectiveness Review Project, an effort by an Oregon nonprofit. It issues reports summarizing all the studies in a particular treatment area and often criticizes individual studies for failings such as inadequate controls and high dropout rates.

Grains of Salt

Some journals are trying themselves to help readers discover marketing messages slipped in amid the scientific data. Last year BMJ, a British journal, published a piece called "Users' guide to detecting misleading claims in clinical research reports," which came with a picture of a reader dumping salt on a medical journal. One piece of advice: Beware when the authors break out one subgroup of patients and claim benefits from the treatment that weren't evident in the whole group.

Executives at Kaiser, a nonprofit based in Oakland, Calif., say they're saving health dollars by vetting drug studies to ensure they choose the drugs that work best. But some doctors and pharmaceutical companies say the sleuthing of people like Dr. Kubota is designed to give insurers and HMOs an excuse for blocking patients from useful but expensive treatments. A resolution in June by the American Medical Association said some Medicaid programs were trying to cut costs and devalue doctors' judgment under the guise of "evidence-based medicine."

Doug Williamson, a research psychiatrist at Lilly, says Cymbalta is good at treating painful physical symptoms that sometimes accompany depression. He says patients and their doctors should have access to different options and "the clinicians should make the best choice."

At Kaiser, physician panels make the final decisions whether a drug is added to the formulary, or list of preferred drugs, but they rely heavily on reports prepared by Kaiser staff. Dr. Kubota and the other Kaiser pharmacists who sift medical evidence are thus powerful gatekeepers for the six million Kaiser members in California and two million more nationwide.

Dr. Kubota, a 26-year Kaiser veteran who holds a doctor of pharmacy degree from the University of Southern California, is based in the industrial city of Downey, southeast of Los Angeles, across the street from an abandoned movie set. Kaiser tries hard to keep her and her colleagues away from the influence of the companies whose products they evaluate. Before meeting with Dr. Kubota, a representative of a drug company must fill out a form indicating who will be coming, what they plan to discuss and why the information can't be relayed in written form. No gift pens, mugs or other trinkets are allowed. Dr. Kubota's business card doesn't show her direct phone number.

'I Was Naive'

When Dr. Kubota started her current job in 1997, she says she "would just read the abstract," the summary at the beginning of a study. "I guess I was naive," she says. "You kind of assume everything is there for you in the abstract." Today, she quickly homes in on details that aren't mentioned in the abstract and generates a 6-inch stack of papers studded with Post-it notes for each drug.

When she reviewed Adderall, a stimulant now marketed by Shire Pharma- ceuticals Group PLC to treat attention deficit hyperactivity disorder, she noted that one of the major trials included only people who had responded well to Ritalin, another ADHD drug. She thought the move likely improved the results. Dr. Kubota recommended leaving Adderall off Kaiser's formulary. The physician committees partially overruled her, putting Adderall on the Northern and Southern California formularies but only for patients who failed to respond to another stimulant. Adderall later was added to the preferred list as a first-line treatment after a once-daily formulation went on the market.

A spokesman for Shire, Matt Cabrey, said that the designers of the trial, which was conducted when Adderall belonged to another company, felt that for ethical reasons they should give the drug to people with a "reasonable anticipated reaction." James Swanson, the lead author of the article that described the trial's results, said the trial was designed to show whether Adderall works faster than Ritalin.

Dr. Kubota has also reviewed studies of Lyrica, a new Pfizer Inc. drug for epilepsy and pain that is similar to the company's blockbuster Neurontin. Pfizer is hoping to persuade insurers that they should cover Lyrica rather than steering patients toward Neurontin, which is now available in generic copies that cost less than a dollar per pill.

At least four articles in leading medical journals, authored by scientists at Pfizer and in academia, described Lyrica as effective in treating pain. One last year in the journal Neurology stated in the summary that Lyrica "demonstrated early and sustained improvement in pain" for patients with pain caused by diabetes.

But Dr. Kubota discovered that in a section with a smaller font size, the Neurology paper said the trial excluded patients who hadn't responded to Neurontin. A similar caveat appeared in the other articles. That meant the trials failed to include patients who were likely to pull down Lyrica's results.

Cathryn Clary, a Pfizer vice president, said that the goal of the Lyrica studies was to demonstrate that the drug worked, and that including patients who failed to respond to Neurontin would have biased the trial against Lyrica. Some later studies included them, and they also showed favorable results for Lyrica. Kaiser hasn't ruled on Lyrica yet. Pfizer says the drug will go on sale this fall.

Tracking Cymbalta

In 1997, Dr. Kubota began tracking Cymbalta, the Lilly antidepressant. Cymbalta works by altering the presence of two brain chemicals, norepinephrine and serotonin. That puts it in head-to-head competition with Effexor, a Wyeth drug that works similarly. Cymbalta also competes with older drugs, including Lilly's Prozac, that work on serotonin alone. Prozac lost patent protection in 2001 and now costs as little as 26 cents per capsule in generic form at Drugstore.com versus more than $3 per capsule for Cymbalta.

Last Aug. 4, the Food and Drug Administration approved Cymbalta for treating depression. To help decide whether Kaiser should put the drug on its recommended list as an antidepressant, Dr. Kubota says she focused on a simple question: "Does it have any advantage over other existing agents in the treatment of depression? What added value does this have?"

In a meeting with Dr. Kubota, Lilly offered a 133-page binder that summarized its research. One article, in the Journal of Clinical Psychiatry, said in the summary that Cymbalta was "efficacious" and proved "numerically superior" to Prozac in treating depression. Another, in the Journal of Clinical Psycho- pharmacology, said in its summary that a daily 80-milligram dose of Cymbalta appeared "superior" to GlaxoSmithKline PLC's Paxil in the estimated likelihood of remission and improvement on a scale rating depression.

However, Dr. Kubota was struck by a statement in the first study saying it "was not designed to be a comparison" of Cymbalta and Prozac. The group of those taking Prozac was "underpowered," meaning it likely didn't include enough people to prove anything about the drug's effectiveness. She noticed another detail: Cymbalta was given at a daily dose of 120 milligrams -- twice as high as the daily 60-milligram dose recommended on its label. "It's not a fair comparison at all," she says. "They come right out and say that it's not."

Alan Gelenberg, editor in chief of the Journal of Clinical Psychiatry and a professor at the University of Arizona, said that while he hadn't reviewed the article recently, Dr. Kubota seemed to have a valid point. "It is misleading if in fact a study is not designed for comparison, and you like the way it turns out, to banner the comparison," Dr. Gelenberg said. He said the journal's editors and peer reviewers have gotten tougher in the last two years and are "taking a skeptical, jaundiced view" to prevent "spinning that can distort the data."

The study comparing Cymbalta to Paxil raised similar questions. It had two groups of patients receiving Cymbalta, half on a daily 40-milligram dose and half on the daily 80-milligram dose. The article claimed superior results versus Paxil only for the 80-milligram group, but those patients were receiving a higher dose than the recommended one. The article also noted that "the primary goal" of the study wasn't to compare Cymbalta and Paxil, so the conclusions were tentative, and higher doses of Paxil "might have proven more effective."

Several of the Cymbalta studies used an unusual statistical method to account for patients who dropped out partway through. Though she thought the method was legitimate, Dr. Kubota found that Cymbalta generally fared better when it was used. When she reviewed the results using the normal method, Cymbalta's performance was less impressive.

Lilly officials argued that reducing pain was an important factor in easing depression. (Cymbalta is also approved for treating diabetes-related pain.) Dr. Kubota thought that if this aspect of Cymbalta's action made a difference in depression, it ought to show up in the numbers.

The one study Dr. Kubota wanted most -- a head-to-head human trial comparing Cymbalta and Effexor -- wasn't available at that point. Lilly offered evidence from test-tube studies suggesting Cymbalta might have a more balanced effect on the two brain chemicals than Effexor, but Dr. Kubota wasn't buying it. She wanted to see results in humans.

A 24-Page Review

In the end, her 24-page review argued that Lilly needed more evidence before it could prove that Cymbalta was superior to Effexor. David Campen, a Kaiser medical director involved in deciding drug coverage, says Cymbalta also didn't offer any price advantage over Effexor. Given this information, Kaiser's physician committees voted to reject Cymbalta as an antidepressant for Kaiser's formularies in Northern and Southern California.

Michael Detke, a Lilly medical director, says the studies Dr. Kubota examined weren't designed to provide full comparisons between Cymbalta and the other drugs -- as disclosed in the articles -- but primarily to meet the needs of regulators. The doses were selected before the company had pinpointed the best dosing for Cymbalta, he said. Regarding the statistical analysis of trial dropouts, he says it was chosen because it was "considered a better statistical test," but Lilly "tried to provide all of the meaningful "findings with the old method.

"These are imperfect studies for payers to use to evaluate relative efficacy and relative safety and tolerability," he says. But he added that they weren't "designed to produce any kind of favorable outcome."

In the first half of this year, Cymbalta had $269 million in U.S. sales compared with $1.29 billion for Effexor XR, an extended-release form of the drug that is the most commonly prescribed version, according to IMS Health, a medical information firm. That puts Cymbalta in the top five antidepressants in terms of revenue.

At a conference this June, Lilly quietly released the results of two head-to-head trials of the kind Dr. Kubota had long sought. They showed Cymbalta wasn't significantly different from Effexor in treating depression.

Write to Anna Wilde Mathews at "anna.mathews@wsj.com"