Super Crunchers (12 page)

The openness of the Internet is even transforming the culture of medicine. The results of regressions and randomized trials are out and available not just for doctors but for anyone who has time to Google a few keywords. Doctors are feeling pressured to read not just because their (younger) peers are telling them to, but because increasingly they read to stay ahead of their patients. Just as car buyers are jumping on the Internet before they visit the showroom, many patients are going to sites like Medline to figure out what might be ailing them. The Medline website was originally intended for physicians and researchers. Now, more than a third of its visitors come from the general public.

And Medline has responded by adding twelve consumer health journals and MedlinePlus, a sister site specifically for patients. The Internet is thus not just changing the mechanism by which information is disseminated to physicians, it is changing the technology of influence, the mechanisms by which patients can affect what their physicians do.

Technology can be critical for Super Crunching to change real-world decisions. When a decision maker in business or government commissions a study, the mechanism for transmitting the results is usually not an issue. In that case, the “technology” might be as direct as handing a copy of the results to your boss. But when there are hundreds and even thousands of unsolicited studies on the general topic of interest, the question of how to quickly retrieve the most relevant result often will determine whether the result will even have a chance of altering a decision.

The Future Is Now

The success of evidence-based medicine is the rise of data-based decision making par excellence. It is decision making based not on intuition or personal experience, but on systematic statistical studies. It is Super Crunching that reversed conventional wisdom and found that beta blockers can actually help patients with heart failure. It is Super Crunching that showed that estrogen therapy does not help aging women. And it is Super Crunching that led to the 100,000 Lives Campaign.

So far, the rise of data-based decision making in medicine has largely been limited to questions of treatment. The next wave will almost certainly concern diagnosis.

The database of information that we call the Internet is already having a bizarre impact on diagnosis. The
New England Journal of Medicine
published a description of rounds at a New York teaching hospital. A “fellow in allergy and immunology presented the case of an infant with diarrhea; an unusual rash (‘alligator skin'); multiple immunologic abnormalities, including low T-cell function; tissue eosinophilia (of the gastric mucosa) as well as peripheral eosinophilia; and an apparent X-linked genetic pattern (several male relatives died in infancy).” The attending physicians and house staff, after a long discussion, couldn't reach any consensus as to the correct diagnosis. Finally, the professor asked the fellow if she had made a diagnosis, and she reported that she had indeed and mentioned a rare syndrome known as IPEX which fit the symptoms perfectly. When the fellow was asked to explain how she arrived at her diagnosis, she answered: “I entered the salient features into Google, and it popped right up.” The attending physician was flabbergasted. “William Osler must be turning over in his grave. You googled the diagnosis?…Are we physicians no longer needed?”

The attending's extemporaneous reference to William Osler is particularly apt. Osler, who was one of the founders of Johns Hopkins, is the father of the medical residency program—the continuing cornerstone of all clinical training. Osler would be turning in his grave at the thought of Google diagnoses and Google treatments because the Internet disrupts the dependence of young doctors on the teaching staff as the dominant source of wisdom. Young doctors don't need to defer to the sage experience of their superiors. They can use sources that won't take joy in harassing them.

A bunch of medical schools and private corporations are developing the first generation of “diagnostic-decision support” software. A diagnostic program named “Isabel” allows physicians to enter a patient's symptoms and receive a list of the most likely causes. It will even tell the doctor whether a patient's symptoms might be caused by the use of over 4,000 drugs. The Isabel database associates more than 11,000 specific illnesses with a host of clinical findings, lab results, patient histories, and the symptoms themselves. The Isabel programmers created a taxonomy for all of the diseases and then tutored a database by statistically searching for word patterns in journal articles that were most likely to be associated with each disease. This statistical search algorithm dramatically increased the efficiency of coding particular disease/symptom associations. And it also allows the database to continually update as new articles emerge having a high prediction of relevance. Instead of all-or-nothing Boolean searches, Super Crunching predictions about relevance are crucial to Isabel's success.

The Isabel program grew out of a stockbroker's own experience with suffering caused by misdiagnosis. In 1999, Jason Maude's three-year-old daughter Isabel was misdiagnosed by a London resident as having chicken pox and sent home. It was only the next day when her organs began shutting down that Joseph Britto, an attending intensive care doctor at the hospital, realized that she in fact had a potentially fatal flesh-eating virus. Though Isabel ultimately recovered, her father was so shaken by the experience that he quit his finance job. Maude and Britto together founded a company and started to develop the Isabel software to fight misdiagnosis.

Misdiagnosis accounts for about one-third of all medical error. Autopsy studies show that doctors seriously misdiagnose fatal illnesses about 20 percent of the time. “If you look at settled malpractice claims,” Britto said, “diagnosis error is about twice or three times as common as prescription error.” The bottom line is that millions of patients are being treated for the wrong disease. And even more troubling, a 2005 editorial in the
Journal of the American Medical Association
concludes that there hasn't been a perceptible improvement in the misdiagnosis rate in the last several decades.

The ambition of Isabel is to change the stagnation in the science of diagnosis. Maude puts it simply: “Computers are better at remembering things than we are.” There are more than 11,000 diseases in the world and the human brain is not adept at remembering all the symptoms that give rise to each. Isabel actually markets itself as the Google of medical diagnosis. Like Google, it aids us in searching for and retrieving information from a large database.

The biggest reason for misdiagnosis is “premature closure.” Doctors think they have a bead on the correct diagnosis—like the resident's idea that Isabel Maude had chicken pox—and they close their minds to other possibilities. Isabel is a reminder system of other possibilities. It actually produces a page that asks, “Have you considered…?” Just proactively reminding doctors of other possibilities can have profound effects.

In 2003, a four-year-old boy from rural Georgia was admitted to a children's hospital in Atlanta. The boy had been sick for months, with a fever that just would not go away. The doctors on duty that day ordered blood tests, which showed that the boy had leukemia. They ordered a strong course of chemotherapy to start the very next day.

John Bergsagel, a senior oncologist at the hospital, was troubled by light brown spots on the boy's skin that didn't quite fit the normal symptoms of leukemia. Still, Bergsagel had lots of paperwork to get through and was tempted to rely on the blood test that clearly indicated leukemia. “Once you start down one of these clinical pathways,” Bergsagel said, “it's very hard to step off.”

By chance, Bergsagel had recently seen an article about Isabel and had signed up to be one of the beta testers of the software. So instead of turning to the next case, Bergsagel sat down at a computer and entered the boy's symptoms. Near the top of the “Have you considered…?” list was a rare form of leukemia that chemotherapy does not cure. Bergsagel had never heard of it before, but sure enough, it often presented with brown skin spots.

Researchers have found that about 10 percent of the time, Isabel helps doctors include a major diagnosis that they would not have considered but should have. Isabel is constantly putting itself to the test. Every week the
New England Journal of Medicine
includes a diagnostic puzzler in its pages. Simply cutting and pasting the patient's case history into the input section allows Isabel to produce a list of ten to thirty diagnoses. Seventy-five percent of the time these lists include what the
Journal
(usually via autopsies) verifies as the correct diagnosis. And manually entering findings into more tailored input fields raises Isabel's success rate to 96 percent. The program doesn't pick out a single diagnosis. “Isabel is not an oracle,” Britto says. It doesn't even give the probable likelihood or rank the most likely diagnosis. Still, narrowing the likely causes from 11,000 to 30 unranked diseases is substantial progress.

I love the TV show
House.
But the central character, who has unsur-passed powers as a diagnostician, never does any research. He relies on his experience and Sherlockian deductive powers to pull a diagnostic rabbit out of the hat each week.
House
makes excellent drama, but it's no way to run a health care system. I've suggested to my friend Lisa Sanders, who recommends script ideas for the series, that
House
should have an episode in which the protagonist vies against data-based diagnostics—á la Kasparov vs. the IBM computer. Isabel's Dr. Joseph Britto doesn't think it would work. “Each episode would be five or seven minutes instead of an hour,” he explains. “I could see Isabel working much better with
Grey's Anatomy
or
ER
where they have to make a lot of decisions under a lot of time pressure.” Only in fiction does man beat the machine.

And Super Crunching is going to make the diagnostic predictions even better. At the moment these softwares are still basically crunching journal articles. Isabel's database has tens of thousands of associations but at the end of the day it is solely a compilation of information published in medical journal articles. A team of doctors aided with Google-like natural language searches for published symptoms that have been associated with a particular disease and enters the results into the diagnostic database.

As it currently stands, if you go to see a doctor or are admitted to the hospital, the results of your experience have absolutely no value to our collective knowledge of medicine—save for the exceptional case in which your doctor decides to write it up for a journal or when your case happens to be included in a specialized study. From an information perspective, most of us die in vain. Nothing about our life or death helps the next generation.

The rapid digitalization of medical records means that for the first time ever, doctors are going to be able to exploit the rich information that is embedded in our aggregate health care experience. Instead of giving an undifferentiated list of possible diagnoses, Isabel will, within one or two years, be able to give the likelihood of particular diseases that are conditioned on your particular symptoms, patient history, and test results. Britto grows animated as he describes the possibilities. “You have someone who comes in with chest pains, sweating, palpitations, and is over fifty years old,” he says. “You as a doctor might be interested to know that in the last year, at Kaiser Permanente Mid-Atlantic, these symptoms have turned out to be much more often myocardial infarction and perhaps less commonly a dissecting aneurysm.”

With digital medical records, doctors don't need to type in symptoms and query their computer. Isabel can automatically extract the information from the records and generate its prediction. In fact, Isabel has recently teamed with NextGen to create a software with a structured flexible set of input fields to capture essential data. Instead of the traditional record keeping where doctors would non-systematically dictate what in retrospect seemed relevant, NextGen collects much more systematic data from the get-go. “I don't like saying this loudly to my colleagues,” Britto confides, “but in a sense you engineer this physician out of the role of having to enter these data. If you have structured fields, you then force a physician to go through them and therefore the data that you get are much richer than had you left him on his own to write case notes, where we tend to be very brief.”

Super Crunching these massive new databases will give doctors for the first time the chance to engage in real-time epidemiology. “Imagine,” Britto said, “Isabel might tell you that an hour ago on the fourth floor of your hospital a patient was admitted who had similar features of infection and blisters.” Some patterns are much easier to see in aggregate than from casual observation by individual participants.

Instead of relying solely on expert-filtered data, diagnosis should also be based on the experience of the millions of people who use the health care system. Indeed, database analysis might ultimately lead to better decision making about how to investigate a diagnosis. For people with your symptoms, what tests produced useful information? What questions were most helpful? We might even learn the best order in which to ask questions.

When Britto started learning how to fly an airplane back in 1999, he was struck by how much easier it was for pilots to accept flight support software. “I asked my flight instructor what he thought accounted for the difference,” Britto said. “He told me, ‘It is very simple, Joseph. Unlike pilots, doctors don't go down with their planes.'”

This is a great line. However, I think physician resistance to evidence-based medicine has much more to do with the fact that no one likes to change the basic way that they have been operating. Ignaz Semmelweis found that out a long time ago when he had the gall to suggest that doctors should wash their hands repeatedly throughout the day. The same reaction is at play when the EBM crowd suggests that doctors should do patient-specific research about the most appropriate treatment. Many physicians have effectively ceded a large chunk of control of treatment choice to Super Crunchers. Lisa Sanders distinguishes diagnosis, which she calls “my end,” from the question of the appropriate therapy, which she says “is really in the hands of the experts.” When she says “the experts,” she means Super Crunchers, the Ph.D.s who are cranking out the statistical studies showing which treatment works best. Very soon, however, Isabel will start to invade the physicians' end of the process. We will see the fight move to evidence-based diagnosis. Isabel Healthcare is careful to emphasize that it only provides diagnostic support. But the writing is on the wall. Structured electronic input software may soon force physicians to literally answer the computer's questions.