Skip to content

Why the ‘Moneyball’ Approach Isn’t a Home Run for Health Care

A decade ago, the Oakland Athletics were the toast of baseball. Despite a decrepit stadium and tiny payroll, the A’s won just as many games as other Major League Baseball teams that could — and did — spend six times more money acquiring players.

The A’s secret: Using data to uncover the real, undervalued drivers that would make for a winning season. And the architect of that revolutionary approach: Billy Beane, who remains the team’s general manager.

The 2003 book “Moneyball” made Beane a star — and gave statistical savants a platform to call for “data-driven insights” across U.S. industries. And what sector could benefit more from an objective approach than health care, where subjective evaluations often cloud clinical judgment?

There’s just one hitch: Statistics are imperfect, too.

And whether in a pitching progression or disease progression, humans have a way of deviating from their predicted path.

Stats-Driven Approach Wins Champions — and Can Also Make Them

Health care leaders remain intrigued by the “Moneyball” method, which has been getting renewed attention since a movie version was released in September. (The Brad Pitt-starring film has cleared $50 million at the box office — or about $10 million more than the 2002 A’s spent for the magical season chronicled in the movie.)

James Knickman, CEO of the New York State Health Foundation, last week urged the industry to “take the Moneyball approach! … We need to generate, share, and analyze data that will help us make smarter decisions about how to invest our health care dollars and how to structure a reformed health system.” Mark Graban of the Lean Blog suggests that Moneyball holds lessons for improving frontline quality.

Beane, too, has argued that his approach is a model for health reform. Ahead of the 2008 election, Beane co-wrote an eye-catching New York Times opinion piece with GOP stalwart Newt Gingrich and Sen. John Kerry (D-Mass.), calling for a “data-driven information revolution” to sweep the nation’s “overpriced, underperforming health care system.”

Limitations Exist

Of course, it’s easier to urge an evidence-based revolution than to actually implement one. But there is considerable evidence that data-driven solutions already are shaping the future, both in baseball and health care.

When Beane began his work with Oakland, his team’s dedication to a statistical-based approach was unique; now, every Major League Baseball team employs statistical analysts.

Melanie Evans’ thorough article in Modern Healthcare likewise details how evidence-based medicine is becoming the norm in the health care industry. More hospitals and providers are adopting a range of algorithms to identify high-cost patients, improve their care and reduce spending. Select Health, an innovative health plan owned by Intermountain Healthcare, is using predictive modeling to focus on the top 2% of potentially high-cost patients.

Yet predictive statistics are not all-powerful, and insights drawn from data may be fleeting — which Beane quickly learned and as Bay Area residents and baseball fans know too well. The A’s haven’t been near the World Series in years, unless you count the waning moments of the new “Moneyball” film.

Similarly, Dr. Stephen Barlow, SelectHealth’s medical director, told Evans that predictive modeling will help explain only 30% of the organization’s costs. In addition, more than half of high-cost patients drive up spending for one year but not in subsequent years.

Many physicians also caution against overly depending on data, but finding the right balance between statistical insight and human intuition. “I have countless times been at a bedside when all the ‘numbers’ were heading in the wrong direction … and when statistics would tell us that the patient’s chances for recovery were virtually nil,” before the patient made significant improvements, UC-Davis’s Erich Loewy noted in 2007.

Three Common Hurdles Facing an Evidence-Based Approach

Advocates of statistically driven analysis in health care must surmount at least three major barriers.

Generational resistance

To change his team’s approach, Beane had to battle baseball lifers who often cared more about a player’s raw appearance than his advanced statistical numbers. (Tony Lucadello, one of the greatest baseball scouts of all time, theorized that every athlete has “eight sides” and that a scout should scurry around the field until he could construct a player’s three-dimensional portrait from memory.)

Health care leaders seeking to adopt evidence-based medicine must face seasoned skeptics who rely on a mix of instinct and science, too.

Sure, physicians acknowledge the value of statistical indicators — but “every clinician with some experience develops a sixth sense about many of his or her patients,” Loewy wrote in 2007.

User doubts

Consumers also have taken a skeptical view of evidence-based medicine — especially when it threatens to affect their own treatment. A 2010 study found that many patients simply don’t understand what an EBM approach would entail and feared that it would lead to reduced care.

(Baseball fans, in contrast, are significantly less discriminating; so long as their team wins, most fans don’t care how they do it.)

There’s been real-world backlash against some recent decisions influenced by evidence-based medicine. As Margot Sanger-Katz notes in National Journal, an effort to curb prostate-specific antigen testing for men has sparked controversy, despite comprehensive evidence showing no benefits of routine PSA testing.

A similar evidence-based recommendation to reduce annual mammograms for women provoked a much larger storm of criticism in 2009.

Unproven measures

Complying with evidence-based medicine assumes that health care leaders know what’s worth tracking. But what if the metrics we’re looking at are unclear — or even worse, wrong?

Using a database that built on years of baseball history, Beane was able to draw on a proven correlation between certain statistics, like a player’s on-base percentage and total runs scored, to figure out how his team could win more games.

Yet medical data are not always apparent or even accurate. A 2010 study published in the Archives of Internal Medicine suggested that publicly available physician information simply could not “predict which physicians will deliver high quality, evidenced-based care.”

There’s also a potential gap in how providers should be recognized and reimbursed. As David Whelan notes in Forbes, the Joint Commission’s recent list of top-performing hospitals — which recognized 405 institutions for following the commission’s best practices — had some puzzling omissions. No Harvard teaching hospitals, University of California facilities, or even Johns Hopkins made the list.

Is a small community hospital in a Midwestern hamlet really a better place to seek care than Stanford Hospital & Clinics?

Change is Coming

The federal reform law will advance evidence-based principles in health care, through channels like planned reimbursement for accountable care organizations. HHS also recently awarded $224 million in state grants to provide evidence-based home health programs that it says will lead to better outcomes and savings.

But federal dollars alone can’t sway public opinion. For evidence-based medicine to truly catch on, data-driven decisions need to be appropriately messaged to the public, analysts must uncover important connections within reams of new health data — and the current generation of clinicians may need to be steadily replaced by a younger crop of physicians.

Here’s what else is happening around the nation.

On the Hill

Rolling Out Reform

  • Last week, HHS began listing on the federal HealthCare.gov website information about proposed health insurance rate increases categorized by state. Previously, such information was available only in a few states (Kennedy, USA Today, 10/7). The federal health reform law requires all insurers to alert HHS if they wish to raise rates by 10% or more for individual and small group plans, as well as to provide justification for the proposed increase (Winfield Cunningham, “Inside Politics,” Washington Times, 10/7). The website provides a platform for consumers to comment about the proposed rate increases (USA Today, 10/7).
  • Last week, HHS officials released data showing that more than 20 million Medicare beneficiaries have benefited from provisions in the federal health reform law. According to the statistics, 20 million Medicare beneficiaries have received preventive health care services. In addition, as of August, 1.8 million beneficiaries who reached the Medicare Part D “doughnut hole” received prescription drug discounts totaling $1 billion, or $540 per person (CQ HealthBeat, 10/6). HHS did not reveal the cost of providing such benefits (Pecquet, “Healthwatch,” The Hill, 10/6).

Spotlight on ACOs

  • Uncertainty over Medicare payment reforms and an unpredictable implementation schedule have discouraged some hospitals from seeking certification as accountable care organizations. Soon after CMS issued its proposed rule for ACOs, a vast number of negative responses to the rule’s potential financial risks prompted HHS to roll out the Pioneer program. The department promised that it would put participating hospitals on a fast-track for ACO qualifications and allow high-performing health systems to access their savings up front in exchange for taking on greater financial risk.  However, hospitals have hesitated. Last month, Henry Ford Health System in Michigan withdrew from the Pioneer program because it had better and more predictable alternatives to the ACO program (Feder, Politico, 10/10).

In the States

  • Florida is preparing to launch a health insurance marketplace in 2012, two years before it is mandatory under the federal health reform law. Details about the program — called Florida Health Choices — indicate that it will not resemble the insurance exchanges being established in other states. State health officials have declined to acknowledge the program as an exchange partly because the state is part of a multistate lawsuit against the reform law and the program includes differences from the marketplaces mandated by the law (Galewitz, Kaiser Health News, 10/8).

On the Campaign Trail

  • Republican presidential candidate and Texas Gov. Rick Perry recently released a new campaign ad criticizing fellow candidate Mitt Romney for signing the 2006 Massachusetts health reform law when he was governor. The ad is particularly critical of the individual mandate in the law, which observers say had served as a basis for the 2010 federal health reform law signed by President Obama. The ad uses video and audio clips to link Romney’s and Obama’s positions on health reform and suggests that Romney has shifted his positions several times on the issue (Sink, “Healthwatch,” The Hill, 10/10).

Related Topics

Health Care Costs Road to Reform The Health Law