Health knowledge made personal
Join this community!
› Share page:
Search posts:

When is a comparison not a comparison?

Posted Feb 08 2012 4:56pm
I guess we shouldn't be surprised when a website called "Hospital Compare" chooses to adopt a methodology that presents comparisons, but leaves us with less-than-useful information.  According to Modern Healthcare :

The CMS' Hospital Compare website has been updated with facility-specific data on central line-associated bloodstream infections, a move the agency says will “hold hospitals accountable for bringing down these rates, saving thousands of lives and millions of dollars.

Here is the approach used by CMS :
The Central Line Associated Blood Stream Infections (CLABSI) Score is reported using a Standardized Infection Ratio (SIR). This calculation compares the number of central line infections in a hospital’s intensive care unit to a national benchmark based on data reported to NHSN from 2006 – 2008. The result is adjusted based on certain factors such as the type and size of a hospital or ICU.
  • A score of less than 1 means that the hospital had fewer CLABSIs than hospitals of similar type and size.
  • A score of 1 means the hospital's CLABSI score was no different than hospitals of similar type and size.
  • A score of more than 1 means the hospital had more CLABSIs than hospitals of similar type and size.
  • What's wrong?  The hospital data presented "reflect experiences of patients treated between October 2008 and June 2010 ," an untimeliness that would render them useless if were we interested in comparisons.  But, what is this business about benchmarking to a period of time four to six years ago, an ancient era in the world of hospital safety and quality?  Further, why are the data also adjusted based on the type and size of a hospital?  These three flaws act together to present an untrue impression of precision and relevance.
    If there was ever a metric that did not need a benchmark or an adjustment, it is the rate of central line infections.  The target for this metric should be zero.  As noted here by Jason Hickok, a registered nurse who helps direct patient safety and infection prevention at the Hospital Corporation of America: “We are making sure that people understand that zero is our goal.  It is a permanent culture change that zero is our target.”  The numbers reported on the CMS site should be the raw rate, the number of cases per thousand patient days.
    But, why are we comparing hospitals to one another?  The more valuable presentation would be one that showed the trend for each hospital, comparing its performance against its own previous performance.  We don't have to wait several years for data to arrive.  Each hospital knows its infection rate and calculates it in real time, every month.  If these numbers were presented, consumers could see if the hospital was getting better and sustaining its gains.  That would tell you more about a hospital than a flawed comparison presenting a useless metric based on a faulty methodology.
    Post a comment
    Write a comment: