Defending Imperfect Data

“There’s something wrong with your data.  We’re not that bad.”

“Your data’s too old.”

“I don’t agree with your methodology.”

“These reports are useless because they don’t include data from our main competitor.”


These statements are typical of the objections hurled by various people within healthcare organizations, especially when the implications may be unpleasant.  How should the people responsible for generating these reports respond?


The first thing is to acknowledge the partial validity of their charges.  No data set, analytic approach, or methodology is perfect.  The overriding principle for minimizing negative reactions is to be careful about how your present the results.  You should portray them as indicators and starting points for discussion.  Acknowledge the possible limitations right from the start, and invite others to understand both the validity of the basic approach and the limitations.

Data enthusiasts can sometimes get so wrapped up in their analyses that they come across as if they believe the results are absolute truth.  “After all, it’s data.”

Here are some specific ideas:

Data Accuracy – To minimize the impact of imperfect data you should diligently work to bring it to the highest possible accuracy level.  Then invite others to review your data cleansing process and the data accuracy reports that quantify the possible errors.  Sometimes these reports point to systematic errors which, ideally, can be corrected before the reports are generated.  It’s unlikely that error rates in the 1% - 2% range would materially affect results.

Age of Data –  There is always a trade-off between timeliness and accuracy.  When I was Executive Vice President at Georgia Hospital Association, I led a data project that included a time lag of nearly 18 months.  This delay was necessary because of age-of-data requirements of anti-trust laws and the multiple data sets that had to close before the analysis could be completed.  Not surprisingly, we received some complaints about the time lag.  We worked hard to get the participating hospitals’ technical and financial staff to agree to compress the schedule, but they firmly believed that doing so would not allow for the incoming data to “stabilize” to the point for the results to be useful.

Methodology – People can have legitimate concerns about underlying methodology and logic.  If that’s the case, you should hold open discussions and see if there you can reach a resolution that satisfies all parties.  This may not always be possible, but if agreement can’t be reached, this should clue everyone in to the “firmness” with which you apply whatever conclusions the reports suggest.  The more disagreement, the less forceful.

Missing Participants – Comparative data reports are always most useful when all relevant parties participate.  If a main competing organization is missing, certainly that diminishes the reports’ value, but useful information can still be gleaned. 

Another Thought –  Remind everyone of the value of a series of reports over time.  Even if the data is not perfect, assuming data flaws remain constant from report to report, you can track performance trends – very helpful!


The key to maximizing support for your data results is to be humble and moderate the “fervor” with which you present your conclusions in light of possible data issues.  Reports and approaches that are fairly bulletproof can be used with a high degree of confidence.  To the degree that data issues exist, results should be handled more delicately.

Vendors' 3 Most Common ROI Mistakes - Part 2

Last time, I presented the first of three fallacies vendors often succumb to in their ROI projections to potential clients, namely the idea that saving time in a worker’s day results in cost savings to the hospital.  The typical methodology they use is identifying a certain number of minutes per procedure the new approach will save, multiplying the time saved by the number of procedures per day that job function performs, and then multiplying the result by annual salary plus benefits.  Viola!  Savings.

Not so fast.  As I pointed out, very rarely will a worker be sent home after seven hours and 45 minutes (instead of working a full eight-hour shift), thereby truly reducing costs.  When I point this out, the vendor typically counters by stating there is value in freeing up someone’s time to perform other neglected tasks.  True, but that doesn’t save a dime.

The example from last time concerned a vendor who had a product he claimed would save physical therapists’ time.  He was a little rattled after I explained Fallacy #1, so his next line of reasoning was, “Well, even you don’t send the therapist home early, reducing the time per procedure ends up increasing the therapist’s capacity so he can see more patients and bring in new revenue.” 

Fallacy #2

Nice thought – if the facility is literally turning patients away due to capacity constraints.  That may hold true in for some clinical areas, but it’s certainly not universally true across all hospital services.  Depending on the service line, some providers may be idle for parts of their days due to low patient volume.  So increased efficiency doesn’t necessarily mean bringing in new patients, meaning incremental revenue will probably not materialize.

There is a further complication to the “increased throughput argument.”  There may be other staff involved in a clinical or operational process who wouldn’t be directly affected by the positive effect of the new technology.  A good example relates to Operating Room schedules.  Even if a technology saves a few minutes per procedure for the surgeon, those saved minutes evaporate if housekeeping is understaffed and can’t turn the room can’t over quickly enough to slip in an additional procedure.  

So increasing patient throughput is Fallacy #2.

What It Takes to Avoid Fallacy #2

·         A revenue-producing service where the hospital is literally turning away patients due to capacity limitations

·         No other volume-based bottlenecks in related service areas

So demonstrating credible and persuasive ROI numbers is harder than it seems.

Vendors' 3 Most Common ROI Mistakes - Part 1

For more than 19 years, part of my job as Executive Vice President at Georgia Hospital Association was serving as gatekeeper for vendors seeking the association’s endorsement for promotional purposes.  Believe me, I have heard just about every vendor approach imaginable, and I would say the biggest mistake I consistently saw was vendors presenting questionable ROI numbers.  I call those “ROI numbers only your mother would believe.”

Let me present the three most common ROI fallacies.  A couple of years ago, I met with a vendor who asserted that his product could greatly enhance throughput from increased efficiency.  The product was fairly expensive, but he promised a five-month ROI.  “You’re on,” I said.  (I’m changing the facts of his product, but let me walk through his logic.) 

Fallacy #1

 “Our product can save a physical therapist five minutes per procedure, and the average PT sees 15 patients a day.  So that saves 75 minutes per eight-hour shift.”  He did the math assuming fulltime PT’s salary and benefits at $90,000 to get to his five-month breakeven.

What’s wrong with this picture?  His math works but his logic doesn’t.  Assuming his product did truly save five minutes per procedure, could the hospital really capture those savings?  In other words, he was asserting a reduced workload – and by implication reduced costs – of 75 minutes per day.   He then declared you could apply the savings to pay for his product.  But since the PT isn’t going home after 6 hours and 45 minutes, he is still on the clock and getting paid.  You can’t write a check from the reduced minutes..

After I explained to my vendor friend that capturing savings can be tough, he said, “Well, maybe, but the therapist can do other things that are really important that he hasn’t had time to do.”  True, formerly undone tasks may get done, but he is still getting paid, so there is no salary reduction. 

I should point out that this logic could work in an area with many employees performing the same job function which dominates most of their days.  In that case, you could potentially aggregate the saved minutes.  For example, if you have six FTEs in the same job function and you can eliminate 75 minutes per day each, that translates to 450 minutes or 7.5 hours per day.  Now you’re talking. 

This only works, though, if demand is pretty steady throughout the day.  Departments like outpatient Physical Therapy where procedures are somewhat elective may face uneven demand through the day as patients choose visits based on their pre-work or lunchtime schedules.

Also, keep in mind the nature of the job function.  A technology that reduces minutes for a billing clerk is likely to be more promising than one that saves time for nurses.  Billing staff have one primary task, and larger hospitals typically have enough billing clerks that a streamlined process could potentially eliminate part of a position.  On the other hand, nurses conduct many different functions during their shifts, so it is unlikely they do the same activity enough to aggregate enough “saved” minutes to cut staffing.

So Fallacy #1 is claiming that the few minutes saved per procedure can pay for the product.


What’s Required to Avoid Fallacy #1

·         A technology that introduces efficiency for positions with high volume of a single activity and fairly steady demand throughout the work shift

·         Enough employees in the affected job category to “gather” the savings to the point where headcount can be reduced


Understanding Health Tech: The 2 Big Buckets and the 6 Sub-Buckets

When I tell people I consult in emerging healthcare technology, their first question is usually whether I help hospitals with their EHRs.  No.  Their next question is, “Are you a CIO?”  No.  “Well then what do you do?”

The best way to explain my focus is to think in terms of two big buckets and four sub-buckets of health technology.


Bucket 1 – Traditional Healthcare Technology

·         Sub-bucket 1 – Electronic Medical Records

·         Sub-bucket 2 – IT infrastructure

Both of these are virtually universal.  It’s hard to have a hospital without them.  There is no competitive advantage in having either an EHR or IT infrastructure.  That would be like saying, “Come to our hospital.  We have elevators.”  Or “We have an HVAC system.”  These are pretty much cost-of-doing-business items. 


Bucket 2 – Emerging, Disruptive, mHealth, Digital Health Technology

You can see by the multiple adjectives that it is a bit difficult to describe this bucket.  It’s largely every kind of tech except what is in the traditional healthcare technology bucket.  Much of emerging, disruptive technology is enabled by the Internet and distributed delivery approaches such as Software as a Service (SaaS) and Cloud-based storage.

The four sub-buckets in this category are:

·         Sub-bucket 1 – Patient-Touching – This technology realm has been around for centuries, but ongoing scientific breakthroughs introduce new products and approaches virtually every day in the following areas:

o   Diagnostics – Everything from bedside lab analytics to peripheral smartphone devices that perform EKGs or other clinical functions

o   Intervention – Multitudes of new treatment modalities

o   Implantables and Devices – Such as 3-D printing or bionic limbs

·         Sub-bucket 2 – Personalized Medicine – Evidence-based and data-driven care customization based on my personal traits and circumstances

o   Clinical – Such as IBM Watson, genomics and automating evidence-based protocols to tailor care plans based on my personal physiology

o   Coordination – Such as customized post-acute care plans incorporating psychographic/demographic data that factors in preferred communications approaches

·         Sub-bucket 3 – Communications

o   Between patients and providers – To monitor patient progress, allowing for early intervention when needed

o   Among providers – Either in the referral process or post-discharge to track patient progress and determine whether the patient is actually following the care plan to allow the various caregivers to regroup if necessary

·         Sub-bucket 4 – Business Functions

o   Clinical applications – Such as Lean analytics for things like improving efficiencies in handling ER throughput

o   Standard business functions – Such as OR scheduling or administering hospital HR functions

Since “technology” is such a broad term that encompasses so much, this analytical framework provides a helpful structure to help sort out the many, many aspects of healthcare technology, especially for hospital executives who are not particularly technology oriented. 

So for the record, I don’t do anything with the first big bucket of traditional healthcare technology, but I’m all over the emerging, disruptive, mHealth, digital health technology group.  I love helping developers and entrepreneurs develop strategies for successful product development and introduction into the healthcare marketplace.