Bill Bithoney, MD, spent several years as the chief of general pediatrics primary care at Boston Children's Hospital, in addition to serving as a professor at Harvard University School of Medicine.
"I thought that I was an expert," says Dr. Bithoney, now managing director with BDO Consulting and a member of the company's healthcare practice. "Whenever there was anything important in the world of pediatrics, different folks would call me up from the media and television channels."
However, when he moved to a smaller, less prestigious medical school, his phone stopped ringing. "I was the same person," he says. "I was just as smart. It all had to do with reputation."
That wasn't the only change. Dr. Bithoney says he also found negotiating managed care contracts was more difficult at a hospital that wasn't widely viewed as one of the best, even if it still delivered quality care. "It's a lot easier at a prestigious hospital than it is at a community hospital, even if your outcomes are as good, and that's disappointing," he says.
A Health Affairs study published in January raised questions about whether hospitals' reputations match the quality of care they provide. The study examined the differences between high-price and low-price hospitals and found the more costly providers were the clear winners in U.S. News & World Report rankings, which are partly (32.5 percent for the Best Hospitals 2013-14) based on their reputation with specialists. However, low-price hospitals performed better on certain outcomes-based readmissions and patient safety measures, such as postoperative blood clots. If high-price hospitals tend to have better reputations, the study raises the question of whether there's a disconnect between how hospitals are perceived and how they perform and whether reputation should play a part in lists such as Truven Health Analytics' 100 Top Hospitals, Becker's Hospital Review's "100 Great Hospitals" and U.S. News' Best Hospitals.
U.S. News & World Report has criticized the study for saying its hospital rankings are "largely based on reputation." Ben Harder, managing editor and head of the U.S. News healthcare analysis team, wrote a letter to the editor of Health Affairs stating, "U.S. News uses objective, primarily Medicare-derived quality measures such as mortality, volume and patient safety indicators to calculate 67.5% of each hospital's rank-determining score in the four specialties analyzed."
Chapin White, PhD — a RAND Corp. senior policy researcher and co-author of the Health Affairs study on high- and low-price hospitals —says the study's assertion was based on a study of U.S. News rankings from 2009 conducted by Ashwini Sehgal, MD, a professor at Case Western Reserve University in Cleveland.
"I gather that U.S. News has been refining their methodology (including changes since my article was written), and they’re obviously trying to move away from relying on reputation reporting," Dr. White says.
Mr. Harder confirms that shift, saying the publication has steadily reduced reputation's impact as more reliable quality metrics become public. In fact, he hopes to reduce its role entirely one day, assuming enough quality data become available. "At some point, reputation may be completely removed from the Best Hospitals methodology," he says.
Other hospital industry experts agree that more objective quality data should be given more weight than reputation (if reputation gets any weight at all) when assessing healthcare providers. Although hospitals and health systems that conduct significant amounts of groundbreaking research and have the best and brightest staff members may stand out to clinicians and the general public, a community hospital can often perform just as well, depending on what the patient needs, says Nancy Foster, vice president for quality and patient safety policy at the American Hospital Association.
"If we've got a fairly routine thing that's the matter with us…that can be done superbly in a community hospital," she says. "If you're going in for a gallbladder removal, you're going in to deliver a baby…they know how to do that, and they can do it well. That comes with repetition."
Mr. Harder points out that U.S. News currently doesn't evaluate hospitals on routine care. "Our methodology is designed specifically to identify hospitals that excel in treating the most challenging patients, the kind for whom hospital choice holds the highest possible stakes," he says.
The rise in quality data and potential discrepancies with reputation
It's difficult to reliably measure the quality of care a hospital provides, Dr. White says. "It's a lot easier to look at whether the building is new, and whether it has the latest equipment and whether the lobby is attractive," he says.
Dr. Sehgal agrees that factors other than quality can have a significant effect on a hospital or health system's image. He says physicians — especially those evaluating hospitals and health systems far away from themselves —don't necessarily have a reliable handle on which facilities truly provide the best care.
"There are all these others things that have to do with size, research, training and marketing that are going to be more prominent in the minds of physicians," he says.
Dr. Sehgal's research, cited by Dr. White's study, looked at the relative standings of the top 50 hospitals in the 2009 issue of U.S. News and concluded that, in terms of where they placed in the top 50, the rankings favored hospitals with national recognition, since those were the facilities that specialists nationwide tended to name frequently. "These data suggest that hospitals lacking national recognition are unlikely to be highly ranked by U.S. News & World Report," he wrote in the study, which was published in Annals of Internal Medicine.
Mr. Harder acknowledges reputation makes a difference in terms of the order of the top 50. However, he says it doesn't have a significant impact in determining which hospitals actually make the top 50. "If you're in the top 50, you may move up or move down based on what your reputation is," he says. "But whether you're in the top 50, it has little impact."
According to Mr. Harder, a recent U.S. News analysis found that more than 98 percent of the hospitals it currently ranks nationally perform in the top 2 percent nationwide on objective measures. More than five out of every six U.S. News-ranked hospitals are in the top 50 entirely on the basis of their outstanding performance on objective measures.
The significance of hospital reputation seems to be shrinking as it gets progressively easier to assess providers based on factors other than how patients or specialists perceive them. Medicare's Hospital Compare website — which displayed its first set of 10 process of care measures in 2005 — makes HCAHPS and CMS data accessible to consumers and others looking to find out how well hospitals perform according to various measures, including readmissions, linking quality to payment and delivering timely and effective care.
Truven measures both improvement and performance on a balanced scorecard to compare leaders' impact on the hospital's performance and value produced. Clinical outcomes and performance measures centered on factors such as financial health, HCAHPS results and how well the facility follows accepted care protocols are combined to sort out the best providers through its 100 Top Hospitals program. Jean Chenoweth, senior vice president of Truven's Center for Performance Improvement, oversees the program and says the study has gotten better as the publicly available data has increased and improved. She says the "brand name" providers don't always end up on top, either.
"Winning the World Series every year is really hard," she says. "When you're using pure data rather than softer metrics that don't change as much, then it's harder to repeat."
She says Truven prefers not to use reputation or other survey measures: "We have elected to use hard public data because it is objective, precise and includes the scores of all hospitals — not just those who elect to answer the survey."
Becker's Hospital Review's annual list of "100 Great Hospitals in America" is developed based on Truven's 100 Top Hospitals, as well as U.S. News rankings, HealthGrades, Magnet Recognition by the American Nurses Credentialing Center, the Studer Group and Malcolm Baldrige National Quality Award recipients. The editorial staff also accepts nominations for the list.
As more objective quality data becomes available, Dr. White says the facilities regarded as the best in the nation might not necessarily come out on top. That would raise several questions about whether people's perceptions of which facilities are the best are off, the quality measures are off or if the dissonance results from some combination of both factors.
The case for reputation's role in determining the best hospitals
Despite the increase in publicly available quality data and new performance measures, Mr. Harder says, for now, reputation still has value as a factor in determining which hospitals are the best at treating the most challenging patients. First of all, he says it helps differentiate among hospitals that have similarly phenomenal objective metrics. This approach allows a hospital that deals with incredibly complex patients, for example, to rank above one that has the same objective performance but deals with less challenging cases. "We wouldn't want or expect there to be perfect agreement between reputation and objective measures," he says. "If there is no difference between the two, it wouldn’t belong in the model. We include it because there is a difference, and we believe that difference is meaningful."
Second, he says the quality metrics out there today are "just not good enough" and can misrepresent a hospital's quality. For instance, a 2013 JAMA study, titled "Evaluation of Surveillance Bias and the Validity of the Venous Thromboembolism Quality Measure," found quality metrics for blood clots can suffer from "surveillance bias," meaning that providers who use imaging studies to detect clots more frequently find more, and therefore perform poorly. "That could, in fact, be an artifact that they're looking for it more aggressively, and they're on the road to improving," Mr. Harder says.
Notably, Mr. Harder says, one of the strongest correlations reported in the Health Affairs study was a link between high cost and high rates of blood clots that were diagnosed following surgery. He says the authors inferred that high-cost hospitals are worse at preventing blood clots. But the JAMA study suggests a different interpretation: More expensive hospitals may be better at detecting them.
Additionally, he says hospital leaders support the idea of throwing reputation into the mix when deciding which providers are the best. A 2012 U.S. News survey of hospital CMOs found more than 86 percent felt that hospital reputation, as determined by U.S. News' nationally representative survey of specialists, should continue to be a factor in the publication's ranking methodology, although nearly all of them also thought it should have only minor influence.
As objective quality measures become more robust, Mr. Harder says U.S. News will feel comfortable increasing their weight when ranking hospitals. The publication already plans to reduce reputation's weight this year to just 27.5 percent.
"When we began publishing Best Hospitals in 1990, it was entirely reputation driven because there were no quality metrics in the public domain at that time," he says. "With every improvement in the quality metrics that we're using or the addition of new metrics, we always take a look: 'Ok, is it reasonable for us to reduce the role of reputation now that we have better or more objective data?'"
More Articles on Hospital Reputation:
3 Key Findings on Hospital Reputation
14 Findings on the Correlation Between Hospital Reputation, Patient Experience
Study: High-Price Doesn't Mean High-Quality for Hospitals