Since U.S. News & World Report first published Best Hospitals in 1990, the annual series has become the most publicized and coveted ranking among healthcare organizations. For this we have Avery Comarow, editor and director of Best Hospitals, to thank.
Avery Comarow, director of U.S. News & World Report's Best Hospitals and other healthcare rankings, retired Dec. 27 after 31 years with the organization.
For a leader in the methodical world of rankings, Mr. Comarow's career path to healthcare journalism was hardly calculated. He says he "stumbled" into journalism, although he paid his undergraduate tuition at the University of Maryland as a part-time proofreader for a nearby news production plant of The Wall Street Journal. He attended graduate school at Indiana University to study and teach folklore before changing paths and working at a Bloomington, Ind.-based daily newspaper and then a larger daily in Middletown, N.Y. From there, he gained experience in consumer-driven reporting as a writer for Money magazine and Washington editor for Consumer Reports before landing as an editor at U.S. News.
Mr. Comarow, who ran the "News You Can Use" column of U.S. News, drafted the first steps to develop the Best Hospitals rankings after Mortimer Zuckerman, owner of U.S. News, approached him with the idea in 1987. However, Mr. Comarow worried it was impossible to rank hospitals — at the time, there was no available data to evaluate them.
To start gathering data, Elizabeth Gross, the magazine's then director of research, helped Mr. Comarow partner with a research team at Georgetown University to develop a plan for surveying a small number of physicians. The initial survey gathered opinions from 480 physicians — 40 physicians across 12 specialties. Researchers asked the physicians to name the hospitals they would choose to send patients with challenging medical conditions.
The survey was based strictly on the physician's perception of a hospital's reputation. "Whatever the physicians thought, we used that," Mr. Comarow said. "I refused to use the word 'rankings' on the cover because statistically there was no way we could take the small number of responses from the 480 physicians and characterize them as robust rankings."
To build a more credible system, Ms. Gross then connected Mr. Comarow with the National Opinion Research Center at the University of Chicago. From 1990-1992, U.S. News published rankings driven by physician surveys as the NORC worked to develop the model the magazine uses to organize rankings today — the Donabedian paradigm. The move proved to be critical in evolving the list from a limited, reputation-based tool to a credible, data-driven resource.
The Donabedian model — a three-part framework to evaluate health services and healthcare quality — establishes a hierarchy for rankings based on a health system's structure, process and outcomes. The framework, which U.S. News first used in 1993, was based on data from physicians and nurses who participated in focus groups and ranked the attributes of hospitals they found most important.
Structure analyzes aspects such as the hospital's technology or number of staff, process examines how few mistakes are made during care and outcomes assess whether patients live or die under the hospital's care. But a model informed by focus groups presented challenges — Mr. Comarow said it was difficult to determine if physicians and nurses accounted for how well patients were taken care of, not just whether they lived or died.
All while the NORC conducted analyses and provided data, Mr. Comarow worked as the one-man team at U.S. News to complete the published lists. Nonetheless, he is sure to emphasize the team effort it took to complete the rankings. "If I had tried to do this on my own, I'm not sure this ever would have been possible," he said. U.S. News parted from NORC in 2005 to contract with RTI International in North Carolina.
As Best Hospitals continued to evolve, Mr. Comarow was vocal in emphasizing the importance of transparency to both consumers and hospitals, which guided his role at the helm of an increasing number of U.S. News healthcare rankings.
"We have to treat consumers as if they are as smart as we are but don't happen to know as much as we do," Mr. Comarow said. "We give them information and help them understand what it's about. Six different people may make six different choices, but that's what you want. You don't want to make the choices for them."
In 2006, Mr. Comarow proposed a session for quality analysts and clinical executives to give U.S. News feedback on its hospital rankings. "We invited professionals from 40-50 hospitals to let them throw rocks at us for a whole day, and it was amazing," Mr. Comarow said.
If U.S. News hadn't been open to receiving constructive criticism from hospitals, "we would have lost out on input from the people who are in the trenches," Mr. Comarow said. "They know much more than we do and it would have been idiotic to pretend we were the authority."
To keep the consumer's interests and needs top of mind, Mr. Comarow had to push back on data and make it as accessible as possible. "My role was to be an informed medical reporter who tries to wear the mask of a reasonably smart but naïve consumer," he said. "Our contractors did all the behind-the-scenes work, but I would push NORC and RTI to come up with data that I could understand."
Mr. Comarow remained the only member of the hospital rankings team until current health analysis editor Ben Harder joined U.S. News in 2007. Mr. Harder has since grown the health analysis team to five people. "[Mr. Harder] has been running the show for the last three or four years, but he insisted I keep my title," Mr. Comarow said.
Now that Mr. Comarow is retired, he's planning to volunteer at a fire department near his home in Maryland, where he will help with administrative work and recently participated as a simulated patient in an emergency call. "I'm not going to be an EMT, I'm 72 years old," he said. "But I know how to write and I know something about putting numbers together."
And the numbers Mr. Comarow put together in the decades he spent with U.S. News endure as the true purpose of the Best Hospitals rankings. "These rankings really can matter in the most fundamental way — whether patients live or die," Mr. Comarow said. "Whatever harm we might have helped people avoid, that's the real legacy of these rankings."