To measure an IT project's success, hospital and health system CIOs often opt for traditional metrics such as project costs, implementation timelines and the functionality delivered. But many are shifting their focus to the value it will deliver to their organizations and the outcomes and benefits to the end user.
Becker's spoke to four CIOs and a chief technology officer to discuss how they balance traditional and unconventional approaches to measuring whether IT projects deliver on their promises.
Note: Their responses have been lightly edited for clarity and length.
Denise Zabawski. CIO of Nationwide Children's Hospital (Columbus, Ohio): I used to say "on time, on budget," but now it's: "Have we produced the best outcome for the project and for the recipient, whether it's the patient or the provider?" So best outcomes.
Bobbie Byrne, MD. CIO of Advocate Aurora Health (Downers Grove, Ill., and Milwaukee): You've got to hit your budget. You've got to hit your date. You've got to hit your specific key performance indicators for whatever that project is.
If you said we were going to increase revenue or [improve] patient safety, you have to hit those goals and measure them ahead of time to implement. Then you've got to track them afterwards.
But my personal measure of IT success is when somebody asks how our Epic system helps us solve this problem. Because I don't want the system to be a capital asset that gets depreciated — to me, it is an investment that you have to leverage.
So if somebody says, "Hey, we got a problem with turnaround time," or, "We have a problem with patient satisfaction," I want them to say, "How can Epic help us solve this problem?"
I love the concept that whatever the problem is, no matter how near or far to the patient, we're going to use our systems to work that problem.
Charles Worthington. Chief Technology Officer of the U.S. Department of Veterans Affairs (Washington, D.C.): How well does it solve the end user's problem? Every technology we put in front of an end user should help them accomplish a task. And we should measure how well it's doing that and how well they liked it along the way. The only reason we should be deploying technology is to help make the end user better at whatever they're trying to accomplish.
Matt MacVey. Executive Vice President and CIO of Children's National Hospital (Washington, D.C.): In the ultimate sense, it's adoption, but with some [key performance indicator] measurement. Did we create the change we were seeking in improving something, or did we just put in technology on top of process? The latter just doesn't cut it anymore. Have we made a business process improvement? That's how you measure success of an IT rollout, and ideally with some KPIs related.
Ellen Pollack, MSN, RN. CIO of UCLA Health (Los Angeles): I judge our success by what the patients think of us and what the business thinks of us. How did they rate us? If we achieved what the business was looking for, in a way they view was a success, then to me that means we're doing a good job.