One of the keys to performance improvement is collecting and analyzing data on different measures. To determine quality and patient safety, hospitals track several measurements, including complication, mortality and readmission rates. Capturing data on specific events, such as deaths, is relatively straightforward. But what about other, "softer" determinants of quality and patient safety, such as patient safety culture and patient engagement? These concepts are complex; there are several indicators of patient safety, and they are often abstract, such as having a "just" culture in which people feel comfortable reporting adverse events.
Why should healthcare organizations measure "soft" aspects of patient safety?
While much of healthcare data are "hard" measures — measures that are concrete and easily quantifiable — there is a growing awareness of the importance of "soft," more qualitative measures, such as patient safety culture, in improving the U.S. healthcare system. In fact, what has typically been viewed as a soft measure — patient satisfaction — now factors into hospitals' reimbursement under CMS' Value-Based Purchasing program. It is important to measure and benchmark these multifactor concepts to identify opportunities for improvement.
Patient safety culture speaks to healthcare providers' intrinsic motivation, according to Peter J. Pronovost, MD, PhD, senior vice president for patient safety and quality and director of the Armstrong Institute for Patient Safety and Quality at Baltimore-based Johns Hopkins Medicine. Dr. Pronovost is well known for developing a checklist for intensive care units that has dramatically reduced central line-associated bloodstream infections in Michigan, saving an estimated 1,500 lives and $100 million annually. "Our work in reducing ICU infections was almost entirely driven by intrinsic motivation. The softer side — this intrinsic motivation, the way we speak to each other, empowerment of the frontline — is the magic sauce of what it takes to improve safety and quality," he says.
How can healthcare organizations measure "soft" aspects of patient safety?
Unlike patient satisfaction, which hospitals have been required to report using the Hospital Consumer Assessment of Healthcare Providers and Systems survey for roughly six years, standardized patient safety culture measures are not widespread in the U.S.
However, there are some surveys of patient safety culture that are gaining ground among hospitals. The Agency for Healthcare Research and Quality released the Hospital Survey on Patient Safety Culture in 2004 to help hospitals assess their safety climate. AHRQ benchmarks data from all hospitals who submit data from the survey. In the 2012 User Comparative Database Report, AHRQ compares results from 1,128 hospitals, compared with 1,032 hospitals in 2011. HSOPS includes 42 items designed to measure 12 areas of patient safety, including teamwork within units, and supervisor/manager expectations and actions promoting patient safety.
"AHRQ has developed and offers the use of validated surveys, as well as benchmarking data. This is a huge step in the attempt to measure a complex construct," says Marti Beltz, PhD, LSSMBB, a faculty member and senior healthcare quality consultant at the American Society for Quality.
Another survey, the Safety Attitudes Questionnaire, was developed by Bryan Sexton, Eric Thomas and Bob Helmreich with funding from the Robert Wood Johnson Foundation and Agency for Healthcare Research and Quality. This survey includes items designed to measure factors such as teamwork climate and job satisfaction.
Do surveys accurately measure culture?
Although surveys can reveal hospital employees' perceptions of safety, they have flaws that prevent a completely accurate picture of an organization's culture. "While validated (particularly those offered by [AHRQ]), these instruments are only proxies used to estimate the degree to which a facility is committed to safety at all levels," Dr. Beltz says. "In addition, significant variations in safety culture may exist within an organization (e.g., between leadership and frontline workers, between one unit and another) so institutional level scores may mask these significant differences."
Conducting the survey at the unit level can be more useful than a hospital-wide survey in identifying where patient safety culture is successful and where it faces challenges. Dr. Pronovost says variation in survey results can vary as much as six- to eight-fold across a hospital. To gain a more accurate assessment of patient safety culture in the hospital, Johns Hopkins Medicine issues the surveys to individual units.
However, measuring safety culture at the unit level also presents challenges because physicians often work in many units. To include physicians in the surveys, Johns Hopkins surveys all physicians as one unit.
In addition, response rates for patient safety culture surveys can be low, which diminishes its validity as a measure of the organization's culture. Dr. Pronovost says hospitals should aim for at least a 60-percent response rate to attain representative data. "You have to be cautious about using these [data] too quantitatively," he says.
Case example: Johns Hopkins Medicine
To improve the safety culture at Johns Hopkins Medicine, the system conducts HSOPS surveys at the unit level, provides feedback on results and uses the responses to begin a discussion on safety. "By far the biggest benefit of the surveys is they start the conversation," Dr. Pronovost says. Johns Hopkins leaders work with staff at the unit level to address items in the survey.
Typically, leaders ask staff about the three highest-scored questions on the survey and the three lowest-scored items. For example, when presenting the high-scoring items, leaders can determine the unit's best practices that improved its culture and that can be spread to other units. Similarly, when discussing a low-scoring item, a leader may say, "Only 30 percent of nurses on the unit said they were able to speak up. Is that what it feels like?" Dr. Pronovost says.
"You ask staff their perceptions of how well they work together, and if they feel comfortable speaking up about mistakes — that's the superficial side. What we care about is the [underlying] values, beliefs and ultimately, behaviors," he says. "The survey takes a vague concept like safety culture and translates it into specific questions with responses we can then have a conversation with staff about."
He suggests having more in-depth discussions on safety culture through smaller focus groups. "It's different saying 'Safety culture is important; we have to improve it,' than saying 'Only 20 percent of nurses felt comfortable speaking up.' It allows you to get into these issues in a much more real and concrete way than if you didn't have these conversation starters," Dr. Pronovost says.
Tips for measuring patient safety culture
As evident from Johns Hopkins' experience, measuring patient safety culture can still be valuable despite the flaws of the measurement tool. Dr. Beltz says, "The best piece of advice I can offer to leaders is acknowledge that there is no one measure of patient safety culture and that all measures are only proxy indices of the construct." She suggests two best practices for measuring the softer aspects of patient safety:
1. Measure the root cause, not the symptom. Hospitals should conduct root cause analyses to determine the reason for adverse events to make long-lasting change. For example, to measure a commitment to safety, leaders should go beyond just counting the number of fall risk assessments conducted, according to Dr. Beltz. If a patient passed a fall risk assessment but fell anyway, the root cause may be that although the patient knew he or she needed help walking, the wait time for a response to the call light was too long, Dr. Beltz says.
Leaders need to identify the root cause, implement a targeted intervention and measure its effectiveness. To address the root cause in the patient fall example, a hospital may decrease call light response time or increase patients' accessibility to the call button and personal items, Dr. Beltz suggests. Hospitals can then measure the rate at which call lights are responded to and the rate at which patients use the call light to retrieve personal items. "If an RCA improvement action is still working in six months, that measure would be a more robust indication of commitment to safety," Dr. Beltz says. Responding to 100 percent of call lights within three minutes would be a stronger sign of safety commitment than distributing a certain number of fall risk assessments.
2. Be creative in developing measures of safety culture and engagement. Since there is no single measure of patient safety culture or patient engagement, leaders should look at several measures that can combine to reflect culture or engagement. Dr. Beltz says some creative measures she has seen for safety culture include the number of responses to a safety culture survey and the number and kind of disciplines represented in root cause analyses. Some examples of creative patient engagement measures include the number of patients participating in process improvement teams, community attendance at health fairs and educational events and the number of "hits" on the quality measures tab of the hospital's website, Dr. Beltz says.
It's the journey that matters
Methods to measure soft aspects of patient safety such as patient safety culture and patient engagement are far from perfect; they rely on staff perceptions and often have low response rates. However, measuring soft patient safety constructs can be useful in launching discussions about safety, identifying areas for improvement and increasing awareness of patient safety. "The commitment to trying to measure (albeit imperfectly) [safety culture] in the face of its complexity is in itself an indication of the organization's commitment to safety," Dr. Beltz says.
Does Patient Safety Culture Always Mean Safe Hospitals? Nurses Say No
3 Interventions to Improve Patient Safety Culture
Why should healthcare organizations measure "soft" aspects of patient safety?While much of healthcare data are "hard" measures — measures that are concrete and easily quantifiable — there is a growing awareness of the importance of "soft," more qualitative measures, such as patient safety culture, in improving the U.S. healthcare system. In fact, what has typically been viewed as a soft measure — patient satisfaction — now factors into hospitals' reimbursement under CMS' Value-Based Purchasing program. It is important to measure and benchmark these multifactor concepts to identify opportunities for improvement.
Patient safety culture speaks to healthcare providers' intrinsic motivation, according to Peter J. Pronovost, MD, PhD, senior vice president for patient safety and quality and director of the Armstrong Institute for Patient Safety and Quality at Baltimore-based Johns Hopkins Medicine. Dr. Pronovost is well known for developing a checklist for intensive care units that has dramatically reduced central line-associated bloodstream infections in Michigan, saving an estimated 1,500 lives and $100 million annually. "Our work in reducing ICU infections was almost entirely driven by intrinsic motivation. The softer side — this intrinsic motivation, the way we speak to each other, empowerment of the frontline — is the magic sauce of what it takes to improve safety and quality," he says.
How can healthcare organizations measure "soft" aspects of patient safety?
Unlike patient satisfaction, which hospitals have been required to report using the Hospital Consumer Assessment of Healthcare Providers and Systems survey for roughly six years, standardized patient safety culture measures are not widespread in the U.S.
However, there are some surveys of patient safety culture that are gaining ground among hospitals. The Agency for Healthcare Research and Quality released the Hospital Survey on Patient Safety Culture in 2004 to help hospitals assess their safety climate. AHRQ benchmarks data from all hospitals who submit data from the survey. In the 2012 User Comparative Database Report, AHRQ compares results from 1,128 hospitals, compared with 1,032 hospitals in 2011. HSOPS includes 42 items designed to measure 12 areas of patient safety, including teamwork within units, and supervisor/manager expectations and actions promoting patient safety.
"AHRQ has developed and offers the use of validated surveys, as well as benchmarking data. This is a huge step in the attempt to measure a complex construct," says Marti Beltz, PhD, LSSMBB, a faculty member and senior healthcare quality consultant at the American Society for Quality.
Another survey, the Safety Attitudes Questionnaire, was developed by Bryan Sexton, Eric Thomas and Bob Helmreich with funding from the Robert Wood Johnson Foundation and Agency for Healthcare Research and Quality. This survey includes items designed to measure factors such as teamwork climate and job satisfaction.
Do surveys accurately measure culture?
Although surveys can reveal hospital employees' perceptions of safety, they have flaws that prevent a completely accurate picture of an organization's culture. "While validated (particularly those offered by [AHRQ]), these instruments are only proxies used to estimate the degree to which a facility is committed to safety at all levels," Dr. Beltz says. "In addition, significant variations in safety culture may exist within an organization (e.g., between leadership and frontline workers, between one unit and another) so institutional level scores may mask these significant differences."
Conducting the survey at the unit level can be more useful than a hospital-wide survey in identifying where patient safety culture is successful and where it faces challenges. Dr. Pronovost says variation in survey results can vary as much as six- to eight-fold across a hospital. To gain a more accurate assessment of patient safety culture in the hospital, Johns Hopkins Medicine issues the surveys to individual units.
However, measuring safety culture at the unit level also presents challenges because physicians often work in many units. To include physicians in the surveys, Johns Hopkins surveys all physicians as one unit.
In addition, response rates for patient safety culture surveys can be low, which diminishes its validity as a measure of the organization's culture. Dr. Pronovost says hospitals should aim for at least a 60-percent response rate to attain representative data. "You have to be cautious about using these [data] too quantitatively," he says.
Case example: Johns Hopkins Medicine
To improve the safety culture at Johns Hopkins Medicine, the system conducts HSOPS surveys at the unit level, provides feedback on results and uses the responses to begin a discussion on safety. "By far the biggest benefit of the surveys is they start the conversation," Dr. Pronovost says. Johns Hopkins leaders work with staff at the unit level to address items in the survey.
Typically, leaders ask staff about the three highest-scored questions on the survey and the three lowest-scored items. For example, when presenting the high-scoring items, leaders can determine the unit's best practices that improved its culture and that can be spread to other units. Similarly, when discussing a low-scoring item, a leader may say, "Only 30 percent of nurses on the unit said they were able to speak up. Is that what it feels like?" Dr. Pronovost says.
"You ask staff their perceptions of how well they work together, and if they feel comfortable speaking up about mistakes — that's the superficial side. What we care about is the [underlying] values, beliefs and ultimately, behaviors," he says. "The survey takes a vague concept like safety culture and translates it into specific questions with responses we can then have a conversation with staff about."
He suggests having more in-depth discussions on safety culture through smaller focus groups. "It's different saying 'Safety culture is important; we have to improve it,' than saying 'Only 20 percent of nurses felt comfortable speaking up.' It allows you to get into these issues in a much more real and concrete way than if you didn't have these conversation starters," Dr. Pronovost says.
Tips for measuring patient safety culture
As evident from Johns Hopkins' experience, measuring patient safety culture can still be valuable despite the flaws of the measurement tool. Dr. Beltz says, "The best piece of advice I can offer to leaders is acknowledge that there is no one measure of patient safety culture and that all measures are only proxy indices of the construct." She suggests two best practices for measuring the softer aspects of patient safety:
1. Measure the root cause, not the symptom. Hospitals should conduct root cause analyses to determine the reason for adverse events to make long-lasting change. For example, to measure a commitment to safety, leaders should go beyond just counting the number of fall risk assessments conducted, according to Dr. Beltz. If a patient passed a fall risk assessment but fell anyway, the root cause may be that although the patient knew he or she needed help walking, the wait time for a response to the call light was too long, Dr. Beltz says.
Leaders need to identify the root cause, implement a targeted intervention and measure its effectiveness. To address the root cause in the patient fall example, a hospital may decrease call light response time or increase patients' accessibility to the call button and personal items, Dr. Beltz suggests. Hospitals can then measure the rate at which call lights are responded to and the rate at which patients use the call light to retrieve personal items. "If an RCA improvement action is still working in six months, that measure would be a more robust indication of commitment to safety," Dr. Beltz says. Responding to 100 percent of call lights within three minutes would be a stronger sign of safety commitment than distributing a certain number of fall risk assessments.
2. Be creative in developing measures of safety culture and engagement. Since there is no single measure of patient safety culture or patient engagement, leaders should look at several measures that can combine to reflect culture or engagement. Dr. Beltz says some creative measures she has seen for safety culture include the number of responses to a safety culture survey and the number and kind of disciplines represented in root cause analyses. Some examples of creative patient engagement measures include the number of patients participating in process improvement teams, community attendance at health fairs and educational events and the number of "hits" on the quality measures tab of the hospital's website, Dr. Beltz says.
It's the journey that matters
Methods to measure soft aspects of patient safety such as patient safety culture and patient engagement are far from perfect; they rely on staff perceptions and often have low response rates. However, measuring soft patient safety constructs can be useful in launching discussions about safety, identifying areas for improvement and increasing awareness of patient safety. "The commitment to trying to measure (albeit imperfectly) [safety culture] in the face of its complexity is in itself an indication of the organization's commitment to safety," Dr. Beltz says.
More Articles on Patient Safety Culture:
3 Steps to Address Underlying Causes of Adverse EventsDoes Patient Safety Culture Always Mean Safe Hospitals? Nurses Say No
3 Interventions to Improve Patient Safety Culture