Successful development of a direct observation program to measure health care worker hand hygiene using multiple trained volunteers

Major article

Successful development of a direct observation program to measure health care worker hand hygiene using multiple trained volunteers

W. Matthew Linam MD, MS a,*, Michele D. Honeycutt BSN, RN, CIC b, Craig H. Gilliam BSMT, CIC c, Christy M. Wisdom BSN, RN, CIC b, Shasha Bai PhD d, Jayant K. Deshpande MD, MPH e

a Pediatric Infectious Diseases Section, Department of Pediatrics, University of Arkansas for Medical Sciences, Little Rock, AR b Infection Prevention and Control Department, Arkansas Children’s Hospital, Little Rock, AR c Infection Prevention and Control Department, St. Jude Children’s Research Hospital, Memphis, TN d Biostatistics Section, Department of Pediatrics, University of Arkansas for Medical Sciences, Little Rock, AR e Departments of Pediatrics and Anesthesiology, University of Arkansas for Medical Sciences, Little Rock, AR

Key Words: Quality improvement Performance measurement Patient safety measures

Background: Direct observation of health care worker (HCW) hand hygiene (HH) remains the gold stan- dard, but implementation is challenging. Our objective was to develop an accurate HH observation program using multiple HCW volunteers. Methods: HH compliance was defined as correct HH performed before and after contact with a patient or a patient’s environment. HCW volunteers from each unit at our children’s hospital were trained by infection preventionists to covertly collect HH observations during routine care using an electronic tool. Questionnaires sent to observers in February and December 2014 recorded demographic characteris- tics, observation time, and scenarios assessing accuracy. HCWs were surveyed regarding their awareness that their HH behavior was being recorded. Results: There were 146 HH observers. The majority of observers reported making 1-2 observations per shift (65%) and taking ≤10 minutes recording an observation (85%). Between January 2012 and December 2014 there were 22,484 HH observations (average, 622 per month), including nurses (46%), physicians (21%), and other HCWs (33%). Observers correctly recorded HH behavior more than 90% of the time in 5 of the 6 scenarios. Most HCWs (86%) were unaware they were being observed. Conclusion: A direct observation program staffed by multiple HCW volunteers can inexpensively and accurately collect HCW HH data. © 2016 Association for Professionals in Infection Control and Epidemiology, Inc. Published by Elsevier

Inc. All rights reserved.

Accurate measurement of hand hygiene behavior by health care workers (HCWs) is crucial to improvement efforts.1,2 Covert direct observation of hand hygiene practices during routine patient care remains the gold standard, but it presents a number of challenges, including significant cost and time investment.3,4 In addition, direct observation programs usually only capture a small sample of all hand hygiene opportunities, may not accurately measure the hand hygiene events, and may be biased due to Hawthorne effect.3-8

Despite these challenges, direct observation is currently the only strategy capable of measuring all 5 key indications for hand hygiene and evaluating technique. It is also among the few strategies that can differentiate compliance by HCW type.4 Electronic applications have been developed capable of assisting observers and reducing the time requirementsof directobservation.9,10 Attemptstoobtainamorerep- resentative sample of hand hygiene data and reduce observation bias have resulted in the development of a number of different auto- mated hand hygiene monitoring systems.4,9 Unfortunately, these systems often require significant cost to install and maintain.4,9 In ad- dition, because situational context is not accounted for, data may be biased toward lower compliance.11 A direct observation program capable of inexpensively collecting a representative sample of HCW hand hygiene data and minimizing bias is needed.

Our objective was to develop a hand hygiene observation program using multiple trained HCW volunteers capable of accurately mea- suring hand hygiene behavior and minimizing Hawthorne effect.

* Address correspondence to W. Matthew Linam, MD, MS, Pediatric Infectious Diseases Section, Department of Pediatrics, University of Arkansas for Medical Sciences, 1 Children’s Way, Slot 512-11, Little Rock, AR 72202-3500.

E-mail address: wlinam@uams.edu (W.M. Linam). Financial support: None. Presented in part at the Association for Professionals in Infection Control and

Epidemiology 41st Annual Conference, Anaheim, California, June 7-9, 2014. Conflicts of interest: None to report.

0196-6553/© 2016 Association for Professionals in Infection Control and Epidemiology, Inc. Published by Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.ajic.2015.12.019

American Journal of Infection Control 44 (2016) 544-7

Contents lists available at ScienceDirect

American Journal of Infection Control

j o u r n a l h o m e p a g e : w w w. a j i c j o u r n a l . o r g

American Journal of Infection Control

METHODS

This program was developed at Arkansas Children’s Hospital, a 370-bed tertiary children’s hospital. There are 14 inpatient units, including 4 critical care units and a hematology–oncology unit.

Appropriate hand hygiene practices of HCWs were defined based on published guidelines.1,2 Hand hygiene compliance was defined as correct hand hygiene performed before and after contact with a patient or a patient’s care area. For patients on transmission- based isolation precautions, hand hygiene was required before donning and after doffing personal protective equipment.

HCW volunteers from day, night, and weekend shifts were re- cruited from each inpatient unit, with a goal of at least 4 observers on each unit. In general, the identities of the observers remained secret. Hand hygiene observers were trained by infection preventionists (IPs) before recording hand hygiene observations. A single hand hygiene observation required the ability to witness the hand hygiene practices of the HCW both before and after contact with a patient or a patient’s care environment. Partial observa- tions were aborted. Each observer was expected to make at least 10 observations each month. Observations were to be collected on a variety of HCW types. All observers were required to complete annual retraining and attend quarterly observer team meetings.

As part of the ongoing education and interrater reliability as- sessment, a 16-item electronic questionnaire was sent to each hand hygiene observer in February and December 2014. Questions in- cluded observer demographic characteristics and time spent making observations. The questionnaire also included 6 scenarios repre- senting common observation situations and assessed the accuracy of their observations. Correct responses for the scenarios were com- pared by 2-sample proportion test (February vs December). P values < .05 were considered significant. All statistical analysis was per- formed using SAS 9.4 (SAS Institute Inc, Cary, NC), or R version 3.1.2 (R Foundation for Statistical Computing, Vienna, Austria).

A separate electronic questionnaire was sent to all HCWs during April and May 2014 to assess their level of awareness that their hand hygiene behavior was being recorded by observers. HCWs rated their awareness that their hand hygiene behavior was being recorded at the time it was being observed using a Likert scale (never aware, rarely aware, occasionally aware, frequently aware, always aware, or not sure). Data were summarized as frequencies and percentages.

Hand hygiene observations were recorded electronically on touch screens located throughout the units. Additional data collected in- cluded date, time and shift of the observation, unit location, HCW type, and patient’s transmission-based isolation status. Observa- tion data were transmitted real-time to an electronic data visualization program that was available to all staff and was capable of sorting the data in a variety of ways to better inform improve- ment efforts. A run chart was created using Microsoft Excel (Redmond, WA) to display monthly hand hygiene compliance over time and annotated to show the relationship between interven- tions and the monthly hand hygiene compliance. Because hand hygiene compliance has been shown to be higher when measured by unit-based observers compared with data from nonunit-based observers, we compared hand hygiene compliance data from unit- based and nonunit-based observers.12

RESULTS

Hand hygiene observers

Of the 146 hand hygiene observers, 101 (69%) completed the questionnaire during December 2014. The HCW hand hygiene ob- servers were mostly nurses (90%) and represented all inpatient units. Most (65%) reported being hand hygiene observers for more than

12 months. The majority of observers reported making 1-2 obser- vations (65%) or making 3-5 observations (30%) during a single shift. A single hand hygiene observation required 10 minutes or less for 85% of the observers. The remaining observers required 11-15 minutes. Eighty percent of the 102 observers in February com- pleted the survey. Responses were similar.

Observer interrater reliability

Hand hygiene observer responses on the validation question- naire in February and December 2014 are shown in Table 1. Observers recorded hand hygiene behavior correctly more than 90% of the time in 5 of the 6 hand hygiene scenarios. Scenario 6 had the fewest number of correct responses (36% in February and 47% in December). This scenario involved a HCW briefly entering a patient room without touching the patient or patient care environment. Al- though observers were asked to record this scenario in a specific way (abort the observation), depending on interpretation, any of the responses could be considered correct. Ongoing education resulted in an 11% improvement.

Hawthorne effect

There were 681 HCWs (63% nurses, 15% physicians, and 23% other HCW types) who completed the separate observation awareness questionnaire. Most (86%) were never aware or rarely aware that their hand hygiene practices were being observed at the time the observation was being made.

Hand hygiene observation data

Between January 1, 2012, and December 31, 2014, there were 22,484 complete hand hygiene observations recorded with an average of 622 observations per month. This included the obser- vation of the hand hygiene behavior of 10,323 nurses (46%), 4,692 physicians (21%), and 7,469 observations (33%) of other HCW types (eg, patient care technicians, respiratory therapists, and various an- cillary staff). Almost one-third (28%) of the hand hygiene observations were recorded for patients on transmission-based isolation pre- cautions. Half the observations (53%) were recorded during day shifts and 24% of the observations were recorded during weekend shifts. The annotated run chart shows the change in hand hygiene com- pliance over time (Fig 1). Hand hygiene compliance gradually increased from a baseline of 75% to sustained compliance of 95%. Hand hygiene compliance averaged 9% higher for unit-based ob- servers compared with nonunit-based observers (range by year, 4%-12%).

DISCUSSION

We successfully developed a program to directly measure HCW hand hygiene compliance using more than 100 trained HCW ob- servers. Observations were collected on all units, shifts, and HCW types. In general, HCWs were not aware that they were being ob- served. Thus, Hawthorne effect was minimized.

Compared with other measurement strategies, direct observa- tion of hand hygiene behavior provides the greatest detail regarding HCW hand hygiene, which allows tailoring of improvement efforts.3,4

Despite these benefits, there are important limitations. Direct ob- servation programs reported in the literature rarely describe details of observer training and whether interrater reliability is assessed.4,13,14

The time and associated costs required for employees to monitor hand hygiene limit the number of observations that can be made. At best, direct observation programs only collect 1%-3% of hand hygiene opportunities.4,8,15,16 Unfortunately, this may not accurately

545W.M. Linam et al. / American Journal of Infection Control 44 (2016) 544-7

Table 1 Hand hygiene observer responses during February and December 2014 on an electronic questionnaire assessing the accuracy of their hand hygiene observations

Scenario Key point(s) February response December response P value*

1. The HCW enters a room and performs hand hygiene with ETOH gel. The observer is called away and unable to observe whether or not hand hygiene is performed upon exiting the room

The observer must be able to witness hand hygiene behavior both before and after contact with the patient or patient care area. Otherwise the observation should be aborted

Compliant: 0 (0) Noncompliant: 0 (0) Abort: 88 (100)

Compliant: 1 (1) Noncompliant: 0 (0) Abort: 99 (99) Missing: 1

.99

2. The HCW is observed entering a patient room. No hand hygiene is performed. A surgical wound is examined. The HCW performs hand hygiene with the ETOH gel upon exiting the room

Correct hand hygiene must be performed both before and after contact with a patient or patient care area to be considered compliant

Compliant: 0 (0) Noncompliant: 83 (94) Abort: 5 (6)

Compliant: 1 (1) Noncompliant: 96 (95) Abort: 4 (4)

.86

3. The HCW performs hand hygiene with soap and water upon entering a patient’s room. A paper towel is used to turn off the faucet. A physical exam is performed. The HCW performs hand hygiene with the ETOH gel upon exiting the room

An HCW does not need to use the same hand hygiene product for before and after contact with the patient or patient care area. If an HCW washes his or her hands with soap and water, the faucet must be turned off without recontamination of the hands

Compliant: 86 (99) Noncompliant: 1 (1) Abort: 0 (0) Missing: 1

Compliant: 98 (99) Noncompliant: 1 (1) Abort: 0 (0) Missing: 2

.99

4. The HCW enters the room of a patient on transmission-based isolation precautions. Hand hygiene is not performed before PPE is donned. The HCW administers a breathing treatment. PPE is removed and hand hygiene is performed using the ETOH gel upon exiting the room

Hand hygiene should be performed before donning and after doffing personal protective equipment

Compliant: 2 (2) Noncompliant: 80 (92) Abort: 5 (6) Missing: 1

Compliant: 2 (2) Noncompliant: 94 (93) Abort: 5 (5)

.99

5. The HCW enters a patient’s room to silence an intravenous pump alarm. No hand hygiene is performed before entering or exiting the patient’s room to silence the alarm

An HCW must perform hand hygiene both before and after contact with the patient care area even if there is no contact with the patient

Compliant: 1 (1) Noncompliant: 84 (95) Abort: 3 (3)

Compliant: 5 (5) Noncompliant: 92 (93) Abort: 2 (2) Missing: 2

.28

6. The HCW enters a patient’s room to relay information regarding scheduling of a diagnostic imaging procedure. No contact is made with the patient or the patient’s care environment. No hand hygiene is performed before entering or exiting the patient’s room

If an HCW enters a patient care area and makes no contact with the patient or the patient care area, the observation should be aborted

Compliant: 35 (40) Noncompliant: 21 (24) Abort: 32 (36)

Compliant: 36 (36) Noncompliant: 17 (17) Abort: 47 (47) Missing: 1

.28

NOTE. Values are presented as n (%). Correct responses are in boldface type. ETOH, alcohol; HCW, health care worker; PPE, personal protective equipment. *Significant at P < .05.

30%

40%

50%

60%

70%

80%

90%

100%

01 /0

1/ 20

12

02 /0

1/ 20

12

03 /0

1/ 20

12

04 /0

1/ 20

12

05 /0

1/ 20

12

06 /0

1/ 20

12

07 /0

1/ 20

12

08 /0

1/ 20

12

09 /0

1/ 20

12

10 /0

1/ 20

12

11 /0

1/ 20

12

12 /0

1/ 20

12

01 /0

1/ 20

13

02 /0

1/ 20

13

03 /0

1/ 20

13

04 /0

1/ 20

13

05 /0

1/ 20

13

06 /0

1/ 20

13

07 /0

1/ 20

13

08 /0

1/ 20

13

09 /0

1/ 20

13

10 /0

1/ 20

13

11 /0

1/ 20

13

12 /0

1/ 20

13

01 /0

1/ 20

14

02 /0

1/ 20

14

03 /0

1/ 20

14

04 /0

1/ 20

14

05 /0

1/ 20

14

06 /0

1/ 20

14

07 /0

1/ 20

14

08 /0

1/ 20

14

09 /0

1/ 20

14

10 /0

1/ 20

14

11 /0

1/ 20

14

12 /0

1/ 20

14

Final Values

Median

Goal

Month

P er

ce n

ta ge

H an

d H

yg ie

n e

C o

m p

lia n

ce

Hand hygiene improvement project ini�ated. Mul�ple interven�ons implemented.

Interven�on Interven�on

Fig 1. Run chart showing health care worker hand hygiene compliance by month, January 2012-December 2014.

546 W.M. Linam et al. / American Journal of Infection Control 44 (2016) 544-7

capture hand hygiene behavior, which frequently varies by day, shift, location, and HCW type.2,17,18 Observer bias, which can occur when unit-based staff preferentially observe HCWs who are being com- pliant or give coworkers the benefit of the doubt, results in overestimation of hand hygiene compliance. In 1 study, hand hygiene data of unit-based observers were 22.8% higher than nonunit- based observers.12 Finally, the Hawthorne effect, which describes a subject’s tendency to alter his or her behavior based on the awareness that they are being observed, can also affect the accu- racy of direct hand hygiene observations.5-7,19 Hand hygiene compliance often increases the longer an observer remains in a par- ticular location,6 and this can occur as early as 15 minutes into an observation period.20

Our direct observation program addressed many of these limi- tations. By using more than 100 trained HCW observers, we were able to collect a large amount of hand hygiene data, including all days, shifts, units, and HCW types while limiting the time require- ment of individual observers. Although our number of observations likely represented a small percentage of all hand hygiene oppor- tunities, the distribution of observations by shift and HCW type was similar to published data recording HCW hand hygiene opportunities.8 Minimizing observation time also allowed the ob- server team to be sustained by HCW volunteers at no added cost. The IP leading the observation program development spent 20% of her time during the first 3 months and then 5% or less of her time afterward maintaining the observation program. Ensuring that hand hygiene data are recorded consistently is a challenge for any direct observation program, especially with a large number of observers. We developed a standardized process to provide initial and ongoing training for our observers. Data from our validation survey showed most observers recorded hand hygiene scenarios correctly, and this persisted over time, suggesting that, overall, our training process was effective. The exception to this was scenario 6. Despite some improvement with education, inconsistent interpretation of this sce- nario persisted. Hand hygiene scenarios that are less clear-cut may require more training to ensure they are recorded accurately. Because most observers made only a few observations per shift during routine care, the majority of HCWs were unaware they were being ob- served. This suggests the Hawthorne effect was limited. Finally, the use of electronic data collection tools streamlined data collection and allowed for real-time feedback to staff.

There were some limitations with this program. Our hand hygiene observation program was implemented in a single children’s hos- pital and may not be easily spread to other settings. We only measured moments 1, 4, and 5 of the World Health Organization “My Five Moments for Hand Hygiene.”2 Despite ongoing training, our validation assessment suggests that some observations may have been recorded incorrectly. Most of the incorrect responses were related to scenario 6. Excluding scenario 6, an average of 96% and 95% of observers answered the remaining 5 scenarios correctly in February and December, respectively. It is possible that responses may have differed between HCWs completing the questionnaires and those who did not. Similar to other studies, we found in- creased compliance reported by unit-based observers compared with nonunit-based observers, but this was less than seen in other reports.12 Although HCWs reported minimal awareness of being ob- served, it is still possible their hand hygiene may have been positively affected.

CONCLUSIONS

We developed a direct observation program staffed by a large number of HCW volunteer observers that inexpensively collects a representative sample of HCW hand hygiene data with minimal Hawthorne effect. A standardized process for ongoing training is

essential to maintain data accuracy and minimize observer bias and should be tailored over time to address drift and misinterpreta- tion. Because many health care organizations continue to struggle to collect accurate HCW hand hygiene data, our hope is that this program may provide a framework that can be successfully adopted at other hospitals.

Acknowledgements

The authors thank the numerous HCW volunteers who make the hand hygiene program possible. The authors also thank the members of the Systems Development Group at Arkansas Children’s Hospi- tal for their assistance in development of the electronic data collection and observation programs and Angela Green, PhD, RNc, CPHQ, FAHA, FAAN, for providing editorial support.

References

1. Boyce JM, Pittet D. Guideline for hand hygiene in health-care settings: recommendations of the healthcare infection control practices advisory committee and the HICPAC/SHEA/APIC/IDSA hand hygiene task force. Infect Control Hosp Epidemiol 2002;23(12 Suppl):S3-40.

2. Pittet D, Allegranzi B, Boyce J. The World Health Organization guidelines on hand hygiene in health care and their consensus recommendations. Infect Control Hosp Epidemiol 2009;30:611-22.

3. Haas JP, Larson EL. Measurement of compliance with hand hygiene. J Hosp Infect 2007;66:6-14.

4. Boyce JM. Measuring healthcare worker hand hygiene activity: current practices and emerging technologies. Infect Control Hosp Epidemiol 2011;32:1016-28.

5. Eckmanns T, Bessert J, Behnke M, Gastmeier P, Ruden H. Compliance with antiseptic hand rub use in intensive care units: the Hawthorne effect. Infect Control Hosp Epidemiol 2006;27:931-4.

6. Stone S, Fuller C, Michie S, McAteer J, Charlett A. What is the optimal period for measuring hand hygiene compliance: are longer periods better than 20- minute periods? Infect Control Hosp Epidemiol 2012;33:1174-6.

7. Kohli E, Ptak J, Smith R, Taylor E, Talbot EA, Kirkland KB. Variability in the Hawthorne effect with regard to hand hygiene performance in high- and low-performing inpatient care units. Infect Control Hosp Epidemiol 2009;30:222- 5.

8. Fries J, Segre AM, Thomas G, Herman T, Ellingson K, Polgreen PM. Monitoring hand hygiene via human observers: how should we be sampling? Infect Control Hosp Epidemiol 2012;33:689-95.

9. Ward MA, Schweizer ML, Polgreen PM, Gupta K, Reisinger HS, Perencevich EN. Automated and electronically assisted hand hygiene monitoring systems: a systematic review. Am J Infect Control 2014;42:472-8.

10. Hlady CS, Severson MA, Segre AM, Polgreen PM. A mobile handheld computing application for recording hand hygiene observations. Infect Control Hosp Epidemiol 2010;31:975-7.

11. Swoboda SM, Earsing K, Strauss K, Lane S, Lipsett PA. Electronic monitoring and voice prompts improve hand hygiene and decrease nosocomial infections in an intermediate care unit. Crit Care Med 2004;32:358-63.

12. Dhar S, Tansek R, Toftey EA, Dziekan BA, Chevalier TC, Bohlinger CG, et al. Observer bias in hand hygiene compliance reporting. Infect Control Hosp Epidemiol 2010;31:869-70.

13. Gould DJ, Chudleigh J, Drey NS, Moralejo D. Measuring handwashing performance in health service audits and research studies. J Hosp Infect 2007;66:109-15.

14. Jeanes A, Coen PG, Wilson AP, Drey NS, Gould DJ. Collecting the data but missing the point: validity of hand hygiene audit data. J Hosp Infect 2015;90:156-62.

15. van de Mortel T, Murgo M. An examination of covert observation and solution audit as tools to measure the success of hand hygiene interventions. Am J Infect Control 2006;34:95-9.

16. Marra AR, Moura DF Jr, Paes AT, dos Santos OF, Edmond MB. Measuring rates of hand hygiene adherence in the intensive care setting: a comparative study of direct observation, product usage, and electronic counting devices. Infect Control Hosp Epidemiol 2010;31:796-801.

17. Erasmus V, Daha TJ, Brug H, Richardus JH, Behrendt MD, Vos MC, et al. Systematic review of studies on compliance with hand hygiene guidelines in hospital care. Infect Control Hosp Epidemiol 2010;31:283-94.

18. Pittet D, Mourouga P, Perneger TV. Compliance with handwashing in a teaching hospital. Infection control program. Ann Intern Med 1999;130:126-30.

19. Hagel S, Reischke J, Kesselmeier M, Winning J, Gastmeier P, Brunkhorst FM, et al. Quantifying the Hawthorne effect in hand hygiene compliance through comparing direct observation with automated hand hygiene monitoring. Infect Control Hosp Epidemiol 2015;36:957-62.

20. Yin J, Reisinger HS, Vander Weg M, Schweizer ML, Jesson A, Morgan DJ, et al. Establishing evidence-based criteria for directly observed hand hygiene compliance monitoring programs: a prospective, multicenter cohort study. Infect Control Hosp Epidemiol 2014;35:1163-8.

547W.M. Linam et al. / American Journal of Infection Control 44 (2016) 544-7