Background: WHO recommends repeated measurement of patient safety climate in health care and to support monitoring an 11 item questionnaire on sustainable safety engagement (HSE) has been developed by the Swedish Association of Local Authorities and Regions. This study aimed to validate the psychometric properties of the HSE.

Methods: Survey responses (n = 761) from a specialist care provider organization in Sweden was used to evaluate psychometric properties of the HSE 11-item questionnaire. A Rasch model analysis was applied in a stepwise process to evaluate evidence of validity and precision/reliability in relation to rating scale functioning, internal structure, response processes, and precision in estimates.

Results: Rating scales met the criteria for monotonical advancement and fit. Local independence was demonstrated for all HSE items. The first latent variable explained 52.2% of the variance. The first ten items demonstrated good fit to the Rasch model and were included in the further analysis and calculation of an index measure based on the raw scores. Less than 5% of the respondents demonstrated low person goodness-of-fit. Person separation index > 2. The flooring effect was negligible and the ceiling effect 5.7%. No differential item functioning was shown regarding gender, time of employment, role within organization or employee net promotor scores. The correlation coefficient between the HSE mean value index and the Rasch-generated unidimensional measures of the HSE 10-item scale was r = .95 (p < .01).

Conclusions: This study shows that an eleven-item questionnaire can be used to measure a common dimension of staff perceptions on patient safety. The responses can be used to calculate an index that enables benchmarking and identification of at least three different levels of patient safety climate. This study explores a single point in time, but further studies may support the use of the instrument to follow development of the patient safety climate over time by repeated measurement.