Speaking up culture of medical students within an academic teaching hospital: Need of faculty working in patient safety
David Schwappach aff001; Gerald Sendlhofer aff003; Lars-Peter Kamolz aff004; Wolfgang Köle aff005; Gernot Brunner aff004
Authors place of work:
Swiss Patient Safety Foundation, Zurich, Switzerland
aff001; Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
aff002; Executive Department for Quality and Risk Management, University Hospital Graz, Graz, Austria
aff003; Research Unit for Safety in Health, c/o Division of Plastic, Aesthetic and Reconstructive Surgery, Department of Surgery, Medical University of Graz, Graz, Austria
aff004; Department of General Otorhinolaryngology, Medical University of Graz, Graz, Austria
aff005; Medical Directorate, University Hospital Graz, Graz, Austria
Published in the journal:
PLoS ONE 14(9)
Speaking up behavior is a manifestation the culture of safety in an organization; however, withholding voice is commonly observed. Within one academic teaching hospital, it was the aim to assess students’ speaking up behaviors and perceived culture in order to stimulation of the academic development in terms of patient safety.
Survey amongst medical students using a validated questionnaire. Data were analysed using descriptive statistics.
326 individuals completed the questionnaire (response rate 24%). 37% of responders were in their 5th- 6th clinical term, 32% were in their 7th-8th term and 31% were in the 9th-12th term. 69% of students had a specific safety concern in the past four weeks, 48% had observed an error and 68% noticed the violation of a patient safety rule. Though students perceived specific patient safety concerns, 56% did not speak up in a critical situation. All predefined barriers seemed to play an important role in inhibiting students’ voicing concerns. The scores on the psychological safety scale were overall moderately favourable. Students felt little encouraged by colleagues and, in particular, by supervisors to speak up.
Speaking up behaviour of students was assessed for the first time in an Austrian academic teaching hospital. The higher the term the more frequent students reported perceived patient safety concerns or rule violations and withholding voice. These results suggest the need to adapt the curriculum concept of the faculty in order to address patient safety as a relevant topic.
Medicine and health sciences – Health care – Patients – Health care providers – Nurses – Medical humanities – Research and analysis methods – Research design – Survey research – Surveys – Mathematical and statistical techniques – Statistical methods – Analysis of variance – Regression analysis – People and places – Population groupings – Professions – Medical personnel – Supervisors – Biology and life sciences – Psychology – Behavior – Social sciences – Sociology – Education – Medical education – Physical sciences – Mathematics – Statistics
Speaking up can be defined as assertive communication of quality and patient safety concerns by a team member through information, questions, or opinions in situations where clinical rules are neglected, forgotten or even unknown by a healthcare professional (HCP) in order to avoid patient harm [1–9]. Effective teamwork and non-technical skills are essential for the reduction of medical errors . Speaking up by HCPs to their colleagues is increasingly acknowledged as an important way to reduce risks [8, 11,12]. However, withholding voice is common and was confirmed recently within our university hospital . More than half of HCPs perceived specific concerns about patient safety at least once in the last four weeks, however more than 40% remained silent in a critical situation . This HCPs’ silence or withholding voice may have many reasons and the existence of steep hierarchies within HCPs may be the most dominant .
Within our academic teaching hospital, more than 7.000 employees care for patients and more than 3.000 employees are responsible for science and teaching. As medical students have to attend several clinical internships in their academic career, they become automatically junior members of care teams and may therefore also play an important role in supporting patient safety . In 2012 a study in Australia revealed that more than 80% of students were willing to speak up towards others students, however, this number decreased for those who were willing to do so to HCPs . Reasons for not speaking up amongst students had been reluctance to question senior staff, unwillingness to interrupt and embarrassment [10,13]. A nursing study also showed that during internships students were frequently asked to condone poor practice .
It is obvious that courage is needed to speak up as it demands the readiness to stand up and to do the right thing at the right time [15,16]. However, it is also known that students diminish their role during internships to that of being “just a student” . Students are, to a certain degree, in a bind. On the one hand, they have to complete and pass the internships and, on the other hand, they are evaluated by HCPs. Speaking up could therefore cause a conflict for medical students as they are alerting a doctor to non-compliance of a clinical rule . And most important, a safety culture needs to be in place giving students and HCPs the certainty to do so.
So far, in German speaking countries no data are available concerning speaking up behaviors of medical students, in particular, in relation to their study progress and years of training. To assess the speaking up behavior and perceived culture in one academic teaching hospital in Austria a validated questionnaire was used to assess self-reported speaking up related behaviors, climate and perceived barriers. Results may give potential implications on the development of a patient safety curriculum.
Materials and method
The Ethics Committee of Medical University of Graz approved the study (vote#: 30–303 ex17/18).
A cross-sectional survey study was conducted among medical students doing their clinical terms at one large Austrian academic teaching hospital. The diploma in human medicine prepares students for their future professions as doctors in all specialties. Theoretical and practical skills are taught in an integral, subject-centred and patient-oriented way. The first part of the human medicine study (four terms) provides knowledge and basic understanding of the human organism. Essential components into general medicine, insight into nursing in the context of a ward internship, which ensures early patient contact are part of the first term. In the second part of the human medicine study (six terms) students develop the knowledge about the healthy and sick organism. It is based on topic-centered, patient-centered, interdisciplinary teaching, including clinical presentations and using new forms of teaching such as problem-based learning. Increasing training of medical skills and abilities continuously ensures the practical application of theoretically acquired knowledge. Students have to complete clinical terms for at least of 18 weeks during the second part of the study. Each clinical term lasts for four weeks and they rotate between units. Finally, during the clinical practical year (2 terms) students are actively integrated as learning team members in everyday clinical practice. In total, students must be clinically active for 48 weeks in a teaching hospital. They deepen and acquire the competences listed in the “Austrian competence level catalogue” for medical skills in accordance with European requirements .
In the present study, students in their 5th to 12th term were included and invited by the Organisational Unit for Studies and Teaching to participate via an online survey. It could have happened that not all students between the 5th and 10th term were recently exposed to a four weeks clinical term. They were instructed that their responses were collected anonymously and used only in aggregate. The survey was open for four weeks. As proposed by Burns et al., weekly reminders were automatically sent by the system to non-responders in order to increase the response rate . There was no sample size estimation as the study had an explorative character, solely.
Survey instrument and measures
We used the ‘Speaking Up about Patient Safety Questionnaire (SUPS-Q)’. Development and psychometric properties of this survey have been reported in detail . The instrument was originally developed in Switzerland and has been applied to health care staff in diverse hospitals [1, 11]. For this study, the Austrian version of the SUPS-Q was used . Few minor language adjustments were made to ensure that the survey fits the clinical context of medical students. In particular, questions addressing personal data were omitted, e.g., occupational group and clinical function. A pre-test among 100 medical students revealed no problems (e.g., no excessive missing data or ceiling effects). Qualitative content analysis of open-ended responses were categorized using the software program “MAXQDA 12”. Of all open-ended responses, some outstanding comments were translated into English. If names or units where mentioned, they were deleted.
The SUPS-Q is a brief questionnaire that assesses health care workers’ perceived patient safety concerns, their past speaking up behaviors, perceived barriers for speaking up, their evaluations of the speaking up climate at their workplace and their anticipated speaking up behavior.
The frequency of speak up-related behaviours includes three scales addressing 1) the frequency of perceived safety concerns (3 items), 2) the frequency of withholding voice, i.e., not to speak up in specified situations (4 items) and 3) the frequency of speaking up (4 items). Response options for the items in these scales are anchored to “in the last four weeks” and include “never” (0 times in the last 4 weeks), “rarely” (1–2 times), “sometimes” (3–5 times), “often” (6–10 times) and “very often” (more than 10 times during the last 4 weeks). Thus, higher mean scale values indicate higher frequencies of past speaking up and withholding voice behaviours respectively.
Barriers for speaking up are assessed with one multiple choice item asking for the relevance of 6 predefined barriers in bringing up patient safety concerns (yes/no).
Speak up-related climate is assessed by 11 items organized in 3 sub-scales: 1) the psychological safety for speaking up scale (5 items), 2) the encouraging environment for speaking up scale (3 items) and 3) the resignation scale (3 items). The answers are coded in a 7-point-Likert scale from “strongly disagree with this statement” to “strongly agree with this statement”. Thus, higher mean scale scores indicate higher levels of perceived psychological safety at workplace, higher levels of perceiving the workplace as encouraging speaking up and higher levels of resignation with speaking up respectively. A high level of resignation is indicative of a poorer safety culture. Survey items are presented in the tables in this manuscript.
The survey also covers a clinical vignette that describes a hypothetical speaking up situation. The vignette reads “You are on a daily round with several doctors and nurses. During the round, the attending doctor shakes hands with a patient who recently had surgery. He wants to examine the patient’s wound. However, the attending does not apply gloves and/or does not disinfect their hands.” Responders were instructed to consider their anticipated behaviours if they would find themselves in the situation. They were asked to complete four questions addressing the realism of the situation, the potential for patient harm, their discomfort with and likelihood of speaking up. These questions each used a 1–7 response scale with specifically labelled poles. The survey closes with an open-end question inviting responders to provide feedback, personal experiences with speaking up and comments.
Descriptive statistics (mean, standard deviation (SD)) are reported for all items. For items organized in scales, mean scale scores were computed as aggregate measures. Cronbach’s alpha was computed as a measure of scale reliability. To determine associations of survey responses with study term (i.e., length of experience in the hospital setting) nonparametric test for trend across terms and analysis of variance (ANOVA) were used as appropriate. We used the nonparametric test for trend across ordered groups developed by Cuzick, which is an extension of the Wilcoxon rank-sum test. A correction for ties is incorporated into the test . Regression analysis was conducted to model students’ reported likelihood to speak up in the situation described in the vignette (i.e., the missed hand disinfection) as outcome. Students’ rating of realism of the situation, rating of potential of harm for the patient, past withholding voice and past speaking up behaviours, their evaluation of the speak up related climate at the hospital, and their current term of studies were included as independent variables. All tests were two-sided and a p-value < 0.05 was regarded statistically significant. All analyses were conducted using STATA V14.
Of invited students (n = 1.524), 326 individuals completed the questionnaire (response rate 24%). At the time of survey participation, 37% of responders were in their 5th- 6th clinical term, 32% were in their 7th-8th term and 31% were in the 9th-12th term. Of participating students, 53% were females. The largest fraction of responders spent their last training in the department of surgery (36%), followed by internal medicine (15%), pediatrics (8%), orthopedics and traumatology (7%), anesthesiology and intensive care (6%), gynecology and obstetrics (6%), and neurology (6%).
Safety concerns and speaking up behaviors
Students commonly reported perceived patient safety concerns as shown in Table 1. For example, more than two thirds experienced safety rule violations by colleagues at least once during the last four weeks. Frequency of safety concerns was significantly associated with higher clinical term. Among students in their 5th-6th term, 64% responded they had not observed a potentially harmful error during the past four weeks whereas it were 41% of students in their 9th-12th term (test for trend p<0.001). Rule violations were observed at least once during the past four weeks by 57% of students in their 5th-6th term and by 78% of students in their 9th-12th term (test for trend p = 0.002). The majority of students reported at least one episode of withholding voice despite safety concerns during the past four weeks (Table 1). A trend of remaining silent was seen for students of higher terms. For example, the fraction of students reporting they had chosen not to bring up specific concerns about patient safety was 48%, and 68% of students in their 5th-6th and 9th-12th term, respectively. Nearly half responders reported at least one episode of speaking up in the past four weeks (Table 1).
All predefined barriers seem to play an important role in inhibiting students’ voicing concerns (Fig 1). The reason cited most often was “reaction of the person causing concern is not possible to predict”, indicated by 69%. “Lack of clarity about the risks of a situation”, “fear of negative consequences”, “presence of patients or relatives” and “ineffectiveness of speaking up (it does not make a difference)” were mentioned as factors inhibiting speaking up by 36%–48% of students. Reported barriers were not associated with study term.
Speaking up related climate
Table 2 reports the results of the speak up related climate items. The scores on the psychological safety scale were overall moderately favourable. However, there was only a low level of support for the statement that “the culture in my unit/clinical area makes it easy to speak up about patient safety concerns.” In addition, students felt little encouraged to speak up by colleagues and, in particular, by supervisors. The mean score on the “encouraging environment for speaking up” scale of 3.1 (SD 1.6) is low. Contrary, levels of resignation were already substantial in students (mean score of the “resignation towards speaking up” scale: 3.6 (SD 1.4)). There were no differences in mean responses related to study term on the item and on the scale level, with one exception. Students in their 9th-12th term were more likely to agree with the statement “when I have concerns regarding patient safety, it is difficult to submit them” (mean 5th-6th term: 3.1 (SD 1.7); mean 7th-8th term: 2.9 (SD 1.7); mean 9th-12th term: 3.7 (SD 1.9); p = 0.0022).
When confronted with the missed hand disinfection vignette, most students rated this situation as realistic (mean score 5.3, SD 1.8) and as posing a severe risk to the patient (mean score 4.9, SD 1.3). Students reported a high level of discomfort to speak up in the described situation (mean score 6.0, SD 1.4) and–on average—anticipated their likelihood to speak up as low (mean score 2.3, SD 1.6), despite the perceived potential harm for patients. Anova analysis revealed significant differences in perceived realism and potential for harm relative to study term, but not in level of discomfort and likelihood to speak up (Fig 2).
Table 3 reports the results of the regression analysis with likelihood to speak up as outcome in the vignette. Higher perceived risk of harm for the patient, past episodes of speaking up and a more favourable climate score were all significantly associated with a higher self-reported likelihood to speak up. Contrary, higher ratings of realism of the situation and a history of withholding voice behaviours decreased the self-reported likelihood that students would speak up. After accounting for these factors, study term was not independently related to self-reported likelihood to speak up.
Feedback of students
In sum, students gave 90 open-ended responses. Students’ free text feedback was summarized in two main categories: 1) experiences of students and 2) solutions (Table 4). Within the first category, hierarchy was the most commonly mentioned challenge for speaking up (mentioned by n = 11). Further challenges were unknown consequences (n = 4), fear of verbal attacks of HCPs who received speaking up (n = 3), the dependence of students on their mentors (n = 2) or the violation of polite manners (n = 2). Students also reported positive (n = 6) and negative experiences (n = 16) of patient safety relevant occurrences. In category two, all answers were clustered concerning solutions to improve the speaking up culture. Most often more courageous and more active feedback were mentioned (n = 5). Furthermore, training on patient safety and safety culture was proposed (n = 3).
This study is the first to report speaking up behaviours and climate for medical students in one Austrian academic teaching hospital using the validated SUPS-Q in German speaking countries. Overall, medical students reported perceived patient safety concerns, which increased significantly with students’ term. Furthermore, rule violations were also observed more often by students of higher terms. Although students perceived patient safety concerns and rule violations, they remained silent, which was more obvious in students of higher terms. The majority reported that an important role in inhibiting students’ voicing concerns was the reaction of the person causing a concern is not possible to predict. All predefined barriers seemed to also play an important role in withholding voice. Students felt also less encouraged to speak up which was associated with a missing safety culture in the clinical departments during their internships. It was raised that it is difficult to submit patient safety concerns. Students considered the described vignette to be a realistic scenario in a healthcare setting; however, the likelihood to speak up was low and was accompanied by a high level of discomfort to speak up.
67% of students had at least once a specific concern about patient safety in the past four weeks, 48% had observed an error and 68% noticed the violation of a patient safety rule. Though students perceived specifics patient safety concerns, 56% did not speak up in a critical situation during the last four weeks. Perceived concerns of students were similar to that of surveyed HCPs the year before using the same survey instrument: 69% of HCPs had at least once a specific concern in the past four weeks, 57% had observed an error, 58% noticed HCPs who ignored a patient safety rule, but 42% remained silent . It was also obvious in the latter study that HCPs gave a socially desirable response as nearly 100% of HCPs reported having spoken up. Results of the present study put the previous in a different light. Students are attentive observers and, as student’s feedback indicated, that the everyday life in the hospital is characterized by hierarchical structures, they fear to speak up. Reasons for not speaking up might be that the psychological safety for speaking up was only moderately expressed, accompanied by a less encouraging environment and substantial resignation. Regression analysis showed that the perceived climate influenced if students wanted to speak up. It was demonstrated that withholding voice and speaking up perpetuate. Students who withhold their voice earlier remain to their habit, whereas students who already spoken up would rather do it again. Reasons might be that the experience of speaking up was either not so bad or students were confident.
The vignette was considered as a realistic scenario and was with increasing term even more realistic. However, the higher the term the lower the risk of harm was estimated. Compared to the previous study, the vignette was scored twice as much with regard to the dangerousness of the scenario . However, high-perceived risk for the patient the likelihood of students to speak up was very low and was accompanied with a very high discomfort to do so. Students’ reasons for discomfort ranged from barriers such as presence of the patient, reaction of the HCPs and fear of negative reaction as well as ineffectiveness. Similar results were found by Doyle et al., where 85% of medical students agreed that it is difficult to question the decision or actions of those with more authority .
Medical errors are already the third leading cause of death in in the US hospitals, therefore education and training in patient safety needs to be a prerequisite in healthcare systems . Though students observe many patient safety concerns during their internships, it is a missed opportunity that they do not address their concerns as the organization might learn and profit . However, this might be difficult in a hierarchical organized system. A possible trigger in order not to lose the knowledge of students’ concern, students should be encouraged a) to speak up and b) as a time delayed alternative to use the critical incident reporting system. However, to achieve these goals, two things have to change in general.
On the one hand, the curriculum concept has to be adapted with regard to patient safety aspects. The growing importance of patient safety aspect requires medical education programs as already suggested by WHO and others [24,25]. The WHO Patient Safety Curriculum Guide for Medical Schools includes a manual for teachers on how to best integrate all patient safety relevant areas . Priority areas in education of students should involve knowledge in cause and frequency of patient incidents, the willingness to take responsibility when it comes to patient harm, communication and teamwork skills for effective safe care and care delivery [27,28]. Nevertheless, a main challenge might be how to best combine content from medicine with human-factors and non-technical skills training to create an effective curriculum . Teaching patient safety skills with an inter-professional approach heightened students’ awareness and importance of effective team-working . One possible trigger could be that medical education profits through medical simulation as it was reported that intrinsic motivation of students significantly increase after simulation based training .
On the other hand, it is also necessary to establish a safety culture at the clinic. On the organizational level, academic clinical departments have a rich tradition of improving approaches such as prevention, diagnosis and treatment of disease, however, quality improvement and patient safety expertise has to be incorporated in academic clinical departments . Furthermore, there is also a lack of senior or midcareer faculty members whose career have focused on quality improvement and patient safety who could then also act as positive role models for students and younger colleagues . Therefore, for large-scale change to be successful in creating a culture of safety, a combination of a top-down approach, such as central coordination in educating HCPs and other healthcare providers, with bottom-up engagement where teams on the ground take control and ownership is needed .
This study has several limitations. A major limitation of this survey was that only 24% responded to the survey though students were reminded three times. One reason could have been that the survey was performed in May to June where students have to pass their exams or that not all students between the 5th and 10th term were recently exposed to a four weeks clinical term and skipped the survey. Moreover, the generalizability is significantly limited as the study was performed in a single institution in Austria. The study only focussed on medical students; however, it should be also of interest how nursing students during their internships perceive speaking up behaviours.
Speaking up behaviours of students was assessed for the first time in a German speaking academic teaching hospital. Higher perceived risk of harm for the patient, past episodes of speaking up and a more favorable climate score were all significantly associated with a higher self-reported likelihood to speak up. All predefined barriers for not speaking up seemed to play an important role and was associated with a missing safety culture in attended clinical units during their internships. These results give hints that there is a need to adapt the curriculum concept with regard to patient safety aspects. Finally, there is also the need to establish a safety culture at the organizational and unit level. Speaking up can only work if HCPs and students perceive an open minded setting in their environment.
1. Schwappach D, Richard A. Speak up-related climate and its association with healthcare workers’ speaking up and withholding voice behaviours: a cross-sectional survey in Switzerland. BMJ Qual Saf [Internet]. 2018 [cited 2018 Jul 23];0:1–9. Available from: http://dx.doi.org/10.1136/bmjqs-2017-007388
2. Lyndon A, Sexton JB, Simpson KR, Rosenstein A, Lee KA, Wachter RM. Predictors of likelihood of speaking up about safety concerns in labour and delivery. BMJ Qual Saf [Internet]. 2012 Sep [cited 2017 Nov 4];21(9):791–9. Available from: http://www.ncbi.nlm.nih.gov/pubmed/22927492 doi: 10.1136/bmjqs-2010-050211
3. Morrow KJ, Gustavson AM, Jones J. Speaking up behaviours (safety voices) of healthcare workers: A metasynthesis of qualitative research studies. Int J Nurs Stud [Internet]. 2016 Dec [cited 2017 Nov 4];64:42–51. Available from: http://www.ncbi.nlm.nih.gov/pubmed/27684321 doi: 10.1016/j.ijnurstu.2016.09.014
4. Beament T, Mercer SJ. Speak up! Barriers to challenging erroneous decisions of seniors in anaesthesia. Anaesthesia [Internet]. 2016 Nov [cited 2017 Nov 4];71(11):1332–40. Available from: http://www.ncbi.nlm.nih.gov/pubmed/27734483 doi: 10.1111/anae.13546
5. Schwappach DL, Gehring K. Trade-offs between voice and silence: a qualitative exploration of oncology staff’s decisions to speak up about safety concerns. [cited 2017 Oct 29]; Available from: https://bmchealthservres.biomedcentral.com/track/pdf/10.1186/1472-6963-14-303?site=bmchealthservres.biomedcentral.com
6. Okuyama A, Wagner C, Bijnen B. Speaking up for patient safety by hospital-based health care professionals: a literature review. [cited 2017 Oct 29]; Available from: https://bmchealthservres.biomedcentral.com/track/pdf/10.1186/1472-6963-14-61?site=bmchealthservres.biomedcentral.com
7. Schwappach DLB, Gehring K. “Saying it without words”: a qualitative study of oncology staff’s experiences with speaking up about safety concerns. BMJ Open [Internet]. 2014 May 16 [cited 2017 Dec 2];4(5):e004740. Available from: http://www.ncbi.nlm.nih.gov/pubmed/24838725 doi: 10.1136/bmjopen-2013-004740
8. Schwappach D, Sendlhofer G, Häsler L, Gombotz V, Leitgeb K, Hoffmann M, et al. Speaking up behaviors and safety climate in an Austrian university hospital. Int J Qual Heal Care [Internet]. 2018 Apr 26 [cited 2018 Jul 23]; Available from: https://academic.oup.com/intqhc/advance-article/doi/10.1093/intqhc/mzy089/4987045
9. der Vossen MM, van Mook W, Croiset G, Kusurkar R. Simulated Patient’s Perspective on Speaking up about Unprofessional Behavior: Qual Prim Care [Internet]. 2018 Feb 15 [cited 2018 May 6];26(1). Available from: http://primarycare.imedpub.com/simulated-patients-perspective-on-speaking-up-about-unprofessional-behavior-training-theresponding-muscles-is-key.php?aid=21890
10. Samuel R, Shuen A, Dendle C, Kotsanas D, Scott C, Stuart RL. Hierarchy and Hand Hygiene: Would Medical Students Speak Up to Prevent Hospital-Acquired Infection? Infect Control Hosp Epidemiol [Internet]. 2012 Aug 2 [cited 2018 May 20];33(08):861–3. Available from: https://www.cambridge.org/core/product/identifier/S0195941700031428/type/journal_article
11. Schwappach DLB. Speaking up about hand hygiene failures: A vignette survey study among healthcare professionals. Am J Infect Control [Internet]. 2018 Apr 9 [cited 2018 Jul 23]; Available from: http://www.ncbi.nlm.nih.gov/pubmed/29650487
12. Martinez W, Lehmann LS, Thomas EJ, Etchegaray JM, Shelburne JT, Hickson GB, et al. Speaking up about traditional and professionalism-related patient safety threats: a national survey of interns and residents. BMJ Qual Saf [Internet]. 2017 Nov 1 [cited 2018 Sep 2];26(11):869–80. Available from: http://www.ncbi.nlm.nih.gov/pubmed/28442609 doi: 10.1136/bmjqs-2016-006284
13. Reeves SA, Denault D, Huntington JT, Ogrinc G, Southard DR, Vebell R. Learning to Overcome Hierarchical Pressures to Achieve Safer Patient Care. Nurse Educ [Internet]. 2017 [cited 2018 May 6];42:S27–31. Available from: http://insights.ovid.com/crossref?an=00006223-201709001-00007 28832459
14. Rees CE, Monrouxe LV., McDonald LA. ‘My mentor kicked a dying woman’s bed…’ Analysing UK nursing students’ ‘most memorable’ professionalism dilemmas. J Adv Nurs [Internet]. 2015 Jan [cited 2018 Jul 23];71(1):169–80. Available from: http://www.ncbi.nlm.nih.gov/pubmed/24916272 doi: 10.1111/jan.12457
15. Bickhoff L, Sinclair PM, Levett-Jones T. Moral courage in undergraduate nursing students: A literature review. Collegian [Internet]. 2017 Feb 1 [cited 2018 May 6];24(1):71–83. Available from: http://linkinghub.elsevier.com/retrieve/pii/S1322769615000761 29218965
16. Doyle P, Vandenkerkhof EG, Edge DS, Ginsburg L, Goldstein DH. Self-reported patient safety competence among Canadian medical students and postgraduate trainees: a cross-sectional survey. [cited 2019 May 1]; Available from: http://dx.doi.org/10.1136/bmjqs-2014-003142
17. American Nurses Association., Kent State University. School of Nursing. Online journal of issues in nursing. [Internet]. Kent State University School of Nursing in partnership with the American Nurses’ Association; 1996 [cited 2018 Jul 23]. http://ojin.nursingworld.org/MainMenuCategories/EthicsStandards/Resources/Courage-and-Distress/Moral-Courage-and-Risk.html
18. Bradbury-Jones C, Sambrook S, Irvine F. The meaning of empowerment for nursing students: a critical incident study. J Adv Nurs [Internet]. 2007 Aug [cited 2018 Jul 23];59(4):342–51. Available from: http://www.ncbi.nlm.nih.gov/pubmed/17543007 doi: 10.1111/j.1365-2648.2007.04331.x
19. Human Medicine. cited 2019 Sep 4. Available from: https://www.medunigraz.at/en/studying/range-of-studies/human-medicine/
20. Burns KEA, Duffett M, Kho ME, Meade MO, Adhikari NKJ, Sinuff T, et al. A guide for the design and conduct of self-administered surveys of clinicians. Can Med J 2008; 179(3): 245–252.
21. Richard A, Pfeiffer Y, Schwappach DDL. Development and Psychometric Evaluation of the Speaking Up About Patient Safety Questionnaire. J Patient Saf [Internet]. 2017 Aug 28 [cited 2018 Sep 1];1. Available from: http://www.ncbi.nlm.nih.gov/pubmed/28858000
22. Cuzick J. 1985. A Wilcoxon-type test for trend. Statistics in Medicine 4: 87–90 doi: 10.1002/sim.4780040112 3992076
23. Makary MA, Daniel M. Medical error-the third leading cause of death in the US. BMJ [Internet]. 2016 May 3 [cited 2017 Nov 4];353:i2139. Available from: http://www.ncbi.nlm.nih.gov/pubmed/27143499 doi: 10.1136/bmj.i2139
24. Walton M, Woodward H, Van Staalduinen S, Lemer C, Greaves F, Noble D, et al. The WHO patient safety curriculum guide for medical schools. BMJ Qual Saf [Internet]. 2010 Dec 1 [cited 2018 Sep 9];19(6):542–6. Available from: http://qualitysafety.bmj.com/lookup/doi/10.1136/qshc.2009.036970
25. Coleman DL, Wardrop RM, Levinson WS, Zeidel ML, Parsons PE. Strategies for Developing and Recognizing Faculty Working in Quality Improvement and Patient Safety. Acad Med [Internet]. 2017 Jan [cited 2018 Sep 2];92(1):52–7. Available from: http://insights.ovid.com/crossref?an=00001888-201701000-00020 doi: 10.1097/ACM.0000000000001230 27191838
27. Sandars J, Bax N, Mayer D, Wass V, Vickers R. Educating undergraduate medical students about patient safety: Priority areas for curriculum development. Med Teach [Internet]. 2007 Jan 3 [cited 2018 Sep 9];29(1):60–1. Available from: http://www.ncbi.nlm.nih.gov/pubmed/17538837 doi: 10.1080/01421590601087546
28. Anderson E, Thorpe L, Heney D, Petersen S. Medical students benefit from learning about patient safety in an interprofessional team. Med Educ [Internet]. 2009 Jun [cited 2018 Sep 9];43(6):542–52. Available from: http://www.ncbi.nlm.nih.gov/pubmed/19493178 doi: 10.1111/j.1365-2923.2009.03328.x
29. Dao DK, Goss AL, Hoekzema AS, Kelly LA, Logan AA, Mehta SD, et al. Integrating Theory, Content, and Method to Foster Critical Consciousness in Medical Students. Acad Med [Internet]. 2017 Mar [cited 2018 Sep 2];92(3):335–44. Available from: http://insights.ovid.com/crossref?an=00001888-201703000-00025 27680318
30. Escher C, Creutzfeldt J, Meurling L, Hedman L, Kjellin A, Felländer-Tsai L. Medical students’ situational motivation to participate in simulation based team training is predicted by attitudes to patient safety. BMC Med Educ [Internet]. 2017 Dec 10 [cited 2018 Sep 2];17(1):37. Available from: http://bmcmededuc.biomedcentral.com/articles/10.1186/s12909-017-0876-5 doi: 10.1186/s12909-017-0876-5 28183316
31. Ogunlayi F, Britton P. Achieving a "top-down" change agenda by driving and supporting a collaborative "bottom-up" process: case study of a large enhanced revovery programme. BMJ Open Quality 2017;6:e000008. doi: 10.1136/bmjoq-2017-000008 29450260