|Year : 2019 | Volume
| Issue : 11 | Page : 908-914
Evaluation of resident satisfaction and change in knowledge following use of high-fidelity simulation teaching for anaesthesia residents
S Shailaja, SS Hilda, Prajna A Pinto, Rithesh J D'Cunha, Lulu S Mahmood, Radhesh B Hegde
Department of Anaesthesia, Father Muller Medical College, Mangalore, Karnataka, India
|Date of Submission||20-Feb-2019|
|Date of Decision||26-Jun-2019|
|Date of Acceptance||26-Aug-2019|
|Date of Web Publication||08-Nov-2019|
Dr. S Shailaja
Department of Anaesthesia, Father Muller Medical College, Kankanady, Mangalore - 575 002, Karnataka
Source of Support: None, Conflict of Interest: None
Background and Aims: Anaesthesia practice demands medical knowledge and skills as essential components for patient management in peri-operative emergencies. Since all residents are not exposed to such situations during their residency, training them using simulation technology could bridge this knowledge and skill gap. The aim of this study was to train and evaluate residents to manage anaesthesia emergencies on high fidelity simulators. Methods: Kirkpatrick model of program evaluation was carried out. Resident reaction was captured using a satisfaction questionnaire and the change in knowledge was assessed using pre-test and post-test Multiple Choice Questions (MCQs). Six scenarios were created and executed on a human patient simulator (HPS). All 22 residents participated in this teaching learning method. The steps of simulation teaching included pre-test, pre-briefing, orientation to manikins, performing/scribe, debriefing, feedback questionnaire, and post-test. The satisfaction questionnaire was administered following the second and fourth scenario. Results: 95% residents agreed on overall satisfaction, that it helps in building team dynamics and clinical reasoning. All students agreed that this teaching had positive professional impact. 14% residents felt they were anxious during the class. The items in the questionnaire had a Cronbach's α value of 0.9. The mean score for pre-test was 24.22 ± 7 (Mean ± SD) and the post-test was 47.18 ± 5.6, the difference between the scores were statistically significant (P = 0.007). Conclusion: The use of high-fidelity simulation to train anaesthesia residents resulted in greater satisfaction scores and improved the residents' reasoning skills.
Keywords: Anaesthesia skills, high fidelity simulation, training evaluation
|How to cite this article:|
Shailaja S, Hilda S S, Pinto PA, D'Cunha RJ, Mahmood LS, Hegde RB. Evaluation of resident satisfaction and change in knowledge following use of high-fidelity simulation teaching for anaesthesia residents. Indian J Anaesth 2019;63:908-14
|How to cite this URL:|
Shailaja S, Hilda S S, Pinto PA, D'Cunha RJ, Mahmood LS, Hegde RB. Evaluation of resident satisfaction and change in knowledge following use of high-fidelity simulation teaching for anaesthesia residents. Indian J Anaesth [serial online] 2019 [cited 2019 Nov 12];63:908-14. Available from: http://www.ijaweb.org/text.asp?2019/63/11/908/270605
| Introduction|| |
The use of high-fidelity simulation teaching learning tool is known to impart learning among students through experiential learning with reflection on action. Majority of the existing literature is from the western world who have been using simulation as a modality to teach, assess, for research or recertifying anaesthesiologists.,,,, Although simulation as a teaching tool is new, it has picked up great momentum and utility due to availability of skills and simulation centres in various parts of India. The Medical Council of India (MCI) has proposed the new Competency Based Medical Education (CBME) with attitude, ethics and communication (AETCOM) module has been implemented from August 2019 for the undergraduate and postgraduate curriculum., For CBME to be implemented, skills and simulation centres will play a vital role as a training tool to achieve mastery level of learning. To incorporate AETCOM and team training programs into existing curriculum, skills and simulation centres are essential. Hence it is necessary for the teachers to be aware of the basics of simulation teaching.
Peri-operative emergencies related to anaesthesia requires prompt recognition and early intervention for successful positive outcome in patient care. All residents are not exposed to all these emergencies hence are not intellectually equipped to handle such situations later in their clinical practice. Thus, to bridge this knowledge and practice gap, we have used simulation as a training modality. We hypothesise that simulation helps to integrate learning from books to learning in situ by improving their clinical reasoning and decision-making skills. In the present study, we evaluated the simulation training program delivered to the Anaesthesia residents.
The aim of this study was to incorporate simulation as a teaching learning tool for post graduates in Anaesthesia. Our primary objective was to train anaesthesia residents to recognise and manage perioperative emergencies on high fidelity simulators and secondary objective was to document residents' reaction to such training and record the change in knowledge using pre, post-test multiple choice questions.
| Methods|| |
Institutional Ethics Committee permission was obtained (letter dated – 16-08-2017, number – FMMC/FMIEC/4338/2017). This study was conducted according to the principles of Helsinki. The study participants gave written informed consent to participate in the study.
After obtaining clearance from Institutional Ethics Committee, 'need assessment' was done by interviewing the faculty and conducting a focus group discussion for the residents to find out how many faculties were interested in teaching using these mannequins and to identify the topics of perioperative emergencies that had to be included in the timetable. Based on their inputs, six scenarios were created and implemented. The six perioperative emergencies chosen by consensus were high spinal anaesthesia, local anaesthetic systemic toxicity, intraoperative tachycardia, intraoperative cardiac arrest, amniotic fluid embolism and mitral stenosis.
This was a Prospective observational-Mixed method research study (Qualitative and quantitative) including all anaesthesia residents numbering to 22 from our institute. The pilot testing was done to calculate sample size to document resident's response and for change in knowledge using this innovative teaching methodology. The sample size of 22 anaesthesia residents met the requirement for our primary objective to train the residents using High fidelity simulators to document change in reaction and knowledge.
Steps taken to validate the scenarios created before implementing to the residents:
- Select the case following a thorough literature search including standard text books
- Write the specific learning objectives maximum of three in number
- Prepare the pre-test MCQs and check for content validity with two other anaesthesia faculty
- Create the scenario including the sequencing of events on the software
- Prepare the room with the manikin to simulate the Operating Room setting with the necessary equipment and drugs
- Pilot test the scenario for authenticity and reality with two other faculties and do any necessary changes
- Implement the scenario to the residents. The 22 residents were divided into 3 batches comprising of 8, 8 and 6 students per group per day, respectively. Each case had to be therefore performed thrice.
The schedule of training and the broad topic to be implemented was notified to the residents in advance and were instructed to come prepared/read for the class. On the day of session, they were given a pre-test to assess their baseline understanding of the concepts related to the topic. This is followed by pre-briefing where the residents were given a brief history of the patient details with vitals, lab investigations and the time point when they are anaesthesia care providers. Later, each batch was divided into four residents per group, where two were the performers (actual care givers) and two were the observers/recorders whose job was to note down the sequence of events and interventions. This is followed by debriefing/reflection where all eight residents and two facilitators sat together to reflect on the actions taken by the residents in managing the case. The facilitators used the plus-delta method for debriefing. The performed actions/interventions that were beneficial for patient's outcome were applauded and reinforced while the gaps in performance or knowledge which need remediation were also highlighted for future corrective performance. The residents could discuss muddy concepts and clarify any other queries related to the event. The final step was the post-test and closure where residents verbalised their take home message and feedback about the facilitator and the case scenario.
To evaluate the entire training program, the Kirkpatrick four level training evaluation model was used.
Level 1 (measures Reaction): the degree to which participants find the training favourable, engaging and relevant to their job (measured using post training questionnaires, interviews, printed or oral reports).
Level 2 (measures Learning): the degree to which participants acquire the intended knowledge, skills, attitude, confidence and commitment based on their participation in the training (measured using Pre-test post-test assessment, observation by peers/instructors, having a control group).
Level 3 (measures Behaviour): The degree to which participants apply what they learnt during the training when they are back on their job (measured using 360 degree feedback, workplace based assessment tools, direct observation of procedural skills, observable change in behaviour, interviews).
Level 4 (measures Results): the degree to which targeted outcomes occur as a result of the training and the support and accountability package (measured by calculating patient safety/reduction in litigations following this educational intervention).
After the second and fourth sessions, resident satisfaction regarding the teaching method was captured using a questionnaire (Annexure 1) designed by Calamassi D et al. It included a total of 43 questions covering seven criteria namely-1. Overall satisfaction, 2. Facilitator and debriefing, 3. Clinical reasoning and self-effectiveness, 4. Team dynamics and team factors, 5. Professional impact, 6. Safeguard and materials, 7. Difficulty and distress.
To evaluate the second level of Kirkpatrick model for change in knowledge pre-test and post-test multiple choice questions (MCQs) were used. These questions were validated for content and relevance to the prepared scenario by two other anaesthesiologists not involved in simulation teaching.
Data was analysed using SPSS-IBM. Exploratory factor analysis was used to assess the underlying factor structure of the questionnaire. The reliability of questionnaire to record the resident satisfaction was analysed using the Cronbach's alpha. By definition, a Cronbach's α from 0.61 to 0.80 represents a substantial correlation and from 0.81 to 1.00 a good correlation. To test if intervention (Simulation training) improved critical thinking and decision making skills we used Extended matching questions in the form of Multiple choice questions (MCQs) as pre and post-tests. The average scores of pre-test and post-test scores were compared using the Student's T-test.
| Results|| |
The items on the questionnaire had good correlation with Cronbach's alpha value of 0.90 and the correlations between items ranged from 0.6-0.84.
[Table 1] shows the domains and items of the questionnaire and [Table 2] shows the satisfaction score of participants for session two and session four for the various components used in the questionnaire. Following session four, 95% of the participants were satisfied with overall training programme. All the participants satisfied with the role of facilitator and debriefing session. About 95% of the participants agreed that this training session helps in developing clinical reasoning skills as well as improving team dynamics. All the participants felt that these simulation training will have professional impact. 95% students agreed that materials as well as safety measures are adequate to recreate the scenarios. Only 5% of the participants felt that these training sessions are difficult and can cause distress.
Kirkpatrick level 2 (change in knowledge) was tested using pre-test and post-test with 60 MCQs. The mean values of pre-test and post-test scores are 24.22 ± 7 and 47.18 ± 5.6 respectively. The difference between the pre-test and post-test scores is statistically significant (P = 0.007).
| Discussion|| |
The Human Patient Simulator (HPS) which is a high-fidelity full body human simulator manufactured by CAE company is present in our institute. HPS has inbuilt technology to mimic physiological changes. It has reactive eyes, palpable pulses, heart sounds, bilateral breath sounds and bowel sounds. It can analyse expired CO2 and inhalational agents in real time, along with recognizing drugs delivered through barcoding technology. HPS has realistic upper airway which allows laryngoscopy and intubation, and also difficult airway module can be mimicked with tongue and posterior oropharyngeal swelling. We can perform needle cricothyrotomy, needle thoracocentesis and pericardiocentesis. It can withstand a defibrillator shock, and a nerve stimulator and be attached to elicit thumb twitch response to neuromuscular blockade. These mannequins are designed to automatically determine patient responses to use of interventions with appropriate physiological responses. The scenario performance station is wirelessly connected with instructor workstation for better communication. The instructor workstation provides the instructor with scenario development tools and programmable patient physiology to create an immersive, realistic training environment.
Simulation based medical education (SBME) has proven to be effective in improving the knowledge, skill and behaviour of health care professionals. Simulation has been effectively used to teach anaesthesia, procedural skills, surgical skills and crises resource management.,,,,,,,, There is evidence of simulation being used to train individuals, teams, environments, technical factors, system factors and patient factors.,,,, In this present study, we aimed to teach peri-operative emergencies targeting patient care as our ultimate goal. To document this process, we captured the resident's reaction and change in knowledge to this teaching modality.
This study showed that anaesthesia residents had greater satisfaction following use of high-fidelity simulation as a learning tool for peri-operative emergencies. We also observed that the ratings for all criteria in the satisfaction scores were better for the fourth scenario than the second, which implies that our role as facilitators also improved. The residents have agreed that their clinical reasoning and self-effectiveness have improved, they started recognizing their role as the team leader or team member. Following the session two, many residents were not satisfied with the materials/makeup of the mannequin hence we modified our preparation for the following sessions and tried to make it as real as possible, hence in fourth session the number of responses for this criterion have improved. Thus, this questionnaire has helped us to identify our lacunae and act on it. After session two, about 14% of the residents felt that this learning had induced anxiety. Similar observations were seen by James WP and Thomas G, where students experienced anxiety while using high-fidelity simulators., This amount of anxiety is expected from residents as they are subjected to pressure of performance in front of their peers and teachers. Also, this helped us to modify the method of pre-briefing and tried to make them at ease during performing in the following sessions, which reduced the anxiety among the participants. This study reinforced the findings of studies by Parck CS, McCrossin and Burlacu CL who stated that use of high-fidelity simulation training has increased the confidence and decision-making skills among residents.,,
In this study, we used the MCQs as a means of assessing the efficacy of simulation sessions. The post-test was given immediately after debriefing. There was a significant improvement in the post-test scores compared to the pre-test scores. Similar improved knowledge following simulation class were seen by Gaba DM and Solymos O., Also, Curran VR has shown that simulation training helps in retaining the skills and higher confidence level in performing these skills later. However, in the present study, we did not follow up the residents after the sessions to check the retention level.
A systematic review on best evidence in medical education by Issenberg SB et al. have identified the features of high-fidelity simulation that facilitate learning such as curriculum integration, feedback/debriefing, deliberate practice, mastery learning, capturing clinical variation, individualised learning opportunities and team training. In the present study, we have integrated simulation into the resident curriculum, structured feedback was given, students were allowed to practice repetitively and in teams. The scenarios used also captured the clinical variations. Therefore, we have used many of the principles of best practices in this teaching learning methodology.
We have used purposive sampling and included only residents from our medical college hence our sample size is small to generalise the findings. Hence a larger study with residents from different regions would be ideal to confirm our findings. In the present study, we recorded the participant's reaction and change in knowledge to this educational intervention. We have not tested the change in behaviour (Kirkpatrick level 3) and result on patient care (Kirkpatrick 4). A longer period of follow-up and recording of the observations is required to get the whole effectiveness of this program. The present study also did not incorporate interdisciplinary team work i.e., including surgery residents and nursing faculty etc. Hence the holistic team dynamics was lacking. Future studies incorporating these features would be ideal.
Simulation as an educational tool compliments clinical exposure. It is an effective teaching learning tool for our adult learners in implementing Competence Based Medical Education (CBME). Teachers need to be trained to use this tool for education. However, future research with a larger number of participants is required to document the positive participant's response and record the change in behaviour or improvement in patient's outcome.
| Conclusion|| |
Hence we conclude that the use of high-fidelity simulation to train residents for “Peri-operative Emergencies” has greater satisfaction scores, provides and improves resident's clinical reasoning knowledge and skills. Therefore simulation teaching needs to be incorporated in the regular curriculum.
Financial support and sponsorship
Logistic support-Father Muller Skills and Simulation Centre, Intellectual support-FAIMER regional centre PSG Coimbatore.
Conflicts of interest
There are no conflicts of interest.
| References|| |
Fletcher G, Flin R, McGeorge P, Galvin R, Maran N, Patey R. Anaesthetists' non-technical skills (ANTS): Evaluation of a behavioural marker system. Br J Anaesth 2003;90:580-8.
Flin R, Patey R, Galvin R, Maran N. Anaesthetists' non-technical skills. Br J Anaesth 2010;105:38-44.
Cook DA, Hamstra SJ, Brydges R, Zendejas B, Szostek JH, Wang AT, et al
. Comparative effectiveness of instructional design features in simulation-based education: Systematic review and meta-analysis. Med Teach 2013;35:e867-98.
Park CS, Rochlen LR, Yaghmour E, Higgins N, Bauchat JR, Wojciechowski KG, et al
. Acquisition of critical intraoperative event management skills in novice anaesthesiology residents using high-fidelity simulation-based training. Anesthesiology 2010;112:202-11.
Pavithran P, Rajesh MC, Rekha K, Sajid B. Survey of change in practice following simulation-based training in crisis management. Indian J Anaesth 2018;62:991-4. [Full text]
Friedman Z, You-Ten K, Bould M, Naik V. Teaching lifesaving procedures: The impact of model fidelity on acquisition and transfer of cricothyrotomy skills to performance on cadavers. Anesth Analg 2008;107:1663-9.
Kirkpatrick DL. Evaluating Training Programs: The Four Levels. 2nd
ed. San Francisco, CA: Berrett-Koehler Publ; 1998.
Calamassi D, Nannelli T, Guazzini A, Bambi S, Gioachin G, Pini R, et al
. High-fidelity in urgency-emergency simulation: Validation of a tool to determine the satisfaction of participants. Acta Biomed 2016;87:28-37.
Morgan PJ, Cleane-Hogg D, Desousa S, Lam-McCulloch J. Applying theory to practice in undergraduate education using high fidelity simulation. Med Teach 2006;28:e10-5.
Kurrek MM, Devitt JH, Cohen M. Cardiac arrest in the OR: How are our ACLS skills? Can J Anaesth 1998;45:130-2.
Murray WB, Good ML, Gravenstein JS, van Oostrom JH, Brasfield WG. Learning about new anesthetics using a model driven, full human simulator. J Clin Monit Comput 2002;17:293-300.
Mayo PH, Hackney JE, Mueck JT, Ribaudo V, Schneider RF. Achieving house staff competence in emergency airway management: Results of a teaching program using a computerized patient simulator. Crit Care Med 2004;32:2422-7.
Ault MJ, Rosen BT, Ault B. The use of tissue models for vascular access training: Phase I of the procedural patient safety initiative. J Gen Intern Med 2006;21:514-7.
Bergman S, Deckelbaum D, Lett R, Haas B, Demyttenaere S, Munthali V, et al
. Assessing the impact of trauma team training program in Tanzania. J Trauma 2008;65:879-83.
Mah JW, Bingham K, Dobkin ED, Malchiodi L, Russell A, Donahue S, et al
. Mannequin simulation identifies common surgical intensive care unit teamwork errors long after introduction of sepsis guideline. Simul Healthc 2009;4:193-9.
Howard S, Gaba DM, Fish KJ, Yang G, Sarnquist FH. Anaesthesia crisis resource management training: Teaching anaesthesiologists to handle critical incidents. Aviat Space Environ Med 1992;63:763-70.
Jankouskas T, Bush MC, Murray B, Rudy S, Henry J, Dyer AM, et al
. Crisis resource management: Evaluating outcomes of a multidisciplinary team. Simul Healthc 2007;2:96-101.
Shukla A, Kline D, Cherian A, Lescanec A, Rochman A, Plautz C, et al
. A simulation course on lifesaving techniques for third-year medical students. Simul Healthc 2007;2:11-5.
Forrest FC, Taylor MA, Postlethwaite K, Aspinall R. Use of high-fidelity simulator to develop testing of the technical performance of novice anaesthetists. Br J Anaesth 2002;88:338-44.
Pian-Smith MC, Simon R, Minehart RD, Podraza M, Rudolph J, Walzer T, et al
. Teaching residents the two-challenge rule: A simulation-based approach to improve education and patient safety. Simul Healthc 2009;4:84-91.
Price JW, Price JR, Pratt DD, Collins JB, McDonald J. High-fidelity simulation in anaesthesiology training: A survey of Canadian anesthesiology residents' simulator experience. Can J Anaesth 2010;57:134-42.
Geeraerts T, Roulleau P, Cheisson G, Marhar F, Aidan K, Lallali K, et al
. Physiological and self-assessed psychological stress induced by a high fidelity simulation course among third year anaesthesia and critical care residents: An observational study. Anas Crit Care Pain Med 2017;36:403-6.
McCrossin K, White HT, Sane S. The effect of high-fidelity simulation on the confidence and decision-making ability of anaesthesia trainees in managing subsequrnt simulated 'Can't Intubate, Can't Oxygenate' scenarios. Anaesth Intensive Care 2014;42:207-12.
Burlacu CL, Chin C. Effect of pediatric simulation training on candidate's confidence. Paediatr Anaesth 2008;18:566-7.
Gaba DM, Howard SK, Flanagan B, Smith BE, Fish KJ, Botney R. Assessment of clinical performance during simulated crises using both technical and behavioural ratings. Anaesthesiology 1998;89:8-18.
Solymos O, O'Kelly P, Walshe CM. Pilot study comparing simulation based and didactic lecture based critical care teaching for final year medical students. BMC Anesthesiol 2015;15:153.
Curran VR, Aziz k, O'Young S, Bessell C. Evaluation of the effect of a computarized training simulator (ANAKIN) on the retention of neonatal resuscitation skills. Teach Learn Med 2004;16:157-64.
Issenberg SB, McGaghie WC, Petrusa ER, Lee Gordan D, Scalese RJ. Features and uses of high fidelity medical simulators that lead to effective learning: A BEME systematic review. Med Teach 2005;27:10-28.
[Table 1], [Table 2]