• Home
  • Abstract-presentations

I. Enhancing Education in CME/CPD (16)


Enhancing Education in CME/CPD (15)

Impact of PI-CME activity design characteristics on learner outcomes and sustained improvements in a regional healthcare system: 2015-2019 retrospective analysis

Authors:  Ericka Cunningham, BSBA, Melanie Moore, MAEd, Rosa Alvarez, MPH, CHES, Erika Gutierrez, BA

Institution:  Southern California Permanente Medical Group

Problem/Intervention - Performance Improvement Continuing Medical Education (PI-CME) activities have been increasingly implemented to improve clinical quality throughout Southern California Permanente Medical Group (SCPMG). We conducted a literature search to discover evidence-based practices in PI-CME design across a healthcare system, however findings were limited. A group of continuing education professionals representing 13 geographic areas collaborated to identify the key elements of SCPMG’s PI-CME design associated with performance improvement outcomes. This program evaluation is in early stages of analysis.  We sought to conduct a retrospective analysis of all PI-CME projects in SCPMG spanning a 5-year period to determine if a correlation exists between PI-CME design which yield improved short- and long-term outcomes related to SCPMG Quality Initiatives.

Methods - Data was collected for the total of 105 PI-CME projects from 2015-2019 throughout SCPMG’s 13 geographical areas.  A retrospective analysis was planned to evaluate the correlation between:

Design variables in five key exposure categories:

    1. Group composition (variables: Group size, Collaboration, Location)
    2. Participant engagement (variables: Sponsor title, Motivation to participate, Method of participation, Change implementation design)
    3. Data review process (variable: attestation of retrospective review of patient charts)
    4. Initiative focus (variables: Diabetes, Medications, Vaccination, Other)
    5. Year of completion (variables: 2015-2019)
Outcome Measures:
    • Metric Type: Structure, Process or Patient
    • Short-term: How much improvement (average %)
    • Long-term: (A) Qualitative - Intent to sustain changes for 1 year after (based on post-activity survey), and (B) Quantitative – Data supporting performance improvement sustained after 1 year, in the subset of projects that had data available to track 1 year later.
Key lessons learned for CME/CPD practice - Our research is in early stages. At the time of this briefing, observations from initial data collection provided key insights for enriching PI-CME design, including:
    • Group Composition: Partner with other professions with impact on improvement outcomes (e.g., nurses) 
    • Participant Engagement: Actively engage stakeholders with influence on participants throughout the project; consider intrinsic motivators throughout
    • Process of data review: Weigh benefits of individual chart review with convenience of aggregate chart data
    • Long-Term Outcomes: Consider both qualitative and quantitative long-term outcome measures
Back to top


Competency-based continuing professional development

Authors: Nancy Davis, PhD, Lisa Howley, PhD and Kamilah Weems

Institution: Association of American Medical Colleges

Problem/Intervention - While Competency Based Education (CBE) is a reality in UME and GME, implementation is less straightforward in CME/CPD. The AAMC, in collaboration with other organizations, is developing new competencies in new and emerging areas for medical curricula and for learners in training or those continuing their professional development.

Methods - Each set of new and emerging competencies is developed with broad and diverse input by stakeholders and leaders from across the medical education and clinical practice communities, including CPD. The new competencies help guide curricular and professional development, formative performance assessment, cross-continuum collaborations, and, ultimately, improvements in health care services and outcomes.

Results - New competencies will be shared in four areas: 1) quality improvement and patient safety (QIPS); 2) telehealth; 3) diversity, equity and inclusion (DEI); and 4) clinician educator. The first two sets of competencies in the series have been published: QIPS and Telehealth. The DEI and clinician educator competencies are in draft form. This provides opportunity for the CPD community to assist with ensuring relevance of the new competencies to our field. Barriers to implementation of new competencies include awareness in the CPD community, understanding of the level of competencies for practicing clinicians, and harmonizing the many available competencies in medical education.

Key lessons learned for CME/CPD practice - Competency-based education must be a part of CME/CPD to ensure competent lifelong learners. CPD professionals need to be facile with the new competencies and use them as the foundation for their offerings.

Back to top


Using theory and technology to cultivate an interprofessional community of human trafficking

Authors: Susan E. Farrell, MD, Hanni Marie Stoklosa, MD, MPH, Rahel Samuel Bosson, MD

Institution: Massachusetts General Hospital, Harvard Medical School

Problem/Intervention - To promote a non-hierarchical environment of interprofessional learning, we used social cognitive constructivism, and experiential learning theory to scaffold our participants’ new knowledge. We leveraged various technologies to engage participants and build a community of human trafficking educators.

Methods - We used Whatsapp® and Flipgrid® to connect participants and faculty prior to a 2-day course. Participants’ prior experiences informed small group learning activities that depended on confidential and respectful sharing of information. The program consisted of live case presentations interwoven with pre-recorded didactic videos, Zoom® breakout rooms for small group case analyses, and question and answer sessions with trafficking survivors.

Participants collaborated in small groups to apply their new knowledge to co-create a 15-minute teaching lesson related to educating others about labor and sex trafficking, disclosure, and the law. This activity facilitated integration of the learning outcomes with participants’ prior experiences and authentic work responsibilities as educators in their respective organizations. Each group taught their lessons to other course participants and guests using newly acquired knowledge of learning theory. All groups received peer feedback on the content, clarity, and engagement of their teaching. The resulting teaching lessons were available to all course participants for their future use. We used a pre-post retrospective survey to assess participants’ report of change in knowledge and skill, immediately and three months after the program.

Results - In 2019, 35 physicians, nurses, social workers, and psychologists from the US and UK graduated from the inaugural program. Three-month post-program surveys indicated lasting behavior change in the following course learning objectives: use and teaching of the SOAR® framework, teaching with adult learning principles, and creating organizational trafficking protocols. In 2020, 65 medical students, physicians, nurses, public health workers, physician assistants, and psychologists from the US, Canada, and Trinidad/Tobago attended the virtual program. Technology allowed us to broaden the community, connecting the 2020 cohort to five 2019 program graduates who returned as coaches in 2020.

Key lessons learned for CME/CPD practice - Continuing professional development that addresses global public health problems must be interprofessional in nature. Using theories and strategies to engage interprofessional learners in a non-hierarchical community of shared authentic learning and behavior change will be most effective in achieving this goal.

Back to top


In learning about a new medication: how patients can contribute in translating knowledge into better-informed practice

Authors: Alex Hu, BA, Karson Haigler, BA, Brittany Albright, MD, MPH, Gale Hannigan, PhD, MPH, MLS, and William Rayburn, MD, MBA

Institution: University of New Mexico

Problem/Intervention - The concept of patient education is not necessarily associated with continuing professional development (CPD) in medicine. This project illustrates how patients, as they become more knowledgeable about a new FDA-approved drug (SPRAVATO) for treatment-resistant depression, can make a valuable contribution to the continuing education of health professionals.

Methods - Each patient was encouraged to ask questions during every 2-hour session for a minimum of 8 clinic sessions. Responses to frequently asked questions were developed from a weekly National Library of Medicine PubMed search (key words: esketamine, treatment-resistant depression) and discussions with industry professionals. Patient interviews were conducted to assess clarity and completeness of written responses.

Results: Twenty-six patients underwent 239 treatment sessions. A 40-question online report in question-answer format included these drug-related topics: how it works; dose and route determination; time to act and clear; medications to avoid; time of expected relief; short and long-term hazards; failure and other treatment options. Patients' feedback about the report provided further clarity which, in turn, informed the clinician and improved the report.

This exercise illustrated how, by focusing on patient education and health literacy, providers can enable and encourage patients to be more involved in their care, shared decision-making, and clinic experience. Furthermore, this engagement enhanced the ability of providers to gain new knowledge in a meaningful manner. Preparing the report provided better-informed discussions during the treatment session. This website report allowed for continuous updates with new information.

Key lessons learned for CME/CPD practice - This example of innovative learning about a new medication prompted patients to be agents of change in accelerating the translation of knowledge into practice. This CPD activity structured information from the medical literature into a tool for patients, while reinforcing the knowledge of the practitioner from a patient perspective.

Back to top


White boards: The future of patient education?

Authors: Ariel Jordan BS, Deliabell Hernandez BS, Tahitia Howard BS., Jahan Tajran BS, Elizabeth Nesbitt BS, Colt Moran BS, Eric Ayers MD

Institution: Moorhouse School of Medicine

Problem/Intervention - Health literacy has become known as a public health concern in the United States leading to poor health outcomes for patients, especially those most negatively impacted by the social determinants of health. White Boards: The Future of Patient Education was a survey-based quality improvement project conducted in the med-peds clinic at a large safety-net hospital in downtown Detroit. The project goal was to identify if white boards are an effective patient education tool bolstering health literacy compared to a traditional handout.

Methods - Fifty-eight patients were included in the study, with the average age being 49.8 years. 77% of patients were female, 53.4% listed high school as their highest level of education, and majority 81% of patients were African American. 100% of the patients received a handout and the white board teaching. 5.2% preferred the handout only, 70.7% preferred the white board only, and 24.1% preferred both the handout and the white board. 91.4% thought the white board was better than other clinic resources. 70.7% took a picture of the completed white board. Many patients submitted feedback about the white board teaching; listed are a few of their comments: “Can use to explain to my family”, “I can use to compare with the next visit”, “Helps patient take time to think if missing something to ask. Great idea.”

Results - Adequate communication is challenging in the clinic setting due to time constraints on length of visits and complex medical terms. Our study revealed a majority of patients preferred the white board to the handout and thought it was better than other resources previously received in clinics. Additionally, over half of patients even took a photo of the white board in order to reference back to at home further demonstrating that patients found this a useful method of learning. Teaching from a white board provides an innovative take on patient education and can be particularly useful for patients in low resource settings and limited reading comprehension skills. In considering ways to continue this work, we plan to expand the use of white boards to other clinics and explore the usefulness of the white board in the inpatient setting as part of the discharge process.

Key lessons learned for CME/CPD practice - CME/CPD providers should consider use of a white board in their hospital clinics alone or as an adjunct to a handout in an effort to continue to bolster patient health literacy.

Back to top


Educator, judge, public defender: Conflicting roles for remediators of practicing physicians

Authors: Gisèle Bourgeois-Law MD, PhD; Glenn Regehr PhD, Pim Teunissen MD, PhD; Lara Varpio PhD

Institution: University of British Columbia

Problem/Intervention - As part of a larger program of research exploring social constructions of remediation in practice, this study explored remediators' understanding of the remediation process through their stories of engaging with practicing physicians.

Methods - Using experience-focused narrative research, we elicited the stories of nine clinicians in five provinces asked by regulatory authorities to oversee the learning and practice of physicians with significant competence gaps. We asked them to tell us stories of particularly memorable remediation experiences. We then analyzed these stories, by iteratively reading them, examining the sense-making participants achieved through these narratives, and identifying the roles and responsibilities described.

We used positioning theory as a sensitizing concept. Positions describe how individuals conceptualize and enact a particular role and influence not only how they carry out their role, but also the behavior they expect of others. Our participants positioned themselves in three different ways: educator, judge and public defender or variants thereof, shifting between these positions in response to evolving experiences with the remediate.

Results - Participants expressed preference for the educator position, where they guided and empowered the remediatee in making practice changes. However, they sometimes encountered serious obstacles to enacting that position. Those obstacles were imposed both by regulatory processes which limited the information they were given, and thus sometimes their ability to plan the remediation to include remediatee objectives, as well as by remediatees themselves who persisted in viewing the remediator as an enemy, an agent of the regulatory authority. Younger, female remediators also had to deal with the power differentials involved in remediating older, more experienced, male practitioners.

Key lessons learned for CME/CPD practice - For clinicians, remediating a peer is more complex than remediating a senior learner. Remediators of practicing physicians need targeted faculty development and support to deal with these complexities. They may also need information on how the assessment and remediation system works in their jurisdiction and where their particular role is situated in that process.

Back to top


Continuing professional development in Canada: Preliminary findings from a scoping review of the Grey literature

Authors: Francesca Luconi, PhD, Elizabeth Wooster, PhD (ABD), Morag Paton, MEd, Meron Teferra, MSc, Suzan Schneeweiss, MD, MEd, FRCPC, Andrea Quaiattini, MA, MLIS

Institution: McGill University

Problem/Intervention - Continuing professional development (CPD) as an agent of change in healthcare reform could contribute to resolve misalignments in the Canadian system. CPD provider organizations (i.e., faculties of medicine, medical societies, medical associations) are responsible and accountable for the development, delivery, and evaluation of accredited CPD activities. Due to the nature of their role as decision makers, CPD providers may be able to act as agents of change. Leveraging the value of grey literature (GL), this scoping review (SR) aimed to map the terrain of the Canadian CPD landscape from 2008 to 2019 by identifying terminology, conceptual frameworks, instructional methods, topics, and values present in the grey literature.

Methods/Results - This SR reviewed and coded strategic plans from 16 Canadian universities, 70 national specialty societies, 234 university CPD programs, and 763 CPD conference presentations. A variety of terms have been used in the GL to describe CPD activities (e.g. continuing medical education, CPD, continuing education, lifelong learning, education). CPD was the most frequently reported term which might indicate CPD activities covering multiple competencies in contrast with the term CME that may reflect a medical-centered individual approach to professional development. Didactic instruction is predominant despite its limited impact on practice. Furthermore, results indicate limited reporting of conceptual frameworks in conference abstracts. The GL for CPD demonstrates a partial response to societal needs focusing on topics such as addiction/opioids and Indigenous health. The thematic categories identified in the MacIntosh-Murray analysis of JCEHP content (1981-2004) was used as the basis for our coding. Additional themes emerged from the GL indicating changes in the CPD landscape since 2004.

Key lessons learned for CME/CPD practice - This SR contributes to an understanding of areas of focus and potential gaps in the Canadian CPD landscape, potentially better enabling CPD to act as an agent of change. Knowing what has been represented in the Canadian grey literature can help the Canadian CPD community identify potential over- or underutilized instructional strategies or absences of conceptual frameworks. These findings could lead to improvement of CPD offerings. This SR (2008-2019) could be used as a potential benchmark to track changes in the Canadian CPD field pre-post COVID-19 pandemic.

Back to top


Moving towards mastery: A needs assessment for preparing CPD leaders to develop programs using educational technology

Authors: Heather MacNeill, MD, BSc(PT), MScCH(HPTE), FRCPC, David Wiljer, PhD, Morag Paton, MEd

Institution: University of Toronto

Problem/Intervention. COVID-19 has precipitated rapid change in CPD design and delivery. Provider organizations need to retool and develop new skills and competencies to create effective CPD using technology (Price et al., 2020). In order to do so, faculty and staff need resources and support (Silver & Leslie, 2018). The aim of this study is to understand CPD providers’ knowledge, comfort level, and learning needs for developing CPD programs using educational technology.

Methods. We developed a quantitative survey (5-point Likert scales, MCQ, rank-order responses) exploring domains such as confidence related to transitioning to developing online programs, perceptions of online CPD, and knowledge of supports available. Recognizing the importance that creating an expectation of efficacy can enable subsequent mastery, Bandura’s Theory of Self-Efficacy (1977) was used in constructing the various domains of the survey.

Results. We piloted the survey tool in June 2020 inviting institutional departmental CPD leaders (n=15) to participate (60% response rate). Participants agreed that the survey had faced validity. More survey respondents (33%) perceived that their CPD program participants were confident in making the transition to online than themselves as leaders (16%). The top perceived advantage of utilizing educational technology was increased reach of program access, with the top disadvantage being a lack of informal networking. A revised survey will soon be deployed to institutional Chairs of accredited program/conferences (n=430). In addition, SACME members will be invited to participate in the survey as well.

Key lessons learned for CME/CPD practice. Like many before us, we can learn from the experiences forced upon up by the pandemic. CPD developers and leaders need access to strong faculty development in order to enable change in this new online environment and will need this even after the immediacy of COVID has passed. CPD recognizes the importance of moving towards mastery in educational design.

Back to top


Creation of just in time videos for healthcare educators on using synchronous technologies (Zoom)

Authors: Heather MacNeill, MD, BSc(PT), MScCH (HPTE), FRCPC, Suzan Schneeweiss, MD, MEd

Institution: University of Toronto

Problem/Intervention - The current pandemic has caused a rapid shift to “emergency” online teaching, with little preparation of teachers and learners on best practices, particularly synchronous (webinar) environments. To our knowledge, no comprehensive resources exist for faculty development in synchronous teaching for a healthcare context. In live online (synchronous) environment, educators need to be aware of how their methods and materials need to change, while still incorporating the same effective principles that are important in face to face (F2F) teaching, (e.g., interactivity, feedback, repetition, reflection, social learning).

Methods/Results - We developed a YouTube channel for teaching and learning in synchronous environments. Please visit this channel and its associated resources to view, like, give us feedback on your synchronous pandemic teaching experiences and subscribe to upcoming video releases (learning in online environments- summer 2021 and using twitter in healthcare education- fall 2021).

We chose video to allow bite sized, on the go, just in time, repetitive, practical hands-on and reflective learning for busy physicians, already overwhelmed by pandemic changes. Videos also allow for accessibility and quick dissemination to physicians across Canada across multiple devices. However, it's also important to consider ways to maximize applicability to different learner groups through iterative and collaborative editing processes (for example using an Agile framework), incorporating interactivity/feedback/reflection for learners, partnerships and dissemination strategies, and pedagogical approaches to video creation.

Key lessons learned for CME/CPD practice - We explain a collaborative and iterative approach for video creation which may assist others in building similar technology enhanced faculty development. These videos and associated resources may also help faculty development for our educators, speakers, facilitators, and learners in CPD synchronous education, through exploring best practices in these environments.

Back to top


Physician learning from a summative, longitudinal assessment activity

Authors: David W. Price, MD, FAAFP, FACEHP, FSACME, Ting Wang, PhD, Kevin Rode, BA and Warren Newton, MD, MPH

Institution: American Board of Medical Specialties

Problem/Intervention - In 2019, the American Board of Family Medicine began piloting the Family Medicine Certification Longitudinal Assessment (FMCLA) as a potential alternative to every 7-10 year, secure knowledge examinations for continuing Board Certification. Participants receive 25 questions/quarter, with a 5-minute limit per question. References may be used to answer the questions, but participants are not allowed to confer with others. After each question, participants indicate their confidence in their answer and the question relevance to their practice before receiving receive feedback, a critique, and references. Participant dashboards identify progress, comparison with peers, and content areas of strength and weakness. While intended to be summative in nature, we are exploring how FMCLA has also acted to enhance participant learning.

Methods - Surveys and data extracted from performance on the FMCLA platform in the first cohort of FMCLA pilot physicians.

Results - Of the 8411 eligible participants, 5984 (71%) started FMCLA in 2019. The vast majority of participants favorably reacted to FMCLA, agreed/strongly agreed that it enhanced their learning, and reported making subsequent practice changes. Participants tended to avoid indicating high or little confidence in their answers but were more likely to indicate confidence and correctly answer questions that they felt were relevant to their practice. Compared with a similar, untimed, lower stakes activity, participants were less likely to be confident in their FMCLA answers despite more often answering questions correctly.

Key lessons learned for CME/CPD practice - Longitudinal, spaced assessments with interleaved content is consistent with evidence on CME that is effective in changing practice and outcomes and can serve both formative and summative purposes. Questions should be framed in to help participants perceive the content as relevant to their practice. Those who employ confidence-based formative assessment should look for confidence-correctness mismatches (confidently incorrect or not-confidently correct) as opportunities to provide tailored subsequent learning activities.

Back to top


Using accredited CE and implementation science to build an academic healthcare system in an under-resourced community

Authors: Ellen Rothman, MD, Tasha Dixon, MD, MPH, Ronald Edelstein, EdD, and Sarah Porter, CHPC

Institution: Charles R. Drew University of Medicine and Science and Martin Luther King, Jr. Outpatient Center

Problem/Intervention - This is a case study describing how two healthcare organizations--a health sciences university in South Los Angeles and a proximate safety-net institution--established a partnership with the shared goal of improving patient care and cultivating a clinical learning environment for a family medicine residency and other medical education programs.

The study institutions embraced key approaches for developing a successful, collaborative partnership with shared goals: 1) Healthcare metrics were pre-defined and clearly articulated; 2) Multiple stakeholders were aligned and invested; 3) Leadership made education and implementation science institutional priorities and requirements for all professional healthcare staff; and 4) The social determinants of health and multicultural competencies were critical factors in planning.

Methods - Community healthcare metrics were defined from the outset along with clear goals for achievement. The metrics were tracked across a period of four years. The following metrics were deemed facility goals for which all staff members in all departments were responsible for outcomes: influenza immunization, tobacco assessment and counseling, BMI screening and follow-up, colorectal cancer screening, cervical cancer screening, breast cancer screening, and sexual orientation/gender identity completeness.

Results - Data across the four years the metrics were tracked shows measured improvement, with the most recent target goals for each metric being achieved by the MLKOPC. Comprehensive Diabetes Care: HbA1c Poor Control was at 21.61 for MLKOPC (target: 29.07; lower is better), Screening for Clinical Depression and Follow-up was at 85.79 (target: 83.88), Tobacco Assessment and Counseling was at 91.88 (target: 91.26), Sexual Orientation/Gender Identity Completeness was at 64.54 (target: 25.00), CG-CAHPS: Provider Rating was at 82.60 (target: 76.40), Colorectal Cancer Screening was at 67.47 (target: 64.87), Influenza Immunization was at 85.09 (target: 69.38), Breast Cancer Screening was at 77.42 (target: 70.29), Cervical Cancer Screening was at 60.73 (target: 53.66) and BMI Screening and Follow-up was at 93.10 (target: 88.29).

Key lessons learned for CME/CPD practice: Ongoing commitment, focus and investment from stakeholders across multiple organizations is necessary to achieve real community health improvement. In addition, accredited CE can be a bridge to improving practice when metrics are pre-defined and clearly articulated.

Back to top


Promoting resilience and mental well-being during the Covid-19 pandemic: Using the echo model to provide system support to frontline healthcare professionals

Authors: Sanjeev Sockalingam, MD, MHPE, FRCPC, Chantalle Clarkin, RN, PhD, Cheryl Pereira, MPH, Sarah MacGrath, PMP, Victoria Bond, MSc, Jenny Hardy, OT Reg. (Ont.) Javed Alloo, MD, CCFP, MPLc, Mark Bonta, MD, FRCPC, Andrea Furlan, MD, PhD, Heather Flett, MD, FRCPC, Mona Loutfy, MD, FRCPC, MPH, Terri Rodak, MA, MISt, and Allison Crawford, MD, PhD, FRCPC

Institution: Center for Addiction and Mental Health, University of Toronto.

Problem/Intervention - Frontline healthcare providers (HCPs) are at increased risk of experiencing psychological distress and mental health sequelae secondary to COVID-19. Tele-mentoring interventions have the potential to address emerging educational needs, while providing a sense of community and psychosocial support.

Methods - In March 2020, the Centre for Addiction and Mental Health launched Project Extension for Community Healthcare Outcomes-Coping with COVID (ECHO-CWC), to provide support, build capacity for self-care and wellness, and promote resilience among Canadian HCPs (including residents) throughout all phases of the pandemic. ECHO-CWC brings together HCPs weekly, connecting them with peers and an interdisciplinary specialist team to learn together, share experiences and coping strategies, and foster a virtual community of practice using videoconferencing. Program curriculum was developed in four phases, using a modified Delphi approach. Enrollment in ECHO-CWC is continuous and ongoing.

To evaluate program impact, data on ECHO-CWC participation, satisfaction, and learning needs are collected following each session via survey. Pre-and-post program questionnaires are also administered to examine perceived changes in self-efficacy and perceptions of COVID-19-related risks.

Results - To date, ECHO-CWC has registered 941 HCPs from 398 organizations across 9 provinces/territories in Canada. Evaluation data demonstrate high mean satisfaction ratings (>4.23/5) and a reduction in perceived professional isolation. Pre-ECHO program questionnaires (n=301) demonstrated moderate confidence ratings for self-efficacy items in core program competencies related to managing self-care and coping, as well as elevated perceptions of risk regarding COVID-19.

The ECHO model enabled the rapid implementation of a national virtual education program to support frontline HCPs throughout the pandemic. The findings show high satisfaction and engagement with the ECHO-CWC program. Further evaluation is currently underway to examine longitudinal impact.

Key lessons learned for CME/CPD practice.

  1. HCPs can be connected educationally and supportively through virtual, interprofessional communities of practice during a pandemic
  2. Embedding mindfulness activities and creative coping practices within CPD allows for role modeling, skill development, and the crowdsourcing and curation of new knowledge and resources.
  3. Ongoing evaluation is key to adaptive curriculum, continuous improvement, and pragmatic sequencing that addresses emerging learning needs and an evolving knowledge base.
Back to top


Turning COVID lemons into virtual exhibit hall lemonade: Successfully maintaining compliance while integrating virtual exhibit halls into accredited CME

Authors: Vickie Skinner, DHA, Cheryl Stingily, BA, Kim Chaney, Hannah Copeland, MD, FACC, FACS, Patricia Freeman, BSN, RN, Markita Hall, BA and Bo Lewis, BA

Institution: University of Mississippi Medical Center

Problem/Intervention - The University of Mississippi Medical Center Division of Continuing Health Professional Education (UMMC CHPE) utilized BigBlueButton, a web-based conferencing platform, to host virtual continuing education activities paired with virtual exhibit halls. The problem was to assure ACCME compliance with respect to separation of promotion from educational content in a virtual educational environment.

Methods. In lieu of utilizing virtual ‘breakout rooms,’ UMMC CHPE created separate rooms with individual web links for each vendor’s exhibit. This method solved UMMC CHPE invested significant time and resources in training speakers and vendors on navigation within the Big Blue Button platform. Additional efforts went to engaging exhibitors who had not participated in a virtual exhibit hall previously to orient and ‘sell them’ on the process.

Results - UMMC CHPE hosted numerous virtual conferences (Acute Cardiopulmonary Shock Symposium, Dermatology Symposium, and Diabetes Conference) with virtual exhibit halls. The vendor ability to engage in direct conversations with event attendees during virtual exhibit hall periods provided an alternative method to reach health care providers at a time when live in-person events and travel were restricted. Utilizing virtual exhibit halls also allowed UMMC CHPE to maintain compliance with ACCME standards and criteria while providing a viable source of funding for the planning and hosting of accredited continuing education.

Key lessons learned for CME/CPD practice - Regardless of the platform selected, virtual exhibit halls only work if sufficient attendees elect to visit them. It was also more challenging to offer sponsorship in tiered levels, versus a flat rate exhibit fee for virtual events.

Back to top


Evaluation of health care provider training in tobacco dependence treatment: Moore’s levels 5 and 6

Authors: Elise Tanzini, MA, Sheleza Ahad, MSc, Megan Barker, PhD(c), MA, Stephanie Duench, MPH, Myra Fahim, MSW, RSW, Rosa Dragonetti, MSc, RP, and Peter Selby, MBBS, CCFP(AM), FCFP, MHSc, DipABAM, DFASAM

Institution: Nicotine Dependence Service, Centre for Addiction and Mental Health, University of Toronto

Problem/Intervention - The Training Enhancement in Applied Counselling and Health (TEACH) Project has offered comprehensive training in tobacco dependence treatment to health care providers (HCPs) since 2006. Evidence suggests that training HCPs to deliver cessation interventions has positive effects; however, differences in evaluation methodology, including the inconsistent use of theoretical frameworks, and limited reporting on patient level outcomes, suggests a need for further investigation.

Methods - We implemented Moore’s outcomes framework to examine HCPs’ self-reported performance change across eight competencies related to tobacco dependence treatment, 6 months after completion of the TEACH ‘Core Course’ training in 2015/2016 (n=62; Moore‘s Level 5).1 We also examined 6-month quit outcomes among overlapping patient samples who received most (n=26,590) or all (n=20,986) cessation treatment by one identifiable trained HCP (Moore‘s Level 6).2

Results - A significant increase in the proportion of HCPs who provided tobacco cessation counselling 6 months following completion of the TEACH Core Course was observed (44% vs. 81%, p<0.001). A significant positive association was also observed between receiving treatment from an HCP trained by TEACH and the likelihood of patient smoking abstinence at 6-month follow-up, compared to receiving treatment from an HCP trained in less intensive training, in both analytic samples (most care sample: OR = 1.10, 95% CI = 1.01, 1.20; all care sample: OR = 1.12, 95% CI = 1.02, 1.24). Accessible, intensive, and competency-based training may enhance treatment capacity, as HCPs obtain skills to deliver specialized evidence-based tobacco cessation interventions, and ultimately, support their patients‘ cessation goals.

Key lessons learned for CME/CPD practice - Systematic adoption of theoretical frameworks into program planning, implementation, and evaluation may be used to continuously ensure program accessibility and efficacy, and may further support comparison across studies.

References

  1. Vanova, A., Baliunas, D., Ahad, S., Tanzini, E., Dragonetti, R., Fahim, M., & Selby, P. (2021). Performance Change in Treating Tobacco Addiction: An Online, Interprofessional, Facilitated Continuing Education Course (TEACH) Evaluation at Moore's Level 5. Journal of Continuing Education in the Health Professions, 41(1), 31-38. doi: 10.1097/CEH.0000000000000328
  2. Baliunas, D., Ivanova, A., Tanzini, E., Dragonetti, R., Selby, P. (2020). Impact of comprehensive smoking cessation training of practitioners on patients’ 6-month quit outcome. Canadian Journal of Public Health, 111, 766-774. doi: 10.17269/s41997-020-00318-1

    Back to top


    Educating the educator: A novel model for change and quality education

    Authors: Sandhya Venugopal, MD, MS-HPEd, Erik G. Laurin, MD, Shelley A. Palumbo, MS, CCC-SLP

    Institution: University of California, Davis Health

    Problem/Intervention - Advancing expectations in continuing medical education (CME) and the need for demonstrable change in outcomes can challenge even seasoned educators. The University of California, Davis Health Office of Continuing Medical Education (OCME) developed a novel Educate the Educator model to improve educators’ understanding of CME principles, implement them in courses and foster culture change for the creation of CME curriculum through partnership. This new model was specific to our institution and evolved from experience in course design through evaluation, combined with an in-depth gap analysis and needs assessment.

    Methods - These included an examination of coursework, assessment of existing skills, study of other CME models, and exploration of organizational priorities.

    Results - Based on findings, the OCME restructured the team with new talents including instructional designers and educational specialists, held development trainings, created a wealth of resources to support educators, and created a new faculty liaison role. The faculty liaison worked peer-to-peer, strengthening partnerships and optimizing courses by offering OCME resources and expertise.

    Beginning in 2019, over 150 courses were examined and phased into the new model. Partnerships resulted with the application of the CME construct, resulting in quality educational coursework utilizing educational technologies and greater diversity in instructional formats. Training sessions facilitated further collaboration, particularly in the area of evaluation for health outcomes.

    Key lessons learned for CME/CPD practice - Although many models designed to change educational culture exist, this multifaceted model increased the quality and expanded the impact of CME courses. Evaluations became standardized and strengthened faculty focus on changing practice patterns and evaluating health outcomes The OCME team is positioned to adapt to future educational transformations with the necessary talent to lead, communicate and sustain such changes. By reimagining CME and realigning resources, changing culture was possible and can be adapted by others for their institutions.

    Back to top


    Active learning in continuing education: A Delphi study of in-person and virtual techniques

    Author: Yemisi Jones, MD

    Institution: Cincinnati Children’s Hospital, University of Cincinnati

    Problem - Despite the evidence supporting the use of active learning strategies to enhance learning, they are infrequently used during continuing education (CE) lectures, in both in-person and virtual settings. Studies attempting to measure the use of active learning during didactic sessions have predominantly involved novice learners. Learners at this stage require more intensive active learning strategies, as they need help organizing new information and require guidance on how to retrieve stored information. CE learners are different because they are already content experts and are able to incorporate new information more efficiently. This study aims to describe which active learning strategies are most appropriate for use with CE large-group didactics.

    Methods - We conducted a literature review of active learning techniques in medical education. We also searched lay and business literature for virtual-only techniques. We included those strategies in a questionnaire for rating by our Delphi panels. We convened a panel of CE and active learning experts as well as one of Pediatric Grand Rounds attendees. Each panel completed two rounds of ratings in an effort to achieve consensus.

    Results - Twenty (of the 31 from the literature search) active learning strategies were rated highly by both panels for appropriateness, feasibility, and likelihood to engage.

    Key lessons learned for CME/CPD practice - There are many active learning strategies that are acceptable to CE learners and deemed appropriate by expert consensus for large-group didactic CE. Few strategies should be avoided with this group of learners.

    Back to top


    II. Team-based Learning (3)

    Joint accreditation: Now what?

    Authors: Annette Mallory Donawa, PhD, MS Ed; Sarah J. Carmody, MBA, CPTD

    Institution: Thomas Jefferson University

    Problem/Intervention - The Office of Continuing Professional Development at Thomas Jefferson University was awarded Joint Accreditation in December 2019. The self-study was written by previous leadership. A new Accreditation Specialist, Sarah J. Carmody, was hired, as well as a new Assistant Provost, Annette Mallory Donawa. This new leadership team had the task of implementing the new Joint Accreditation requirements, along with developing new processes, policies, and procedures.

    Education and communications were used as our principal interventions. Informal training was provided during meetings with CPD staff about joint accreditation. We met with several course directors to educate them and share information about JA requirements. Formal educational sessions with the JA Advisory Committee were also held which included opportunity for discussion and Q & A. More formal communication tools are currently in development, such as an annual report to be released by the end of calendar year 2021.

    Methods - The method used for this research was a survey of our key stakeholders in July 2020. Our key stakeholders included course directors, planning committee members and our newly developed Joint Accreditation (JA) Advisory Committee members. Data from this survey will provide baseline metrics for analyses.

    Results - We focused on two data points from our key stakeholder survey. The survey question and responses were:

    Please rate the performance of the OCPD in the following area:

    1. Listened and supported your needs: Strongly Agree/Agree: 78.57%; Neutral: 14.29%; Disagree: 7.14% (n = 14)
    2. Was knowledgeable and informed: Strongly Agree/Agree: 85.72%; Neutral: 7.14%; Disagree: 7.14% (n = 14)

    Key lessons learned for CME/CPD practice - We realized that having JA recognition gives Thomas Jefferson University a competitive edge. Educating staff, faculty, and leadership is an ongoing, continuous process. One educational session is not sufficient. Reminders are given to faculty that the planning committee has to be representative of the target audience. Another lesson learned is that the CPD office has to be aware of instructional design methods to ensure topics on the agenda reflect the target audience. We plan to improve the response levels on the two data points that were considered for this abstract on the next survey. JA has proven to be an exciting learning adventure. Having a positive attitude about JA could have a positive impact on our goal to increase interprofessional educational opportunities.

    Back to top


    Improving UW-Madison ICEP global evaluation survey

    Authors: Laura Kelble, Barbara Anderson, MS and Marianna Shershneva, MD, PhD

    Institution: University of Wisconsin, Madison

    Problem/Intervention - Interprofessional education is a growing area of continuing medical education (CME) and as a result is being emphasized in novel ways in the existing curricula. In order to achieve the best possible educational outcomes and meet accreditation criteria, the University of Wisconsin-Madison Interprofessional Continuing Education Partnership (ICEP) conducts an annual evaluation survey of participants in all activities provided by ICEP. The survey was designed to document outcomes on multiple levels of Moore’s evaluation framework. Questions about interprofessional activities were informed by the four domains of collaborative practice.

    Methods - The project goal was to improve the survey questions and, in particular, convert several free response questions into multiple-choice questions in order to increase response rate and facilitate quantitative analysis. Responses from open-ended questions from the 2019 survey were coded using qualitative software NVivo. Themes emerging from this analysis informed creation of new survey questions and question formats. These changes were then pilot-tested against the data from the most recent survey.

    Results - This work supported revision of the ICEP annual survey. Analysis of responses documented multiple examples of impact on team-based strategies and interprofessional, collaborative practice, regardless of whether respondents attended an activity designated as interprofessional or non-interprofessional activity. This could indicate a shift in CME toward valuing the interprofessional team and a desire to learn about the roles of the team members. The new multiple choice questions include data-driven categories of abilities and behaviors applicable to measuring educational impact across different types of activities. Further, some survey responses emphasized diversity, equity and inclusion (DEI), despite that no questions specifically were asked about DEI. This shows an increasing awareness of the medical community of how cultural and personal backgrounds of patients and peers impact the health care they receive. As a result, a new survey question was offered to measure the nature of the learning environment with respect to DEI.

    Key lessons learned for CME/CPD practice - Providers should consider adding an explicit question to their standard CME/CPD activity evaluation survey, to collect learner feedback on the nature of the curriculum and learning environment.

    Back to top


    III. Education Technology Innovations (9)

    Enhancing high risk procedural skills training through active learning, emerging technology and simulation

    Authors: Caroline Abercrombie, MD, Elizabeth Wolf, MBA, CPPS, CPHRM, CHCP, Robert Becker, BS, L. Michelle Kelley, MPH, Thomas Kwasigroch, PhD

    Institution: East Tennessee State University Quillen College of Medicine

    Problem/Intervention - This series integrates evidence-based procedure training with a review of the applicable anatomy for multi-level learners, promotes competency and improves confidence by embracing active learning principles, emerging technologies, and simulation educational theories. Immediate feedback on procedural skills is provided using a competency rating form (developed with Delphi Method). Intentional pre-briefing and debriefing for the simulation scenarios provide a clear understanding of actions and the opportunity for reflection to enhance future clinical performance. Pre-and post-surveys assess perceived confidence, experience and ability to perform the procedure.

    Methods - Need-based sessions delivered include lumbar puncture, central line, airway management, paracentesis, thoracentesis, and upper extremity, knee, and hip injections. Each session includes an: evidence-based didactic, applied anatomy skill station, procedure skills station, and simulation experience. The didactic covers the clinical anatomy and procedure to ensure a similar foundation. The applied anatomy station integrates a variety of modalities (VR AR, prosection). A clinical procedure skill station with surgical donors provides a realistic, low-stress environment. Standardized Patient Encounters and Hi-Fidelity Simulation Scenarios explore application of the procedure, differentials, complications and contraindications. Skill sessions have less noise and distraction with in-the-moment feedback, while scenarios have uninterrupted action followed by intentional debriefing.

    Results - All participants demonstrated competency. Participants found the simulation valid and transferable to patient care, felt more prepared and confident to perform the skill, and preferred learning through this teaching model. Skill stations were successfully integrated with anatomical resources and simulations to provide an environment well reviewed by learners that improved their perceived confidence and ability to perform the procedure. Baseline levels were not established with the competency checklist, however, future research includes a planned longitudinal study to show an improvement in competency from baseline.

    Back to top


    Advancing continuing professional development in quality improvement using workshops and an online project registry – In Progress

    Authors: C. Michael Fordis, MD, Kyler M. Goodwin, PhD, MPH, Aanand D. Naik, MD, Michael C. O’Connor, BA, Ying Y. King, MHA, Anne N. Perch, MBA, CHCP, Jason E. King, PhD, and Anthony Adams

    Institution: Baylor College of Medicine

    Problem/Intervention - Faced with the need to expand faculty development in quality improvement (QI) application and teaching, we undertook an effort that combined training workshops with the development of an online registry of QI projects. Registries have been widely applied for assessing the effectiveness of condition or disease-specific QI efforts using observational methodologies. Our purpose is to design a registry of QI workshop projects to (1) provide guidance in working through the steps of QI project development; (2) support formative assessment and feedback by coaches throughout workshops followed by summative assessment at workshop conclusion; (3) and facilitate longer-term tracking outcomes thereafter.

    Methods - In this combined effort, we drew upon theoretical underpinnings that included Andragogy (Knowles); Backward design (Wiggins and McTighe); the Precede-Proceed Model (Green and Kreuter); human-centered design; and a framework for project reporting (Ogrinc).

    Results - Our educational approach employed 4 elements: (1) introduction to QI via online modules; (2) two internally developed interactive workshops--one for intermediate level learners and a second for advanced learners—providing “hands-on” training using faculty members’ projects; (3) support by QI coaches; and (4) a newly completed project registry that takes users through the stepwise process of planning and documenting a project’s progress, outcomes, and dissemination. The registry also offers tools for recording individual improvement cycles, captures information to facilitate advancing completed projects to presentation and/or publication, and provides pathways for faculty seeking Maintenance of Certification credit and/or other recognition. While course participants will be required to register projects, any faculty or staff member outside of the courses can register their own projects for tracking and sharing. The registry is fully searchable providing visibility for faculty achievements and can be used to review previous experience, select mentors, and avoid duplicative efforts. Pilot testing of the registry is underway with future plans for formal studies.

    Engagement of stakeholders in iterative development cycles has continually improved the registry for use in workshops and in doing so has led to enthusiasm for additional application of the registry in tracking QI projects and outcomes across the institution--options that are possible because initial planning anticipated and incorporated in development needs for registry scalability and extensibility.

    Key lessons learned for CME/CPD practice - Described here is a tool to assist in tracking and evaluation of QI training outcomes that also offers potential to provide insights into the effectiveness of institutional QI initiatives informing additional training needs.

    Back to top


    Technology and academic medical leadership education: An evaluation of learner feedback of a hybrid leadership module for “learning in an academic health system”

    Authors: Ashleigh Jaggars, MPH, Lauren DeChant PhDc, Richard Rothstein, MD, Mary Turco, EdD

    Problem/Intervention - The purpose of this study was to explore the benefits and barriers of a team-based, hybrid learning format in a two-week ‘Leading in an Academic Health System’ module. 143 senior medical professionals across six interdisciplinary cohorts participated in this hybrid learning module within a ten-month Leadership Development Program in an academic health system.

    Methods - Leaders were required to access learning materials online via Blackboard’s Course Sites, join online live sessions via Adobe Connect, and participate in asynchronistic online discussion boards with cohort peers. Feedback on the technological components of the program was assessed from a post-module survey. Positive feedback was defined as responses ranging from agree to strongly agree.

    Results - Of the 143 leaders, 57 (40%) provided post-module feedback. Most participant responses yielded high rates of positive feedback. Over 90% of individuals reported that the technology was used effectively to support learning and that instructions on the course website were clear and understandable. 88% reported that the online discussion board and live virtual sessions were helpful to their overall learning. Throughout the two-week module, 79% of individuals reported experiencing no technological issues and that technology platforms were used effectively to support learning.

    Key lessons learned for CME/CPD practice - In order to make this hybrid model successful, a customer-service approach should be taken with both faculty and staff. It is important to provide as much support as possible to ensure that learners don’t become discouraged by unanticipated tech barriers. Additionally, learning needs should be addressed first, and technology needs second. Faculty must decide what skills or knowledge they want students to learn so the technology can be designed to fit the desired learning objectives, not vice versa. As we work increasingly to integrate technology into professional education, we must ensure that technology barriers are kept to a minimum to allow for a highly satisfactory and robust learning experience.

    Back to top


    10 Tips for transitioning to a synchronous online CPD environment

    Authors: Heather MacNeill, MD, BSc(PT), MScCH, (HPTE), FRCPC, Suzan Schneeweiss, MD, Med, Trevor Cuddy, BCom, BA, MEd, DBA(c), Marta Guzik, BA

    Institution: University of Toronto

    Problem/Intervention - Many CPD programs have been forced to transition from a face to face to online formats due to social distancing and travel restrictions. This rapid change in delivery method has created many opportunities for learning how to best provide CPD in an online context.

    Methods - Learning online requires significant preparation of learners, speakers, and the online environment. Recognizing the importance of these domains, we used Garrison’s Community of Inquiry for considerations during CPD transition to online delivery. This theory considers interdependent elements: social, cognitive, and teaching presences in an online environment when creating online educational experiences. Within this context, we explore “10 tips” for practical application of online CPD delivery, including 

    Teaching Presence

    1. Optimize your virtual set-up (audio, video, internet, monitor and classroom set-up)

    2. Establish learner environment (expectations, online learning rules)

    3. Be available to your learners (feedback, social connection)

    4. Include participant and peer-peer interactivity in your teaching

    Cognitive Presence

    5. Shorten your content

    6. Consider cognitive load in online environments (include breaks, judicious use of different technologies enhanced modalities, and individual cognitive load considerations)

    7. Use co-facilitation strategies

    Social Presence

    8. Orient learners to the online environment

    9. Create inclusive, equitable and accessible learning environments online

    10. Don’t forget the importance of social interactions in your events

      Key lessons learned for CME/CPD practice - We outline an approach to consider when providing online CPD, based on the Community of Inquiry framework. Within this framework, we provide 10 practical tips to consider both now, and into the future, as we move towards possible blended and hybrid models of CPD in the post COVID era.

      Back to top


      Real time virtual support: Bringing medical education and clinical support to the point-of-care

      Authors: John Pawlovich, MD, CCFP, Kate Meffen, BSc, Laura Beamish, MHA, MSc, Kendall Ho, MD, RCPSC

      Institution: University of British Columbia

      Problem/Intervention - At the onset of the pandemic, there were concerns that health care providers working in rural, remote, and Indigenous communities in British Columbia, Canada would become overwhelmed with COVID-19 cases. Translating the emerging best practices for COVID-19 from high-resource urban settings to rural settings required rapid development of supportive, on-demand CME/CPD opportunities.

      Methods - Provincial Real-Time Virtual Support (RTVS) consultative “pathways” were deployed in April 2020 to provide 24/7 access to specialized clinical peer support through videoconferencing and telephone. The service provides a range of support including collaborative clinical support, case review, and referrals and/or transport support. The RTVS calls were accredited/certified for Mainpro+ and MOC Section 1 Group Learning credits in August 2020, to acknowledge the invaluable education involved in the peer support interactions.

      Results - From April 2020 to March 2021, the RTVS consultative pathways have been accessed over 1900 times. Each consultation represents bidirectional learning through reflection, feedback, coaching, and practice change. Utilization of these pathways continues to grow as awareness of the service increases and best practices are established. The pandemic rapidly accelerated the use and acceptance of virtual technologies and presented an urgent need for health care providers to prepare for and receive real-time support with cases. British Columbia piloted a similar but smaller-scale project in 2019, which acted as a framework for the rapid expansion of RTVS, and there was strong system-level and operational support for this initiative across the province.

      Key lessons learned for CME/CPD practice - This innovative accredited program illustrates that the merging of education and on-demand clinical consultation is feasible and provides benefits to health care providers. Evaluation is underway to inform the expansion of RTVS to a multimodal Education-on-Demand CPD Strategy including, but not limited to, mentorship and virtual simulation.

      Back to top


      Innovative use of technology to enhance point of care ultrasound education

      Authors: Kate Meffen, BSc; Kevin Fairbairn, MD, CFPC (EM), Laura Beamish, MHA, MSc, Tandi Wilkinson, MD, CFPC (EM)

      Institution: University of British Columbia

      Problem/Intervention - The Hands-On Ultrasound Education (HOUSE) Program is a well-established, travelling point-of-care ultrasound (POCUS) program for rural physicians in British Columbia (BC). When the COVID-19 pandemic disrupted the delivery of in-person courses, we were challenged to continue delivering real-time POCUS education. We saw this as an opportunity to enhance the program with available technology including videoconferencing platforms and handheld ultrasound machines.

      Methods - We piloted a two-day virtual HOUSE course in a remote BC community in Fall 2020. Although we undertook a literature search and conducted key informant interviews on best practices for virtual education, there is little evidence to guide virtual real-time POCUS instruction.

      Results - Course instructors attended the session virtually via Zoom, and the rural physicians participated in-person from their community hospital. The introduction of technology allowed us to continue teaching POCUS skills and enhance the learning experience by offering individualized virtual follow-up sessions using the community’s handheld ultrasound machines. We determined that two cameras are required to show instructors both the learner’s ultrasound screen and probe placement. It is also important to have strong information technology (support for both the instructors and learners. The virtual follow-up sessions improved learning by offering longitudinal support; previously we were challenged by significant geographic distances between instructors and learners.

      Key lessons learned for CME/CPD practice - Real-time virtual POCUS education is feasible and enhances some aspects of learning. Advantages include, decreased travel time and costs, enhanced longitudinal education support, and increased access to high-quality POCUS education, especially in very remote communities.

      Back to top


      Virtual learning about community engagement in clinical research

      Authors: William F. Rayburn, MD, MBA, Carolyn Jenkins, DrPH, MSN, MS, RD, LD, Kathleen Lenert, MET, Daniel Lackland, PhD

      Institution: Medical University of South Carolina

      Problem/Intervention - We developed a 15-week course to prepare physician researchers to partner with at-risk communities to address root causes and barriers that contribute to health disparities. Due to social distancing during the COVID-19 pandemic, we redesigned this course from traditional classroom to a virtual learning setting.

      Methods - The instruction used several online formats: weekly preview of learning activities; asynchronous 45-minute lectures (including YouTube) with pauses for reflection and questions; bi-weekly live interactive reviews of principles from lectures; early presentations by students about their research interest and its application to communities; student-course director videoconferencing to discuss research progress; final presentations of community engagement projects with immediate feedback. Student engagement was measured by the time and duration online for attending lectures and contributions to discussion.

      Results - Our cohort of 17 students was primarily fellows and junior faculty in clinical departments. Each developed at least one community-academic partnership research proposal.. Anonymous survey responses revealed students' strong agreements (more than 80%) about the educators' effectiveness, preparation, enthusiasm, availability, encouragement of questions, and clear explanations. All students agreed or strongly agreed that the educational technology was helpful and course content was intellectually challenging and practical in applying principles.

      The interprofessional faculty was considered to be a project strength. The instructors found that times to prepare and present their lectures online were not more than in the classroom. Improvements in technology connectivity are desirable. Additional subjects of interest would require more course time. Learning was often asynchronous, yet Tuesday evenings were the best times for any live educational activities.

      Key lessons learned for CME/CPD practice - The rapid change to virtual education formats demonstrated innovation was well received for interactive, interprofessional learning about community engagement and clinical research.

      Back to top


      Evaluating the future of faculty development: An actor network theory-based analysis of a virtual conference

      Authors: Yusuf Yilmaz, PhD, MSc; Sarrah Lal; Shawn Locke; Ilana Bayer; X. Catherine Tong; Anjali Kundi; Carolyn Stewart, CMP; and Teresa Chan, MD, FRCPC, MHPE, DRCPSC

      Institution: McMaster University

      Problem/Intervention - In the Spring of 2020, many in-person professional development events were canceled due to the COVID-19 pandemic. The McMaster Program for Faculty Development (MacPFD) restructured its 13th annual conference to take place in an online format. This MacPFD13 Virtual Conference had one keynote, two parallel large group sessions, and 12 workshops in four parallel tracks over five hours. This study examines the virtual conference experience from an attendee and speaker perspective.

      Methods - We used Actor Network Theory to examine the experiences of virtual conference attendees and speakers in three stages: before, during and after the conference. Data about attendee participation and experiences was collected through online platforms including Google Forms, Zoom, Slack, social media (e.g. Twitter), and conference registration form.

      Results -. Pre-conference announcements and social media campaigns resulted in 203 registrations with 194 unique attendees. Attendees spent 134 (SD=94) minutes on average in the sessions during the 5-hour virtual conference. Attendees’ post-conference feedback (n=57, response rate of 29%) indicated that the content met their learning expectations (91%) and the virtual conference was more convenient (84%) and affordable (77%) compared to in-person conferences. Attendees responded that in-person conferences provide superior networking opportunities (77%) and Slack-based “virtual hallway” environments were not an adequate substitute. The MacPFD13 Virtual Conference provided an opportunity to compare virtual and their prior experiences with in-person educational conferences. MacPFD13 showed promise in virtual conferencing by high participation and low cost in the virtual format.

      Key lessons learned for CME/CPD practice - Convenience and quality were not compromised through a virtual format. Additional benefits of a virtual conference were affordability and convenience. This study highlights the need to improve virtual networking experiences and integrate mechanisms to track attendee engagement online.

      Back to top


      Just-in-time faculty development: A scoping review

      Authors: Yusuf Yilmaz, PhD, Mark Lee, BHSc, DipCLS, Shera Hosseini, PhD, Teresa M. Chan, MD, FRCPC, MHPE, DRCPSC

      Institution: McMaster University

      Problem/Intervention - Amid the realities of faculty’s modern academic lives, there has been widespread acceptance of the use of technology in faculty development (FacDev) to provide them with hands-on training opportunities. Two questions that faculty developers often ponder upon include: “When do faculty need professional development?” and “How can faculty development be delivered when faculty require support?” With constant developments in online technology and tools, Just-in-Time (JiT) approaches offer novel perspectives to address development needs of faculty. However, the novelty of this approach results in different ways of utilization and design of FacDev programs.

      Methods - The present study is a scoping review of JiT approaches for FacDev development guided by the Arksey and O’Malley framework, and aims to identify JiT approaches that may be solutions for delivering FacDev in a fast, effective, and on-demand way. Authors developed a search strategy and queried several databases (e.g., PubMed, ERIC, Web of Science).

      Results - The literature search yielded 644 studies, of which 31 were included in the study for data extraction. Learning management systems and online modules were used in 10 studies (23%). Websites such as blogs (n=5, 12%) and mobile apps (n=4, 9%) were also reported. The majority of the studies were description study (n=10, 32%), or description of an innovation (n=9, 29%). There were a considerable number of conceptual papers as well (n=8, 26%). Push strategies (n=20, 33%) of content and development activities to faculty were the main way of delivering. Pull strategies (n=11, 18%) were used less compared to the push studies. While the majority of the studies delivered FacDev based on perceived needs (n=21, 35%), unperceived needs (n=5, 8%) were targeted infrequently. Studies using JiT approach for FacDev are generally from outside of health profession sciences. While JiT provides learning and behavior change in FacDev, the area of health profession FacDev requires more research.

      Key lessons learned for CME/CPD practice - Various technology use for JiT FacDev is suggested, especially using micro-content and micro-credentialing. JiT framework needs to be studied using more rigorous methods while incorporating better technology to deliver programs. This process may better address the perceived needs of faculty and emerge unperceived needs in the long term.

      Back to top


      Designing Comprehensive Online CE Courses for Healthcare Professionals Targeting Practice Change and Patient Care Improvement

      Authors: Kim Denny MS, Rima Patel, Bita Zakeri PhD

      Institution: Indiana University School of Medicine

      Problem/Intervention.  Leveraging the “Diagnosis of Cystic Fibrosis: Consensus Guidelines” published in 2017 for the purpose of improving newborn screening diagnosis, in collaboration with the Cystic Fibrosis Foundation (CFF), IUSM CME designed a series of comprehensive online courses to optimally minimize gaps in knowledge and practice in the field and improve patient care, thereby creating a QI pathway to change professional performance. 

      Methods - We used the IU Canvas learning platform as an innovative approach to develop and deliver education to programs across the country and globally. Perspectives informing the design and assessment of the courses and assessment were drawn from adult learning theories. Topics were developed from established clinical and practice educational needs. 

      Course development focused on quick turnaround time to maintain engagement with authors and editors, including the opportunity for lead content experts to serve as advisors to junior mentees as new authors. Design of each course was guided by 3-4 goals for learning outcomes and includes 4-5 modules with a post quiz of 5 questions. We have consistently offered CME and MOC Part 2 credits for these courses. Starting with Course 2, we added numerous CE credit types as due to the interprofessional nature of the content and planning committee, addressing the needs of all healthcare team members in their successful application of Cystic Fibrosis Cinical Practice Guidelines and recommended practices.

      Results - Our findings based on learner data from three completed courses demonstrates success. For Course 1, CFF Registry Data showed the accuracy of diagnosis from newborn screening at first posting had improved by 10% (from 31% to 21%). This project is still ongoing: Course Four launched in February 2021, and during Summer to Fall 2021 we will make Courses 5, 6 and 7 available, followed by Course 8 in 2022.

      Key lessons learned for CME/CPD practice -  Learner feedback per course is utilized during the revision process. During the NACFC annual conference, we learned the need to reach learners more directly. Thus, we utilized the program directors in forwarding course information to all CFF listservs. This drastically increased the enrollment and participation in our courses.  In response to demand for the course, each course is to be available beyond the initial 4-month period for the duration of two year, revised and relaunched as an enduring source of education for existing CF team members and for the purpose of on-boarding new team members.  The success of these courses make the design and delivery of these courses serve as a model for future programs, especially at a global training level. 

      Back to top


      IV. Equity, Diversity and Inclusion (2)

      The potential influence of gender in framing reason for referral to a remediation program

      Authors: Garrett Girard, BA; Liz Moddelmog, Student; Rodderick Williams, Student; Mette Esbensen, BA; Michael Williams, PhD; Betsy Williams, PhD

      Institution: University of Kansas

      Problem/Intervention - There has been considerable work exploring the importance of interpersonal/interprofessional relationships in healthcare delivery. Studies have explored gender-based differences in frequency of complaints in surgical settings. Less work has been done exploring qualitative aspects of concerns in physicians referred for disruptive behavior. The purpose of the current study is to explore potential qualitative differences in the operationalization of disruptive behavior as a function of gender to understand whether there are gender-based stereotypes.

      Methods - A qualitative methodology was utilized. Disruptive behavior of two types was the focus: aggressive/angry behavior and violations in maintaining appropriate interpersonal boundaries. Data were drawn from collateral interviews of workplace respondents who interacted with the identified physician. For this exploratory study, 4 clients were chosen: a similarly aged male and female with the same specialization, Board status, and ethnicity referred for angry/aggressive behavior and a similarly aged male and female with the same specialization, Board status, and ethnicity referred for boundary violations. The data were entered into a text analysis that enumerated the most common words and most common phrases of 5 words or less. The data were concatenated within each client for analysis. The findings were contrasted across, gender, ethnicity, specialty, and referral question. Qualitative findings of word and phrase usage frequency and pattern differences and similarities were identified. The data were analyzed using JMP 14.0.

      Results - Differences were observed by gender and gender within referral type. Qualitative review of the descriptors indicated differences in the frequency and types of words used. A review from the research team members also revealed differences in common themes and patterns within the data. These findings are suggestive of different behavioral standards by gender and referral type. The literature and this preliminary study suggest that there are different expectations around aspects of professional behavior, contributory factors to those behavioral issues, and consequences for engaging in unprofessional behavior.

      Key lessons learned for CME/CPD practice - Findings reflect the need for CME/CPD to address system issues through implicit bias and feedback training. Identified physicians require remedial education targeting problematic behaviors and cultural sensitivity training.

      Back to top


      The role of practitioners in indigenous wellness – participant experience with cultural responsiveness training

      Authors: Stacey Lovo, BScPT , Veronica McKinney, MD, CCFP, MSc, Eman Abdulhadi, BA, BKIN (Hons), Sarah Oosman, BSc, BScPT, MSc, PhD, James Barton, BSc (Hon), MD, FRCPC 

      Institution: University of Saskatchewan

      Problem/Intervention - In Canada, Indigenous populations face systemic racism daily in healthcare settings. Indigenous peoples have known inequities in access to health care and in health outcome disparities compared to non-Indigenous people as a direct result of colonization and systemic racism. It is the responsibility of health practitioners to understand how colonization currently impacts health, to create actions for reconciliation moving forward, and to act against racism every time it is observed. The Role of Practitioners in Indigenous Wellness program is a 25-hour interactive online learning program that supports health practitioners to learn about colonization, intergenerational trauma, racism, cultural humility when integrating traditional Indigenous methods supporting health and wellness, and cultural safety critical to creating safe and responsive healthcare spaces. This course is developed in collaboration with and delivered by Indigenous scholars, community members, elders and health practitioners.

      Methods - Sixty-four health practitioners participated in the course in 2016. Learners engaged in self-reflection and interactive discussion activities, and 36 completed the final assignment. The goal of the final assignment was to develop a culturally responsive communication strategy for Indigenous patients. Iterative thematic analysis was used to examine the final communication strategies from the participants.

      Results - Preliminary analysis revealed five primary themes: 1) Communication: A Cornerstone of Effective Care; 2) Patient and Community are Central to Care; 3) Understanding Access to Community Resources; 4) Importance of Traditional Ways in Indigenous Wellness; 5) Employing a Culturally Safe Approach in a Western System.

      Key lessons learned for CME/CPD practice - As health care providers, decision makers and clinicians, we are all responsible for informing a culturally safe health care environment to ensure care is equitable and free of racism. In order to do this, we must carve out safe spaces for courageous dialogue, for open and inward questioning about our own values and beliefs and how it informs our practice. It is critical that we continue to meaningfully engage with Indigenous communities to learn from their experiences, to receive community perspectives with humility and with the ultimate goal of narrowing the health inequity gap.

      Back to top


      Beyond “mini-me” and #MeToo?: An environmental scan of physician leadership programs that include concepts of diversity, gender and race

      Authors: Sophie Soklaridis, PhD, Elizabeth Lin, PhD, Georgia Black, BA Hons, Yasmin Lalani, PhD, Ayelet Kuper, MD, DPhi,l FRCPC, Morag Paton, MEd, Anna MacLeod, PhD, Constance Le Blanc, MD, FCFP(EM), CCPE, MAEd, Ivan Silver, MD, MEd, FRCP(C), Cynthia Whitehead, MD, PhD

      Institution: University of Toronto

      Problem/Methods - Current literature indicates that there is a lack of training for physicians in relation to equity, diversity and inclusion (EDI). This can impede physicians’ ability to lead a diverse workforce, representing a barrier for certain groups to obtain positions of power within medicine. To explore this gap, we conducted an environmental scan of physician leadership programs, peer reviewed literature and medical education conferences to understand current trends in physician leadership training with regard to EDI.

      Results - We found that leadership programs for physicians did not address issues of EDI. When it was addressed, it was done so in a “fix me” mentality; the actions that women or individuals underrepresented in medicine “can do” when the real barriers are beyond the individual and are systemic. Of the 19 leadership articles included, 9 articles included observable outcomes like promotion or retention rates. With regard to conference content, only 7% of the 1408 titles scanned included EDI content. There was an increase of 16% in 2019 most likely attributed to the #MeToo, #UsToo and TimesUP movements and a rise in activities among social justice groups like Black Lives Matter. Although integrating EDI concepts into leadership programs could hold great promise for redressing health inequalities, our scan indicated EDI content was generally absent from programming.

      Key lessons learned for CME/CPD practice - The findings provide CME/CPD educators an opportunity to make more strategic decisions regarding the content of physician leadership training given the identified gaps. To address the physician leadership diversity gap, programs must be developed to address systemic issues and they must be offered to all physicians (not just women and those underrepresented in medicine).

      Back to top


      Findings from the implementation of r2c2 model of feedback for psychiatry residents within a competency-based medical education framework

      Authors: Anupam Thakur, MD, MSc, Shaheen Darani, MD, FRCPC, Yasmin Lalani, PhD, Csilla Kalocsai, MPhil, PhD, Ivan Silver, MD, , MEd, FRCP(C), Sanjeev Sockalingam, MD, MHPE, FRCPC, Sophie Soklaridis,PhD

      Institution: Center for Addiction and Mental Health, Canada

      Problem/Intervention - Studies demonstrate that feedback in clinical settings often lack the ingredients necessary to support a progressive learner experience. This assumes greater significance within a competency-based medical education (CBME) framework, where structured feedback and coaching play an important role in residents’ progress. R2C2 is an evidence-based model of feedback comprising of four iterative phases: building relationship, exploring reactions, exploring content, coaching. Recognizing the need for structured feedback, this pilot study reports on the implementation of R2C2 model in a CBME context. In addition, this study encourages supervisors to reflect on gender and intersectionality during the feedback sessions, in addition to the elements of the original model.

      Methods - Implementation was carried out in three phases: planning and preparation, implementation and evaluation of the project. This study was carried out between July 2019 to November 2020 at the Centre for Addiction and Mental Health, the largest trainer of psychiatrists in Canada. In addition to training sessions from content experts, 15 supervisors used R2C2 model of feedback with psychiatry residents in the Longitudinal Ambulatory Experience clinic. Support from education leaders was available to solidify learning. Due to COVID-19 pandemic, some of the feedback sessions were carried out virtually. Semi-structured interviews were conducted with supervisors (n=10) to understand their experience of using the model.

      Results - Participants expressed positive attitudes towards R2C2 model and reported that it helped structure feedback conversations. With regards to intersectionality, there were generally two competing views: some participants felt that their social identities or the identities of their residents were not relevant to their working relationship. Other participants expressed that reflecting on the power dynamic and social identities of themselves and their residents is beneficial to the learning relationship. More training in use of the model came up as a separate theme, which has implications for future faculty development.

      Key lessons learned for practice in CME/CPD - R2C2 model is an important model of feedback within a competency-based medical education framework, where coaching for improvement can play a crucial role. Secondly, it is important to consider intersectionality in context of the supervisor-resident relationship. This model can help supervisors with skills needed to contextualize feedback. Future implementation efforts should focus on expanding faculty development, fidelity measures and resident perspectives in evaluation.

      Back to top


      The potential of implicit bias in physicians referred for communication issues

      Authors: Nathaniel V. Williams, BA, Kaylin Ervay, BS, Michael V. Williams, PhD, and Betsy White Williams, PhD, MPH.

      Institution: University of Kansas

      Problem/Intervention - Interpersonal and communication skills are critical to healthcare delivery in a modern healthcare environment. Good patient-physician communication is protective against malpractice and enhances patient compliance. Difficulties with interpersonal and communication skills can lead to referrals for remedial education. Many factors can contribute to difficulties in interpersonal and communication skills, including implicit bias. A growing number of studies found a relationship between implicit racial bias among healthcare professionals (Hall et al., 2015). Given this literature, we were interested in exploring whether such bias is present in understanding of the facial expression of emotions by physicians referred for remedial services secondary to communication issues.

      Methods - Referred physicians completed the Affect Naming of the Advanced Clinical Solutions Social Cognition test as part of the assessment process. Test stimuli are pictures of faces of people of different ethnicities displaying different emotions. The data from 150 physicians completing the test were analyzed. The data were coded for identification of the emotion displayed in the target stimuli and analyzed in a logistic regression.

      Results - There was a significant main effect for observer ethnicity and ethnicity of the stimulus, but no interaction. This indicates that participants had significantly more trouble identifying the emotion displayed on the stimulus based ethnicity, rather than whether or not the stimulus was their own ethnicity or that of another. This lack in group advantage has been noted in the literature (Reyes et al.2018 and Beaupre et al.2005).

      These data must be interpreted with caution due to the small sample size and the ad hoc nature of the stimuli used. These stimuli were not designed to assess cross ethnic sensitivity. In addition, the data were somewhat limited in power and were from a single test site.

      Key lessons learned for CME/CPD practice - Findings are important to CME/CPD as communication is an integral part of the safe delivery of medicine. Consideration of these finds should be considered when planning CME/CPD activities focusing on interpersonal and communication skills. Program content might include reinforcement of the importance of emotional competency and include activities that facilitate development of enhanced recognition of emotions in diverse populations.

      Back to top

      Evaluation of the University of Wisconsin-Madison Diversity, Equity, and Inclusion Toolkit for Interprofessional Continuing Education

      Authors: Barbara Anderson, MS, Marianna Shershneva, MD, PhD

      Institution: Office of Continuing Professional Development, University of Wisconsin School of Medicine and Public Health

      Purpose/Problem Statement/Scope of Inquiry - In January 2021, the University of Wisconsin–Madison Interprofessional Continuing Education Partnership (ICEP) implemented the Diversity, Equity, and Inclusion (DEI) Toolkit for Accredited Continuing Education to foster inclusive learning throughout our accredited continuing education program. This study focused on user feedback from early experiences with the Toolkit and a baseline assessment of applied strategies for creating inclusive learning environments in regularly scheduled series (RSS) and other activity formats.

      Approach(es)/Research Method(s)/Educational Design - The Toolkit design, developed by the ICEP team in collaboration with a School of Medicine and Public Health (SMPH) Shapiro Summer Research student, is based on the foundational work UWSMPH Presenter’s Guide,[1] and is comprised of four components:

      1) Reflective Questions spark conversation and critical exploration during the initial stages of activity planning and later during post-activity debriefings.

      2) DEI Competencies inform professional practice gaps analysis, educational needs assessment, and learning objectives formulation.

      • Engage in Self-reflection,
      • Address Health Disparities, and
      • Value Diversity in the Clinical Encounter

      3) Subject Matter Expert (SME) Tools guide faculty in the development of inclusive content.

      4) Evaluation Questions measure achievement of the DEI competencies.

      Consistent with the experiences of other academic centers [2], our Toolkit is designed to support inclusive learning environments by fully embedding its components into the planning, delivery, and evaluation process of accredited CE activities.

      Following Kirkpatrick's four-level evaluation model, we collected feedback from Toolkit users (reaction and learning), analyzed planning documents and committee meeting notes (behavior), and examined responses to DEI-related questions in program-wide and activity evaluations (results).

      Evaluation/Outcomes/Discussion - The Toolkit was officially released through our planning application, faculty communication, and evaluation in 2021. However, a full awareness-building campaign was not implemented that year because the ongoing COVID-19 pandemic slowed faculty engagement in accredited education.

      Program-based Results

      In the annual survey of participants in the ICEP-accredited activities, respondents were asked to rate their satisfaction regarding the diversity of perspectives and experiences presented within the activity they were evaluating.

      On a scale from 1=very dissatisfied to 5=very satisfied, 87.18% of respondents were either satisfied or very satisfied with the diversity of perspectives presented (Figure 1). Further investigation of the data showed the average satisfaction response by activity type fell within the “satisfied” domain, although the rating appeared to be lower for the performance improvement activities:

      • Live/virtual live: 4.18
      • Enduring: 4.18
      • RSS: 4.38
      • Performance improvement projects: 3.79

      Figure 1. Satisfaction with Diversity Perspectives

      As an interprofessional continuing education program, our planning process emphasizes learning from, with, and about members of the healthcare team. Possibly, the high level of agreement was influenced by our work to promote interprofessional learning and collaborative practice rather than inclusive learning environments in the context of DEI. Qualitative responses provided some insights into how respondents may have interpreted the questions. The examples below describe satisfaction from the DEI and team-based learning perspectives.

                • “The [course] is a multi-state conference that combines research talks with medical approaches and the treatment, but also has a strong component of equity diagnosis, access, and treatment for underserved populations.”
                • “The diversity of perspectives and experiences from the interprofessional teams presenting has been outstanding.”

      Others are examples of suggestions for improvement:

                • “Also include by type of patients served: ultra-rural, rural, suburban but far from academic, suburban near academic, urban nonacademic, urban academic.”
                • “We need to hear perspectives of people of color.”

      The official roll-out of the Toolkit to the RSS community began in February 2022. In preparation for the renewal process, RSS planners responded to survey questions about their familiarity with the Toolkit (Figure 2). Not surprisingly, many respondents indicated this was the first time they were learning about the Toolkit.

      Figure 2. Toolkit Awareness Among RSS Planners.

      Notably, several RSS committees identified specific steps they would take to facilitate creating inclusive learning environments, such as conducting DEI-focused needs assessments, incorporating cases of underrepresented populations, increasing speaker diversity, developing metrics to measure the effectiveness DEI strategies used by speakers, and collaborating with DEI specialists.

      Reflection and Debrief

      Three planning committees responded to the reflection/debrief questions via the Activity Planning Debrief tool. Two of three committees indicated they included all patients who might be impacted by the disease through thoughtful faculty selection and content development. Committees shared strategies for how they integrated the voices and priorities of all affected by the issue, such as:

              • “The planning committee ensures a wide range of topics, needs, presenters, and patient and provider experiences are incorporated into this series.”
              • “Speaker had firsthand knowledge of navigating these issues.”

      DEI Competencies

      In 2021, 72% of 18 activities elected to address DEI competencies in the activity practice gaps, educational needs, and learning objectives. This included 60% of live or virtual live activities and 87% of new RSS. Planners selected the competencies on an equal basis. Engaging in self-reflection was selected seven times, addressing health disparities, eight times, and valuing diversity in the clinical encounter, six times. Outcomes data related to these competencies are not yet available for these activities.

      SME Tools

      SME tools have been underutilized to date. A total of 28 speakers/authors submitted the DEI checklist. RSS faculty used the checklist most often. These faculty wrote notes describing steps they would take that will support inclusive learning environments.

              • “When applicable in my presentation materials, I will aim to appropriately acknowledge the role of psychosocial determinants of health as they relate to my topic.”
              • “Appropriate language will be used to discuss the condition. These patients are underrepresented in research but have important and specific needs related to their care.”
              • “My choice of examples and case studies will be sensitive to the criteria listed above.”

      Evaluation

      Evaluation tools have continued to evolve since the initial release of the Toolkit. Through anecdotal feedback, it was determined that the limited number of the DEI-related evaluation questions did not provide adequate opportunities to measure the intended outcomes and/or perceived inclusivity of learning environments at the individual activity level. As a result, we have expanded the pool of questions, from two to six, that planning committees can select from when building their activity evaluation. In addition, planners are also encouraged to write a new question that best measures the achievement of intended outcomes.

      Discussion

      The study documented progress in the Toolkit implementation, high interest in the DEI tools among education planners and faculty, and opportunities to improve the tools and increase awareness of these resources. For example, to increase utilization of the Debrief Tool, we need Accreditation Specialists to encourage planners to review the tool when closing out the activity and/or we need to centralize the distribution of this tool.

      Key Learnings for CME/CPD Practice - We observed a favorable uptake in the Toolkit utilization, and we continue to collect and analyze data that indicate needed improvements. The Toolkit is available for review and use at the ICEP Learning Portal [3]. Our institution’s experience illustrates the need for DEI tools that are accessible to the CE community. We learned that requests for support from early adopters of best DEI practices need to be honored. This could be accomplished through professional development to increase the knowledge and skills needed to effectively use resources, like the ICEP DEI Toolkit, and to practice meaningful DEI discussions. Considerate implementation of resources organized in a toolkit could serve as a framework that empowers CPD practitioners’ efforts to develop inclusive and equitable continuing education.

      References

      1. Bussan H, Hoang T, Villaruz J, Hernandez JB, Rajan S. University of Wisconsin School of Medicine and Public Health Presenter's Guide. (2019) SMPH Intranet. https://intranet.med.wisc.edu/building-community/. Accessed June 1, 2020.

      2. Centre for Addiction and Mental Health (2021). Health Equity and Inclusion Framework for Education and Training. Toronto: CAMH. https://kmb.camh.ca/eenet/resources/health-equity-and-inclusion-framework-education-and-training. Accessed March 1, 2022.

      3. Nytes C, Shershneva M, Anderson B. UW-Madison ICEP Diversity, Equity, and Inclusion Toolkit for Accredited Continuing Education (2021) https://ce.icep.wisc.edu/content/uw-madison-dei-toolkit-accredited-continuing-education.

      Back to Top

      V. Healthy and Inclusive Workplaces (2)

      Creating a culture of humility: Equity, diversity and inclusion

      Author: Heather L. Clemons, MS, MBA, ATC, CHCP

      Institution: Sharp Healthcare

      Problem/Intervention - The Sharp Equality Alliance (SEA), a grass roots EDI committee created at Sharp HealthCare began as 4 employees asking to get engaged in the local annual Pride Parade in 2014 and has since exploded into 100+ member committee providing a range of educational opportunities across the organization on equity, diversity and inclusion (EDI) and health equity topics. Significant growth came in 2020 and SEA responded to organizational needs by creating a steering committee and developing new opportunities: Safe Speak, Current Conversations and Breakfast Forum - Health Equity Series.

      Methods/Results - Safe Speak, a space where employees can discuss their challenges shortly after George Floyd’s death in response to a call for support by employees. These sessions are facilitated by trained counselors. Conversations, a moderated educational series designed to support organization-wide cultural humility development through themed topics (ex.: What’s in a Name?) and relating it to one’s own experience. Moderated by a content expert, there is a focus on questions and discussion. Breakfast Forum - A Health Equity Series, an accredited continuing education series focused on addressing health disparities by exploring the connections between social determinants and barriers to equitable care. It was accredited for CME, continuing pharmacy education (CPE) and social work credit.

      Additionally, leader education on micro-aggressions, a more engaged executive leadership has consistently created awareness for EDI, and targeted education helps staff better connect with patients.

      Key lessons learned for CME/CPD practice - CME’s collaboration with SEA has demonstrated an ability to support organizational priorities and be a resource for developing impactful education. Creating a culture of humility has become part of the CME Department’s value proposition. The collaborations that have resulted are invaluable. SEA priorities for 2021 include adding entity-based chapters, increasing the frequency of the Safe Speak and educational opportunities, and develop a tracking mechanism describing the impact of SEA across marketing and communications, patient care, and physician and employee engagement.

      Back to top


      The Oregon Wellness Program

      Author: Donald E. Girard, MD, MACP

      Institution: Oregon Health Sciences University

      Problem/Intervention - Healthcare professionals deal with substantial occupational stress, including burnout and now the disastrous effects of the COVID pandemic. Professionals’ outcomes of untreated mental health crises result in poor patient care, negative work relationships, professional attrition, family disruptions, somatic complaints, anxiety, depression, maladaptive behaviors and suicide.

      The Oregon Wellness Program was created by physicians and healthcare leaders to provide professional, confidential, voluntary-only, free mental health support to Oregon practicing healthcare professionals. Its goal is to help these professionals cope with the stresses of their personal and professional lives, regaining or maintaining wellness, and contribute exceptional patient care.

      Methods - A myriad of methods are available as mental health aids, but the gold confidential, voluntary-only services from highly reputed psychologists or psychiatrists experienced in caring for healthcare professionals. Through a central call line, sessions are scheduled within three days. Eight sessions yearly are free. The Oregon Medical Board with legislative approval and Oregon health systems support the financial costs.

      Results - From its 2018 inception, 1400 visits have been provided. Each client has been seen more than once. Most clients have been physicians, more women than men, and most professionals have been in mid-career. Suburban and rural numbers equal demographic densities.

      Key lessons learned for CME/CPD practice - The pilot, and confidential research reveals highly positive results for program value by clients, improved work behavior and relationships, and enhanced professional satisfaction. OWP serves as a replicable model for other states and regions to bring wellness to its healthcare professionals.

      Back to top


      VI. Practice Change (5)

      “Change talk” in small group learning communities – an ethnographic study

      Authors: Heather Armson, MD, MCE, CCFP, FCFP, Kathleen Moncrieff, PhD, MD, Meghan Lofft, MSc, and Stefanie Roder, PhD

      Institution: University of Calgary

      Problem/Intervention - In the context of small group learning communities, this study explored components of small group discussions that contribute to knowledge implementation using an ethnographic approach. Participants were Canadian family physicians who are members of a well-established Practice-based Small Group Learning (PBSGL) program. PBSGL members meet monthly to discuss evidence-based educational modules on various clinical topics under the guidance of a trained peer facilitator.

      Methods - For this study data was collected from two PBSGL groups, one new group (7 members) that had been meeting for 1 year and an established group (8 members) that had been meeting regularly for over 15 years. Nine small group learning sessions were observed, 11 small group participants were interviewed and practice reflections tools (PRTs) documenting decisions for practice change were reviewed.

      Results - Thematic analysis of the field notes showed similarities and differences for observed sessions of the new (5 sessions) and the more established (4 sessions) PBSGL group. Both PBSGL groups with guidance of a facilitator discussed cases provided in evidence-based educational modules, interpreted new information, and documented decisions for practice change. The new group predominantly worked through cases and focused on new information provided to consolidate knowledge. The established group was critical of evidence provided in modules, shared other resources and clinical experiences to consolidate knowledge. Interviews supported these different approaches, emphasizing that sharing of practice experiences were an integral part in the decision to implement new knowledge, helped validate guideline recommendations, and provided strategies for feasible practice changes. Decisions for practice change documented on PRTs overlapped with the field notes.

      Key lessons learned for CME/CPD practice - This study highlighted key elements of small group discussions that facilitate translation of new knowledge into practice. Being part of a community of practice provides the opportunity to share practice experiences, evaluate clinical evidence and challenge guideline recommendations in the context of local practices.

      Back to top


      Teaching trauma tools: Two novel approaches to continuing professional development for healthcare providers in developmental trauma

      Authors: Mushfika Chowdhury, MD, Chantalle Clarkin, RN, PhD, Victoria Bond, Cheryl Pereira, Tali Boritz, Allison Crawford, MD, PhD, FRCPS, and Priya N. Watson

      Institution: University of Toronto

      Problem/Intervention - Developmental trauma (DT) or Adverse Childhood Experiences (ACEs) are chronic childhood stressors related to abuse, neglect, and household dysfunction. DT/ACEs are associated with lifelong poor health outcomes; however, DT/ACEs are not typically addressed in most routine healthcare encounters. Continuing Professional Development (CPD) is therefore needed to address this clinical-practice gap.

      Methods - We describe the design and evaluation of two separate online models for virtual CPD in DT:

      1. The Childhood Trauma Toolkit (CTT), a self-directed, module-based online curriculum.
      2. Project Extension for Community Healthcare Outcomes - Developmental Trauma and Resilience (ECHO DT&R), an educational program delivered through live videoconferencing.


      CTT and ECHO DT&R were designed as independent programs to provide education to healthcare providers (HCPs) regarding the screening, identification, and management of DT/ACEs. The frameworks underpinning both models emphasize the importance of needs assessments to adapt CPD to local learner contexts. The CTT used a needs assessment survey and pre-program focus groups with HCPs to inform program content and delivery. ECHO DT&R employed a needs assessment survey, and administered surveys following each session to assess learning needs, self-efficacy, and satisfaction; this data created a feedback loop for continuous improvement. Summative evaluations took the form of pre-and-post surveys in both programs.

      Results - Participants in both programs reported enhanced knowledge, self-efficacy, and practice change. High satisfaction scores were reported by HCPs in both programs with regards to learning needs, relevance to practice, and evidence-base.

      Key lessons learned for CME/CPD practice - The findings from our program evaluations contribute three broad recommendations for CPD initiative that addresses clinical practice gaps:

      1. The virtual delivery of CPD enhances access and scalability, and can address widespread capacity concerns; 
      2. The assessment of learner needs can provide insights regarding knowledge-to-action gaps; 
      3. Curriculum development requires responsiveness and relevance to learners’ needs.

      The virtual delivery of CPD holds promise for addressing clinical-practice gaps that persist in the screening, identification and management of DT/ACEs, particularly when adapted to local learning contexts. While using different approaches, the findings of these programs demonstrate high satisfaction, enhanced learning and intentions to change practice.

      Back to top


      Mental health training for correctional officers: A systematic review

      Authors: Shaheen A. Darani, MD, FRCPC, Sandy Simpson, MBBS, FRCPC, Elena Wolff, MD, Robert McMaster, MD, FRCPC, Sarah Bonato, MIS, Graham Glancy, M.B, ChB. FRCPC, Jason Quinn, MD, FRCPC

      Institution: University of Toronto

      Problem/Intervention - People with mental illness are overrepresented in correctional facilities. Correctional officers often lack training to respond to inmates with mental illness. Implementing mental health training could improve officer knowledge, skills, and attitudes toward inmates with mental illness.

      Methods - We conducted a systematic review of mental health training programs for correctional officers to identify key factors related to success. Medical and criminal justice databases were searched for scientific articles describing correctional officer mental health training programs. All studies that included a measurable outcome on either correctional officer knowledge or inmate mental health were included in a final analysis. The review adhered to PRISMA guidelines for systematic reviews.

      Results - Of 1492 articles identified using search terms, 11 were included in the analysis. 6 articles described mental health education programs, 2 articles described skill-specific programs, and 3 articles described suicide prevention programs. Training programs reviewed content about mental illness, practical skills, and included didactic and experiential teaching modalities. The programs led to improvements in knowledge, skills, and attitudes amongst officers. Prior mental health attitudes, knowledge, and work experience did not correlate with improvements following training. Officers were more receptive to program facilitators with correctional or lived mental health experience. Experiential teaching was preferred to didactic teaching. Regression occurred several months after training.

      There is limited but positive literature suggesting that structured training programs, particularly involving persons with lived experience and experiential components are beneficial. It is possible articles pertaining to correctional officer mental health training were not available on the databases searched or some programs may not be published. Studies were limited in their outcome measurement, with no consistent tools, and no control groups.

      Key lessons learned for CME/CPD practice - This review can guide the creation and delivery of future training programs. Training that incorporates facilitators with lived experience and experiential teaching modalities can be beneficial. Future studies should address methodological limitations in the studies reviewed. The regression suggests a need for ongoing education and systems change within correctional institutions to ensure sustainability of improvements.

      Back to top


      Evaluations of continuing health provider education for opioid prescribing: A systematic scoping review

      Authors: Abhimanyu Sud, MD, CCFP, Graziella R. Molska, PhD, Fabio Salamanca-Buentello, MD, PhD

      Institution: University of Toronto

      Problem/Intervention - Continuing health provider education (HPE) is an important health policy intervention for the opioid epidemic. We aimed to provide a comprehensive understanding of the evaluation of opioid continuing HPE programs and their appropriateness as population health interventions.

      Methods - We adapted a logic model for medical education programs and considered evaluation outcomes from Moore’s outcomes framework for continuing HPE. We used Cervero and Gaines’ review of systematic reviews on the effectiveness of continuing medical education to categorize evidence-based best practices in continuing HPE. To facilitate consideration of HPE as health policy, we categorized education outcomes as implementation, effectiveness, and impact outcomes according to frameworks for complex interventions.

      We conducted a systematic scoping review on continuing HPE programs for opioid analgesic prescribing and overdose prevention. We extracted a range of data using categories for evaluating complex programs, including use of explicit theory, program purpose, inputs, activities, outputs, outcomes, and industry involvement.

      Results - We analyzed evaluations reporting on 32 continuing HPE programs from 28 distinct author groups. The reports spanned from 1983 to 2020, though 88% were published on or after 2010. The vast majority of North American programs with 78% based in the United States and 19% in Canada. Most studies in our sample, 84% used population-level concerns of opioid-related harms to justify the educational intervention, but only 15% measured patients or population level outcomes related to the educational programs. There is a clear knowledge gap regarding appropriate program design and evaluation. There is further need for conceptual models for the development and evaluation of continuing HPE intending to have population health effects.

      Key lessons learned for CME/CPD practice - We provided two primary recommendations for the practice of continuing HPE programs (especially those that aim to have population health impacts): 1) adoption of evaluation frameworks for complex interventions; and 2) the use of conceptual models that link clinical practice and population health.

      Back to top


      A conceptual model of continuing medical education for population health

      Authors: Abhimanyu Sud, MD, CCFP, Kate Hodgson DVM, MHSC, CCMEP, Gary Bloch MD, CCFP, FCFP, and Ross Upshur MA, MD, MSc

      Institution: University of Toronto

      Problem/Intervention - Continuing medical education (CME) is an important means through which to improve clinical practice. Both individual health care providers and health systems are being increasingly directed towards population health concerns. Using the example of opioid prescribing CME, we argue that existing CME conceptual models and conceptions of CME effectiveness, are inadequate to meet the goals of population health improvement.

      Frameworks for the design, implementation and evaluation of CME consistently reference population health, but fail to adequately conceptualize population health beyond the aggregation of individual patient health changes. Opioid prescribing CME programs use population health concerns to justify their programs, but evaluation approaches are inadequate for demonstrating population health impacts.

      Methods - We draw on the frameworks of Clinical Population Medicine (CPM) and the Social Determinants of Health (SDOH), as a well as philosophies of population health to build conceptual bridges between the typical outcome levels of physician knowledge, physician performance, and patient health to population health.

      Results - We propose that CME must be re-oriented in multiple forms, including interpreting evidence from a population prevention perspective, incorporating SDOH into CME activities and explicitly training health care providers and CME planners to be fluent in population health concepts and practices.

      Key lessons learned for practice in CME/CPD - We provide a framework for how CME practitioners, funders and policy makers can develop, implement, and evaluate CME vis-a-vis population health concerns. We include five practical means for doing this which include: 1) spreading effective clinically oriented CME programs across clinical practices; 2) (re)orient educational content from focusing on only clinical perspectives to clinical & population health perspectives; 3) providing clinical approaches for intervening on social determinants of health; 4) training clinicians and educators in population health to facilitate CME planning to identify and address population health concerns; and 5) address marginalization and intersectionality to ensure CME programs do not increase social inequalities.

      Back to top


      VII. Post-COVID Implications (5)

      Leveraging technology: 5 Steps for effectively selecting and implementing new platforms

      Author: Bonnie Doman

      Institution: AOE Consulting

      Problem/Intervention - AOE Consulting’s (AOE) work helping multiple accredited organizations transition to virtual accredited continuing education (CE) activities in this past year, afforded our team the opportunity to research, analyze and implement a number of technology platforms. Our direct involvement with this implementation allowed us to identify a list of best practices, which promoted development of a concise, step-by-step guide for selecting and implementing new technology platforms.

      Methods - AOE has participated in focused workshops related to technology in accredited CE, and we have engaged in numerous product demonstrations and hands-on testing of many technology platforms. To effectively compare technology platforms, we developed an internal comparative analysis tool to guide selection of technology solutions based on established need(s). We then developed a step-by-step plan for overcoming common challenges and pitfalls that can come with implementation. This plan involves training and timeline considerations for successful onboarding and ongoing maintenance.

      Results - Leveraging technology platforms effectively is critical for maintaining a successful accredited CE program, in terms of improving internal efficiencies, providing effective educational solutions and achieving optimal utilization of resource investments. Following our step-by-step plan outlined, AOE has successfully implemented new technology in a streamlined and efficient manner, that has facilitated achievement of both short-term and long-term organizational goals.

      Key lessons learned for CME/CPD practice - As the COVID-19 crisis has ushered in a new era of dependence on technology, many accredited CE programs have been forced to rapidly implement new software platforms or upgrade existing platforms to adapt to remote work environments internally, and to offer quality accredited CE activities virtually. The wide range of available platforms can be difficult to sort through, and success often requires significant resources and expertise. Challenges of technology selection and implementation can be successfully navigated with a well-devised plan, inclusive of thorough research and analysis, creation of an organized training program and an ongoing maintenance strategy.

      Back to top


      Mental health interventions and supports during COVID-19 and other medical pandemics: A rapid systematic review of the evidence and implications for CME/CPD  

      Authors:  Elizabeth Lin, PhD; Sophie Soklaridis, PhD; Yasmin Lalani, PhD; Terri Rodak, MA, MISt; and Sanjeev Sockalingam, MD, MHPE, FRCPC

      Institution: Center for Addiction and Mental Health, University of Toronto

      Problem/Interventions -  A rapid systematic review (following PRISMA guidelines) was conducted regarding mental health (MH) interventions during medical pandemics – including SARS, Ebola, MERS, influenza pandemics, and COVID-19. The resulting 21 peer-reviewed studies were assessed for methodological quality (KMET criteria). The interventions included: 

      • System-level interventions (national and health-system pandemic protocols and structures; training program designs, rapid-response staff supports).
      • Psychosocial interventions (arts-based support for children, self-care and group interventions for patients and staff; mobile and Internet modalities).

      Quality assessment -

      • 57% of the articles met criteria for high/very-high quality.
      • However, there was a wide range of ratings (40-100%).

      Research findings -

      • Emotional and mental health impact of pandemics seen in everyone, especially clinical staff caring for pandemic cases.
      • Both system-level and psychosocial interventions show promise in addressing these impacts.
      • However, cultural differences affected the performance of some interventions.

      Key lessons learned for CME/CPD practice - 

      • A variety of individual- and system-level interventions show promise in addressing mental health sequalae of medical pandemics.
      • From a global perspective: Cultural and country-specific differences should be considered when adopting interventions.
      • Given the different expectations associated with pandemics (need for quick action) versus evaluation (need for careful assessment) and the varying methodological quality ratings, a ‘hypothesis-testing’ approach when implementing solutions is recommended. A critical ingredient is incorporating evaluation from the start to monitor the impacts of the intervention and provide evidence for other jurisdictions to judge how applicable the results are for them.

      Back to top


      UBC CPD’s rapid COVID-19 CPD response: A story of unprecedented impact and lessons learned

      Authors: Brenna Lynn, PhD, Jennie Barrows, MA, Stephanie Ameyaw, MA, Kate Meffen, BSc, Steph Din, BA, Bruce Hobson, MD, Bob Bluman, MD, CCFP, FCFP

      Institution: University of British Columbia

      Problem/Intervention - UBC’s Division of Continuing Professional Development (UBC CPD) is a trusted source of education for health care providers (HCPs). When faced with COVID-19, UBC CPD successfully addressed urgent and emerging learning needs while pivoting from in-person to virtual delivery.

      Methods - UBC CPD conducted a rapid needs’ assessment and mobilized a comprehensive COVID-19 CPD response including webinars, a resource hub, rapid knowledge translation for HCPs, and online modules. This multimodal approach focused on responsive programming to meet the determined needs of a wide audience.

      Results - There was unprecedented demand for CPD. From March to June 2020, 12,254 HCPs attended 22 webinars and 8400 users viewed online resources. Participant evaluation surveys and internal focus groups identified key success factors: an iterative needs assessment, appropriate virtual learning space and skilled facilitation; presenter diversity and expertise; and flexible delivery formats. UBC CPD rapidly pivoted to meet the needs of HCPs with timely, accessible and requested CPD. While some factors were unique to the early period of the pandemic, UBC CPD and the broader field of CPD can learn from this experience.

      Key lessons learned for CME/CPD practice - Nimble, iterative needs assessments are key for rapidly changing topics; formatting webinars around participant questions allows content to be responsive to learners’ needs and reduces demands on presenters; a panel led format provides a venue for experts to share evolving information; increased frequency of education delivery to meet the wide range of educational needs related to the pandemic; and online CPD reaches a wide audience and enables timely education and collegial connection.

      Back to top


      From the front lines of a pandemic: Interviews with the experts—a six part CME/CPD series created in response to the crisis

      Authors: Vickie Skinner, DHA, Brian Tollefson, MD, and Bo Lewis, BA

      Institution: University of Mississippi Medical Center

      Problem/Intervention - The University of Mississippi Medical Center Division of Continuing Health Professional Education (UMMC CHPE), in collaboration with the Department of Emergency Medicine, recognized a critical need to inform, educate and train clinicians in the diagnosis and treatment of patients with COVID-19. To address this urgent need, UMMC CHPE staff planned and delivered a series of panel discussions with expert physicians from across the institution. A champion emergency medicine physician moderated all sessions and assisted with creation of panel discussion questions. A survey of the target audience also produced additional panel questions. The six panels consisted of experts from the following areas: research, emergency department, medical intensive care unit (MICU), academic leaders/ department chairs, ambulatory leadership, and division of infectious disease.

      Methods - UMMC CHPE staff created, produced, and launched six free 1-hour enduring material modules of accredited continuing education for physicians and nurses in response to the pandemic.

      Results - In record time, UMMC CHPE staff were able to offer six free hours of accredited continuing education on a topic of immediate relevance and importance to physicians and nurses. This effort was one way CHPE was able to demonstrate value to the institution and meet an ongoing educational need.

      Key lessons learned for CME/CPD practice - As an enduring material addressing COVID-19 response actions, protocol and policy changes this information would need to be continually reviewed and updated as the pandemic evolved to be considered current guidance. Without a plan in place to update information as guidance changed, this educational content is best presented and positioned as a snap shot of one moment in time and possibly a benchmark for future responses or ongoing changes.

      Back to top


      Research integrity southeast ’21: In the wake of a global pandemic-crisis response in the form of an interprofessional CPD conference

      Authors: Vickie Skinner, DHA and Samuel Bruton, PhD

      Institution: University of Mississippi Medical Center

      Problem/Intervention - In response to the COVID-19 crisis, the University of Mississippi Medical Center Division of Continuing Health Professional Education (UMMC CHPE), in collaboration with the University of Southern Mississippi Office of Research Integrity (USM ORI), created, produced, and launched a research integrity conference as an enduring material of accredited continuing education for physicians, nurses, and dentists.

      Methods - This conference was planned as a live event and was shifted to a virtual event, then shifted again to an enduring material when severe winter weather impacted our presenters and participants ability to attend. This conference provides six and one-quarter hours of interprofessional continuing education highlighting a wide range of emerging and research integrity challenges across a broad academic landscape. The target audience includes: researchers, research ethicists, physicians, nurse practitioners, nurses, dentists, allied health professionals, public health professionals, health education professionals, research faculty, and graduate students.

      Results - The final conference agenda includes the following topics: The New Normal: COVID-19 Mitigation Risk and Research Risk, The MS CEAL Team: Conducting Community-engaged Research During the COVID-19 Pandemic, Moderna’s COVID Vaccine, Publishing in a Pandemic: COVID-19’s Effect on Preprints and Open Science, Pediatric Clinical Research Challenges: Response to COVID-19 and Directions for Future Research, as well as a panel discussion with research experts and poster presentations.

      Key lessons learned for practice in CME/CPD - Planning and hosting a conference of this caliber during a pandemic requires adaptability and perseverance. This live conference pivoted to a virtual conference, which then pivoted to enduring material. Keeping your presenters and registrants engaged and informed throughout the various changes posed quite challenging.

      Back to top


      The COVID CPD landscape: Market research and insights for planning programmatic offerings

      Authors: Eleftherios K. Soleas, PhD, BEd, Jenny DeBruyn, BA, Katherine Evans, BA, Mary Felix Dip-MA, Kate Kittner, BA and Richard C. van Wylick, MD

      Institution: Queen’s University, Canada

      Problem/Intervention - The COVID pandemic forced CPD offices to begin to enact the recommendations of The Future of Medical Education in Canada-CPD (FMEC-CPD; Campbell & Sisler, 2019). In a rapid series of events stemming from the inability to hold in-person programming, CPD offices hurriedly transitioned to an online paradigm, well outside their comfort zone. This study is a chronicle of one office’s lessons learned.

      Methods - This mixed-methods market research and program evaluation consolidates a program of anonymous surveys taking place pre/post program (15 webinars/6 asynchronous modules) CPD offerings totaling 2784 respondents at Queen’s University. The surveys asked Likert, demographic (age, gender, practice location, health profession), and market research open-ended questions relating to learner preference, comfort, knowledge gain, barriers to practice, and intent to change metrics. These were analyzed thematically using ATLAS.ti and using inferential statistics (SPSS).

      Results - Our thematic findings show learners increasingly adapting to online learning. Learners reported preferring asynchronous as compared to the same duration of online synchronous learning. The analyses of pre-post responses showed that asynchronous offerings resulted in greater change metrics than synchronous programing.

      Key lessons learned for CME/CPD - CPD offices must continue to learn from their successes and failures and we will need to decide whether we will be biding our time to return to in-person primary programming or elect to embrace the possibilities afforded to us by the pandemic-shifted paradigm. Key learnings include:

      1. The preferred price points differ by profession:
        • Nurse practitioners, pharmacists, and physicians prefer standard rate
        • Residents, nurses, and physician assistants prefer discounted rate
        • UGME, nursing, and rehabilitation students prefer heavily discounted rate
      2. Learners across professions tended to select programs with similar reasoning: upfront topic relevance, involving a member of my profession, time-conservative, and reacting badly to biased or non-compelling speakers and topics.
      3. Offer a pre-survey (if you do one) concurrent with the registration process.
      4. Conduct post-surveys immediately after each session, as opposed to only one survey to evaluate all sessions in the series at the end

      Back to top


      Effects of COVID-19 Pandemic on Continuing Medical Education Activities

      Authors: Bita Zakeri PhD, MacKenzie Church

      Institution: Indiana University School of Medicine 

      Problem/Intervention - COVID-19 pandemic set about a forced movement for mass online education. By May 2020, all continuing education activities at IUSM CME were postponed, cancelled, or transitioned to virtual. As a result, we designed a survey to gauge learner perspective and assess the efficacy of this rapid transition, aiming to improve virtual activities based on learner feedback post-COVID. The overarching question for our investigation was whether virtual CME activities are as effective as in-person activities. While this study is intended to assess the overall efficacy of virtual vs. in-person meetings, it is more specifically intended to meet the objectives of assessing the changes in opportunities for participation, feedback, and collaboration. The framework of this study is grounded theory and the perspectives informing this study are drawn from adult and social learning theories pertaining to online education. (Miller, 1990; Bonk 2005, Taylor & Hamdy, 2013; Bowen, 2013). 

      Methods - Preliminary data consisted of a survey distributed to learners who participated in 84 virtual live courses and 314 virtual RSS for a total of 1369 respondents from May - December 2020.  The survey comprised of 6 multiple choice and 3 open ended questions. Results include descriptive quantitative data and qualitative data analyzed thematically.

      Results - Major findings show that:

      • ~80% indicated there is the same level of opportunity or more to participate in virtual meetings
      • Though majority of responses indicated level of feedback to questions were same or more, as in-person activities, ~28% rated level of feedback as less in virtual courses, highlighting the need for more Q&A and response time.
      • ~44% overall did not find virtual activities as collaborative and interactive as in-person. Hence there is a significant need to improve interactive and collaborative elements in online meetings.
      • ~ 56% preferred a hybrid mode of delivery for both live courses and RSS.  Another ~28% indicated their preference depends on the activity, pointing out the topic, need for hands-on practice, and location of in-person meetings as influencing factors.
      • ~48% indicated some important elements were missing in virtual activities, top five being participation, social, professional, difficulty for presenter due to lack of visibility/interaction with audience, and interactivity.
      • ~81% of respondents found CME virtual activities as beneficial as when in-person.

        Key lessons learned for CME/CPD practice - Though responses indicate an overall shift from in-person to virtual as successful, this transition is a major milestone that needs continuous significant improvements to address the needs of learners to enhance opportunities for in-depth discussion and collaboration​, social and professional interaction​, and increased interactivity to entice engagement​. Findings of this study are critical to future design and delivery of CME activities, appropriate resources to enhance virtual or hybrid education and design of educational activities at large across UME, GME and CME.

        Back to top

        Powered by Wild Apricot Membership Software