Syndicated, Virginia Group

Connected learning goes to medical school

Connected Learning has evolved since I wrote my dissertation on it a couple years ago. Back then, the Connected Learning Alliance was promoting an awkward framework that combined learner perspectives and instructional design in the same space (who does that?).  But now they have boiled their messaging down to the essence:

Connected learning is an educational approach that combines learner interests, relationships, and opportunities to promote deeper, inclusive, and powerful learning. To break it down, briefly:

  • Learner interests emerge from a spectrum of contexts including personal passions, community and culture, and career goals and aspirations.
  • Relationships between peers and mentors allow everyone to engage in information sharing, knowledge construction, and feedback for the purpose of benefiting everyone.
  • Opportunities are authentic, real-world activities brokered* by faculty, student mentors, and institutions (*brokered =  sharing information and resources, making introductions, providing funding, smoothing out the logistics, etc).
  • Deeper learning means effective learning, in terms of increased retention, better application, and easier transfer.
  • Inclusive learning means that which clicks with more and more diverse student bodies.
  • Powerful learning is that which makes a difference in terms of empowering learners but also benefiting something or someone beyond the individual learner.

Okay, now that that is explained…

Connected learning has reached medical education, although it is not called by that name. You can find connected learning in the Genes to Society curriculum at Johns Hopkins University.  They call it Precision Education.

Johns Hopkins has prioritized digital platforms that allow students to consume, share, and collaborate in the construction of knowledge with their peers and faculty.  They learn to choose what they consume strategically and based on personal interest.  Their mentors focus on helping students make connections between those interests and career goals. In providing a digitally-driven network of resources (peers, faculty mentors, and well-curated content), they are reaching and supporting students of diverse backgrounds and goals.  That’s my summary, but here’s the article:

Excerpts from Precision Education in the [Johns Hopkins] School of Medicine’s 125th Year

Goldberg and his team are designing a system where medical students can explore facets of particular subjects that interest them before, during and after traditional classroom lectures. An online network of articles, videos, discussions and even notes from other students opens up new options for class preparation and engagement. He adds that not only can students assume greater control over their own learning, but that this approach allows courses to grow into dynamic collections of materials shared by faculty as well as students.

I started to bold the parts of this quote with direct connections to connected learning, but I had more words bolded than not, so I gave up. But note the dynamic collections of materials created by FACULTY AND STUDENTS. To continue here is more about community and peer support…

“The idea is that we are creating a community of learners, that we’re all in this together,” says Goldberg. “It’s often more difficult for the expert to explain something to the person who is not the expert than it is for a colleague to explain it to that non-expert.”

and some more about empowering students to see the connections between personal interests and potential career pathways…

“This has the potential to help students learn more about themselves,” he says. “If a student takes an interest in a particular area, such as biomedical ethics, that may be something he or she could make a career of.”

and let’s end with some diversity and inclusivity…

Precision education also addresses the disparate experiences and interests that exist among students who come to medicine from different academic backgrounds.

“Currently, every student receives the same information regardless of their academic needs. Such standardization compromises our ability to enhance creativity, wonder and academic diversity of thought,” Harry Goldberg says.

The digital pedagogies are sinking in, even if people aren’t naming them correctly.  In fact, I can’t think of anything worse than “precision education” as a name for connected learning. Currently, I’m imagining the connected learning gurus hearing that term and immediately turning off and tuning out, thinking that they are about to hear another promotion for learning analytics. Get Johns Hopkins a new branding team for goodness sake but don’t throw away what they actually mean by it, because it’s all good.

Featured Image: Owen Beard on Unsplash

Syndicated, Virginia Group

Personalized Learning in MedEd: Multiple Learning Pathways

Personalized learning, or the ability to customize the learning experience for the purpose of student satisfaction, effective learning, and better (more + more relevant) outcomes is a trend emerging across every aspect of higher education. In the context of medical education, the concept of personalized learning as a mandate can be linked to its mention in 2010 Carnegie Report, which called for the transformation of the medical education system to include (1) standardized outcomes, (2) better integration between classroom and workplace learning, and (3) more emphasis on the development of professional identity WITH (3a) personalization and/or acceleration of learning once professional identify was established.

I’ve written on (1) the standardized outcomes and (2) clinical-classroom integration before (best done so far in this post).  I’ve also written on how many schools are working on enhancing their formal and informal curriculum around professional identity through learning communities and targeted, proactive – dare I say aggressive? – coaching (see this post, among others).

So let’s talk about 3a:  the personalization and/or acceleration of learning. Between the figure and the text, you’ll see a conflation of two terms: personalized learning and precision education.  As I’ve mentioned before, “precision” medical education may be taking over for “personalized” learning as an official buzzword…or maybe it is going to be one approach or type of personalized learning (most likely, in my opinion). The jury is still out, so feel free to read this post with whatever phrase you prefer.  But regardless of what name you want to use, let’s walk through the components mentioned in the graphic above in more detail. This post will focus on multiple learning pathways.

One of the most common ways to promote personalized learning is the presence of multiple, personal learning pathways. Personal learning pathways can mean a lot of things across – or even within – programs.  This section explores three approaches medical schools take to providing students with the ability to customize their learning pathways

WWW.NORTHBOUND.CO (1)Express Lanes: Accelerated and Dual Degree Programs

Accelerated or dual-degree programs are designed for students who know what they want and how they want it.  Once a student enters the pathway, their experience will be very different from students on the traditional track, but similar to others who share their lane. Yes, best practice dictates that these programs have multiple entry and exit points, but for the most part, they are about fast-tracking.

Acceleration (3-year programs).

Three-year accelerated undergraduate medical education programs were commonplace in the 1970s, but they languished in the US due to legal and logistical challenges.  While accelerated programs continued to grow in Canada (See McMaster University & University of Calgary), the US only recently began to reexamine the idea as a means to support individualized learning,  fast-tracking for much needed primary care physicians, and reduction of student debt.

As of 2018, there are eight U.S.-based undergraduate medical education programs that support three-year accelerated programs.  All are members of the Consortium of Accelerated Medical Pathway Programs (CAMPP).  According to recent CAMPP publications (this and this), lessons learned from the currently participating programs include the following:

  • Multiple entry points. More students have a latent desire to opt-in to accelerated programs, so offering opportunities to opt-in during or after the first year of schooling (rather than offering it only at the beginning of the first year) is helpful.
  • Multiple exit points. Students must have a flexible opt-out option if they change their minds.
  • Mentoring. Early and active mentoring related to professional identity (including opportunities to meet, shadow, work, and research with residents and attending physicians within the same role) is important.
  • Partnerships with residency programs. Students must be assured of opportunity for placement after graduating with an accelerated degree. In most cases, these programs offer contingency acceptance at the school in which they completed undergraduate training.

Who does acceleration well?

The University of Calgary and McMaster University Michael G. DeGroote School of Medicine, both of Canada, have been graduating medical students after three years (131 and 130 weeks, respectively) of study for the last forty years with equal or better student satisfaction and outcomes.  However, since my work focuses on U.S. based medical programs (for better or worse), let’s look at the accelerated program at NYU School of Medicine.  Started in 2013, here is how NYU hits all the best practices listed above – and then goes above and beyond the call of duty.

  • Multiple entry points. At NYU students  offers  entry points: options (reasons) for entering a three-year MD pathway:
    • As part of the medical school application process.
    • Between the first and second years of school (halfway through the first year until halfway through the second year).
    • After completing a PhD (for those interested in research-focused residency programs and careers).
  • Mentoring and identity development. Students have three mentors: a traditional mentor, an accelerated program mentor, and a departmental mentor. The traditional mentor focuses on academic achievement. The accelerated program mentor focuses on professional development and student wellness.  The Departmental advisor helps students identify appropriate departmental activities (such as grand rounds, shadowing opportunities, social events, research experiences) for the student to attend.  Students also spend the summers between year 1 and 2 completing a project in the department of the intended residency.
  • Partnerships with residency programs. While many three-year programs focus on primary care, NYU provides accelerated track students with a conditional match with one of 20 specialties.
  • Multiple exit points. Students may exit the program at any time.  If they decide to pursue a different specialty, they may transfer their conditional placement to a different NYU residency program (pending availability and space) or return to the 4-year curriculum.

Dual degree programs

Dual degree programs used to the be only way to customize undergraduate medical education (beyond the universal ability to select fourth-year clinical electives). Typically, students who pursue dual degrees are those whose interests lie outside basic clinical practice in fields that draw heavily from multiple disciplines (e.g. medical law, public health, or medical education).   Because dual degree programs are so common, I’m not going to spend a lot of time on them here, other than to say that many programs are opening the doors to degree combination that far exceed the traditional MD-JD, MD-MPH, MD-PhD combinations.   However, I have one shout-out…

Who does dual degree programs well?

I chose Stanford University Medical School for the shout-out, not because their dual degree options are too different than other schools of the same caliber.  Rather, it’s the flexibility with which they interpret “dual degree” that caught my eye. According to their website, Standford University’s Discovery Curriculum offers all students the opportunity to take two or three years to complete their pre-clerkship (typically an 18-month to two-year block) coursework.  If students follow a three-year plan, they may use the extra time for independent research, dual degree options (which includes any of the substantial degree offerings Stanford has to offer),  or any other longitudinal scholarship or leadership activities.

31 flavors

31 Flavors: Electives and Scholarship

Medical schools are serving up new and interesting (read “interesting” as interdisciplinary, interprofessional, global, digital, and humanities-focused) opportunities for elective, co-curricular, or self-directed study.  I won’t focus too much on these as the concept of electives are not new; the trend in electives speaks to how the next generation of physicians value diverse types of knowledge and experience  – and this is beyond the scope of the blog post.

But let’s talk about required research – rebranded as scholarship in many schools – as an area for personalized learning.

Who does student-driven scholarship well?

I’ve written previously on Oregon Health & Science University, where students are encouraged to choose to engage in “scholarship” (not necessarily a “research project”) that makes sense.  This means that some students will use this time to take a business class, shadow practice managers, and write a business plan for their private practice while other students might teach or develop health education materials.  If students want to add a little time to their program to engage in clinical or translational research, they will be assigned a special mentor and potentially earn an additional degree.

Success in student-driven projects emerges from a simultaneous increase and decrease in structure over that found in traditional projects.  Structure around instructional design and assessment need to increase; structure around specific details of content (“You must have five sections, starting with a 180-word abstract and typed in 12-point font) must be relaxed. Johns Hopkins does an excellent job of this with their Scholarly Concentrations.  Students are individually mentored in an inquiry project related to one of five areas of study:

  • Basic Science
  • Clinical Research
  • Ethics and the Art of Medicine
  • History of Medicine
  • Public Health and Community Service

Students choose an area of concentration by December of Year One. There are seven core assignments beginning with the project proposal and ending with the final project presentation in March of Year Two. Most students devote the summer after Year One to the scholarly project.

choose your own adventure

Choose Your Own Adventure: Modular Learning

While ‘Express Lane’ approaches put students in several pre-designated tracks and ’31 Flavors’ limits personalization to small chunks of the curriculum, a modular curriculum opens the door for free-range degree programs designed (almost) in real-time by a student and their coach. By modular curriculum, I mean a time-variable, competency-based (or micro-competencies for even more pinpoint personalization) curriculum assessed through badging. We don’t have a medical degree program like this yet.

However, this is where “precision education” comes into the mix.  I’m not going to rewrite the blog post I wrote last week here.  However…look for innovative programs at Stanford, Oregon Health & Science, Johns Hopkins, and National Universities to be writing more about this in the next few years.

Featured Image by Alexander Ramsey on Unsplash

Syndicated, Virginia Group

Reconciling Competency-based and Precision medical education

We have all heard the challenges associated with US medical education.  They include the need for greater flexibility;  the modern student (medical or otherwise) wants to be able to customize their learning experience so that it fits in and around their other life pursuits and obligations. Reduction of cost is another challenge. As costs rise and resources dwindle, the considerable financial burden of schooling is landing on students – and with all the known short- and long-term implications of student debt. Finally, reformers want educators to close the competency gap between undergraduate medical education (UME) and graduate medical education (GME) while also making sure students leave school with the skills they need for the workforce, including but not limited to communication, collaboration, self-regulation, and lifelong learning skills.  Traditionally, medical education does not respond well to external mandates, non-medical expertise, or change. However, for reasons that are not completely known to me, it has taken to competency-based education as a potential solution for its issues related to inclusivity, cost, and efficacy.

According to a 2014 EDUCAUSE article, “competency-based education awards academic credit based on mastery of clearly defined competencies,”  or observable learner outcomes representing the integration of knowledge, skills, and attitudes. (See Competency-based medical education: The basics for additional definitions and background). Competency-based education has its critics, who tend to cite it as encouraging a checklist mentality as detrimental to critical thinking, deeper understanding, and connected learning.  Nevertheless, it has gained traction in the field to the point that the UME programs known for pedagogical innovation consider it a foregone conclusion. While it remains a buzzword to add to the webpage description, competency-based education is no longer something they advertise as new or cutting edge.

In the last year, a new buzzword emerged among perennial innovators such as Stanford, Johns Hopkins, and National Universities. Precision education (styled after the 2015 Obama Administration’s Precision Medicine Initiative) promises customizable curriculum and multiple learning pathways.  It is a semantic variation of personalized learning, which is found on medical school webpages nationwide, including those of Vanderbilt, Michigan State, and Oregon Health and Science Universities.  Precision or personalized learning (I’ll use precision education for the remainder of this post) focuses on customizability and the ways in which students can make a program their own.

Given the recent rise of precision education- in a time where medical schools are just accepting competency-based education –  I want to explore the relationship between the two concepts. Is it an and/or situation?  Do they overlap?  Or are they the same concept but with different names?

In this post, I will show how competency-based and precision education are both facets of the same general educational movement towards student-centered, active, and authentic learning.  When educators think and speak in terms of competency-based education, they are focusing on the educational design.  When they think and speak in terms of precision education, they are focusing on the student experience.  Which the school opts to emphasize (compare Vanderbilt’s Curriculum 2.0 vs. Oregon Health and Science University’s YourMD) have much to do with school culture and branding.

Competency-Based and Precision education.jpg

featured image: Gaelle Marcel on Unsplash

 

Syndicated, Virginia Group

Precision Education

In 2015, the Obama administration launched the Precision Medicine Initiative as a means to support “the emerging approach for disease treatment and prevention that takes into account individual variability in genes, environment, and lifestyle for each person.” Precision medicine signifies a transition away from “one-size-fits-all” medical research and practice and towards that which takes into account individual genetics, environment, and preferences.   While precision medicine and the older, more established “personalized medicine” share the same founding principles, precision medicine is a little more clear: neither precision nor personalized medicine aimed to develop unique treatments for specific individuals.  Rather, they are flexible approaches for the development of treatment plans that take into account genetic, environment, and personal choice factors.

All of this to introduce “Precision Education,” a play on precision medicine that several medical schools – including Johns Hopkins and Stanford – have made in the last several years.  Of note, the precision education world seems to be relatively small:  The former dean of Johns Hopkins School of Education, David Andrews, is the new president of National University.  Other big names in the small field are Sara Hart (Florida State University) and Carole Beal (University of Florida), who work in reading and open data, respectively.

Big Data and Learning Algorithms: National University School of Medicine

At National University, faculty are developing micro-competencies for their basic science courses.  For example, if a course has eight key concepts, each concept is divided into 5-6 micro-competencies. The instructor develops quizzes for each micro-competency.  When students show a weakness in a micro-competency, they are prompted to remediate.

Faculty are encouraged to find open educational resources (OER) and multimedia content – multiple material types for each concept.  As the school collects student learning data, they will work to identify which student profiles match which material type.  Once these patterns are established, they will be able to make recommendations to students on how to best remediate.

More info: Precision Education Hopes to Apply Big Data

Connected and Crowdsourced: Johns Hopkins School of Medicine

Johns Hopkins appears to be taking the Connected Learning route for Precision Education. It is focused on enhancing student agency and success through a focus on personal interests, peer support, and personalized feedback.

As part of the new Genes to Society curriculum, students are taught a new model of health: one that treats health and disease in the context of all factors (genetics – environment – socio-cultural – personal) simultaneously and at every student level.  Digital materials are written by students and instructors to help new students make the connections across basic science, practice, and population medicine. These resources live in an accessible digital space, where students can access and make contributions across space and time.

Additionally, digital networks of crowdsourced articles, videos, discussion, and notes prepared by peers allow students to explore facets of subjects they find personally compelling before, during, and after whole-group learning. but that this approach allows courses to grow into dynamic collections of materials shared by faculty as well as students.

More Info: Precision Education in the School of Medicine

Students were given the option of face-to-face or digital learning activities to supplement their clinical experiences in the required neurology clerkship.  Student learning preferences (abstract v. concrete, teacher- v. student-structured; and type of learning environment) significantly impacted their personal satisfaction with the digital vs. face-to-face formats.  Assessment outcomes were the same across groups.

More Info: Precision Education in Neurology

In the Surgery Department, resident physicians received EMR-generated “electronic scorecards” for how consistently they ordered the appropriate DVT prophylaxis for their patients.  Residents received their scorecards monthly and low performers were provided with individualized remediation.

Optimizing Self-Directed Learning: Standford University School of Medicine

Stanford University is also trying to improve student options for outside-of-class learning by enhancing student access to quality multimedia learning tools.  This includes app-driven case-based learning (with built-in knowledge checks) and a partnership with Osmosis online medical education videos.  The university is attempting to generate a recommendation engine (like Netflix) to help students make good decisions about self-directed study.

More Info: Precision Education

Additionally, Standford University’s Discovery Curriculum offers 2- and 3-year pre-clerkship options.  The three-year option provides time for independent research, dual degree options, or any other longitudinal scholarship or leadership activities.

More Info: Discovery Curriculum

 

Featured Image by rawpixel on Unsplash

Syndicated, Virginia Group

Summer’s Never Off

If one more person says to me, “So, enjoying the summer off?,” I’m gonna scream. True, the number of classes in session is fewer, the number of students on campus has dropped, and the number of meetings has dwindled. Yet June and July are busy months for those of us who oversee general education programs at public universities.

Our office, University Programs, manages the enrollment of all first year students–at present, we expect more than 4,600 freshmen and about 650 transfers (we use the term Firstyears for both populations). That many bodies requires a lot of seats! In the past, we assumed an average of 4.2 general education classes per freshman to make a full-time schedule of 12-15 credits. Today, I use 4 courses per freshman. In round numbers, that’s about 18,500 seats distributed across hundreds of sections of the general education program’s 80-some courses. And that’s not including the seats needed by our continuing students, the sophomores, juniors, and seniors. My office works closely with departments and the Registrar to hold back seats for the freshmen to ensure they can make appropriate schedules when their enrollment period begins.

It’s my job to keep track of how many seats remain available in general education courses, and to do it I monitor JMU’s Peoplesoft system regularly. Incoming freshmen register online using a series of web-based how-to videos, worksheets developed by University Programs, and advisor input. No other students have access at this time. During the 13 days of Summer Springboard, they come to campus as part of JMU’s orientation program, meet their advisors face to face, and make any adjustments to their schedule. The wrinkle is that every year students are bringing more transfer credit from high school dual enrollment courses and AP exams—and there’s no perfect way to know which program requirements they have met with these credits until their transcripts get here. Someday soon I’ll write a post about the perils of transfer credit, but suffice it to say that students are often surprised to learn that credits don’t count the way they think they should.

Summer isn’t just about seats, though. Orientation affords an opportunity to talk directly to students and their parents about the value of a liberal arts and sciences education. JMU’s outcomes-based liberal education program, General Education: The Human Community is 20 years old now; we were one of the first institutions in the nation to abandon the old distributive approach in favor of a coherent curriculum. We are a national leader in assessment practices, and we have won recognition from the AAC&U, CHEA and AGLS. The program has dedicated faculty, administrative support, and an ample budget. And yet even we feel the need to advocate constantly for  .

When I address the class of 2022 and their families during Springboard, I know that, notwithstanding their excitement, many in the audience are worrying about rising tuition and student debt loads. Though the economy is improving, it’s still tough to get a job in Virginia, even with a degree. The anxiety is palpable, and for me it explains not only the dramatic shifts in the majors students now declare, but their recent embrace of dual enrollment. And while I understand these choices, I worry that they are misinformed.

My approach is to herald the good news proclaimed by numerous recent reports–that every part of a bachelor’s degree has value, and further, that a coherent general education program provides the best long term preparation for success. The statistics help: from 2007-2012, 82% of all new jobs created in the USA required a bachelor’s degree; by age 39, the average American with a bachelor’s degree will hold 10-14 different jobs across multiple fields; within five years of graduation, only 50% of degree-holders will still be employed in the field in which they majored. The most powerful bit comes with an activity. First, I ask college-educated parents who still work in their baccalaureate majors to raise their hands, then I ask college-educated parents who have changed careers to raise their hands. The second group is always much larger, and it’s a potent illustration of what the numbers mean for actual people.

Students on the Quad during Orientation

Summer may seem like an unlikely time to talk about post-graduation realities. For freshmen, it’s a good time because it marks a milestone in their transition to independent adulthood. Having just graduated from high school, they look toward the fall with anticipation yet they know that college lasts a mere four years. They expect to graduate and they choose majors, most of them, anyway, with the near future in mind. So they are already thinking about the journey ahead, they just don’t have a detailed map.

My hope is that they will view the general education program as essential to their preparation for life. Instead of a random collection of courses grouped by disciplinary categories, we offer a limited array of choices in areas defined by shared learning outcomes. That is by design. Just as faculty here created curricula for majors based on each field’s specific standards, so they created a core curriculum of general education courses to provide what every student needs to know, do, and understand.

Summer isn’t the only time to make the case for liberal education. We message in a variety of other ways, at myriad points in the academic year, and many program faculty add their voices to the chorus. And there is some evidence that it’s working, at least for some students. A recent set of focus groups with undergraduates, for example, reveals their awareness of the way the program complements their majors. The irony is that, by the time they get it, they are ready to leave.

And so it goes. With each summer comes a new class of freshmen. There’s never a summer off.

Syndicated, Virginia Group

The CBME Resource Database

I’ve created a database for all of the resources I collected for my self-education on competency-based medical education.  Y means yes, N means no, NA means not applicable.

Access the excel spreadsheet here.  I

Some explanations of column names (except where self-explanatory):

  1. Year of Publication 
  2. First Author
  3. Title
  4. Full Citation
  5. DOI or URL
  6. Country of Origin – of the author(s).  ‘International’ was used to indicate authors from different countries.
  7. Learning Level – UME (undergradate medical education), GME (graduate medical education), PG (postgraduate or continuing medical education), ALL (combinations of learning levels or applicable to all).
  8. Assessment Frameworks-
    1. Competencies & Milestones – Relates to ACGME or CanMeds competency frameworks?
    2. EPAs – Relates to Entrustable Professional Activities?
    3. RIME – Relates to RIME?
    4. Combination – Demonstrates alignment of multiple frameworks?
  9. Resource Type 
    1. General Knowledge (GK) – Provides background or foundational knowledge?
      1. GK category – If a GK article, provides general knowledge on what?
    2. Accrediting Body Initiative – Offers recommentations or updates from a big initiative you should know about?
      1. Initiative Name – If related to an initiative or workgroup, which one?
    3. Practice Example – Description of implementation or experience at a specific site
      1. Practice Site – If so, where?
  10. Notable Elements – Items of interest that shows up in the article including
    1. Change management – this includes lessons learned, challenges, etc.
    2. Outcomes and objectives development
    3. Curriculum development
    4. Assessment development
    5. Faculty development
    6. Personal learning plans or pathways
    7. Educational technology

Featured Image by Delano Balten on Unsplash

Syndicated, Virginia Group

Breaking Down Medical Education 2.0_A First Draft

In their article introduction, Spickard, Ahmed, Lomis, Johnson, & Miller (2016) identify the defining characteristics of Medical Education 2.0 as the presence of (1) early clinical exposure, (2) competency-based assessment, and (3) personalized pathways to degree completion.  Normally I might ignore the list, since Spickard et al. (2016) is an IT- rather than an ID-centric article, and in my experience, it makes a difference in how meaningful introductions tend to be.  However, authors Lomis and Miller are architects of the recent curriculum reboot at Vanderbilt University School of Medicine, so I stopped to give it some consideration.

This blog post was born out of the consideration.  The goal was to see if I could fit the design themes I have found in reviewing a dozen recent medical education curricular reboots into the three qualities listed in Spickard et al. (2016). My desire to do so is born entirely out of the fact that I need to create a workshop around these things very soon and a loose, three-spoked organizational structure would suit my needs perfectly.

Disclaimer: There is nothing scientific going on with my sampling.  I started with the AAMC Core EPA pilot schools and snowballed a little bit from there, picking and choosing what looked interesting and good and also well-documented enough for me to write something of substance. If a school didn’t have enough out on the internet (school website, websites or blogs of professional organizations, or in the research literature) for me to understand what was going on, they were also out of luck.

Early Clinical Exposure

I rolled my eyes when I first saw this on the list.  “Early clinical exposure” has been around for a long time – long before Medical Education 2.0. In 1998, my alma mater was touting “early clinical exposure” when they placed us in primary care offices two months into the first year.  Based on my review, preceptor experiences such as mine are still popular and are considered evidence of “early clinical exposure,” but I think we can expand the concept beyond just that of workplace learning. Clinical exposure can stretch to mean applying basic science to clinical problems or thinking like a clinician (e.g. differential diagnoses, clinical workups) as well as workplace learning.

Integrated curriculum

Most medical education reboots describe their curriculum as “integrated,” defined by the intentionally combined presence of :

  • Basic science knowledge and its clinical applications. Applied learning (case- or problem-based) usually takes place in small group settings. Vanderbilt describes this well; see my earlier post for references.
  • Longitudinal threads and block curriculum. Concepts like ethics, bioinformatics, and professionalism are woven through all didactic curriculum. The best infographic illustrating this comes from Oregon Health and Science University.
  • Workplace and didactic learning. While all programs describe this, the Shared Discovery Curriculum at Michigan State University College of Human Medicine offers a unique slant on their mash-up:  they intentionally structure workplace and didactic learning throughout all four years of medical school (as compared to the usual two) with a goal of making “students feel useful to the healthcare team.” In the first phase, students work predominantly with medical assistants and nursing coordinators (not necessarily physicians) in primary care settings to room patients, take vital signs, document chief complaints, etc.  This not only exposes students to other types of healthcare professionals but offers the opportunities to build foundational clinical skills quickly and with the healthcare professionals who typically perform them. In the second phase, students work with physicians on in inpatient settings to develop differential diagnoses and patient care skills.  In the third phase, they engage in core clinical clerkships which are shorter and more learning-dense than traditional clerkships, because students have already established basic competencies.  They also continue with their didactic learning blocks to support the more complex clinical exposure in the third phase.

Learning communities

In Medical Education 2.0, Etienne Wenger’s learning communities are often elevated to learning “societies” or “academies” or “colleges,” but the concept remains the same: learners from diverse academic backgrounds engage in informal, experiential, authentic, social learning experiences around a shared purpose.   In most cases, these learning communities serve at least one of the following purposes:

  • Academic and career advising.  This will be discussed below in ‘personalized learning’
  • Enrichment programming related to clinical and professional identity and culture. I like the OHSU Colleges at Oregon Health and Science University School of Medicine and have described them in-depth in a previous post.
  • Small group learning, reflections, and debriefings. While some schools (Vanderbilt) create multiple sets of small groups (to enhance diversity, facilitate multi-team skill development), Michigan State University School of Human Medicine keeps all small group activities within their learning societies to enhance trust development.  It’s very Hogwarts. Very very Hogwarts.

An interesting twist: 

Workplace learning

Consider this a placeholder.  We all know what workplace learning is in the context of medical education.  It differs from the conversation found in the integrated curriculum section in that it could be used to describe the different ways in which early (i.e. pre-clerkship) workplace learning can look. Preceptorships (individual mentoring-shadowing relationships between practicing clinicians and students) are a well-established approach (as evidenced by my own preceptorship, circa 1998).  However, Medical Education 2.0 programs are recognizing that team approaches, pedagogically structured experiences, or opportunities to debrief with or without the creation of learning artifacts are also beneficial.

  • Team approaches. From my own experience working as a senior learning architect with Mayo Clinic School of Medicine, I know that sending students into the workplace in teams of three to five students can be useful in terms of protecting resources but also as a means for the students to coach and learn from each other. See also Michigan State University and its description of early clinical experiences.
  • Pedagogical structure. The Transition to Clinical Experience at Oregon Health and Science University School of Medicine – and its implications for all clinical experiences thereafter – is my favorite example of pedagogical structure for workplace learning.  As I’ve written before (see this post if you want to read about it in the broader curricular context), the Transition is a two-week simulation bootcamp that aims to get students ready for their clinical clerkships.  The experience acts as a gateway experience; students cannot progress without passing it.  It not only teaches students the practicalities (example: gowning and gloving) but introduces them to the EPAs in which they will need to demonstrate competency before graduation. In that respect, it acts as a pre-assessment.  The researcher in me loves this too.
  • Small group debriefings. Most programs bookend early workplace learning with small group debriefing sessions.  These experiences provide opportunities for peer-to-peer learning, challenging elements of the hidden curriculum, reinforcing learning through simulation or review, reflecting and creation of learning artifacts.

Competency-Based Assessment

My new favorite book, The Practical Guide to the Evaluation of Clinical Competence, does an incredible job of outlining the dimensions of competency-based assessment: (1) Competencies (e.g. core competencies and EPAs); (2) Levels of Assessment (e.g. Miller’s Pyramid); (3) Assessment of Progression (e.g. Dreyfus-driven milestones). I say that here, I think, mainly to promote this book.

Competency-based assessment has emerged from a long tradition of workplace-based assessment, an approach that this earlier post only begins to approach.  Workplace-based assessment is characterized by multi-faceted formative and summative assessment, and while direct observation forms the basis of most of this type of assessment, different settings (e.g. simulation centers, standardized patients, authentic patient care) and different assessors (multisource feedback, peer review, expert assessors) to consider. In the case of undergraduate medical education, where there is a very real need for students to demonstrate medical knowledge science and pass standardized national licensing examinations, written assessments (in the form of quizzes and tests) are part of competency-based assessment too.

There are many ways to organize a conversation on competency-based assessment. However, I think one of the easier ways is to discuss it in terms of formative assessment, summative assessment, and platforms for documentation and tracking.

Formative Assessment

Formative assessments are the learning artifacts that accumulate over time to demonstrate a student’s progression along the competency trajectory (milestones). The quality and success of formative assessment will depend on the program’s ability to do the following:

  • Design learning opportunities that facilitate student expression of the desired behaviors.  I mean curriculum and instructional alignment.  While I started a conversation about curriculum mapping and milestone development in this post, I’m not prepared to go down that rabbit hole today.  My point in bringing it up here is to reinforce the thought that if a school chooses to engage in CBME and competency-based assessment, they may need to redesign their curriculum to align with new desired outcomes. Curriculum 2.0 at Vanderbilt University School of Medicine offers a perfect, well-documented example (See this post for resources).
  • Create meaningful – valid, reliable, standardized – competencies and milestones by which to judge student performance. Again, Curriculum 2.0 at Vanderbilt University School of Medicine is your well-documented friend (See this post for resources).
  • Support student interpretation of and decision-making around their progress. This will be discussed below in personalized learning.

Some program-level core competency lists if you’d like to compare:

Summative Assessment

In the best cases, formative assessment is the building block of summative assessment.  The Michigan State University School of Human Medicine website offers a particularly complete description of its progress testing: a systematic approach to multi-faceted summative assessment.  Since I have not written a post on this (yet), I’m going to outline the contents of MSU’s Progress Suite of Assessments here.  Twice a semester for the duration of the program,  the Student Competence Committee reviews student performance on the following assessments:

  • Progress Clinical Skills Examination. A standardized patient/simulation experience; students move through eight stations, spending 20 minutes on the clinical scenario and another ten minutes answering questions about the clinical and basic science relevant to the scenario. Frequency: four times a year.
  • Comprehensive Necessary Science Examination. NBME exam (multiple choice medical knowledge exam designed to simulate USMLE Steps).  Frequency: four times a year.
  • Multisource Feedback.  360-degree assessments from preceptors (workplace and classroom), peers, nurses, patients, other healthcare team members. Frequency: twice a year.
  • Portfolio. Weekly aggregation of learning artifacts, tagged with SCRIPT competency goals
    • Direct Observations – Assessment checklists and rubrics are found in the JustInTime learning management system.
    • Clinical Documentation – Student artifacts such as an H&P, assessed with a JustInTime rubric
    • Procedural logbooks
    • Skills Certification – BLS and CITI training are required
    • External Examinations – USMLE Step 1&2 scores
    • Formative Assessments – Competency-driven assessments from small group, laboratory, simulation, and other learning activities.
    • Additional artifacts – optional certificates (e.g. Rural Health Certificate),  reflection journals (e.g. from a service learning project) – anything chosen by the student to demonstrate competence.
  • Self-Assessment. Students populate a competency-driven rubric referencing aspects of the portfolio as evidence. Frequency: Twice a year

An interesting twist:

  • As described in my last post, Oregon Health and Science University School of Medicine takes summative workplace-based assessment to a new level with badging.  The details are a bit sparse (probably because it is still in the planning stages), but it appears that OHSU students compile a suite of assessments for each of the 13 core EPAs. When a student is ready to be assessed for an EPA badge, the Entrustment Group (structurally similar to Michigan State’s Student Competence Committee) reviews the evidence and makes a decision to award (or not) the badge.  When a student achieves all the badges, they may graduate, regardless of the time in school.

Digital Assessment Platforms

CBME assessment required real-time longitudinal data collection on student performance and improvement. In all cases, schools are hoping to make the data collection, analysis, and visualization as seamless (and maintenance-free) as possible.  Here are two examples.

The Michigan State University School of Human Medicine Just in Time platform is a mobile, cloud-based application that functions on smartphones, tablets, and laptops. It  acts as a (1) content management system (for holding curricular content and allowing for curriculum reporting; (2) logging system (for entering, viewing, and creating reports around student-patient encounters); and (3) student assessment system (for entering, viewing and creating reports around faculty assessment of student performance).  A demo simulation can be accessed by following the link above. Students must still run the reports to include in their portfolio for assessment.

Vanderbilt University School of Medicine published the story of their portfolio platform development.  They wanted a platform that would allow them to integrate their learning management system, student clinical skills performance, assessments and artifacts, and course evaluations.  Ultimately, they modified Moodle to support the self-directed online modules, social media networks and group work, and electronic portfolio.  This platform interfaces with the electronic health record, where clinical skills performance artifacts reside.  Students are able to track their competency milestones in real time through dynamic dashboards.

Personalized Learning

When done well, multiple pathways to success and diversity in degree offerings support the development of skills and habits of mind required for lifelong learning and self-regulation.  Personalized learning can also promote inclusivity and diversity and (in some cases) reduce student debt. It is both a component and outcome of competency-based medical education in ways that I will probably work out in more depth in the next few blog posts. However, for now, I will address personalized learning in terms of personal learning plans and pathways.

Personalized learning plans

Medical Education 2.0 reboots almost always mention a personalized learning plan.  These flexible and formative plans are developed by the student and a coach as a reflection of goals and performance-to-date.  Coaches go by different names – ‘Learning Society Fellow’ at Michigan State; ‘Portfolio Coach’ at OSHU and Vanderbilt – but they are trained faculty housed within the learning communities (or equivalent) who work with students for the duration of the program.   Students and their coaches meet on a schedule to discuss student goals, interests, and performance on formative and summative assessments before formulating an action plan to be carried out by the student before the next meeting.

Personal learning pathways

If personalized learning plans offer students the opportunity to practice structured and supervised self-assessment and -reflection, personal learning pathways offer students the opportunity to customize their education. To a certain extent, personal learning pathways have always existed in medical education through dual degree programs.  However, Medical Education 2.0 has increased offerings far beyond the traditional MD combinations (M.P.H., Ph.D., M.B.A). I plan to document this in greater detail in my next post, but for now here are some examples. Oregon Health Science University School of Medicine is notable for their multi

  • Accelerated programs. Many programs now have a 3-year accelerated pathway but New York University School of Medicine offers an excellent description of their pathway if you follow the link. Students enrolled in the accelerated program at NYU are offered guaranteed spots in one of the school’s residency programs.
  • Additional concentrations, certificates, or dual degrees. Some programs offer students additional certifications – typically in areas or fields connected to the university’s strategic initiatives or mission.
  • Time-variable degrees. In the spirit of true competency-based education, OHSU is moving towards a time-variable degree program in which students will be able to finish their degree in as few as three years if they are able to achieve competency in the 13 core EPAs in that time period.  Alternatively, students may take longer to complete their degree if necessary or desired.
  • Clinical experience choice. Some programs – including Vanderbilt, Michigan State, and OHSU – allow students to mix their required and elective clerkships in ways that make sense for their personal interests and goals.
  • Multi-tiered student choice. The Your MD program at OHSU is notable for the number of ways personalized learning is built into their program design.  In many ways, they’ve developed personalized learning to support CBME (rather than the other way around). I’ve written about much of this before and will do so again, so I’m going to leave things as a bullet list just to give you an idea. Students customize their education by choosing:
    • the OHSU College that caters to their clinical and professional interests.
    • the content and order of their clinical experiences
    • the ‘pathway’ and content of their scholarly project (see my previous post)
    • additional certifications or degrees (more information to come)
    • the length of the program (this is still in the planning stages and is dependent on their capacity to achieve competencies in a shorter timeframe).

More to come.

Featured Image by Ian Schneider on Unsplash

Syndicated, Virginia Group

CBME at Oregon Health & Science University

Competency-Based Medical Education (CBME) focuses design (and the designer) on learner objectives and outcomes.  Milestones, which are created through the study of typical development, help track a learner’s progress from novice to expert across any number of tasks. Information about a learner’s progress is collected from diverse sources in a variety of ways and kept in a developmental portfolio.  The portfolio is reviewed by the learner and their academic coach on a regular basis for the purpose of developing action plans. 

Hallmarks of Competency-Based Education

  • Competencies with milestones (+/- EPAs)
  • Curriculum and instructional approaches that allow for the assessment of competencies.  Typically involves combinations of active and social learning: case-based, problem-based, team-based, and workplace-based.
  • Formative and authentic assessment
  • Longitudinal portfolios to track progress

Implications of Competency-Based Assessment

  • Portfolios are typically digital platforms which interface with other university IT systems [e.g. learning management system, registrar, electronic medical record].  For more information about how Vanderbilt integrated their portfolio platform see Spickard et al. (2016). 
  • Best practices in portfolio assessment demand the inclusion of academic coaching. A number of studies indicate that students – especially medical students – require training in self-assessment, reflection, and regulation.  Portfolios do not work without guidance.
  • Portfolio assessment best supports self-regulation and personalized learning when there are multiple learning pathways available to students. 
  • While milestones suggest the ‘typical’ developmental trajectory of learner progress, not all students will progress at the ‘typical’ speed.  Performance is prioritized over hours/time spent on learning a task.  A pure CBME program will allow for time- variable progression. 

This is my third CBME-related post. The first post reviews the basic vocabulary and North American players.  It also reviews Vanderbilt University School of Medicine as the best-documented (as in “most published”) example.  The second post delves into the relationships between competencies and EPAs (entrustable professional activities) and the role of workplace-based assessment in CBME. This post will focus on the personalized learning facet of CBME by describing one university’s approach to portfolios, personal learning pathways, and time-variable progression.

As I’ve said before, this is a blog post, not a peer-reviewed article. It consists entirely of information gleaned from the internet and my early attempts to make sense of it all.  If I got something wrong about OHSU’s programming, first – my apologies and second – please contact me on Twitter. If you have more information, I would love to talk! 

Oregon Health & Science University School of Medicine

OHSU School of Medicine is a Core EPA Pilot school. Unlike its pilot partner, Vanderbilt University, OHSU had not started its CBME transformation before the pilot (see this post for a description of Vanderbilt’s UME reforms and the Core EPA Pilot Program).  

Why did they do it?

Beyond the general context for reform described in my first post, it appears that the educational reform at OHSU may have been driven by the results of a 2012 LCME site visit.

Who did the work?

Best resource for learning more

Mejicano, G. C., & Bumsted, T. N. (2018). Describing the journey and lessons learned implementing a competency-based, time-variable undergraduate medical education curriculum. Academic Medicine, 93(3), S42-S48.

Curriculum Transformation Steering Committee (CTSU)

At the direction of the Dean of the School of Medicine, the CTSU spent six months researching medical education before settling on these desired qualities:

  • Learner-centered education
  • Emphasis on critical thinking, inquiry, and lifelong learning 
  • Personalized learning
  • Integrated curriculum
  • Active learning
  • Competency-based education
  • 21st-century skills

The committee received a $1 million grant from the AAMC to continue their work and the school was later selected to be a part of the Core EPA project. 

The Foundational Phase Development Team 

At the direction of the Associate Dean for UME, a multidisciplinary team (scientists – clinicians – educational specialists) met weekly for 10 months to accomplish the following tasks:

  • Developed a new integrated curriculum (seven integrated course blocks with longitudinal threads).
  • Mapped new course goals to the ACGME competencies
  • Wrote session-level objectives
  • Described a general framework for instruction and student assessment that were consistent with the qualities laid out by the CTSU.

The Foundations Operation Team

The operation team consisted of the course directors who would select faculty to lead individual sessions and develop assessments consistent with the framework laid out by the development team.

What did they do?

Best resource for learning more

OHSU School of Medicine MD Program. (2018). Your MD Student Information. Retrieved from: https://www.ohsu.edu/xd/education/schools/school-of-medicine/academic-programs/md-program/curriculum/your-md-details.cfm

The Operation Team launched the new curriculum (“Your MD”) in 2014. Details are best described in the program website, but briefly:

Pre-Matriculation Self-Assessment.  Students do this before they start the program.  I would like to know more but cannot find it online.

Foundations of Medicine.  For the first 18 months, students engage in seven integrated science blocks, one longitudinal preceptor experience, College activities, and assessments. 

  • Didactic Blocks. Each of the seven didactic blocks integrate basic and medical science and addresses relevant combinations of the following conceptual themes: anatomy, biochemistry, cell biology, embryology, genetics, histology, microbiology, pharmacology, professionalism, ethics, communication, epidemiology, biostatistics, evidence-based medicine, informatics, health policy, patient interviewing, and physical exam. The curriculum is case-based.  Typical instructional approaches include large group, small group, laboratory groups, clinical skills, and independent learning.  
  • Preceptorship and Narrative Medicine. A longitudinal clinical experience that takes place in the foundational medicine phase of the program.  Students work directly with practicing physicians in a clinical setting and focus on all ACGME core competencies. 
  • Colleges Activities. A robust, personalized, longitudinal approach to student affairs curricula that continues throughout the program (Foundations & Clinical). 
    • Learning Communities.  Students choose one of seven learning communities (e.g. Acute Care Medicine, Global Health and Urban Underserved, Surgical Specialities). All learning communities engage students in academic opportunities, career advising, financial planning, interprofessional education, leadership, service learning, and wellness but will align the content of such activities to meet the interests of the group.  Students may attend any of the activities across colleges, but they must select one “home base” from which to attend regularly.  Approaches to curricula & instruction generally include expert panels, field trips, and workshops.
    • Portfolio Coaching. The colleges provide individual and group coaching around professionalism, professional identity, career counseling, academic advising, reflection and self-assessment.
  • Assessment.  Takes place every Friday and includes a combination of ExamSoft quizzes, clinical skills quizzes, and science skills quizzes.

Transition to Clinical Experiences (2 weeks). A simulation ‘bootcamp’ for students entering the clinical years, the experience aligns with the core EPAS, which enables pre-post assessment and sets student expectations around what they are meant to accomplish.  This is a gateway course – students cannot proceed without passing this experience.  

Clinical Experiences.  After completion of the Foundations of Medicine, students engage in clinical experiences earlier than the traditional timeline (e.g. at 18 months instead of  24 months). In some cases, this will mean (in future cohorts) that students can finish their undergraduate medical training in as little as three years instead of the traditional four years.

  • Traditional core clinical experiences.  Traditional core experiences (e.g. pediatrics, surgery, etc) are required
  • More electives, earlier. Elective options are available from the beginning of the clinical experience (as compared with the traditional model which limits electives to the fourth year of training). 
  • Intersessions. Two-week sessions between clinical experiences that integrate foundational science into the clinical experiences.  Also address issues such as health disparities and communication. 
  • Scholarship. Students choose from two different pathways for this dedicated time. 
    • Scholarly Project.  A mentored inquiry-based research project designed by the student.  Past projects have ranged from performing traditional clinical or basic science research to developing a business plan for a medically-related startup or proposing public health-related legislation. 
    • Physician-Scientist Experience.  Students interested in pursuing clinical translational research or research medicine can use this time to be mentored and gain research experience.  Those who extend this experience to a year can earn a Masters Degree in Clinical Research.

How will they…implement competency-based assessment? 

Best resource for learning more

Mejicano, G. C., & Bumsted, T. N. (2018). Describing the journey and lessons learned implementing a competency-based, time-variable undergraduate medical education curriculum. Academic Medicine, 93(3), S42-S48.

Portfolios.  “The Research in Evaluation Data for Educational Improvement” digital portfolio provides real-time visualization of a learner’s academic progress and all the data points associated with that progress.  Eventually, this will consist of documentation of achieved competencies and EPAs.  

Badging. Portfolios will also contain digital badges for achieved Core EPAs.  Students must earn all 14 core EPA badges via summative assessments made by the Entrustment Group [Note: I think this process is still very much in research & development; I found this pdf – updated 2-26-2017 online but cannot verify if this is still the current plan…I will keep looking].  Summative assessment appears to rely on a variety of data points.  Mejicano & Bumstead (2018) provide this list but with minimal explanation:

  • Qualified assessor evaluation
  • Peer and/or patient input
  • Instruments provided by the AAMC
  • Simulation experiences
  • Reflection exercises
  • Gateway assessments
  • Procedure and required encounters
  • OHSU’s four components from clinical experiences

Individualized Learning Plan. Portfolio coaches (assigned to students through the College for the duration of the program) and students create an individualized learning plan.  While the time spent in Foundations will be fixed at 18 months, the clinical experiences portion of the curriculum provides significant temporal flexibility.

Featured Image by Matheus Ferrero on Unsplash

 

Syndicated, Virginia Group

Workplace-Based Assessment in Competency-Based Medical Education

Competency-based education expresses competency as a progression through milestones: criterion-referenced, performance-based trajectory from novice to expert for any number of behaviors.  The most commonly used development framework used for expressing milestones in clinical contexts is Miller’s Pyramid of Clinical Skills, Competency, and Performance:  Know, Know How, Show How, Do.

As discussed in my last blog post, competency-based education prioritizes the objectives and outcomes (and outcome-based objectives) aspects of instructional design.  Technically, it does not dictate specific curricula or approaches to instruction.  However, the very nature of outcome-based objectives implies best-fits with certain learning and assessment philosophies, approaches, and activities.

As Vanderbilt University School of Medicine discovered (Miller et al., 2010), if a medical school wants to anchor their curriculum to the ACGME core competencies, they need to provide students with opportunities to practice and demonstrate behaviors that align with those competencies.  While input activities such as listening (e.g. lectures), reading, or watching are appropriate for the introduction of new information, social learning settings for case- or problem-based learning are best-suited for factual integration and application.

While medical knowledge can be assessed through a series of frequent low-stakes assessments and less frequent high-stakes exams (which mimic standardized national licensing exams), multi-faceted qualitative rubric assessments work best for assessing other competencies.  Vanderbilt has published at length on these (Pettepher et al. 2016) and how those data points contribute to a developmental portfolio in which students and their coaches can follow the student’s progress from novice to expert (Lomis et al.,2017).

CBME chart
Summary Table of Competency-Based Medical Education Instructional Design (DRAFT)

There is more to the story – much more.

This blog post will focus on the assessment focus and tools that facilitate competency-based education at undergraduate, graduate, and postgraduate levels.   Not much time will be spent on knowledge assessments (e.g. quizzes and tests; because they are familiar) or qualitative rubric assessments (because I focused there my last post).  Instead, I will focus how we assess competence in the workplace – at the “show how” and “do” levels of Miller’s pyramid.

“Show How” marks the transition from classroom to workplace-based learning. In the Flexner years, undergraduate medical students spent the first two years in the classroom (“preclinical years”) before engaging in hospital- and outpatient-based clinical clerkships for their third and fourth years (“clinical years”) of schooling.

In the Beyond-Flexner era of medical education, the “preclinical” label is no longer accurate, because students are moving into clinical settings and demonstrating basic clinical skills as early as the first month of medical school. Additionally, current practice conditions require clinicians to learn new procedures and demonstrate their competence through formalized and documented processes. Therefore the “Show How” levels of clinical competence are no longer the sole purview of graduate medical education. Learners at all levels will encounter “Show How” assessments.

Competencies versus Entrustable Professional Activities

One of the most consistently-voiced critiques of the ACGME core competencies relates to the nature of the competencies themselves. While they are meaningful at the individual skill level, successful clinical practice is more than the sum of its parts; practitioners must be able to integrate the individual skills and exercise context-based judgment on how to use them. Critics argue that “checking off competencies” will never achieve appropriate levels of authentic assessment.  Thus, Entrustable Professional Activities (EPAs) were born.

Best source for learning more: 

Englander, R. & Carraccio, C. (2014). EPAs, Competencies and Milestones: Putting it all Together. 2014 Fall APPD Meeting. Retrieved from:  https://pdfs.semanticscholar.org/presentation/a3fe/6090d11619a87585aa94bba9660ce24ce31f.pdf

As described by ten Cate, Snell, and Carraccio (2010), EPAs are the interplay of core competencies and authentic practice.  They are founded in ‘entrustment’ which has been defined by Kennedy et al. (2008) by these criteria:

  • Ability or level of Knowledge, Skills, &/or Attitudes
  • Conscientiousness
  • Truthfulness
  • Discernment (Knowing one’s limits).

As Englander & Carraccio (2014) explain, entrustment implies competence through a lens of supervision – a lens which is consistent with the culture of health care and intuitive for its practitioners.  The following table, which is a mashup of Englander & Carraccio (2014) and ten Cate and Scheele (2007), differentiates between competencies and EPAs.

COMPETENCY EPA
Describes the ability of the practitioner  Describes the outcome of the work
Context-independent Embedded in a clinical setting and situation 
Addresses the knowledge/skills/attitudes of a specific task Integrates multiple tasks required to provide successful patient care

Example of a Competency: Gathers and synthesizes essential and accurate information to define the clinical problem.

Example of an EPA:  Care of a complicated pregnancy.

Workplace-Based Assessment

“Can you be entrusted with care?” translates to “Do you need to be supervised to make sure things are done correctly?” The binary response (yes or no) is particularly useful in assessment.

Best source for learning more: 

Ten Cate, O. (2013). Nuts and bolts of entrustable professional activities. Journal of Graduate Medical Education, 5(1), 157-158.

In the ten years that Olle ten Cate has been writing on EPAs, he has developed levels of supervision that allow for a progression from novice to expert.  Although they have evolved over time, this list was published in the Journal of Graduate Medical Education as part of a practice bulletin (of sorts) in 2013:

  • Level 1.  Observation of task with no involvement in execution.
  • Level 2.  Execution of task with proactive, direct supervision.
  • Level 3.  Execution of task with supervision easily accessed on request.
  • Level 4.  Execution of the task with off-site or post-hoc supervision.
  • Level 5.  Providing supervision for more junior colleagues.

Achievement of an EPA takes place when the learner moves from Level 3 to 4.

ten Cate (2013) explains that the ultimate aim of EPAs is not to result in independent practice in the literal sense since healthcare is an interdependent act.  [Author’s Note: Writing this has made me understand that I need to edit the Draft Summary Table for ‘Do’ reads as “Qualified to Teach” rather than “Unsupervised Practice.”]

Assessment Tools for Workplace-Based Assessment

The ability to “Show How” requires assessments that allow for demonstration. Norcini & Burch (2007) outline a handful of approaches to workplace-based assessment.

Best source for learning more: 

Norcini, J. & Burch, V. (2007) Workplace-based assessment as an educational tool: AMEE Guide No. 31. Medical Teacher. 29(9-10), 855-871. DOI: 10.1080/01421590701775453

  • Mini-CEX (Mini Clinical Examination).  The mini-CEX was first described by the American Board of Internal Medicine (Norcini et al., 1995) as a formative assessment process for faculty evaluation of physicians-in-training as they interact with patients. Subsequent research suggests mini-CEX assessments are similar to similar simulation-based assessments but with higher fidelity and potentially lower cost (Norcini et al., 2003).  The process involves the following sequence:
    • Direct Observation
    • Learner Self-Evaluation
    • Structured, faculty-driven written and verbal feedback
    • Development of an action plan for improvement
    • Follow-up with multiple mini-CEXs through the year [with different faculty assessors].
  • Clinical Encounter Cards (CEC).  The CEC system was developed at McMaster University (Hatala & Norman, 1999) before its implementation and study at other centers.  It is similar to the mini-CEX; faculty assessors use pre-printed notecards to score performance and provide structured feedback.  The system has been shown to provide feasible, valid, and reliable measurements.
  • Blinded Patient Encounters. First described in undergraduate clinical clerkships in South Africa (Burch, Seggie, & Gary, 2006), trainees are assessed during bedside sessions in which they take a history, perform an exam, and formulate a plan of care without prior access to the medical record or diagnosis.   This allows students to develop their clinical diagnostic skills without leaning on the work previously done by more experienced practitioners.  The assessment includes direct observation and structured feedback using performance rating scales.
  • Direct Observation of Procedural Skills.  Developed in the UK, this system involves the direct observation and scoring of performance on a 6-point rating scale.  Physicians-in-training are given the list of procedures for which they will need to be assessed in advance. For a resident physician, typical procedures might include endotracheal intubation or arterial blood sampling.  For medical students, typical procedures might include venipuncture or urethral catheterization. The sequence is designed to involve an (approximately) 15-minute, direct observation and a 5-minute, structured feedback session.
  • 360-Degree Assessments.  These structured questionnaires are filled out by multiple stakeholders surrounding the physician-in-training, including peers, supervisors, and ancillary staff. 

Next blog post: Personalized Learning, Competency-Based Assessment and Oregon Health and Science University

 

Featured Image: rawpixel on Unsplash

The network is the deliverable