Geriatrics Recruitment Resources

at the University of Wisconsin-Madison School of Medicine and Public Health

Program Overview

The Year-end Professional Skills Assessment program in geriatrics was created to evaluate third-year medical students to ensure that they have developed the clinical competencies in geriatrics necessary to proceed with their medical education.

These include:

• recognizing how the features of the aging process are distinct from those of age-related illness
• identifying psychosocial and economic risk factors for older adults that predispose them to illness and loss of function
• performing the basic components of a geriatric assessment
• recommending age-appropriate preventive care for older adults
• demonstrating knowledge of basic approaches in screening for and managing geriatric syndromes/illnesses
• adjusting treatment strategies.

Incorporating a geriatrics-specific case in the overall Year-end Professional Skills Assessment program provides greater visibility for geriatrics as an important area of specialization, with the possibility of interesting more medical students in careers in the field.

Program Operations

A comprehensive, objective-structured clinical examination is administered to students at the end of Year 3 of medical school. It is a degree requirement and must be passed before the student is allowed to graduate from the School of Medicine.

Students rotate through multiple 15-minute clinical stations during which time they have ten minutes to interview the patient. Each case aims to assess the student’s communication skills, knowledge, and ability to synthesize information elicited from the standardized patient. During the final five minutes, faculty ask programmed questions to assess the students' synthesis of the information elicited. A case evaluator videotapes each student’s performance. These cases are intended to assess competency rather than differentiate average from excellent students. As such, they attempt to screen for the small subset of students who do not have the interviewing skills, knowledge, or capacity to synthesize the key concepts of the case.

Each station is based on a clinical core competency list that is developed by the third- year clerkship directors and the geriatrics education liaison. It is distributed to students at the beginning of Year Three. At Wisconsin, geriatrics is one of the few non-departmental cases in the Year-end Professional Skills Assessment.

The geriatrics case can also be used for assessment of performance in other learning venues such as a clerkship or an elective rotation. To incorporate it into a larger skills examination requires the following steps:

• meeting with faculty who oversee the skills assessment at the medical school for buy-in and consensus
• identifying the competencies to be tested
• creating the case with school experts in instructional design and geriatrics
• reviewing the case with the Year-end Professional Skills Assessment committee
• educating standardized patients and case evaluators for uniformity in the administration and evaluation of the case
• scheduling evaluators if multiple cases are to be run simultaneously
• storing the assessments into an electronic database, e.g., PC tablets and concurrent video capture of all cases
• creating remediation processes for those students who obtain marginal or failing grades on this case.

Once there is completion of a case description, a script for standardized patients and a checklist for evaluators, then the case is embedded into the larger skills examination and infrastructure.

Staffing Requirements

Different staffing requirements exist within the three phases of the program. The case development phase requires geriatrics faculty member content expert(s), an administrative staff member with expertise in instructional design, and a seasoned standardized patient to help translate the key concepts and details into a script. The evaluators’ checklist must also be created by the case developer.

The case administration phase covers the actual six half-day testing session. Over 150 medical students are tested. Typically, two geriatrics stations are run simultaneously for 12 students per half day. This requires two case evaluators per half day and one back-up faculty to fill in for any evaluator who cannot be present. A Year-end Skills Professional Assessment administrator needs to coordinate this schedule and ensure that the faculty adjust their schedules and are reminded several days prior to the examination.

The remediation phase typically requires one faculty member or a small working group. They must identify the component(s) of the case that a student performed poorly on and create a remediation plan to help improve the student’s skills or knowledge base prior to readminstering the examination five months later.

Program Costs and Funding Sources

The following personnel and training are required:

• Six to eight hours of development time by a geriatrics faculty member
• An instructional design expert (either internal staff or a consultant)
• Paid time of an experienced standardized patient who helps develop the case script
• Faculty time to administer the cases within the examination and proctoring time of the Year-end Professional Skills Assessment administrators (the exact program costs for this aspect of the program depend on the number of students tested)
• Extensive training for the volunteer standardized patients (the cost for training these individuals is usually integrated within the infrastructure of the schools testing center)
• The cost for the faculty time to conduct the remediation dimension of the program, with a typical remediation meeting and follow-up to enact the plan taking between two and three hours per student

In addition to funds from the school testing center, earmarked faculty time and money are released from the Geriatric Research Education and Clinical Center and the geriatrics section. The GRECC has faculty time earmarked for education and the geriatrics section releases faculty time and money for these activities.

Process and Outcomes Data

Out of the 750-plus students who have been assessed over the past five years, approximately 2% have had marginal or failed performances on their geriatrics cases. No students have argued that the case was unfair or did not test the competencies that were emphasized.

Four distinct geriatrics cases have been developed since 2002; a fifth case is currently being adapted from an earlier case. The geriatrics case directly aligns with topics from the written geriatrics competencies of the medical school curriculum. Information from the case examination is analyzed and given back to course directors as feedback and presented to the Educational Policy Committee to determine if the curriculum is meeting the needs of our students.

Several levels of evaluation occur to test the internal validity of the cases to distinguish between students who do and don’t demonstrate the core competencies tested. During the initial case development, test students evaluate each case. Once the cases are employed in the larger examination, analysis occurs to determine how failures on multiple cases correlate. Typically, students who struggle in one case also show inadequate performance in other cases. If a specific case fails more students than the mean or does not detect any marginal or failing students, then closer scrutiny occurs regarding that case. This has not occurred to date with the geriatrics case. Video capture also helps to corroborate if the evaluator appropriately assessed the student by allowing another evaluator to independently judge the student’s performance.

Long-term educational outcomes have not yet been gathered regarding the students’ performance on the geriatrics case and the capacity of this evaluation tool to predict performance on standardized tests or measures of proficiency in caring for older persons at the residency or practicing-physician level.

Implementation Lessons

• For initial acceptance of the geriatrics case, the geriatrics program director needed to work closely with the Year-end Professional Skills Assessment committee and key administrators of the student testing site to gain buy-in. The program director also worked with the Associate Dean for Curriculum to demonstrate the need for a geriatrics case that is distinct from cases delivered by the Departments of Medicine and Family Medicine. The Year-end Professional Assessment Skills program has flourished due to strong institutional support from key curriculum leaders. Aligning the case topic with both local competencies and the forthcoming geriatric competencies supported by the Association of American Medical Colleges persuaded leadership to proceed with support for the geriatrics case.
• It typically takes two to three months for case development from conception to administration of the beta-cases to students acting as test subjects. The case developer needs to identify a test population that will be comparable to the students being evaluated. This typically has been either fourth-year medical students or third-year students from other schools who have the integrity to maintain the security of the case. Feedback from the 3-5 test students has resulted in cases with higher ratings for quality than cases tested by fewer student test subjects.
• The evaluation of these cases leads to three categories of performance: pass, marginal, and fail. Given the wide variety of faculty views regarding student performance, it is essential that evaluation criteria and an explanation of these criteria occur with all case evaluators. The evaluation checklist attempts to help the evaluators identify the presence or absence of discrete skills or behaviors to enhance uniformity amongst testers.

Available Materials

Tools/Resources

• Example of geriatric competencies for third-year medical students
• Example case
• Script for standardized patient
• Checklist and direction for case evaluators
• Schedule for evaluators

For More Information

Steven Barczi, MD
Associate Professor of Medicine
Division of Geriatrics and Gerontology
University of Wisconsin School of Medicine and Public Health
Madison VA GRECC (11G)
2500 Overlook Terrace
Madison, WI 53705
(608) 280-7000
steven.barczi@med.va.gov

Download the program as a PDF file