Residency Program Interview Process Best Practices 2020-2021
Table of ContentsI. Introduction II. Background and Identification of Need
- The beginning: The era of "travel agents"
- The next step: ERAS Launches
- Today: The age of interview scheduling software
- Why this guide is necessary
- The number of candidates to invite to interview
- The ideal times to send out interview invitations
- How to announce your program's interview invitations
- How to communicate your program’s invitation message
- How to best accommodate your candidates
- How scheduling flexibility optimizes completed interviews
I. IntroductionA. The purpose of this guide:
Over the past decade, the residency interview, scheduling, and match processes have become increasingly more competitive and anxiety-producing. The reasons for these challenges are varied and nuanced, but include application inflation and residency program over-invitation (e.g. inviting 600 applicants to 400 residency positions) as major contributing factors.1 As a result, candidates have been observed to be “glued to their phones” during the months-long interview season, to ensure optimal scheduling opportunities.
As the premier and most preferred residency interview scheduling software, Thalamus has studied these processes in over 70 residency and fellowship specialties, across all 50 states, at over 125 academic medical centers, and covering all regions, geographies, and practice areas. With data on over 100,000 candidates scheduling over 1,000,000 interviews, we have been analyzing these practices to better understand the shortcomings and to promote best practices. And practices do vary!
Given our knowledge and experience, we are in a unique position to share our findings and advice. In raising this awareness, our hope is to improve the process for everyone involved. Specifically, we hope that following these best practices will improve efficiency while maintaining emotional well-being, increase accessibility to scheduling while keeping costs at bay, and ultimately optimize candidate selection and matching to training programs.
We invite all stakeholders to consider this guide. Program directors and applicants can optimize their selectivity of one other, specialty organizations can enact policies to drive best practices, and medical education governing bodies can better streamline resources to gather data and lead holistic change.
B. Why are we doing this?
Thalamus was founded by a grassroots collaboration of applicants, current residents, program coordinators, program directors and GME administrators who have been on the front lines of this process. In fact, our company was inspired by the experiences of our founders, a former applicant and program director, who personally grappled with a particularly chaotic interview season in 2012 disrupted by Hurricane Sandy.
What follows is backed by years of observational and data-science driven research. We invite you to review our findings and advice. We have always valued your feedback and we hope you continue to participate in improving this process with us.
II. Background and Identification of Need
During the era of “travel agents” and prior to the advent of online interview scheduling, most of this process was accomplished by email and phone calls. A typical residency coordinator spent 30% of administrative time (or more) managing the recruitment season. Candidates typically applied to ~20 programs across all specialties, eventually interviewing with a subset of those programs. For ease of workflow, bulk invitations would be sent to the “best” candidates early, and operations occurred during weekday “banking” hours. Candidates would race to ensure concise and politely-worded emails arrived at a program coordinator’s inbox prior to the other several hundred of responses from counterparts. The coordinator would then fill out a date planner or an Excel spreadsheet, going email by email through Inboxes. If a candidate’s preferred days were filled, more email notifications were sent requesting additional preferential dates of attendance. Several electronic exchanges over weeks and months would pass as the dates filled. And then late season cancellations would occur.
2.) The next step: ERAS Launches
When the Electronic Residency Application Service (ERAS) launched in the late 1990’s2, communications surrounding interview scheduling grew exponentially as the process of applying became easier and more efficient. Over the last 20 years, GME applications and their costs have quintupled. Domestic graduates are now applying to 60 programs and international medical graduates (IMGs) are applying to 150 programs on average across all specialties. In the 2018-2019 ERAS cycle, nearly 4,000,000 applications were submitted by ~47,000 applicants, incurring costs of greater than $80M in total application fees. Medical schools have partially fueled this inflation by promoting broad application strategies in an effort to secure 100% match rates amongst their 4th year classes. The growing applicant base per program (supply) is not being met by a similar increase in available residency positions. In essence, more programs are just offering more interviews to the same pool of ~47,000 applicants (or rather the subset of applicants applying to any given specialty). Thus, the chance of any one candidate matching at any given program decreases further.
3.) Today: The age of interview scheduling software
The introduction of interview scheduling software has allowed for ease of scheduling and cancellation of interviews, but has also opened up the process to new challenges. Eager to fill spots early, those programs that rush to invite candidates usually experience greater overall flux in their interview schedules.
Top tier applicants are eager to schedule as many interviews as they can, early on.3 Anxious about prospects, they are hesitant to cancel until they are invited to their most preferred programs and/or they have completed a significant amount of interviews.
These spots may change rapidly or remain taken until the middle or end of the season. And as these applicants cancel, interview spots open for less competitive applicants.
Many programs are attempting to adjust for this flux by grossly over-inviting for the number of spots available each year to the tune of 100-200 more candidates than they have available interview positions. Being invited to a filled interview calendar creates significant applicant anxiety further fueling application-inflation, increased costs, and applicants being “glued” to their smartphones.
So, while on-line scheduling software has operationally improved efficiency and accessibility, it has also translated into increased applicant anxiety and distractibility during the final year of medical school.4
4.) Why this guide is necessary
The lack of transparency confounded by these phenomena, described as both “application inflation” and “over-invitation” is resulting in a suboptimal experience for all stakeholders. This problem is process-driven, fueled by applicants and programs alike struggling to find an ideal match, and sincerely lacking in knowledge of preference. The outcome, to overcome “Match uncertainty” is an inefficient system, contrasting antiquated recruitment practices with newer technology. We believe that targeted adjustments in recruiting strategies, when adopted in the context of evidence-based data, will dramatically improve this process.
What follows are Team Thalamus’ best practices for residency interview scheduling. Not only will we explore what to do (and not to do), we’ll explore the reasons and the consequences that result based on over a million historical data points.
III. Best Practices
Best Practice #1The number of candidates to invite to interview
Do not initially invite more candidates than available interview positions. EVER.
A. In fact, invite fewer candidates than you have total positions.
B. Invite in multiple waves, spread out over a number of days to weeks.
Why: Currently, the gravest concern of candidates is that they will arrive at a program’s calendar to find all interview dates full. If your program invites 600 candidates at once to 400 positions, this fear will be realized by 33% of your candidates.
Instead, invite 50-100 of your top candidates to your calendar to the open 400 positions. Give this group a set amount of scheduling time before inviting the next wave of candidates. This will allow this tier of applicants to view multiple scheduling options, and will signal their value to your program, and all within the confines of the NRMP match guidelines.6 After the set period of time expires, invite the next wave of candidates. And repeat the process until the schedule is full. Maintaining open dates allows for continued flexibility and candidates will appreciate it.
Programs that use our software and maintain this strategy have decreased cancellation rates, improved interview completion rates and generate higher candidate satisfaction.
Best Practice #2The ideal times to send out interview invitations
Why: There are four time zones in the continental United States (seven when including Alaska, Hawaii and Puerto Rico, and more when including other US territories), creating a rather large spread of time throughout the day for medical students across the country. Given that international medical graduates (IMGs) are applying from all over the world, it is nearly impossible to send invitations at a time that works for all candidates. Yet, regardless of a program director’s orders to “Get invitations out now,” there are more optimal times to send interviews dependent on your program’s geographic location.
Eastern Time Zone: Do not invite prior to 12pm EST, or risk biasing against the west coast.
Central Time Zone/Mountain Time Zone: Recognize that candidates exist in areas that are both earlier and later in the day with regards to your program. Plan accordingly.
Pacific Time Zone: Do not invite in the evening. For instance, at 5pm PST, MST applicants are in rush hour, CST applicants are likely eating dinner, and EST applicants are winding down their days.
Note: While historically there has been critique of disrupting educational activities during the day, we highly recommend combining item #2 (time zone recommendations) with item #1 (inviting less candidates than available positions, and in waves). Inviting in waves and providing ample available positions that are greater than the number of candidates invited, gives candidates choices and adequate time to access their invitation during a free moment in their day. By giving applicants the ability to not have to disrupt their day to schedule (because spots remain free until the next wave), you are allowing the candidate to choose an optimal interview day when schedules allow and without time pressure. This leads to more optimal and planned decisions on which day to interview based on travel schedules, flight prices and interviews at other programs.
Similarly, blanket statements that candidates should be invited in the evening, morning or any other hour are counterproductive. Many medical students work long days and evening hours, similar to residents and attendings, and the assumption that all students are free at any one given time is inaccurate. Similarly, candidates may be in transit, on airplanes, spending time with their families or any other activity. The key isn’t to find the window when most candidates are free (although a noble goal). The key is to provide ample time to schedule (and slowing down the rate at which calendars fill), so that the candidate can choose an optimal interview date.
Additionally, inviting candidates when staff is still in the office ensures that any questions or issues may be dealt with in a timely manner, rather than having to ask a candidate to wait to the start of the next business day for assistance. If a program makes an inadvertent error in setting up a schedule then pushing off the issue to the next day is also more difficult to manage.
Finally, there are internet outages, travel disruptions and any other number of interruptions that may preclude a candidate from scheduling an interview within minutes. Slowing down the scheduling process and providing options to a smaller number of candidates also minimizes the effect of these unpredictable disruptions.
Best Practice #3How to announce your program's interview invitations
Why: Imagine a radio station announces that it is going to award $100,000 to the hundredth caller who calls in at noon the following day as part of a promotion. Quite a few listeners with a phone instantly become anxious and start planning to be available at noon the following day, so they can frantically start dialing their phone.
The same can be said of residency interviews.
1.) Informing candidates they’ll receive an invite at a certain date and a certain time instantly increases anxiety.
2.) The candidates then have to wait and start worrying if they will be able to secure an interview position.
3.) When the waiting period is completed and scheduling opens, all candidates are trying to sign up for all spots… and it is mass chaos. While, technically, scheduling software can handle this process, the race is larger and quicker than ever.
4.) All this accomplishes is delaying the rat race.
5.) When a new institution sends out invitations, the process is repeated and this prior program’s calendar changes as candidates reschedule the previously booked interview dates.
And… if your program doesn’t invite at exactly the pre-defined and communicated date/time, your program will receive a large influx of emails and phone calls from candidates inquiring if something is wrong. Again, slowing the process down by offering ample positions, and inviting in waves, bypasses the need for any and all of this.
Let your interview invitation be the announcement.
Best Practice #4How to communicate your program’s invitation message
Clearly communicate with your candidates the time they have to schedule an interview as well as if there are any positions left.
A. In your interview invitation, state that there are enough positions for all candidates invited and that they will have up to “x” hours (or days, weeks, etc.) to schedule an interview position before risking losing their interview invitation.
B. In the event that your calendar is full, clearly communicate to candidates that they are being invited for a wait list position.
Why: Giving candidates a guaranteed window within a clearly defined time period ensures that they do not need to rush nor leave clinical activities, and provides reassurance that they may take their time in scheduling. Similarly, in the event that a calendar is already full, candidates then know they are being invited to a wait list position and will arrive to a full calendar (rather than being surprised and thinking something is broken or not set up correctly… or they were too slow). This seems straightforward, but there are many programs that fail to communicate this effectively.
Note: There is a common belief on internet message boards and forums that all candidates are invited to a program at the same time. While online scheduling systems have provided greater transparency into the number of positions remaining, candidates are not provided information about when they were invited compared to other applicants. When scheduling any event (such as getting on a bus vs. scheduling a residency interview) someone will always be last. Previously, when invitations were sent by email (or phone), candidates were not aware of whether they were the first or last to be invited. Presently, candidates arriving to a full calendar may assume their invitations were last, but really have no insight into whether the schedule filled in the seconds prior or weeks before.
Best Practice #5How to best accommodate your candidates
Why: In nearly all other industries when someone is invited to interview at a company, institution or otherwise, that position is guaranteed. This is not historically or currently true in graduate medical education. Granting that additional interview (especially during the last month of the season) does not really stress the “system” as much as it serves as a buffer for cancellations. In fact, the a priori scheduling of increased end of season interview slots allows for more candidates to schedule and greater malleability overall.
In expanding our network of users, we have the benefit of studying other industries. Through ThalamusLaw, for instance, several of the nation’s top law firms have realized the benefits of accommodating candidates as much as possible. Of course, there are candidates who may try to take advantage of such situations, so this is completely at the discretion of your program.
Best Practice #6How scheduling flexibility optimizes completed interviews
The practice of offering a subset of interviews days to a portion of your candidates (i.e. early invited candidates to fill your earliest interview dates) is an inefficient practice. Residency recruitment does not happen in a vacuum and candidates scheduling interviews at your program are also potentially scheduling dozens more. Candidates have limited finances and are attempting to schedule many interviews limited by number of trips and dollars spent. If an applicant wants to interview at your program, they will come as long as there is availability on a convenient day. Limiting options limits the chances of your mutual match. Conversely, if a candidate does cancel and doesn’t interview at your program, you were never going to match that candidate anyway.
Note: If you’re recruiting for multiple NRMP tracks or various specialty programs that have their own interview date, this logic does not apply (because you may need certain faculty or specialty programming to be available). This point applies to instances and dates where any candidate may be able to schedule across your general calendar.
Optimal recruitment practices
1.) Initially invite fewer candidates for the total number of interviews slots.
2.) Invite candidates during business hours, taking time zone into account.
3.) Do not set a date and/or time for interview invitations to be sent or scheduling to “open.”
4.) Clearly communicate that candidates are guaranteed a position (time-limited) and/or that they are being invited to a wait list position.
5.) Try your best to accommodate candidates when possible. It will be appreciated.
6.) Do not force candidates to interview on a particular date, unless for a special program.
Utilize and optimize interview scheduling software to your program’s benefit. This should reduce inefficiency, increase convenience, and enhance candidate satisfaction. Adopt new recruliting strategies which fit the technology and current recruitment trends. Such practices will provide candidates with an improved interview invitation and scheduling process, decrease cancellations and candidate frustrations, and optimize program interview yields.
Join with us in making the interview process fairer and more equitable for all.
For more information about Thalamus, visit our blog at thalamusgme.com/blog
Have questions? Comments? Inquiries? Or want to learn more about our research? Then please contact us at firstname.lastname@example.org
Thanks again and have a great interview season.
- Team Thalamus
1. Berger JS, Cioletti A. Viewpoint From 2 Graduate Medical Education Deans Application Overload in the Residency Match Process. J Grad Med Educ. Jul 2016;8(3):317-321.
2. Mandel LP, Spadoni LR, Hewitson LA, Vontver LA. One residency's experience with the electronic residency application service. Family medicine. Mar 1997;29(3):209-212.
3. Whipple ME, Law AB, Bly RA. A Computer Simulation Model to Analyze the Application Process for Competitive Residency Programs. J Grad Med Educ. Feb 2019;11(1):30-35.
4. Luftig D. The Residency Interview Scheduling Process: Unintended Consequences and a Proposal for Change. J Grad Med Educ. Mar 2015;7(1):134.
5. Weissbart SJ, Hall SJ, Fultz BR, Stock JA. The urology match as a prisoner's dilemma: a game theory perspective. Urology. Oct 2013;82(4):791-797.
6. NRMP Match Communication Code of Conduct http://www.nrmp.org/communication-code-of-conduct/