The Ultimate Guide to Preference Signaling for Medical Residency Applicants and Programs 2022-2023. Part 4: Recommendations for Applicants and Programs on how to use Preference Signaling

This 5-part blog series, The Ultimate Guide to Preference Signaling for Medical Residency for Applicants and Programs 2022-2023 includes:

This is Part 4: Recommendations for Applicants and Programs on how to use Preference Signaling

Recommendations for applicants and programs on how to use preference signaling:

As of the writing of this article, ERAS has announced that the following 16 specialties will be participating (on an opt-in basis for applicants and programs) in preference signaling for the 2022-2023 residency application season .

Otolaryngology will continue hosting preference signaling on the OPDO website. Urology will also continue preference signaling. Finally, it appears Plastic Surgery will be including preference signaling as part of their new Plastic Surgery Common Application (PSCA).

In total, that would mean that 19 specialties will use a form of preference signaling in 2022-2023. Our analysis here will focus predominantly on the specialties signaling through ERAS, although the lessons learned can be applicable to all.

As shown above, the number of preference (or program) signals allocated to applicants in specialties will vary widely, from 2 to 30. Therefore, while there are common recommendations applicable to all groups, we also break down specific recommendations for specialties using few (2-3), more (4-8) or many (18-30) preference signals. Note: Each specialty has unique instructions for applicants as to whether they should signal a) their home institution and b) sub-internships/away rotations. This will provide an additional variable to consider, as this further changes the signal allocation in each specialty due to differences in program-expectations.

Universal Recommendations for Applicants and Programs:

How Preference Signals can be accessed/used: For the 2023 application cycle, students will submit preference signals for the allocated specialties via ERAS. Programs will receive signal information in the ERAS Program Director Workstation (PDWS), and be able to filter and run reports.

How to view the use a Preference Signal: Given preference signaling is incredibly new to GME, applicants should consider sending preference signals to programs they for which they have a strong preference to interview. This could include programs they really want to attend and/or programs that they are interested in but are unlikely to receive an interview at otherwise (e.g. they have no geographic connection to the area). Preference signals can also be used by applicants to distinguish themselves for programs where many applicants similar to them will apply (e.g. many applicants from the same medical school). Programs should be aware of this information and recognize that the receipt of a preference signal is an expression of interest in the program (Note: The level of interest designated by a preference signal varies based on the number of signals the specialty allocates to each applicant, which is further analyzed below).

If a preference signal is sent, is it certain an interview will result? No. A preference signal, while designating interest in a program, is not a golden ticket. It does not guarantee an interview for an applicant, even if their application is a strong fit to attend that program. Programs should view preference signals similarly and not feel obligated to interview applicants sending signals. For applicants, Medical School Advisory Deans and other mentors are great resources to gain a better understanding of their likely fit at a program within each specialty. These advisors can also assist applicants (programs) in networking and share interests to programs (applicants) through other channels (i.e. outside of preference signals). We strongly urge all applicants to consider against sending a preference signal to highly ranked programs where their academic performance or other characteristics are not aligned with the program (e.g. they require a visa, and the program explicitly does not sponsor visas). This is especially true for specialties with a small number of allocated preference signals.

How do Preference Signals fit with the Match Participation Agreement: What is and what is not a match violation when it comes to preference signals?

Per the NRMP Match Agreement for Programs: “Programs are not authorized at any time during the interview, matching, or onboarding processes to… request that applicants reveal preference signal(s) if in a specialty participating in preference signaling.” Programs also have the right to “Keep confidential all information pertaining to preference signals. Similarly per the NRMP Match Agreement for Applicants, “Applicants have the right to keep confidential all information pertaining to preference signals sent to programs if the applicant is participating in a specialty that has implemented preference signaling.”

Beyond the Match Agreement, there are anecdotal reports, like in this Emergency Medicine podcast, that programs in certain specialties may only use preference signaling to make decisions about who to invite for interview. They may not use preference signals after the interview process or to make decisions about their rank list. The program directors on this podcast also suggested that since signaling is optional, applicants opting-out of preference signaling were instructed to state why they did not partake in signaling in their personal statement, although this does not appear to be an official specialty-wide recommendation.

Overall, the confidentiality of preference signals are protected by the Match Agreement. A program will know whether an applicant signaled their program (but the only way a program would learn an applicant didn’t partake in preference signaling would be in their personal statement). Thus, programs may ask why you signaled their program or did not signal their program, but they CANNOT ask where else an applicant signaled. Therefore, our advice is identical to that of our economics colleagues’: “Applicants are encouraged to be able to explain to the program why a preference signal was sent. Conversely, if asked by a program why they did not send a preference signal, applicants are instructed to communicate that they believed that adequate signaling of interest could be achieved through other channels.”

Specific Advice for Applicants and Programs by Specialty Preference Signal Allocation:

Specialties Using Few Preference Signals (2-3): Adult Neurology, Dermatology, Internal Medicine/Psychiatry, Preventive Medicine.

This group is using preference signaling most closely to what was proposed in economics and therefore our recommendations are aligned with those, with some caveats.

Advice for Applicants in Specialties Using Few Preference Signals:

Preference Signal Value: High. Consider that there are few preference signals you can send. This means that a signal has the built-in, and intended opportunity cost, as described in economics, and as such, you are telling a program “I only had two (or three) preference signals, and I chose to send one to you.” This is a powerful message as described previously.
Will programs to whom I do not send a preference signal still interview me? Yes. The average applicant in all specialties interviews at many more than 2 or 3 programs. And while the preliminary data presented by ENT shows that preference signaling may result in a higher likelihood of interview invitation, not signaling a program will likely still result in an interview for the greater majority of applicants (although this can vary between US grad and IMG applicants). There may be a concentrating of higher value (i.e. greater likelihood of match interviews) and lessening of lower value interviews (i.e. less likelihood of match interviews), which follows the Thalamus mean interview percentage overlap data. Thus, there will be a large number of interviews that occur that were not driven by a preference signal.

Advice for Programs in Specialties Using Few Preference Signals:

Preference Signal Value: High. Consider that there are only a few preference signals that each applicant can send. This means that a signal has the intended opportunity cost and in doing so, receipt of a signal from an applicant means “I only had two (or three) preference signals, and I chose to send one to you.” This is a powerful message as described previously. Yet, the distribution of preference signals (as supported by the ENT data) will vary widely due to applicant perception of programs by prestige, geography, competitiveness, etc. Overall, programs are encouraged to use preference signals to help break ties in determining which applicants receive interview offers and/or allocate additional application review resources to applicants who send a preference signal. As in economics, we also expect that “signals sent to lower ranked programs by highly sought-after applicants increase the probability of receiving an interview.”
Could applicants that do not send preference signals to us be interested in our program? Yes. Similar to in economics, since applicants may only send two to three total preference signals (and some may send none), not getting a preference signal from an applicant, provides limited information. However, the opposite occurs upon receiving a preference signal, as it conveys valuable information about the applicant’s interest in that program.

Specialties Using More Preference Signals (4-8): Anesthesiology, Diagnostic Radiology, Interventional Radiology, Emergency Medicine, General Surgery, Internal Medicine (Categorical), Neurological Surgery, Pediatrics, Physical Medicine and Rehabilitation and Psychiatry.

Note: Given the overlap in applicants between Diagnostic and Interventional Radiology, applicants will be allocated a total of 6 preference signals across both specialties (e.g., 4 diagnostic/2 interventional, 5 diagnostic/1 interventional, etc.).

This group is using preference signaling with an increased number (2x-4x) of what is used in economics and therefore our recommendations are amended as follows:

Advice for Applicants in Specialties Using More Preference Signals:

Preference Signal Value: Medium. An increasing number of preference signals decreases the opportunity cost of sending additional preference signals to each program. Overall, this weakens the value of any signal, as applicants have an increased ability to signal more programs. However, this also increases the number of programs to which an applicant can send a preference signal, and therefore allows a more risk-tolerant behavior of signaling programs that may be slightly “out of reach” or expand signals to programs where an applicant may be overqualified. The exact strategy used by an applicant should reflect the overall strength of their application, as well as their specific career goals.
Will programs to whom I do not send a preference signal still interview me? Yes. The average applicant in these specialties interviews with at least 4 to 8 programs. The increased number of preference signals will result in a higher count of interviews attributable to preference signals. However, there will still be interviews offered where no preference signal was used. Given the preliminary ENT data presented above, it appears that a larger percentage of a given applicant’s interview offers will be attributed to programs to which they provided signals than those they did not. Significant decreases in mean interview overlap may also occur as shown in the Thalamus data above (it is unclear how much of the observed effect was due to signaling).

Advice for Programs in Specialties Using More Preference Signals:

Preference Signal Value: Medium. Similarly, as noted for applicants, an increasing number of preference signals weakens the strength of any one signal. However, preference signaling information remains beneficial to programs, because it identifies a smaller subset of applicants with increased interest vs. the hundreds to thousands of applications a program may receive in a recruitment cycle (without signal data). This provides a smaller subset of applicants to consider with an increased interest. However, the level of interest can vary widely. In the most extreme case, a signal may inform a program that they are an applicant’s first choice at best, or eighth choice at worst. This is a significant spread, outside the range of where applicants usually match within each specialty. Programs are encouraged, similar to economics, to use this information to help break ties in determining the group of applicants they interview. But here, further, programs must be cognizant that the distribution of preference signals vary widely due to applicant perception of prestige, geography, competitiveness, etc., and the opportunity cost is not as high as when smaller number of preference signals are used, and hence, are less reliable. Similarly, signals sent to lower ranked programs by highly sought-after applicants will not have as much of an impact on the probability of receiving an interview.
Could applicants that do not send preference signals to us be interested in our program? Yes. The larger number of preference signals means that there will be a greater overall amount of preference signaling resulting in an interview. However, this then means that a lesser number of interviews will occur from non-signaled applicant-program pairings. They will still occur, but likely with overall less frequently. It also brings forth further questions as to why applicants did not signal a program, particularly in a scenario such as between an applicants, 8^th (signaled) vs. 9^th (non-signaled) choice (using neurosurgery as an example). As such, a non-signal has more value than in a specialty allocating 2 or 3 signals per applicant. Here, not signaling a programs represents either a larger signal of disinterest or that the applicant thinks they have another way to denote interest in the program (outside of signaling). Regardless there will be applicants who interview and ultimately match at programs where they did not signal.

Specialties Using Many Preference Signals (18-30): Orthopaedic Surgery, Obstetrics and Gynecology

This group is perhaps the most interesting as well as the most divergent from economics preference signaling implementation due to the abundance of preference signals being used.

Obstetrics and Gynecology is using a two-tiered system, as announced in their Signaling Guidelines, published April 10^th, 2022. The specialty will use 3 “Gold” (tier 1) and 15 “Silver” (tier 2) preference signals to denote “highest” and “very high” interest in programs respectively. Orthopaedic Surgery will allocate 30 signals per applicant as announced via their specialty website. Of note, instructions and recommendations on both the OB GYN and Orthopaedic Surgery sites are identical.

Specific advice to OB GYN/Ortho applicants from specialty leadership includes:

“Applicants should signal programs that they have both a strong interest in and where they have a reasonable ability to receive an interview INCLUDING their home programs (and away rotations).”
- Note: While this seems to be the majority practice in the 2022-2023 preference signaling system (used in all specialties in some capacity either than Dermatology, Internal Medicine and Psychiatry), this interest can also be expressed via other mechanisms (outside of signaling).
Applicants should plan to use all their signals. The instructions report that, “There is no known advantage to not using all their available signals.”

Specific advice to OB GYN/Ortho programs from specialty leadership includes:

“Signals are not intended to be used as a screening tool, however some programs may choose to use them in the initial application review.” The instructions note that signaling may be used as a part of a recommended holistic review of applications.
Programs MUST NOT:
- Require a signal to interview an applicant.
- Disclose preference signaling information to any person outside their selection committee.
- Ask applicants to disclose where they sent signals (or the number of signals sent).
- Disclose the number of signals received.

There is also substantial national discussion that this system was created to serve as “proxy” to emulate an application cap, though applicants are still able to apply to as many programs in each specialty as desired.

The recommendations in this group are the most difficult, as this novel use of preference signaling has not been studied previously.

Advice for Applicants in Specialties Using Many Preference Signals:

Preference Signal Value: Unclear. With a multitude of preference signals that can be sent, the opportunity cost declines substantially (by 9x to 15x vs. economics). Mathematically, this further weakens the value of any given preference signal. Yet, the applicants are able to send signals to many programs, which results in being able to signal to the number of programs that cover nearly the entire range of applicant-rank list lengths.
- OB-GYN: is unique in that the 3 gold tokens, create a hybrid market, that encompasses a similar number of preference signals equivalent to the few preference signal specialties (e.g. dermatology). However, this effect is modified by the presence of the 15 silver tokens. Hence there is a large gap not only between choice 3 and 4, but also an indistinguishable “silver” status assigned to choices 4 through 18. This represents an incredibly wide margin of preference. And further, non-signaled programs then becomes a 3^rd tier. Our advice is that applicants use a strategy similar to few preference signal above for the 3 gold tokens, while using the 15 silver to denote any other program that they have a preference vs. those where they have no preference.
- Orthopaedic Surgery: With 30 preference signals, applicants should signal any and all programs within which they have an interest (and/or fall within their top 30). If they are applying to a number above 30 (Average Apps/Applicant in Ortho in 2022 was 86), there will likely be a bimodal strategy of applicants signaling to their most desired programs and/or allocating a portion of their preference signals for programs where they are more likely to receive interviews (based on academic performance, geographic interest, program visa sponsorship, etc.).

As in the other groups, applicants should not use more than a few preference signals to overreach in an attempt to gain an interview at a program where their likelihood of interview is low to start. The exact strategy used by an applicant should consider the overall strength of their application, as well as their specific career goals.

Will programs to whom I do not send a preference signal still interview me? Yes, but less likely. Here the tables are likely somewhat turned, as the number of signals eclipses the average number of interviews completed per applicant, and the significant increase in preference signals likely means most (although not all) interviews will be attributable to a preference signal. It is also unclear how this will affect interview overlap, as it will be very dependent on how preference signals are assigned across each specialty.

Advice for Programs in Specialties Using Many Preference Signals:

Preference Signal Value: Unclear to low to confusing. Signal value will vary significantly based on program characteristics (perceived prestige, geography, etc.). It is likely that the distribution of preference signals across programs will be even more widely distributed. Highly sought-after programs will likely receive hundreds of signals while less desired programs will likely receive much fewer (in a similar pattern to what was observed in ENT). Further, the value of any given preference signal declines with the abundance of preference signals allocated. This means that programs that receive few signals will have a challenge in interpreting them (as applicants likely signaled many other programs as well). Further, programs that receive a large number of signals will have an even harder time discerning a value given the large number of applicants expressing interest. It is unclear how this will affect the interview overlap amongst programs. As such, it is also unclear how programs should use signals (e.g. break ties, etc.). It is difficult to provide strong recommendations to this group as a result, other than that OB GYN programs should follow the recommendation for few preference signal programs above when interpreting their gold signals.

Could applicants that do not send preference signals to us be interested in our program? Yes, likely a small percentage of them. The abundance of preference signals means that there will be a much larger overall amount of preference signaling resulting in interviews. As a result, the number of interviews that will occur from non-signaled applicant-program pairings will be minimal, although will still occur. Non-signals have a higher negative predictive value here in terms of applicant interest, as the applicant forwent sending a signal to a non-signaled program vs. all of the other programs that they did signal. The one caveat again is that OB GYN programs will certainly see an abundance of interviews form non-gold-signaling applicants (and a smaller amount of neither-gold-nor-silver-signaling applicants).

In Summary, applicant and program use of preference signals should be based on the amount of signals afforded to applicants by each specialty.

How is Thalamus going to support preference signaling? Find out in Part 5!