Surgery Fellow: Let’s Check Out of Hotel Rooms for Oral Board Exams

Imagine taking the highest-stakes test of your career. No matter how prepared you are, you’re likely operating on very little sleep, fueled by nervous energy, caffeine and the contrails of recent anxiety dreams.

Now imagine taking the test in a hotel room with two examiners you’ve likely never met before. Maybe there’s a bed in view, or maybe you’re lucky and you’re in the separate seating area of a suite.

For surgeons having taken the American Board of Surgery’s oral exams up until this year, that scenario was very real—including the hotel room.

Meredith Barrett, M.D., a clinical lecturer and transplant surgeon at Michigan Medicine, had experienced the scenario. When she took the boards in 2019 at a hotel in Philadelphia, she did not pass. Taking them again remotely in December 2020, she succeeded and reflected on the experience.

“When I was describing the scenario to a resident here, she said that sounds like an unfair testing environment,” Barrett says. 

She thought about it and decided it was indeed an unfair environment. Then she wrote about it for the Annals of Surgery. 

If at first you don’t succeed, try again—and share your experience 

Barrett isn’t sure if the setting of her first test, a hotel room in Philadelphia, affected her result. She is sure it didn’t help her mindset. The experience was so strange that she distinctly remembered the room number: 1438.

Standards cited for oral exams to objectively assess surgeon competence are that they be valid, reliable and fair. Barrett says in her commentary that the hotel room setup doesn’t pass the fairness test. 

There’s the simple fact that the setup is not just awkward but bizarre. 

“I remember some of the test questions and being distracted by the fact that I was in a hotel room. I can still remember that this man was sitting awkwardly next to a very large plant. The scene doesn’t allow for focusing,” Barrett says.

Then there are the more serious considerations, like the connotations hotel rooms have and the extreme discomfort they bring. That discomfort may be heightened if the examiners are male and the examinees are female. 

“People have innate feelings about hotel rooms. Things are done in hotel rooms that bring certain feelings to certain people. I can only imagine if someone had been a survivor of sexual assault how the environment could have created significant emotional feelings,” Barrett says.

Reflecting on that aspect in particular, particularly after having experienced the virtual experience, drove Barrett to sound off.

“I couldn't fathom how there wasn’t a better solution. The more I thought about it the more I got frustrated by it,” Barrett says.

The upside of disruption 

Aside from the occasional “You’re still muted” hiccups, the world of surgery has adjusted to performing many meeting functions via videoconference.

COVID forced the American Board of Surgery to follow suit with the oral boards, disrupting the hotel room tradition. Barrett says in her paper that maintaining the hotel location for testing served only tradition—one that some who’d run the uncomfortable gauntlet would even agree was not a good one: 

“Rites of passage don’t test ability. They are not objective. COVID has gotten us to disrupt the policies that have always been in place and I think it has shown us the benefit of always being flexible in your approach to things.”

The virtual exam experience came with some obvious upsides compared to the in-person hotel room experience: No chance of seeing a suitcase or shoes in a corner, the opportunity to arrive to the test rested and a physical environment more suited to clear thinking. (Barrett took her exam in an office in the hospital.)

There was also significant cost savings from not having to travel and the emotional discount of not being surrounded by fretting residents waiting in a hallway.

“As someone who’s taken both exams, the only thing that seemed different was the incredible tension in the room from being in this foreign place. The conversations I had back and forth with the faculty were just as engaging, and I didn’t feel like I lost anything in the virtual format,” Barrett says.

Some body language can be hard to read in a virtual format, though it doesn’t necessarily impact the test. Barrett points out that the examiners would be less likely to see examinees’ fidgeting habits, for example.  

For every action, there’s a reaction

Early data from a study from the American Board of Surgery Research Committee on the virtual exams provide insight into the experience and impressions of examiners and examinees. The study found similar pass rates from early rounds of the virtual exams (82%) compared to the most recent in-person exams (85%).

Both examiners and examinees rated the overall virtual experience highly. When participants were asked whether they preferred the virtual experience to the in-person one, however, there was more divergence; 78% of examinees preferred the virtual experience, and 79% of examiners preferred the in-person experience. Examinees cited the lack of travel as an upside. Examiners missed the camaraderie and networking opportunities that come with in-person exams. 

In addition to eliminating the burden of travel on examinees, Barrett sees the virtual exam as a chance to diversify the examiner pool. It would allow examiners who only have so many days to travel, or can’t travel due to family obligations, the opportunity to participate if invited.

Reactions to Barrett’s paper have been positive—and in the case of Twitter, public and anecdote-filled. The sources of that feedback have been consistent, perhaps highlighting how the setup negatively impacted some groups more than others.

“I haven’t gotten a lot of people who aren’t women or minorities. I hope we look back on this in five years and ask why that was ever a thing,” Barrett said. 

The president of the American Board of Surgery, Dr. Jo Buyske, also weighed in: 

“100% agree. ABS orals will be virtual again in 2021-2022. While the exams are not yet in their final state, I can guarantee we won’t be going backwards to hotel bedrooms,” Buyske wrote on Twitter in response to the viewpoint. 

The ABS has crafted a commentary, not yet published, on the paper. 

Some feedback has revealed other disparities, and reasons the virtual format is preferable for some. Consider the case of a woman, pregnant at the time of her exams, who shared her experience with Barrett. Too late in her pregnancy to fly safely, she drove a great distance to the testing site. The challenge didn’t stop when she parked. 

“When she got to the testing site, she was told that it was ill advised to take the elevators because they might be too slow and she might be late for her exams,” Barrett says, pointing out the challenge such a scenario would pose for anyone with mobility issues. 

Vulnerability required 

Speaking up about the exam setting was intimidating. The issue wasn’t calling out a tradition, but admitting failure.

In order to provide the unique perspective she did, Barrett had to share her experience. In a high-achieving, competitive culture, that can be scary. 

“Was I super excited that for the story to be told, I had to include that I failed the first time? No. I think surgery has a lot of room for vulnerability and I think being vulnerable and speaking out when you see things that are unfair or unjust is part of our job,” Barrett says.

In speaking up, Barrett hopes to create more space for vulnerability, normalize the non-linear path that can produce success every bit as much as a linear one and help more people succeed.

If traditions aren’t helping, they aren’t worth keeping.

Learn More

• Read Dr. Meredith Barrett's piece "Hotel Room #1438" in the Annals of Surgery →
• Herbert Chen, M.D., F.A.C.S., Department of Surgery chair at the University of Alabama at Birmingham and the American Board of Surgery Research Committee chair, discusses the study findings. Watch the American College of Surgeons From the Frontlines conversation →

##

By Colleen Stone