Pictured (L to R) Dominic Vincentini, Janie Herrema, Kristin Thompson, Stephen Johnson, Matt Donahue, Tim Weesner, Chad Kinart, Melanie McGrath, Brian Brewster, Jenna Street, Mike Neumann and Lora Atchison
When I explained to my friends and family what I would be doing at the BOC Cut Score/Standard Setting meeting, the first question they asked me was: “You accepted and volunteered to take an exam and analyze it over and over again?” Yes. Yes I did, and I was honored and excited to be one of 10 individuals to help set the Cut Score for the BOC for the next 5-6 years.
The BOC Cut Score Standard/Setting meeting occurs once every 5-6 years, when the new RD is released. Once the RD is released, the BOC gathers 10 Athletic Trainers (ATs) who have been certified for 3-7 years from a wide variety of backgrounds. This year, this meeting was held in Omaha, NE for the first time at the new location for the BOC headquarters. The BOC Cut Score Standard Setting panel was comprised of ATs from Florida, Michigan, Indiana, Nebraska, Ohio, Idaho, Rhode Island, Iowa, Colorado, and Virginia who worked in a variety of settings including college, military, high school, Olympic, and clinic/outreach.
We gathered in the loft meeting room in the historic building (which used to be a bowling alley at one time) that serves as the BOC headquarters. This is where we spent the next 3 days sitting around a table, working to establish three different standards by assessing two different exams and reviewing, scoring, debating, rescoring, and debating some more. Prior to my arrival, I was expecting the committee to be a significantly larger group, but I found the small size allowed for great discussions.
Day 1 consisted of becoming familiar with the first exam and setting a standard of “minimally proficient” and “minimally advanced." We were presented with the task of identifying which question out of 125 would be the point where 2 out of 3 minimally proficient athletic trainers would start getting the questions wrong. And then we had to determine at which point 2 out of 3 minimally advanced athletic trainers would start to answer the questions wrong. We engaged in group breakouts, full group discussions, individual analysis, and hearty debate to guide ourselves to a consensus, allowing us to set both standards. At the end of day 1, we were told to get a good night’s rest because over the next two days, we would be setting the Cut Score Standard for the national exam. The night was rounded out with a group dinner with the staff from the BOC at a local restaurant.
Day 2 started off with an instant flashback to sitting for the exam and being in class. We were given the BOC exam to take in order to get a feel for the content and language used. We were instructed to take breaks, walk around, and talk, as it was not being graded and it was not an official test. But 5 minutes into looking at the exam, and you could hear a pin drop in the room. We all automatically went into test-taking mode: no talking, no restroom breaks—we even delayed going to lunch until we had completed…it was test taking time! After we finished “taking” the exam, we were handed the answers and the debate began! We went through the exam a total of 4 times on day 2, and by the end of the day, we were all pretty well spent.
Day 3 mimicked the previous day, with the exception that we did not have to “take” the exam; rather, we picked up where we left off. We went through one more solid analysis of each question, debating language, answers, and whether or not an entry-level athletic trainer WOULD get the answer correct, not SHOULD get the answer correct. We adjourned early on day 3 once we had completed the task. With half a day free to do whatever we wanted, we all went our separate ways for the afternoon and met up for dinner that evening. Once at dinner, I asked around to see what everyone did with their free afternoon. The answer, more often than not, was: “I took a nap! I can’t remember the last time I had time to take a nap…and it was wonderful!” A nap: the perfect reward for staring at, thinking about, debating, and assessing 250 test questions and formats for 3 straight days!
Reflecting back upon this trip and experience, I am extremely honored that I was asked to and happy that I was able to participate in this process. I met colleagues from all over the US and was able to immerse myself in an environment filled with consistent high-level thinking and discussion revolving around our profession. Knowing the behind-the-scenes of what an individual question goes through in order to be able to appear on the exam is astounding! If asked again, there would be no hesitation: yes, I will volunteer to sit and take, analysis, debate, and do it all over again for an exam!
Written By: Jenna Street, MS, ATC, PES