Phillip McMahon, CSC Docket No. 2023-2348

BackgroundPhillip McMahon appeals his score on the oral portion of the promotional examination for Fire Lieutenant (PM2385C), Vineland. It is noted that the appellant passed the subject examination with a score of 83.430 and ranks fourth on the subject eligible list.

This two-part examination consisted of a written multiple-choice portion and an oral portion. Candidates were required to pass the written portion of the examination, and then were ranked on their performance on both portions of the examination. The test was worth 80 percent of the final score and seniority was worth the remaining 20 percent. Of the test weights, 35.90% of the score was the written multiple-choice portion, 22.04% was the technical score for the evolving exercise, 7.45% was the supervision score for the evolving exercise, 5.71% was the oral communication score for the evolving exercise, 23.20% was the technical score for the arriving exercise, 5.71% was the oral communication score for the arriving exercise.

The oral portion of the Fire Lieutenant examination consisted of two scenarios: a fire scene simulation with questions designed to measure the knowledge of safe rescue tactics and procedures to safeguard citizens, supervision of fire fighters and the ability to assess fire conditions and hazards in an evolving incident on the fireground (Evolving Scenario); and a fire scene simulation designed to measure the knowledge of safe rescue tactics and procedures to safeguard citizens, supervision of firefighters and the ability to plan strategies and tactics based upon a building’s structure and condition (Arriving Scenario). Knowledge of supervision was measured by a question in the Evolving Scenario, and was scored for that scenario. For the Evolving Scenario, candidates were provided with a 15-minute preparation period, and candidates had 10 minutes to respond. For the Arriving Scenario, a five-minute preparation period was given, and candidates had 10 minutes to respond.

The candidates’ responses were scored on technical knowledge and oral communication ability. Prior to the administration of the exam, a panel of Subject Matter Experts (SMEs) determined the scoring criteria, using generally approved fire command practices, firefighting practices, and reference materials. Scoring decisions were based on SME-approved possible courses of action (PCAs) including those actions that must be taken to resolve the situation as presented. Only those oral responses that depicted relevant behaviors that were observable and could be quantified were assessed in the scoring process. It is noted that candidates were told the following prior to beginning their presentations for each scenario: “In responding to the questions, be as specific as possible. Do not assume or take for granted that general actions will contribute to your score.”

Candidates were rated on a five-point scale, with 5 as the optimal response, 4 as a more than acceptable passing response, 3 as a minimally acceptable passing response, 2 as a less than acceptable response, and 1 as a much less than acceptable response. For each of the scenes, and for oral communication, the requirements for each score were defined.

On the Evolving Scenario, the appellant scored a 2 on the technical component, a 4 on the supervision component, and a 3 on the oral communication component. For the Arriving Scenario, the appellant scored a 5 on the technical component and a 3 on the oral communication component.

Appellant's argumentThe appellant challenges his scores for the technical component of the Evolving Scenario and the oral communication component of the Arriving Scenario. As a result, the appellant’s test material, video, and a listing of PCAs for the scenarios were reviewed.

The Evolving Scenario involved the response to a two-car motor vehicle accident in which the candidate, the first-level supervisor of Ladder 5, will be the incident commander (IC) and will establish command. The prompt advises that Engines 2 and 3 are delayed and will arrive in 10 minutes. Upon arrival, the driver of a crossover SUV that hit a sedan head-on is seen sitting on the ground by his vehicle, appearing dazed and having a large gash on his forehead. The sedan driver is still in her vehicle and appears to be unresponsive and entrapped. A dog is barking loudly in the backseat of the sedan. Question 1 asks what specific actions the candidate would take to address this incident. The prompt for Question 2 provides that a small pickup truck flying down the shoulder of the roadway slams into the back of parked Engine 3. The front end of the pickup truck has been smashed and Engine 3 skids forward a few inches. The driver stumbles out of the cab of the pickup truck, looking bewildered. Question 2 then asks what actions the candidate would take to address this development.

The SME awarded the appellant a score of 2 on the technical component of the Evolving Scenario, based upon a finding that he failed to ensure that the driver received emergency medical services (EMS) in response to Question 2 and missed a number of additional opportunities, including, in part, requesting air medical be put on standby in response to Question 1 and stabilizing the truck in response to Question 2. On appeal, the appellant argues that he covered the Question 2 PCAs of stabilizing the truck and ensuring that the driver received EMS at specified points during his presentation. He avers that requesting an air medical unit be put on standby was something that only EMS should ask for under their medical direction, rather than the fire department.

In reply, at the outset, it is noted that the appellant received credit for ordering the stabilization of both vehicles involved in the collision presented in Question 1, which was a distinct PCA from stabilizing the truck involved in the crash presented in Question 2 at issue with the instant appeal. Here, a review of the appellant’s presentation on appeal fails to demonstrate that he ordered the stabilization of the truck involved in the collision detailed in Question 2. Concerning the Question 2 PCA of ensuring that the truck driver received EMS attention, a review of the appellant’s appeal confirms that the appellant was properly denied credit for this PCA. In this regard, while the appellant called for additional EMS units during the portion of his response covering Question 2, he did not specifically state that he would ensure that the truck driver received medical attention or otherwise indicate how he would utilize these additional EMS units. As noted above, candidates were told the following prior to beginning their presentations for each scenario: “In responding to the questions, be as specific as possible. Do not assume or take for granted that general actions will contribute to your score.” Therefore, he was properly denied credit for the Question 2 PCA of ensuring that the truck driver received EMS attention. With respect to the PCA of requesting an air medical unit be put on standby, the appellant offers no source to support his contention that it would not be something an IC could request. As such, he has failed to sustain his burden of proving that this PCA, which was identified by the Division of Test Development, Analytics and Administration (TDAA), in consultation with subject matter experts, was invalid. Further, even assuming arguendo that it would be EMS making the direct request to put an air medical unit on standby, there is nothing in the record to suggest that it would be improper or imprudent for the candidate, as the IC, to discuss the need to put an air medical unit on standby with appropriate EMS personnel. Finally, upon review of the appellant's appeal, TDAA has determined that the scoring of two Evolving Scenario PCAs should be revised. Specifically, TDAA advises that the appellant should have been credited with the additional PCA of requesting animal control services to handle the dog on scene. However, TDAA also states that the appellant was erroneously credited with the Question 2 PCA of repositioning Ladder 5 and Engine 2. Based upon the foregoing, TDAA submits that the appellant’s Evolving Scenario technical score of 2 should remain unchanged. The Civil Service Commission (Commission) agrees with TDAA’s assessment.

On the oral communication component of the Arriving Scenario, the assessor awarded the appellant a score of 3, based upon findings that the appellant displayed a major weakness in word usage/grammar and a minor weakness in organization. Specifically, with regard to word usage/grammar, the assessor stated that the appellant frequently utilized filler words like “um” and “uh” through his response, used incorrect grammar, and repeated words and phrases. As to organization, the assessor indicated that the appellant displayed a minor weakness by taking nearly three minutes of response time to continuing writing notes and preparing for his presentation. On appeal, the appellant states that he did not know that using three minutes of presentation time for continued preparation would be problematic. He advises that he believed the extra time he used for preparation would be a wise and effective way to ensure that he would give a proper technical response without running out of time or rushing himself. He also contends that “[o]n previous exams this same technique was used and not penalized.”

In reply, at the outset, the appellant does not appear to contest the finding that he displayed a major weakness in word usage/grammar. Further, he does not dispute that he used three minutes of presentation time to continue his preparation. Accordingly, it is unnecessary to review the recording of the appellant’s Arriving Scenario presentation and the Commission need only consider the appellant’s written arguments. With the subject examination, the 2022 First-Level Fire Supervisor Orientation Guide clearly apprised candidates that they would have a five-minute preparation period inside of the test room for the Arriving Scenario and a separate 10-minute response period for that scenario. For each of the scenes, and for oral communication, the requirements for each score were defined. Here, a finding that the appellant displayed a minor weakness in organization by using the first three minutes of the Arriving Scenario response period to review his notes was consistent with the scoring standard for oral communication on the subject examination. Given this and the assessor’s uncontested determination that the appellant displayed a major weakness in word usage/grammar, it was appropriate for the assessor to find that the appellant provided an “acceptable passing response,” rather than an “optimal response,” and award him a score of 3 for the oral communication component of the Arriving Scenario.

CONCLUSION

A thorough review of the appellant’s submissions and the test materials indicates that the decision below is amply supported by the record and, except as indicated above, the appellant has failed to meet his burden of proof in this matter.

ORDER

Therefore, it is ordered that this appeal be granted in part and that appropriate agency records be revised to reflect the above-noted adjustments to the appellant’s scoring records for the technical component of the Evolving Scenario, but that the appellant’s overall score for this component remain unchanged at 2. It is further ordered that the appellant’s appeal of his Arriving Scenario oral communication component score be denied.

ClosingThis is the final administrative determination in this matter. Any further review should be pursued in a judicial forum.