Michael Martini, CSC Docket No. 2023-2401

BackgroundMichael Martini appeals his score on the oral portion of the promotional examination for Fire Captain (PM2344C), Perth Amboy. It is noted that the appellant failed the subject examination.

This two-part examination consisted of a written multiple-choice portion and an oral portion. Candidates were required to pass the written portion of the examination, and then were ranked on their performance on both portions of the examination. The test was worth 80 percent of the final score and seniority was worth the remaining 20 percent. Of the test weights, 35.90% of the score was the written multiple-choice portion, 22.04% was the technical score for the evolving exercise, 7.45% was the supervision score for the evolving exercise, 5.71% was the oral communication score for the evolving exercise, 23.20% was the technical score for the arriving exercise, 5.71% was the oral communication score for the arriving exercise.

The oral portion of the Fire Captain examination consisted of two scenarios: a fire scene simulation with questions designed to measure the knowledge of safe rescue tactics and procedures to safeguard citizens, supervision of fire fighters and the ability to assess fire conditions and hazards in an evolving incident on the fireground (Evolving Scenario); and a fire scene simulation designed to measure the knowledge of safe rescue tactics and procedures to safeguard citizens, supervision of firefighters and the ability to plan strategies and tactics based upon a building’s structure and condition (Arriving Scenario). Knowledge of supervision was measured by a question in the Evolving Scenario, and was scored for that scenario. For the Evolving Scenario, candidates were provided with a 15-minute preparation period, and candidates had 10 minutes to respond. For the Arriving Scenario, a five-minute preparation period was given, and candidates had 10 minutes to respond.

The candidates’ responses were scored on technical knowledge and oral communication ability. Prior to the administration of the exam, a panel of Subject Matter Experts (SMEs) determined the scoring criteria, using generally approved fire command practices, firefighting practices, and reference materials. Scoring decisions were based on SME-approved possible courses of action (PCAs) including those actions that must be taken to resolve the situation as presented. Only those oral responses that depicted relevant behaviors that were observable and could be quantified were assessed in the scoring process. It is noted that candidates were told the following prior to beginning their presentations for each scenario: “In responding to the questions, be as specific as possible. Do not assume or take for granted that general actions will contribute to your score.”

Candidates were rated on a five-point scale, with 5 as the optimal response, 4 as a more than acceptable passing response, 3 as a minimally acceptable passing response, 2 as a less than acceptable response, and 1 as a much less than acceptable response. For each of the scenes, and for oral communication, the requirements for each score were defined.

On the Evolving Scenario, the appellant scored a 2 on the technical component, a 1 on the supervision component, and a 3 on the oral communication component. On the Arriving Scenario, the appellant scored a 4 on the technical component and a 3 on the oral communication component.

Appellant's argumentThe appellant challenges his scores for the oral communication, technical and supervision components of the Evolving Scenario, and the oral communication component of the Arriving Scenario. As a result, the appellant’s test material, video, and a listing of PCAs for the scenarios were reviewed.

The Evolving Scenario involves the response to a fire reported at a high school building where the candidate is the first-level supervisor of the first responding ladder company, Ladder 1. Battalion 6 and two engine companies are arriving with the candidate and Battalion 6 has established command. As the candidate arrives on scene, the students and the faculty are in the process of evacuating to the side of the road opposite the high school. The principal tells the incident commander (IC) that the fire started in a computer lab on the first floor. The IC orders the candidate to perform a primary search with ventilation and he requests two additional alarms. Question 1 asks the candidate, as the supervisor of Ladder 1, to describe, in detail, what orders they should give their crew to carry out their assignment from the IC. The prompt for Question 2 provides that during the candidate’s search, their crew finds two adult victims on the first floor outside of the involved room, who are unconscious and breathing shallowly. It also advises that one minute before this discovery, the IC had radioed that Ladder 2 had arrived. Question 2 then asks the candidate to give their initial actions and to detail the specific procedures required to safely remove victims.

The assessor awarded the appellant an oral communication component score of 3, based upon a finding that the appellant displayed minor weaknesses in organization and brevity. Specifically, regarding organization, the assessor cited pauses by the appellant to gather his thoughts during his presentation. In terms of brevity, the assessor stated that the appellant’s response was too brief to demonstrate his oral communicative ability. On appeal, the appellant expresses concern that he was “misgraded due to the subject matter expert’s focus on oral communication instead of the fireground knowledge and actions for the response.” He denies pausing to look at his notes, maintaining that he recognized that doing so would have made it more likely that he would be docked points for turning his attention away from the camera and the audience.

In reply, the appellant suggestion that he was “misgraded due to the subject matter expert’s focus on oral communication instead of the fireground knowledge and actions” is without merit. The assessor who graded the appellant’s Evolving Scenario technical component was not responsible for scoring the appellant's Evolving Scenario oral communication score. Rather, the scoring of the appellant’s oral communication component was performed by a different examiner. Thus, a suggestion that his technical component score was somehow artificially low because the assessor was “focus[ed]” on the appellant’s oral communication performance is inaccurate. Additionally, a review of the appellant’s Evolving Scenario presentation confirms the accuracy of the assessor’s findings with respect to the appellant’s oral communication. Notably, there were multiple instances where the appellant paused mid-sentence to gather his thoughts, rendering his delivery stilted at times. Moreover, although the appellant denies stopping to review his notes, the recording of his presentation shows at least one lengthy pause to review his notes approximately 2 minutes and 35 minutes into his presentation. Accordingly, the appellant has failed to sustain his burden of proof and his oral communication score of 3 on the Evolving Scenario is affirmed.

On the technical component of the Evolving Scenario, the SME awarded the appellant a score of 2, based upon a finding that he missed multiple mandatory responses, including, in part, coordinating with the engine company/hoseline operations. On appeal, the appellant contends that it was “unfair” that the SME made such a finding because the prompt merely asked for the roles he would perform as a truck company officer and did not specify that he was expected to communicate with an engine company to extinguish the fire. He further submits that he stated that the fire was extinguished and clearly stated the actions that he would take as a truck company officer.

In reply, a review of the appellant’s presentation confirms the assessor’s findings regarding the appellant’s Evolving Scenario technical component score. Although the appellant challenges the validity of the PCA of coordinating with the engine company/hoseline operations, it is observed that with the appellant’s crew performing a primary search in a school with an active fire, there was an inescapable need to coordinate with the engine company/hoseline operations to ensure scene safety. As noted by Vincent Dunn, Safety and Survival on the Fireground 66-67 (2nd ed. 2015) regarding searching and venting alongside an advancing hose team:

When the hose stream is discharging a quarter ton (180 gallons) of water per minute, and if horizontal outside window venting is coordinated with the hoseline advance, there is less chance of a firefighter being caught or trapped, flashover, fire extension, or a wind-driven fire What we want to hear on the radio during a structure fire is, “Engine 1, Ladder 1, let me know when you get water.” “Ladder 1, Engine 1 go ahead and vent.

We are moving in.”

Based upon the foregoing, the appellant has failed to sustain his burden of proof and his technical component score of 2 is affirmed.

The supervision component of the Evolving Scenario states that while at rehab, the candidate notices that a member of their crew is not wearing their radio. When the candidate asks the firefighter where his radio is, he reports that he left it in his locker. It then asks what actions the candidate should take immediately and back at the firehouse.

On the supervision component of the Evolving Scenario, the assessor awarded the appellant a score of 1, based upon a finding that the appellant missed a significant number of PCAs, including, in part, opportunities to review the firefighter’s personnel/training file, reassign/replace the firefighter, document any actions taken, and monitor the firefighter’s progress. On appeal, the appellant avers that “[t]here is no reason to review a training file for a radio that has been forgotten.” The appellant maintains that other actions the assessor indicated that the appellant failed to identify were covered under his statement that he would “follow[ ] what the chief officer recommend[ed].”

In reply, the review of the firefighter’s personnel/training file is essential here, as it would speak to the prior training and disciplinary action, if any, the firefighter had received. This information would inform what additional training and/or disciplinary action would be appropriate for this firefighter following this incident. Thus, the appellant’s contention that “[t]here is no reason to review a training file for a radio that has been forgotten” is erroneous and must be rejected. Further, the appellant’s argument his statement that he would “follow[ ] what the chief officer recommend[ed]” must be rejected for several key reasons. Critically, among the examples of work listed in the job specification for Fire Captain is “responsibility for employee evaluations and for effectively recommending the hiring, firing, promoting, demoting, and/or disciplining of employees” and, as noted in the 2022 1st Level Fire Supervisor Orientation Guide, the Evolving Scenario was designed, in part, to elicit responses associated with supervisory knowledge and abilities. Substantially deferring to the judgment of the chief officer is inconsistent with the aforementioned duties of a Fire Captain and does not demonstrate supervisory knowledge and abilities. Moreover, as noted above, candidates were told the following prior to beginning their presentations for each scenario: “In responding to the questions, be as specific as possible. Do not assume or take for granted that general actions will contribute to your score.” Here, the appellant’s statement about “following what the chief officer recommend[ed]” was too vague to award him credit for the specific PCAs at issue. As such, the appellant’s arguments regarding the supervision component of the Evolving Scenario must be rejected and his score of 1 must be sustained.

On the oral communication component of the Arriving Scenario, the assessor awarded the appellant a score of 3 based upon a finding that he displayed minor weaknesses in organization and word usage/grammar. In particular, with respect to organization, the assessor indicated that the appellant started sentences and then changed midway. Concerning word usage/grammar, the assessor stated that the appellant used grammatically incorrect sentences. On appeal, the appellant complains about his score being based upon oral communication, maintaining that “fireground knowledge and actions far outweigh minor grammatical errors on the fireground” and that “minor grammatical errors should not have such a great impact on [his] score on the promotional exam.” Regarding organization, he concedes that he “chang[ed his] tactics to better suit the fireground during the scenario,” but emphasizes that his “attention was never taken away from the camera or audience.”

In reply, oral communication performance can clearly distinguish candidates, including those delivering presentations with comparable technical details. To wit, it would be disingenuous to argue that the presentation of one candidate who spoke at a low rate of volume, had their speech punctuated by the frequent use of filler words like “ah” and “um,” rarely made eye contact with their audience and routinely made distracting hand gestures would be as understandable, effective and well-received as the presentation of another candidate who gave a speech with a comparable level of detail, but without these same oral communication issues. Given these considerations, the Civil Service Commission finds the appellant’s arguments, to the extent they challenge the validity of the scoring of nonverbal communication, are without merit and must be denied. Further, a review of the appellant’s presentation confirms the assessor’s findings. Accordingly, the appellant has failed to sustain his burden of proof and his score of 3 on the oral communication component of the Arriving Scenario is sustained.

CONCLUSION

A thorough review of the appellant’s submissions and the test materials indicates that the decision below is amply supported by the record, and that the appellant has failed to meet his burden of proof in this matter.

ORDER

Therefore, it is ordered that this appeal be denied.

ClosingThis is the final administrative determination in this matter. Any further review should be pursued in a judicial forum.