Jarnel Williams, CSC Docket No. 2023-2258

BackgroundJarnel Williams appeals his score on the oral portion of the promotional examination for Fire Captain (PM2329C), Hillside. It is noted that the appellant failed the subject examination.

This two-part examination consisted of a written multiple-choice portion and an oral portion. Candidates were required to pass the written portion of the examination, and then were ranked on their performance on both portions of the examination. The test was worth 80 percent of the final score and seniority was worth the remaining 20 percent. Of the test weights, 35.90% of the score was the written multiple-choice portion, 22.04% was the technical score for the evolving exercise, 7.45% was the supervision score for the evolving exercise, 5.71% was the oral communication score for the evolving exercise, 23.20% was the technical score for the arriving exercise, 5.71% was the oral communication score for the arriving exercise.

The oral portion of the Fire Captain examination consisted of two scenarios: a fire scene simulation with questions designed to measure the knowledge of safe rescue tactics and procedures to safeguard citizens, supervision of fire fighters and the ability to assess fire conditions and hazards in an evolving incident on the fireground (Evolving Scenario); and a fire scene simulation designed to measure the knowledge of safe rescue tactics and procedures to safeguard citizens, supervision of firefighters and the ability to plan strategies and tactics based upon a building’s structure and condition (Arriving Scenario). Knowledge of supervision was measured by a question in the Evolving Scenario, and was scored for that scenario. For the Evolving Scenario, candidates were provided with a 15-minute preparation period, and candidates had 10 minutes to respond. For the Arriving Scenario, a five-minute preparation period was given, and candidates had 10 minutes to respond.

The candidates’ responses were scored on technical knowledge and oral communication ability. Prior to the administration of the exam, a panel of Subject Matter Experts (SMEs) determined the scoring criteria, using generally approved fire command practices, firefighting practices, and reference materials. Scoring decisions were based on SME-approved possible courses of action (PCAs) including those actions that must be taken to resolve the situation as presented. Only those oral responses that depicted relevant behaviors that were observable and could be quantified were assessed in the scoring process. It is noted that candidates were told the following prior to beginning their presentations for each scenario: “In responding to the questions, be as specific as possible. Do not assume or take for granted that general actions will contribute to your score.”

Candidates were rated on a five-point scale, with 5 as the optimal response, 4 as a more than acceptable passing response, 3 as a minimally acceptable passing response, 2 as a less than acceptable response, and 1 as a much less than acceptable response. For each of the scenes, and for oral communication, the requirements for each score were defined.

On the Evolving Scenario, the appellant scored a 3 on the technical component, a 2 on the supervision component, and a 3 on the oral communication component. On the Arriving Scenario, the appellant scored a 2 on the technical component and a 4 on the oral communication component.

Appellant's argumentThe appellant challenges his scores for the supervision component of the Evolving Scenario and the technical component of the Arriving Scenario. As a result, the appellant’s test material, video, and a listing of PCAs for the scenarios were reviewed.

The supervision component of the Evolving Scenario provides that during overhaul procedures, the candidate notices a firefighter joking around with another firefighter, behaving recklessly, removing his self-contained breathing apparatus (SCBA) prematurely and performing actions sloppily with his attention not fully on the matter at hand. The question asks what actions the candidate should take to handle this both on-scene and back at the firehouse.

The assessor awarded the appellant a score of 2 on the supervision component of the Evolving Scenario, based, in part, on a finding that the appellant failed to identify a sizeable number of PCAs, including, in part, opportunities to ensure the rest of the crew was wearing personal protective equipment (PPE), meeting with the firefighter back at the station and documenting all findings and actions/getting a written statement from the firefighter. On appeal, the appellant questions his rating of 2, given that his review sheet did not contain any “failed to” statements (i.e., did not indicate that he missed any mandatory responses. Regarding the PCA of ensuring that the rest of his crew was wearing PPE, the appellant maintains that he covered this during a specified point during his presentation. The appellant maintains that he covered the PCAs of meeting with the firefighter back at the station and documenting all findings by stating that he would work with the firefighter to help him improve, and that he informed the firefighter, corrected him, trained him, orally reprimanded him and informed the chief.

In reply, at the outset, with the scoring standard for the subject examination, the supervision component PCAs were equally weighted, rather than categorized as mandatory or additional responses. As a result, the absence of any “candidate failed to” statements on the appellant’s Evolving Scenario supervision component review sheet cannot be said to suggest any error in the scoring of the supervision component. Regarding the PCA of ensuring that the rest of his crew was wearing PPE, the statement the appellant cited came during his technical component response in the context of the actions and orders he would give to fully address the fire itself. He did not give any indication that he would subsequently ensure that his crew was still wearing PPE after seeing a crew member remove their SCBA during the salvage and overhaul operations later in the response. Therefore, he was properly denied credit for the supervision component PCA of ensuring the rest of the crew was wearing proper PPE after the events described in the supervision component prompt. As to the PCAs of meeting with the firefighter back at the station and documenting all findings and actions/getting a written statement from the firefighter, it is noted that the appellant indicated that he would remove himself and the firefighter from overhaul operations and have the incident commander send two other members in his place to complete overhaul while he orally reprimanded the firefighter on scene and explained proper safety protocols. However, it would not be practicable to conduct such a meeting on the fireground, particularly as the candidate would be expected to review pertinent records prior to a meeting with the candidate, so as to determine the appropriate progressive discipline and any other potential corrective actions following the incident at issue. Further, interviewing the firefighter about his actions and training would be an integral part of such a meeting and it would be imprudent to step away from operations on scene for the length of time needed to conduct such an interview. Moreover, the appellant’s stated course of action was not an acceptable alternative to documenting all findings and actions and/or getting a written statement from the firefighter following this incident. It was important for the appellant to ensure that there was adequate documentation of the relevant findings, actions and statements from the firefighter to support a thorough and fair departmental review of the appropriate progressive discipline and/or remedial training that should follow this incident. To wit, if the firefighter had a history of prior discipline, more significant discipline than an oral reprimand may have been warranted here. Accordingly, the appellant has failed to sustain his burden of proof with respect to the supervision component of the Evolving Scenario and his score of 2 is affirmed.

The Arriving Scenario involves an incident where the candidate is a first-level supervisor who will be the highest-ranking officer and incident commander at a gas station fire. Upon arrival, a gas station employee reports that a portable kerosene heater in the gas station’s convenience store tipped over and the fire spread quickly. Additionally, another employee is trapped inside. Question 1 directed candidates to perform their initial report to the camera as they would upon arrival at the incident. Question 2 directed candidates to give their initial actions and then describe in detail the specific procedures required to safely remove the victims.

The SME awarded the appellant a score of 2 on the technical component of the Arriving Scenario based on a finding that the appellant failed to identify the mandatory response of addressing smoke/fire showing and missed a number of additional opportunities, including, in part, the opportunity to begin offensive operations with Question 1. On appeal, the appellant avers that he covered starting an offensive attack by stating that he would use a 2 ½’ hoseline for reach and penetration, in coordination with truck company operations, including the offensive tactic of cutting and venting the roof and making it tenable for the rescue that took place.

In reply, candidates were supposed to indicate that they were beginning offensive operations as part of their initial report using proper radio protocols in Question 1. The appellant’s statements about using a 2 ½” hoseline for reach and penetration, and about cutting and venting the roof were not part of his initial report. Rather, they came during Question 2. Further, the Division of Test Development, Analytics and Administration (TDAA) advises that the appellant’s statement about using a 2 ½” hoseline for reach and penetration could be interpreted as a statement about conducting a defensive operation, as a member operating such a line could be doing so on the exterior of the structure using reach and penetration to keep a safe distance from the outside of the structure. As such, they were not sufficient to cover the PCA at issue. Accordingly, the appellant has failed to sustain his burden of proof regarding this PCA. However, TDAA advises that the appellant was not initially credited with the additional response of requesting the police department, but that upon review of the appellant's presentation on appeal, it has determined that he should have been credited with this PCA. Nevertheless, TDAA advises that even with this scoring change, the appellant’s Arriving Scenario technical component remaining would remain 2. The Commission agrees with TDAA’s assessment and affirms the appellant’s Arriving Scenario technical component score of 2.

CONCLUSION

A thorough review of the appellant’s submissions and the test materials indicates that the decision below is amply supported by the record and, except as indicated above, the appellant has failed to meet his burden of proof in this matter.

ORDER

Therefore, it is ordered that this appeal be granted in part and that appropriate agency records be revised to reflect the above-noted adjustment to the appellant’s scoring records for the technical component of the Arriving Scenario, but that the appellant’s overall score for this component remain unchanged at 2. It is further ordered that the appellant’s appeal of his Evolving Scenario supervision component score be denied.

ClosingThis is the final administrative determination in this matter. Any further review should be pursued in a judicial forum.