Steven White, CSC Docket No. 2023-2523

BackgroundSteven White appeals his score on the oral portion of the promotional examination for Fire Captain (PM2324C), East Orange. It is noted that the appellant failed the subject examination.

This two-part examination consisted of a written multiple-choice portion and an oral portion. Candidates were required to pass the written portion of the examination, and then were ranked on their performance on both portions of the examination. The test was worth 80 percent of the final score and seniority was worth the remaining 20 percent. Of the test weights, 35.90% of the score was the written multiple-choice portion, 22.04% was the technical score for the evolving exercise, 7.45% was the supervision score for the evolving exercise, 5.71% was the oral communication score for the evolving exercise, 23.20% was the technical score for the arriving exercise, 5.71% was the oral communication score for the arriving exercise.

The oral portion of the Fire Captain examination consisted of two scenarios: a fire scene simulation with questions designed to measure the knowledge of safe rescue tactics and procedures to safeguard citizens, supervision of fire fighters and the ability to assess fire conditions and hazards in an evolving incident on the fireground (Evolving Scenario); and a fire scene simulation designed to measure the knowledge of safe rescue tactics and procedures to safeguard citizens, supervision of firefighters and the ability to plan strategies and tactics based upon a building’s structure and condition (Arriving Scenario). Knowledge of supervision was measured by a question in the Evolving Scenario, and was scored for that scenario. For the Evolving Scenario, candidates were provided with a 15-minute preparation period, and candidates had 10 minutes to respond. For the Arriving Scenario, a five-minute preparation period was given, and candidates had 10 minutes to respond.

The candidates’ responses were scored on technical knowledge and oral communication ability. Prior to the administration of the exam, a panel of Subject Matter Experts (SMEs) determined the scoring criteria, using generally approved fire command practices, firefighting practices, and reference materials. Scoring decisions were based on SME-approved possible courses of action (PCAs) including those actions that must be taken to resolve the situation as presented. Only those oral responses that depicted relevant behaviors that were observable and could be quantified were assessed in the scoring process. It is noted that candidates were told the following prior to beginning their presentations for each scenario: “In responding to the questions, be as specific as possible. Do not assume or take for granted that general actions will contribute to your score.”

Candidates were rated on a five-point scale, with 5 as the optimal response, 4 as a more than acceptable passing response, 3 as a minimally acceptable passing response, 2 as a less than acceptable response, and 1 as a much less than acceptable response. For each of the scenes, and for oral communication, the requirements for each score were defined.

On the Evolving Scenario, the appellant scored a 2 on the technical component, a 2 on the supervision component, and a 3 on the oral communication component. On the Arriving Scenario, the appellant scored a 2 on the technical component and a 3 on the oral communication component.

The appellant appeals his scores for the supervision component of the Evolving Scenario and the technical component of the Arriving Scenario. As a result, the appellant’s test material, video, and a listing of PCAs for the scenarios were reviewed.

The supervision component of the Evolving Scenario states that a victim at the scene of the Evolving Scenario was not found in time and perished from smoke inhalation. It indicates that this is the first time that one of the Fire Fighters serving under the candidate has experienced anything like this and that the Fire Fighter is taking this loss very hard. It then asks what actions the candidate should take to handle this situation.

The assessor awarded the appellant a score of 2 for the supervision component of the Evolving Scenario based upon a finding that the appellant missed a number of PCAs, including, in part, providing a critical incident stress debriefing (CISD) to the Fire Fighter. On appeal, the appellant argues that he covered the PCA at issue by offering counseling to the Fire Fighter and the other members of his crew and following up with his members.

In reply, upon review of the appellant’s presentation and appeal, the Division of Test Development, Analytics and Administration (TDAA) has determined that the appellant should have been credited with the PCA of providing a CISD. TDAA advises that with the award of this additional PCA, the appellant’s score for the supervision component of the Evolving Scenario should be raised from 2 to 3. The Commission agrees with this assessment.

The Arriving Scenario involves a scenario where the appellant is the first-level fire supervisor at a fire in a college dormitory where the candidate is the highest- ranking officer on the scene and will serve as the incident commander. Question 1 asks the candidate to describe, in detail, their size-up of this incident. Question 2 asks what actions, orders and requests are needed to fully address this incident.

The SME awarded the appellant a score of 2 for the technical component of the Arriving Scenario, based upon a finding that he missed a number of mandatory and additional responses, including, in part, the acknowledgement during his size-up of the unknown number of potential victims inside (life hazard and potential victims), a mandatory response to Question 1. On appeal, the appellant contends that he covered the subject mandatory response by stating that he would have a “primary search and a secondary search (for life hazard).”

In reply, a review of the appellant’s presentation fails to demonstrate that he should have been credited with the mandatory response at issue. Since the response at issue was a mandatory response to Question 1, it had to be identified during the portion of the appellant’s presentation covering Question 1. A review of the appellant's response confirms that he did not mention the unknown number of victims or the life hazard during the portion of his presentation addressing Question 1. It is further noted that ordering primary and secondary searches were distinct PCA in response to Question 2 for which the appellant received credit.

Finally, TDAA advises that even with the scoring change for the supervision component of the Evolving Scenario, the appellant still failed the subject examination.

CONCLUSION

A thorough review of the appellant’s submissions and the test materials indicates that the decision below is amply supported by the record and, except as noted above, the appellant has failed to meet his burden of proof in this matter.

ORDER

Therefore, it is ordered that this appeal be granted in part and that the appellant’s score on the supervision component of the Evolving Scenario be raised from 2 to 3 with retroactive effect. It is further ordered that the appellant’s appeal of his Arriving Scenario technical component score be denied.

ClosingThis is the final administrative determination in this matter. Any further review should be pursued in a judicial forum.