Thomas Flores, CSC Docket No. 2024-1101

BackgroundThomas Flores appeals his score on the oral portion of the promotional examination for Fire Captain (PM5018D), Camden. It is noted that the appellant achieved a passing score of 82.710 and ranks 20th on the subject eligible list.

This two-part examination consisted of a written multiple-choice portion and an oral portion. Candidates were required to pass the written portion of the examination, and then were ranked on their performance on both portions of the examination. The test was worth 80 percent of the final score and seniority was worth the remaining 20 percent. Of the test weights, 35.90% of the score was the written multiple-choice portion, 22.04% was the technical score for the evolving exercise, 7.45% was the supervision score for the evolving exercise, 5.71% was the oral communication score for the evolving exercise, 23.20% was the technical score for the arriving exercise, 5.71% was the oral communication score for the arriving exercise.

The oral portion of the first-level Fire Captain examination consisted of two scenarios: a fire scene simulation with questions designed to measure the knowledge of safe rescue tactics and procedures to safeguard citizens, supervision of fire fighters and the ability to assess fire conditions and hazards in an evolving incident on the fireground (Evolving Scenario); and a fire scene simulation designed to measure the knowledge of safe rescue tactics and procedures to safeguard citizens, supervision of firefighters and the ability to plan strategies and tactics based upon a building’s structure and condition (Arriving Scenario). Knowledge of supervision was measured by a question in the Evolving Scenario, and was scored for that scenario. For the Evolving Scenario, candidates were provided with a 15-minute preparation period, and candidates had 10 minutes to respond. For the Arriving Scenario, a five-minute preparation period was given, and candidates had 10 minutes to respond.

The candidates’ responses were scored on technical knowledge and oral communication ability. Prior to the administration of the exam, a panel of Subject Matter Experts (SMEs) determined the scoring criteria, using generally approved fire command practices, firefighting practices, and reference materials. Scoring decisions were based on SME-approved possible courses of action (PCAs) including those actions that must be taken to resolve the situation as presented. For a performance to be acceptable, other than for oral communication, a candidate needed to present the mandatory courses of action for that scenario. Only those oral responses that depicted relevant behaviors that were observable and could be quantified were assessed in the scoring process. Scores were then converted to standardized scores.

Candidates were rated on a five-point scale, with 5 as the optimal response, 4 as a more than acceptable passing response, 3 as a minimally acceptable passing response, 2 as a less than acceptable response, and 1 as a much less than acceptable response. For each of the scenes, and for oral communication, the requirements for each score were defined.

On the Evolving Scenario, the appellant scored a 4 on the technical component, a 3 on the supervision component, and a 5 on the oral communication component. On the Arriving Scenario, the appellant scored a 4 on the technical component and a 4 on the oral communication component.

Appellant's argumentThe appellant challenges his scores for the technical and supervision components of the Evolving Scenario, and the oral component of the Arriving Scenario. As a result, the appellant’s test material, video, and a listing of PCAs for the scenarios were reviewed.

The Evolving Scenario involved a reported fire at a two-family wood-frame residence where the candidate is the first-level supervisor of Engine 2 and that upon the candidate’s arrival, Battalion 2 is establishing command. Question 1 asks the candidate, as the supervisor of Engine 2, what orders they will give their crew to complete their orders from the incident commander (IC). Question 2 states that Firefighters Franco and Phillips are conducting fire suppression operations and that Firefighter Phillips slips down the stairs and his improperly secured helmet is dislodged. He proceeds to hit his head hard and is unresponsive. Question 2 then asks what actions the candidate should take or ensure are taken.

The SME awarded the appellant a score of 4 on the technical component of the Evolving Scenario based upon a finding that the appellant failed to indicate that he would instruct the crew to stay low as they advance. On appeal, the appellant argues that John Norman, Fire Officer’s Handbook of Tactics 42 (5th ed. 2019) discusses getting down low in the context of rollovers but that such conditions were not present upon arrival. The appellant presents that with the fire conditions on his priorities were to locate, confine and extinguish the fire as quickly and safely as possible. He adds that he executed key objectives, particularly performing offensive operations to get water flowing as soon as possible to the main body of the fire at a point between the fire and the likely locations of any occupants in accordance with Norman, supra at 380, and International Association of Fire Chiefs and National Fire Protection, Fundamentals of Fire Fighter Skills and Hazardous Materials Response 650 (4th ed. 2019).

In reply, upon review of the appellant’s presentation on appeal, the Division of Test Development, Analytics and Administration (TDAA) has determined that the appellant should have been credited with the additional response of coordinating with other companies. However, with regard to the PCA of instructing the crew to stay low as they advance, TDAA presents that this is a common fireground tactic. In this regard, TDAA proffers that firefighters and trapped civilians are instructed to stay low to the ground because it will keep them away from the heat and toxins rising as combustion occurs. The Civil Service Commission agrees with this assessment and notes that even with this scoring change, the appellant’s rating of 4 on the technical component of the Evolving Scenario remains unchanged.

The supervision component of the Evolving Scenario indicates that the candidate is dispatched to the same residence again later in the same shift. The building has rekindled and the candidate is tasked with extinguishing the fire. Afterward, it was discovered that the rekindling was due to improper overhaul by the candidate’s crew members. The prompt then asks what specific actions the candidate should take.

On the supervision component of the Evolving Scenario, the assessor awarded the appellant a score of 3, based upon a finding that the appellant failed to review applicable standard operating procedures/standard operating guidelines (SOPs/SOGs), and to check their crew’s training and personnel records. On appeal, the appellant maintains that he covered reviewing applicable SOPs/SOGs by indicating at a stated point that he would “go over the files of overhaul.” Additionally, the appellant argues that he covered reviewing his crew’s training and personnel records by stating at a specified point during his presentation that he would go back to his crew members’ individual files to review if anything similar had occurred previously.

In reply, the appellant’s statement about “go[ing] over the files of overhaul” was an ambiguous statement that was insufficient to cover the PCA of reviewing applicable SOPs and/or SOGs. Significantly, “files” without context does not necessarily refer to the procedures, guidelines, rules, regulations or anything similar and could refer, for example to documentation from a prior event, rather than the SOPs or SOGs contemplated by the scenario. Accordingly, the appellant was appropriately denied credit for this PCA. As to the PCA of checking the crew’s training and personnel records, the appellant stated that he would “go back to [his] crew members’ each and [sic] individual files to review if this has occurred again, or before . . . I’m sorry.” This statement was too general to specifically address the need to review the training and disciplinary histories of his crew members, which would inform the appropriate corrective action and retraining. As such, he was properly denied credit for this PCA. Accordingly, the appellant has failed to sustain his burden of proof with respect to the supervision component of the Evolving Scenario and his score of 3 is affirmed.

On the oral communication component of the Arriving Scenario, the assessor awarded the appellant a score of 4, based upon a finding that the candidate displayed a minor weakness in organization, as evidenced by multiple brief and long pauses within his response. On appeal, the appellant states that because none of the source materials in the 2022 1st Level Fire Supervisor Orientation Guide directly addressed a scenario involving lithium ion bike fires,1 he paused to review his notes to ensure he covered all necessary actions.

In reply, a review of the appellant’s presentation confirms the assessor’s determination that the appellant displayed a minor weakness in organization by pausing multiple time throughout his presentation. Although the appellant maintains that the scenario was not squarely addressed in any of the sources on the suggested reading list in the 2022 1st Level Fire Supervisor Orientation Guide, it is noted that candidates were advised:

[W]hile the justification for correct responses may be drawn from materials on the reading list, it is not limited to that material.

Justification may also be based on the collective Fire Service expertise of Subject Matter Experts. When justification is based on experience, care is taken to ensure that correct answers do not conflict with the sources on the reading list.

As such, even assuming, arguendo, that the suggested reading list materials did not directly cover the scenario at issue, it did not negate its validity or the need for the appellant to provide a coherent, organized response. Since the appellant’s presentation on an area of knowledge relevant to the Fire Service was punctuated by multiple pauses, his rating of 4 for oral communication, corresponding to a “more than acceptable passing response,” was appropriate and is affirmed.

1 The appellant presents, however, that International Association of Fire Chiefs and National Fire

Protection, Fundamentals of Fire Fighter Skills and Hazardous Materials Response (4th ed. 2019) discusses fire suppression on electric battery and hybrid electric vehicles.

CONCLUSION

A thorough review of the appellant’s submissions and the test materials indicates that the decision below is amply supported by the record and, except as indicated above, the appellant has failed to meet his burden of proof in this matter.

ORDER

Therefore, it is ordered that this appeal be granted in part and that appropriate agency records be revised to reflect the above-noted adjustments to the appellant’s scoring records for the technical component of the Evolving Scenario, but that the appellant’s overall score for this component remain unchanged at 4. It is further ordered that the appellant’s appeal of his Evolving Scenario supervision component and Arriving Scenario oral communication component scores be denied.

ClosingThis is the final administrative determination in this matter. Any further review should be pursued in a judicial forum.