Angelina Tirado, CSC Docket No. 2024-1142

BackgroundAngelina Tirado appeals her score on the oral portion of the promotional examination for Fire Captain (PM5020D), Clifton. It is noted that the appellant achieved a passing score of 78.650 on the examination and ranks ninth on the subject eligible list.

This two-part examination consisted of a written multiple-choice portion and an oral portion. Candidates were required to pass the written portion of the examination, and then were ranked on their performance on both portions of the examination. The test was worth 70 percent of the final score and seniority was worth the remaining 30 percent. Of the test weights, 35.26% of the score was the written multiple-choice portion, 22.04% was the technical score for the evolving exercise, 2.79% was the oral communication score for the evolving exercise, 13.56% was the technical score for the administration exercise, 2.79% was the oral communication score for the administration exercise, 22.04% was the technical score for the arriving exercise, 2.79% was the oral communication score for the arriving exercise.

The oral portion of the second-level Fire Captain examination consisted of three scenarios: a fire scenario simulation with questions designed to measure knowledge and abilities in assessing risk (Evolving); a simulation designed to measure technical knowledge and abilities in administrative duties (Administration); and a fire scenario simulation designed to measure technical knowledge and abilities in strategy and attack plan and hazmat (Arriving). For the Evolving and

Administration scenarios, candidates were provided with a 25-minute preparation period for both, and candidates had 10 minutes to respond to each. For the Arriving scenario, a five-minute preparation period was given and candidates had 10 minutes to respond.

The candidates’ responses were scored on technical knowledge and oral communication ability. Prior to the administration of the exam, a panel of Subject Matter Experts (SMEs) determined the scoring criteria, using generally approved fire command practices, firefighting practices, and reference materials. Scoring decisions were based on SME-approved possible courses of action (PCAs) including those actions that must be taken to resolve the situation as presented. For a performance to be acceptable, other than for oral communication, a candidate needed to present the mandatory courses of action for that scenario. Only those oral responses that depicted relevant behaviors that were observable and could be quantified were assessed in the scoring process. Scores were then converted to standardized scores.

Candidates were rated on a five-point scale, with 5 as the optimal response, 4 as a more than acceptable passing response, 3 as a minimally acceptable passing response, 2 as a less than acceptable response, and 1 as a much less than acceptable response. For each of the scenes, and for oral communication, the requirements for each score were defined. It is noted that candidates were told the following prior to beginning their presentations for each scenario: “In responding to the questions, be as specific as possible. Do not assume or take for granted that general actions will contribute to your score.”

On the Evolving Scenario, the appellant scored a 1 on the technical component, and a 4 on the oral communication component. On the Administration Scenario, the appellant scored a 5 on the technical component and a 3 on the oral communication component. On the Arriving Scenario, the appellant scored a 5 on the technical component and a 4 on the oral communication component.

Appellant's argumentThe appellant challenges her score for the technical component of the Evolving Scenario. As a result, the appellant’s test material, video, and a listing of PCAs for the scenario were reviewed.

The Evolving Scenario involves the response to a fire reported at a home improvement store where the candidate, a second-level supervisor, will be the incident commander and will establish command on scene. Upon arrival, the candidate is greeted by the store manager, who reports that the fire is on the roof and indicates concern that the solar panels on the roof may be the cause of the fire. Question 1 then asks what the candidate’s actions, orders and requests are to fully address the incident. Question 2 advises that multiple crews are reporting that the roof is failing. It then asks the candidate what actions they should take to handle this new information.

The SME awarded the appellant a score of 1 on the technical component of the Evolving Scenario, based upon a finding that she failed to identify multiple mandatory and additional responses, including, in part, ordering the solar panel system to be de-energized/activate emergency shutoff, ordering crews to use master streams to attack the fire on the roof, ordering a personnel accountability report (PAR) and the opportunity to sound evacuation tones. On appeal, the appellant maintains that she stated she would conduct a PAR at two specified points. She also avers that she covered ordering crews to use master streams to attack fire on the roof by stating that 2.5-inch hoselines would be utilized along with apparatus in flanking positions and water would be used from the safety of aerials and tower ladders to locate, confine and extinguish fire from corner safe zones using volume, reach and penetration.

In reply, upon review of the appellant's appeal, the Division of Test Development, Analytics and Administration (TDAA) agrees that the appellant covered the mandatory response of ordering a PAR. However, TDAA has also determined that the appellant was erroneously awarded credit for the mandatory response of ordering an evacuation in response to Question 2. In this regard, TDAA notes that the appellant stated that she would order an “orderly withdrawal leaving tools and report to accountability to get a PAR of all members” and followed up shortly thereafter with a statement that she would send additional resources to the area and “proceed with offensive [operations].” Critically, calling for an “orderly withdrawal” is not the same as ordering an emergency evacuation. Specifically, as discussed in In the Matter of Daniel Dornacker, Jr., (CSC, decided October 19, 2016), “[o]rdering an ‘orderly withdrawal’ does not have the same sense of urgency of an evacuation, and is not the same.” John Norman, Fire Officer’s Handbook of Tactics 555 (5th ed. 2019) notes that “[a]n orderly withdrawal allows time to locate all of the members and ensure that everyone brings out their equipment. In an emergency evacuation, it’s drop your tools and run!” By calling for an “orderly withdrawal leaving tools,” the appellant gave a contradictory order, as “leaving tools” would be associated with an emergency evacuation, but an “orderly withdrawal” would involve less urgency. Her follow-up statement about sending additional resources to the area and “proceed[ing] with offensive” operations further indicated that she did not consider the withdrawal urgent and was problematic because doing so could have resulted in loss of life. Accordingly, based upon the foregoing, TDAA presents that the appellants score of 1 for the technical component of the Evolving Scenario should remain unchanged. The Civil Service Commission agrees with TDAA’s assessment on appeal.

CONCLUSION

A thorough review of the appellant’s submissions and the test materials indicates that the decision below is amply supported by the record and, except as indicated above, the appellant has failed to meet her burden of proof in this matter.

ORDER

Therefore, it is ordered that this appeal be granted in part and that appropriate agency records be revised to reflect the above-noted adjustment to the appellant’s scoring records for the technical component of the Evolving Scenario, but that the appellant’s overall score for this component remain unchanged at 1.

ClosingThis is the final administrative determination in this matter. Any further review should be pursued in a judicial forum.