Rusty Litterer, CSC Docket No. 2024-1083

BackgroundRusty Litterer appeals his score on the oral portion of the promotional examination for Fire Captain (PM5032D), New Brunswick. It is noted that the appellant failed the subject examination.

This two-part examination consisted of a written multiple-choice portion and an oral portion. Candidates were required to pass the written portion of the examination, and then were ranked on their performance on both portions of the examination. The test was worth 70 percent of the final score and seniority was worth the remaining 30 percent. Of the test weights, 35.26% of the score was the written multiple-choice portion, 22.04% was the technical score for the evolving exercise, 2.79% was the oral communication score for the evolving exercise, 13.56% was the technical score for the administration exercise, 2.79% was the oral communication score for the administration exercise, 22.04% was the technical score for the arriving exercise, 2.79% was the oral communication score for the arriving exercise.

The oral portion of the second-level Fire Captain examination consisted of three scenarios: a fire scenario simulation with questions designed to measure knowledge and abilities in assessing risk (Evolving); a simulation designed to measure technical knowledge and abilities in administrative duties (Administration); and a fire scenario simulation designed to measure technical knowledge and abilities in strategy and attack plan and hazmat (Arriving). For the Evolving and Administration scenarios, candidates were provided with a 25-minute preparation period for both, and candidates had 10 minutes to respond to each. For the Arriving scenario, a five-minute preparation period was given and candidates had 10 minutes to respond.

The candidates’ responses were scored on technical knowledge and oral communication ability. Prior to the administration of the exam, a panel of Subject Matter Experts (SMEs) determined the scoring criteria, using generally approved fire command practices, firefighting practices, and reference materials. Scoring decisions were based on SME-approved possible courses of action (PCAs) including those actions that must be taken to resolve the situation as presented. For a performance to be acceptable, other than for oral communication, a candidate needed to present the mandatory courses of action for that scenario. Only those oral responses that depicted relevant behaviors that were observable and could be quantified were assessed in the scoring process. Scores were then converted to standardized scores.

Candidates were rated on a five-point scale, with 5 as the optimal response, 4 as a more than acceptable passing response, 3 as a minimally acceptable passing response, 2 as a less than acceptable response, and 1 as a much less than acceptable response. For each of the scenes, and for oral communication, the requirements for each score were defined. It is noted that candidates were told the following prior to beginning their presentations for each scenario: “In responding to the questions, be as specific as possible. Do not assume or take for granted that general actions will contribute to your score.”

On the Evolving Scenario, the appellant scored a 2 on the technical component and a 2 on the oral communication component. On the Administration Scenario, the appellant scored a 4 on the technical component and a 4 on the oral communication component. On the Arriving Scenario, the appellant scored a 2 on the technical component and a 2 on the oral communication component.

Appellant's argumentThe appellant challenges his scores for the technical component of the Evolving Scenario.

The Evolving Scenario involves the response to a fire reported at a home improvement store where the candidate, a second-level supervisor, will be the incident commander and will establish command on scene. Upon arrival, the candidate is greeted by the store manager, who reports that the fire on the roof and indicates concern that the solar panels on the roof may be the cause of the fire. Question 1 then asks what the candidate’s actions, orders and requests are to fully address the incident. Question 2 advises that multiple crews are reporting that the roof is failing. It then asks the candidate what actions they should take to handle this new information.

The SMEs awarded the appellant a score of 2, based upon a finding that the appellant missed a significant number of mandatory and additional responses, including, in part, failing to order crews to use master streams to attack the fire on the roof in response to Question 1. On appeal, the appellant asserts that he ordered crews to use master streams at a specified point during his presentation.

In reply, upon review of the appellant’s appeal, the Division of Test Development, Analytics and Administration (TDAA) agrees that the appellant should have been credited with the mandatory response of ordering crews to use master streams to attack the fire on the roof. Based upon this change, TDAA advises that the appellant’s technical component score should be raised from 2 to 3, pursuant to the “flex rule.”1 Finally, TDAA advises that with the foregoing scoring changes, the appellant has achieved a passing score on the subject examination. The Civil Service Commission agrees with TDAA’s assessment on appeal. Accordingly, the appellant’s name shall be added to the subject eligible list with retroactive effect.

CONCLUSION

A thorough review of the appellant’s submissions and the test materials indicates that the decision below is amply supported by the record and that the appellant has met his burden of proof in this matter.

ORDER

Therefore, it is ordered that this appeal be granted and that the appellant’s score on the technical component of the Evolving Scenario be raised from 2 to 3 with retroactive effect. It is further ordered that, since the appellant passed the subject examination based upon the foregoing scoring change, that the appellant’s name be added to the Fire Captain (PM5032D), New Brunswick eligible list with retroactive effect.

ClosingThis is the final administrative determination in this matter. Any further review should be pursued in a judicial forum.

1 Generally, candidates must identify all mandatory responses to receive, at minimum, a score of 3.

However, a score of 3 may also be achieved via the “flex rule,” where a candidate provides many additional responses, but does not give a mandatory response. However, a score higher than a 3 cannot be provided utilizing the flex rule.