Jason Brown, CSC Docket No. 2025-1484

BackgroundJason Brown appeals his score on the oral portion of the promotional examination for Battalion Fire Chief (PM2230F), Winslow. It is noted that the appellant achieved a passing score of 77.060 on the subject examination and ranks third on the subject eligible list.

This two-part examination consisted of an integrated system of simulations designed to generate behavior similar to that required for success on the job. The first part consisted of multiple-choice items that measured specific work components identified and weighted by the job analysis. The second part consisted of three oral scenarios: Supervision, Administration and Incident Command. The examination was based on a comprehensive job analysis conducted by the Civil Service Commission (Commission), which identified the critical areas of the job. The weighting of the test components was derived from the job analysis data. It is noted that candidates were told the following prior to beginning their presentations for each scenario: “In responding to the questions, be as specific as possible. Do not assume or take for granted that general actions will contribute to your score.”

Each candidate in a given jurisdiction was scored by a team of three different Subject Matter Experts (SMEs), who were trained in current technical scoring procedures. Each of these SMEs were current or retired fire officers who held the title of Battalion Fire Chief (or Fire Officer 2) or higher. Candidates were also assessed by three Commission employees trained in oral communication assessment. As part of the scoring process, an SME observed and noted the responses of a candidate relative to the knowledge, skills and abilities (KSAs) that each exercise was designed to measure. An assessor also noted any weaknesses that detracted from the candidate’s overall oral communication ability. Each assessor then rated the candidate’s performance according to the rating standards and assigned the candidate a technical or oral communication score on that exercise.

In order to preserve the relative weighting of each of the components of the examination, the ratings for each portion were adjusted by a well-recognized statistical process known as “standardization.” Under this process, the ratings are standardized by converting the raw scores to z-scores, an expression of the deviation of the score from the mean score of the group in relation to the standard deviation of scores for the group. Each portion of the examination had a relative weight in its relation to the whole examination. Thus, the z-score for the multiple-choice portion was multiplied by a test weight of 36.53%, the oral technical scores were multiplied by a test weight of 53.91% and the oral communication scores were multiplied by a test weight of 9.56%. The weighted z-scores were summed and this became the overall final test score. This was weighted and added to the weighted seniority score. The result was standardized, then normalized, and rounded up to the third decimal place to arrive at a final average.

On the Supervision Scenario, the appellant scored a 3 on the technical component and a 5 on the oral communication component. On the Administration Scenario, the appellant scored a 4 on the technical component and a 4 on the oral communication component. Finally, on the Incident Command scenario, the appellant scored a 2 on the technical component and a 4 on the oral communication component.

Appellant's argumentThe appellant challenges his score on the technical component of the Incident Command scenario. As a result, the appellant’s test material and a listing of possible courses of action (PCAs) for the scenario were reviewed.

The Incident Command scenario involves the response to a report of smoke at a local county fairground hosting a large big top tent for circus acts where the candidate, a newly-appointed Battalion Fire Chief. Upon arrival, the candidate finds fire and black smoke drifting up from Side A of the big top tent. A woman who sat near Side A approaches the candidate and tells the candidate that during the show, she saw two clowns near the entrance with a red plastic gasoline container, which she thought was part of the act. Approximately 10 minutes later, one of the entrances was engulfed in flames. Question 1 asks what the candidate’s specific actions are upon arrival at the scene. The prompt for Question 2 presents that five minutes after arriving on scene, the candidate hears multiple snapping noises. It adds that some of the steel cables holding the cover in place are beginning to fail and that a portion of the tent collapsed, cutting off a crew’s means of egress and that the crew issues a

MAYDAY. Question 2 then asks, based on this new information, what specific actions the candidate should now take.

On the technical component, the assessor awarded the appellant a score of 2, based on a finding that he failed to request an emergency medical services (EMS) strike team for a potential mass casualty event in response to Question 1 and missed a number of additional PCAs, including, in part, the opportunity to consider foam operations. On appeal, the appellant maintains that he requested an EMS strike team at a specified point during his presentation. As to the PCA of considering foam operations, he avers that foam operations would not be necessary because the fire would be considered a Class A fire, rather than a Class B flammable liquid fire, as the tent material involved would be tarpaulin made from materials like canvas or plastic. In this regard, the appellant avers that while a witness observed an accelerant being used to start the fire, it wasn’t to an extent that required foam operations.

In reply, a review of the appellant’s presentation confirms the SME’s finding that that the appellant failed to request an EMS strike team in response to Question 1. Critically, because this was a mandatory response to Question 1, the appellant had to specifically identify this action during the portion of his response covering Question 1. A review of his presentation on appeal confirms that he only made a general reference to calling for EMS during the portion of his presentation covering Question 1. While the appellant is correct that he specifically called for an EMS strike team later in his presentation, specifically stating that "[w]ith the members being trapped, going back, [he’d] also initiate a EMS strike force for members that were down, for transport," this was clearly a response to the crew becoming trapped in Question 2 and, thus, was insufficient to award him credit for the Question 1 mandatory response. As to the Question 1 additional response of considering foam operations, the Division of Test Development, Analytics and Administration (TDAA) states that reasons to consider foam operations include the observed gasoline container, the black smoke condition, and the fact that foam can be more effective than water for treating the class of fire. TDAA presents that the PCA specifically speaks to considering foam operations and does not necessarily mean that candidates would be expected to utilize foam on scene. The Commission agrees with TDAA’s rationale for the validity of this PCA, as John Norman, Fire Officer’s Handbook of Tactics 30 (5th ed. 2019) advises that “[b]lack smoke suggests the presence of petroleum-based products” which should indicate that firefighters need to “bring a Class B (for flammable or combustible liquids) extinguisher with them” and International Association of Fire Chiefs and National Fire Protection, Fundamentals of Fire Fighter Skills and Hazardous Materials Response 169 (4th ed. 2019) states that Class B fires “can be extinguished by shutting off the supply of fuel or by using foam to exclude oxygen from the fuel.” As such, the Commission finds the appellant has failed to sustain his burden of proof regarding this PCA.

Finally, upon review of the appellant’s appeal, TDAA has determined that the appellant was erroneously credited with the mandatory response of ordering a primary search in response to Question 1. The appellant’s relevant Question 1 statements were that he would “coordinate with the ladder company on their searching” and add a rescue company. However, these did not specifically indicate that he would conduct a primary search, as required. While the appellant gave a more direct statement about having the RIT company perform an oriented search following the events set forth in Question 2, because the PCA of conducting a primary search was a mandatory response to Question 1, his statement about his actions following the events set forth in Question 2 could not be used to award him credit for a mandatory response to Question 1. Accordingly, credit for this PCA shall be stricken. Nevertheless, even with this reversal of credit, the appellant’s overall score on the technical component of the Incident Command scenario remains 2.

CONCLUSION

A thorough review of the appellant’s submissions and the test materials indicates that the decision below is amply supported by the record and the appellant has failed to meet his burden of proof in this matter.

ORDER

Therefore, it is ordered that this appeal be denied and that appropriate agency records be revised to reflect the above-noted adjustment to the appellant’s scoring records for the technical component of the Incident Command scenario, but that the appellant’s overall score for this component remain unchanged at 2.

ClosingThis is the final administrative determination in this matter. Any further review should be pursued in a judicial forum.