Brett Petillo, CSC Docket No. 2023-2687

BackgroundBrett Petillo appeals his score on the oral portion of the promotional examination for Fire Officer 1 (PM2389C), Jersey City. It is noted that the appellant passed the examination with a final average of 85.670 and ranks 64th on the eligible list.

This two-part examination consisted of a written multiple-choice portion and an oral portion. Candidates were required to pass the written portion of the examination, and then were ranked on their performance on both portions of the examination. The test was worth 80 percent of the final score and seniority was worth the remaining 20 percent. Of the test weights, 35.90% of the score was the written multiple-choice portion, 22.04% was the technical score for the evolving exercise, 7.45% was the supervision score for the evolving exercise, 5.71% was the oral communication score for the evolving exercise, 23.20% was the technical score for the arriving exercise, 5.71% was the oral communication score for the arriving exercise.

The oral portion of the Fire Officer 1 examination consisted of two scenarios: a fire scene simulation with questions designed to measure the knowledge of safe rescue tactics and procedures to safeguard citizens, supervision of fire fighters and the ability to assess fire conditions and hazards in an evolving incident on the fireground (Evolving Scenario); and a fire scene simulation designed to measure the knowledge of safe rescue tactics and procedures to safeguard citizens, supervision of firefighters and the ability to plan strategies and tactics based upon a building’s structure and condition (Arriving Scenario). Knowledge of supervision was measured by a question in the Evolving Scenario, and was scored for that scenario. For the Evolving Scenario, candidates were provided with a 15-minute preparation period, and candidates had 10 minutes to respond. For the Arriving Scenario, a five-minute preparation period was given, and candidates had 10 minutes to respond.

The candidates’ responses were scored on technical knowledge and oral communication ability. Prior to the administration of the exam, a panel of Subject Matter Experts (SMEs) determined the scoring criteria, using generally approved fire command practices, firefighting practices, and reference materials. Scoring decisions were based on SME-approved possible courses of action (PCAs) including those actions that must be taken to resolve the situation as presented. Only those oral responses that depicted relevant behaviors that were observable and could be quantified were assessed in the scoring process.

Candidates were rated on a five-point scale, with 5 as the optimal response, 4 as a more than acceptable passing response, 3 as a minimally acceptable passing response, 2 as a less than acceptable response, and 1 as a much less than acceptable response. For each of the scenes, and for oral communication, the requirements for each score were defined.

On the Evolving Scenario, the appellant scored a 5 for the technical component, a 5 for the supervision component, and a 5 for the oral communication component. On the Arriving Scenario, the appellant scored a 2 for the technical component and a 4 for the oral communication component.

Appellant's argumentThe appellant challenges his score for the oral communication and technical components of the Arriving Scenario. As a result, the appellant’s test material, video, and a listing of PCAs for the scenario were reviewed.

The Arriving Scenario involved a report of a fire in a storage unit in a storage facility where the candidate will be the incident commander throughout the incident and will establish command. The question asks what the candidate’s concerns are when sizing up this incident and what specific actions the candidate should take to fully address this incident.

On the oral communication component of the Arriving Scenario, the assessor awarded the appellant a score of 4 based upon a finding that he displayed a minor weakness in inflection/rate/volume of speech. Specifically, the assessor indicated that the appellant spoke mostly in a monotone voice and did not convey an emphasis on the points he was making. On appeal, the appellant complains that the findings were “unfair and ha[d] nothing to do with [his] knowledge on firefighting.” He maintains that he should not have been required to change his voice or style of speech for the subject presentation. In addition, he notes that he was not similarly marked down for his tone of voice on the Evolving Scenario.

On the technical component of the Arriving Scenario, the assessor found that the appellant failed to identify, in part, the mandatory response of securing a water supply and the additional opportunity of indicating that Engine 6 was delayed. Based upon the foregoing, the assessor awarded the appellant a technical component score of 2. On appeal, the appellant argues that he covered the subject mandatory response by assigning a water supply officer at a specified point during his presentation. He contends that he addressed the additional PCA of acknowledging the delay to Engine 6 by stating at a specified point during his presentation that he was calling for a second alarm and noting that Engine 6 was delayed.

CONCLUSION

In the instant matter, the appellant has failed to sustain his burden of proof with respect to the oral communication component of the Arriving Scenario. In this regard, it is this agency’s longstanding policy that technical and oral communication component scores are independent ratings on the examination and that an exemplary or poor technical rating does not have a bearing on oral communication scoring and vice versa. Furthermore, oral communication performance can clearly distinguish candidates, including those delivering presentations with comparable technical details. To wit, it would be disingenuous to argue that the presentation of one candidate who spoke at a low rate of volume, had their speech punctuated by the frequent use of filler words like “ah” and “um,” rarely made eye contact with their audience and routinely made distracting hand gestures would be as understandable, effective and well-received as the presentation of another candidate who gave a speech with a comparable level of detail, but without these same oral communication issues. For these reasons, mere arguments that because a candidate received a certain technical score on a scenario, they should have received a corresponding oral communication rating for that same scenario are invalid. Beyond this, a review of the appellant’s presentation confirms that he displayed a minor weakness in inflection/rate/volume, as noted by the assessor. Accordingly, the appellant’s score of 4 on the oral communication component of the Arriving Scenario is sustained.

As to the technical component of the Arriving Scenario, upon review of the appellant's presentation, the Division of Test Development, Analytics and Administration (TDAA) has determined that the appellant should have been awarded credit for the additional PCA of acknowledging the delay to Engine 6. The Civil Service Commission agrees with TDAA’s finding regarding this item. However, a review of the appellant’s presentation fails to demonstrate that the appellant should have been credited with the mandatory response of establishing a water supply. In this regard, designating a water supply officer is considered a distinct action from securing a water supply and the appellant failed to specifically address the challenge of securing a water supply based upon the information the scenario presented about the fire hydrant locations. As such, the appellant was properly denied credit for this PCA.1 Further, a review of the appellant’s presentation demonstrates that he was erroneously credited with the mandatory response of ordering a hoseline stretched to gain entry into Unit 209, as he ordered a hoseline stretched to the “202 unit,” rather than Unit 209. Critically, with the reversal of credit for this mandatory response and the award of credit for the additional PCA of acknowledging the delay to Engine 6, the appellant’s rating of 2 for the technical component of the Arriving scenario remains correct.

ORDER

Therefore, it is ordered that the appellant’s score of 4 on the oral communication component of the Arriving Scenario and score of 2 the technical component of the Arriving Scenario shall remain unchanged, but that any appropriate agency records be revised to reflect the above-noted changes to the PCAs awarded to the appellant for the technical component of the Arriving Scenario on the subject examination.

ClosingThis is the final administrative determination in this matter. Any further review should be pursued in a judicial forum.