Phillip Johnson, CSC Docket No. 2023-1841

BackgroundPhillip Johnson appeals his score on the promotional examination for Deputy Fire Chief (PM5172C), West Orange. It is noted that the appellant passed the examination with a final average of 78.820 and ranks sixth on the eligible list.

The subject promotional examination was held on April 23, 2022, and six candidates passed. This was an oral examination designed to generate behaviors similar to those required for success in a job. The examination consisted of four scenario-based oral exercises. Each exercise was developed to simulate tasks and assess the knowledge, skills and abilities (KSAs) important to job performance. These exercises covered four topic areas: 1) Incident Command: Non-fire Incident, 2) Supervision, 3) Administration, and 4) Incident Command: Fire Incident. The test was worth 70 percent of the final score and seniority was worth the remaining 30 percent. The various portions of the test were weighted as follows: technical score for the Incident Command: Non-Fire scenario, 24.42%; oral communication score for the Incident Command: Non-Fire scenario, 3.155%; technical score for the Supervision scenario, 14.17%; oral communication score for the Supervision scenario, 3.155%; technical score for the Administration scenario, 11.81%; oral communication score for the Administration scenario, 3.155%; technical score for the Incident Command: Fire Incident scenario, 36.98%; and oral communication score for the Incident Command: Fire Incident scenario, 3.155%.

The candidates’ responses were scored on technical knowledge and oral communication ability. Prior to the administration of the exam, a panel of Subject Matter Experts (SMEs) determined the scoring criteria, using generally approved fire command practices, fire fighting practices, and reference materials. Scoring decisions were based on SME-approved possible courses of action (PCAs) including those actions that must be taken to resolve the situation as presented. For a performance to be acceptable in the technical component for some scenarios, a candidate needed to present the mandatory courses of action for that scenario. Only those oral responses that depicted relevant behaviors that were observable and could be quantified were assessed in the scoring process.

This examination was given using the chain oral testing process, and candidates were given 10 minutes to respond to each question. Candidate responses to each question were rated on a five-point scale (1 to 5) from no response through optimum according to determinations made by the SMEs. Oral communication for each question was also rated on the five-point scale. This five-point scale includes 5 as the optimal response, 4 as a more than acceptable passing response, 3 as a minimally acceptable passing response, 2 as a less than acceptable response, and 1 as a much less than acceptable response. It is noted that candidates were told the following prior to beginning their presentations for each scenario: “In responding to the questions, be as specific as possible. Do not assume or take for granted that general actions will contribute to your score.”

For the Incident Command: Non-Fire Incident scenario, the appellant scored a 2 on the technical component and a 5 on the oral communication component. For the Supervision scenario, the appellant scored a 3 on the technical component and a 4 on the oral communication component. On the Administration scenario, the appellant scored a 4 on the technical component and a 5 on the oral communication component. Finally, for the Incident Command: Fire Incident scenario, the appellant scored a 2 on the technical component and a 4 on the oral communication component.

Appellant's argumentThe appellant challenges his scores for the oral communication component of the Incident Command: Fire Incident scenario and the technical components of the Incident Command: Non-Fire Incident and Supervision scenarios. As a result, the appellant’s test material, video recording and a list of possible courses of action for the scenarios were reviewed.

For the oral communication component of the Incident Command: Fire Incident scenario, the assessor found that the appellant displayed a minor weakness in nonverbal communication, as he failed to make sufficient eye contact when speaking and was constantly bobbing/bouncing/fidgeting. On appeal, the appellant argues that his nonverbal movements were no more of a weakness in this scenario than in others.

In reply, a review of the appellant’s presentation from the Incident Command: Fire Incident demonstrates that the assessor appropriately determined that the appellant displayed a minor weakness in nonverbal communication. The Civil Service Commission (Commission) observes that the appellant’s issues with eye contact, in and of itself, supports the assessor’s conclusion in this regard. When the appellant was not reading from his notes, he alternated between making eye contact with the camera and looking down below it. While the appellant claims that the quality of his nonverbal communication was not any different from any of the other scenarios, a review of the recording of his presentations shows that he looked downward and away from the camera much more noticeably during his Incident Command: Fire Incident response than he did in other scenarios. Similarly, the Commission agrees that the appellant’s bobbing and bouncing was accurately noted by the assessor and adds further support to the assessor’s conclusion that the appellant displayed a minor weakness in nonverbal communication. Accordingly, the appellant’s score of 4 for the oral communication component of the Incident Command: Fire Incident scenario is correct.

The Incident Command: Non-Fire Incident involves the response to a car submerged in a pool at a residence. The prompt asks what actions the candidate would take in response to the incident.

For the technical component of the Incident Command: Non-Fire Incident, the assessor found that the appellant failed to identify the mandatory response of determining the number of victims/if the pool was occupied when the accident took place and several additional PCAs. On appeal, the appellant argues that he should have been credited with the PCA of determining the number of victims. Specifically, he asserts that early in his presentation, he stated that he would establish command with a possible victim in the pool and that he later stated that he would have additional companies confirm the safety of the homeowners and children. He also avers that his statement that he would have urban search and rescue teams bring the rescued patient to EMS for triage, transport and treatment also covers this PCA.

In reply, upon review of the appellant’s presentation for the Incident Command: Non-Fire Incident, the Division of Test Development, Analytics and Administration (TDAA) agrees that the appellant should have been credited with the mandatory response of determining the number of victims/if the pool was occupied when the accident took place. The Commission agrees with TDAA’s assessment. Accordingly, based upon the additional credit for this mandatory response, the appellant’s score for the technical component of the Incident Command: Non-Fire Incident should be raised from 2 to 4.

The Supervision scenario involves the issue of a subordinate, a Battalion Fire Chief (BFC), being late in turning in his reports to the candidate, while two others holding the same rank have not had the same issue. Question 1 asks what initial and specific steps the candidate should take to address this issue. Question 2 presents that third parties have informed the candidate that this same BFC has been delayed or slow in responding to emergency incidents and that this may be contributing to his delays in providing reports to the candidate. It adds that some also suspect something outside of the fire department is happening with this BFC. Question 2 then asks, based on this new information, what actions the candidate should take.

For the technical component of the Supervision scenario, the assessor found that the appellant missed opportunities to offer the Employee Assistance Program/Employee Advisory Service (EAP/EAS). On appeal, the appellant maintains that he should have been credited with documenting his findings because he stated that he would “inform the chief with a written report of the meetings [he and his subordinate] had and the disciplines that have been handed out and how [they] plan to move forward” and offering EAP/EAS because he indicated that he would “contact the health department/HR department and offer all resources available.”

In reply, a review of the appellant’s presentation for the Supervision scenario demonstrates that the appellant should have been credited with the PCA of interviewing the BFC’s previous supervisor to see if the BFC had any issues. Regardless, even crediting this PCA does not elevate his score for this scenario above 3. As to the other PCAs disputed by the appellant, the Commission notes that the instructions to candidates stated: “In responding to the questions, be as specific as possible. Do not assume or take for granted that general actions will contribute to your score.” Concerning the PCA of documenting his findings, the Commission notes that the PCA was to “[d]ocument all interviews and findings” and that keeping the fire chief informed was a distinct PCA for which the appellant did receive credit. Since the appellant only spoke to documenting the meetings he and his subordinate had and not, for example, a meeting with BFC’s previous supervisor, his statement was too general to award him credit for the separate PCA of documenting all interviews and findings. Finally, with regard to the PCA of offering EAP/EAS, the Commission finds that the appellant’s statements were too general to award him credit for this PCA, as he did not specifically identify a resource like EAP or EAS.

CONCLUSION

A thorough review of the appellant’s submissions and the test materials indicates that, except for the technical component of the Incident Command: Non- Fire Incident scenario, the decision below is amply supported by the record, and the appellant has failed to meet his burden of proof in this matter.

ORDER

Therefore, it is ordered that the appellant’s score for the technical component of the Incident Command: Non-Fire Incident scenario be raised from 2 to 4 and that the remainder of his appeal be denied.

ClosingThis is the final administrative determination in this matter. Any further review should be pursued in a judicial forum.