Richard Smith, CSC Docket No. 2023-1890

BackgroundRichard Smith appeals his score on the promotional examination for Deputy Fire Chief (PM5172C), West Orange. It is noted that the appellant passed the examination with a final average of 81.180 and ranks fifth on the eligible list.

The subject promotional examination was held on April 23, 2022, and six candidates passed. This was an oral examination designed to generate behaviors similar to those required for success in a job. The examination consisted of four scenario-based oral exercises. Each exercise was developed to simulate tasks and assess the knowledge, skills and abilities (KSAs) important to job performance. These exercises covered four topic areas: 1) Incident Command: Non-Fire Incident, 2) Supervision, 3) Administration, and 4) Incident Command: Fire Incident. The test was worth 70 percent of the final score and seniority was worth the remaining 30 percent. The various portions of the test were weighted as follows: technical score for the Incident Command: Non-Fire scenario, 24.42%; oral communication score for the Incident Command: Non-Fire scenario, 3.155%; technical score for the Supervision scenario, 14.17%; oral communication score for the Supervision scenario, 3.155%; technical score for the Administration scenario, 11.81%; oral communication score for the Administration scenario, 3.155%; technical score for the Incident Command: Fire Incident scenario, 36.98%; and oral communication score for the Incident Command: Fire Incident scenario, 3.155%.

The candidates’ responses were scored on technical knowledge and oral communication ability. Prior to the administration of the exam, a panel of Subject Matter Experts (SMEs) determined the scoring criteria, using generally approved fire command practices, fire fighting practices, and reference materials. Scoring decisions were based on SME-approved possible courses of action (PCAs) including those actions that must be taken to resolve the situation as presented. For a performance to be acceptable in the technical component for some scenarios, a candidate needed to present the mandatory courses of action for that scenario. Only those oral responses that depicted relevant behaviors that were observable and could be quantified were assessed in the scoring process.

This examination was given using the chain oral testing process, and candidates were given 10 minutes to respond to each question. Candidate responses to each question were rated on a five-point scale (1 to 5) from no response through optimum according to determinations made by the SMEs. Oral communication for each question was also rated on the five-point scale. This five-point scale includes 5 as the optimal response, 4 as a more than acceptable passing response, 3 as a minimally acceptable passing response, 2 as a less than acceptable response, and 1 as a much less than acceptable response. It is noted that candidates were told the following prior to beginning their presentations for each scenario: “In responding to the questions, be as specific as possible. Do not assume or take for granted that general actions will contribute to your score.”

For the Incident Command: Non-Fire Incident scenario, the appellant scored a 3 on the technical component and a 5 on the oral communication component. For the Supervision scenario, the appellant scored a 4 on the technical component and a 5 on the oral communication component. On the Administration scenario, the appellant scored a 4 on the technical component and a 4 on the oral communication component. Finally, for the Incident Command: Fire Incident scenario, the appellant scored a 1 on the technical component and a 3 on the oral communication component.

Appellant's argumentThe appellant challenges his score for the oral communication component of the Incident Command: Fire Incident scenario. As a result, the appellant’s test material, video recording and a list of possible courses of action for the scenario was reviewed.

The Incident Command: Fire Incident scenario involves a fire at a one-story recreation center that consists, in part, of attached classrooms on Side C utilizing steel bar joists. The Fire Department Connection (FDC) is on Side C. Question 1 asks what actions the candidate should take to full address the incident. Question 2 provides that during the incident someone busts out of a window from one of the Side C classrooms and a recreation center counselor yells that there are still kids in there. It also states that crews are reporting that water is not flowing from the sprinkler heads. Question 2 then asks what actions should be taken based on this new information.

For the oral communication component of the Incident Command: Fire Incident scenario, the assessor found, in part, that the appellant displayed a major weakness in specificity/brevity, as evidenced by the appellant failing to complete or touch upon Question 2. The assessor also found that the appellant displayed a minor weakness in nonverbal communication. Accordingly, the assessor awarded the appellant a score of 3 for the oral communication component of this scenario. On appeal, the appellant argues that he should have received credit for addressing Question 2. Specifically, he points to his statement during his presentation that “[i]t states that there were still people trapped inside” as the point where he began answering Question 2. He also maintains that he addressed Question 2 by indicating that he would send additional rescue and ladder companies to the C side of the building, indicating that he would send another engine company with an additional 2.5-inch hose line and referring to “once the children are removed” at a later point in his presentation.

In reply, the Civil Service Commission (Commission) agrees with the assessor’s findings with respect to the oral communication component of the appellant's score for the subject scenario. The appellant did not clearly signal during his presentation when he was beginning Question 2. The full statement the appellant relies upon to support his contention that he signaled his transition to Question 2 was: “They said that . . . it states that there are still people trapped inside, so we would send rescue teams uh uh…additional rescue teams and ladder companies into the building deter . . . due to possible multiple victims.” Based on the above, it was reasonable for the assessor to conclude that this was a weakness in specificity/brevity in the appellant’s oral presentation, as there was not a reasonably clear signal to the average listener that he was beginning his response to Question 2 and his description of numerous actions he would take, particularly at this latter stage, were vague. As such, the oral communication component of the appellant’s score for the Incident Command: Fire Incident scenario is appropriate.

Further, the appellant should be mindful that the oral communication component of his score for the Incident Command: Fire Incident is distinct from the technical component of his presentation score. The appellant’s score of 1 on the technical component of this scenario was based upon the number of mandatory PCAs he missed for the entire scenario (i.e., in his responses to both Questions 1 and 2). In the interest of completeness, the Commission has reviewed the technical component of his score for the subject scenario and it finds that the appellant should have been credited with the mandatory PCA of performing a 360-degree size-up in response to Question 1. However, even with the award of this additional PCA, based upon the remaining responses the appellant missed, the correct score for the technical component of his presentation for this scenario remains 1.

ORDER

Therefore, it is ordered that this appeal be denied.

ClosingThis is the final administrative determination in this matter. Any further review should be pursued in a judicial forum.