Joseph Di Tommaso, CSC Docket No. 2023-1921

BackgroundJoseph Di Tommaso appeals his score on the promotional examination for Fire Officer 3 (PM5162C), Jersey City. It is noted that the appellant passed the examination with a final average of 86.010 and ranks ninth on the eligible list.

The subject promotional examination was held on April 23, 2022, and 13 candidates passed. This was an oral examination designed to generate behaviors similar to those required for success in a job. The examination consisted of four scenario-based oral exercises. Each exercise was developed to simulate tasks and assess the knowledge, skills and abilities (KSAs) important to job performance. These exercises covered four topic areas: 1) Incident Command: Non-Fire Incident, 2) Supervision, 3) Administration, and 4) Incident Command: Fire Incident. The test was worth 70 percent of the final score and seniority was worth the remaining 30 percent. The candidates’ responses were scored on technical knowledge and oral communication ability. The various portions of the test were weighted as follows: technical score for the Incident Command: Non-Fire scenario, 24.42%; oral communication score for the Incident Command: Non-Fire scenario, 3.155%; technical score for the Supervision scenario, 14.17%; oral communication score for the Supervision scenario, 3.155%; technical score for the Administration scenario, 11.81%; oral communication score for the Administration scenario, 3.155%; technical score for the Incident Command: Fire scenario, 36.98%; and oral communication score for the Incident Command: Fire scenario, 3.155%.

Prior to the administration of the exam, a panel of Subject Matter Experts (SMEs) determined the scoring criteria, using generally approved fire command practices, fire fighting practices, and reference materials. Scoring decisions were based on SME-approved possible courses of action (PCAs) including those actions that must be taken to resolve the situation as presented. For a performance to be acceptable in the technical component for some scenarios, a candidate needed to present the mandatory courses of action for that scenario. Only those oral responses that depicted relevant behaviors that were observable and could be quantified were assessed in the scoring process.

This examination was given using the chain oral testing process, and candidates were given ten minutes to respond to each question. Candidate responses to each question were rated on a five-point scale (1 to 5) from nil response through optimum according to determinations made by the SMEs. Oral communication for each question was also rated on the five-point scale. This five-point scale includes 5 as the optimal response, 4 as a more than acceptable passing response, 3 as a minimally acceptable passing response, 2 as a less than acceptable response, and 1 as a much less than acceptable response. It is noted that candidates were told the following prior to beginning their presentations for each scenario: “In responding to the questions, be as specific as possible. Do not assume or take for granted that general actions will contribute to your score.”

For the Incident Command: Non-Fire Incident scenario, the appellant scored a 3 on the technical component and a 4 on the oral communication component. For the Supervision scenario, the appellant scored a 5 on the technical component and a 5 on the oral communication component. On the Administration scenario, the appellant scored a 5 on the technical component and a 5 on the oral communication component. Finally, with the Incident Command: Fire Incident scenario, the appellant scored a 3 on the technical component and a 5 on the oral communication component.

Appellant's argumentThe appellant challenges his score for the technical component of the Incident Command: Non-Fire Incident. As a result, the appellant’s test material, video recording and a list of PCAs for the scenario was reviewed.

The Incident Command: Non-Fire Incident involves the response to a car submerged in a pool at a residence. The prompt asks what actions the candidate would take in response to the incident.

For the technical component of the Incident Command: Non-Fire Incident, the assessor found that the appellant failed to identify the mandatory response of determining the number of victims/if the pool was occupied when the accident took place and several additional PCAs. The assessor used the “flex rule” to give a score of 3. On appeal, the appellant argues that he covered this PCA by asking where the homeowner and the two children were and when he last saw them.

Generally, candidates must identify all mandatory responses to receive, at minimum, a score of 3. However, a score of 3 may also be achieved via the “flex rule,” where a candidate provides many additional responses, but does not give a mandatory response. However, a score higher than a 3 cannot be provided utilizing the flex rule.

BackgroundThe Division of Test Development Analytics and Administration (TDAA) reviewed the appellant’s presentation and stated that the appellant should have been credited with the mandatory response of determining the number of victims, as the appellant made statements during his presentation which demonstrated that he would order a primary search of the pool area and residence in an effort to locate the homeowners and their children. The Civil Service Commission agrees with TDAA’s assessment on appeal. As such, the appellant’s score for the technical component of the Incident Command: Non-Fire Incident should be raised from 3 to 5.

CONCLUSION

A thorough review of the appellant’s submissions and the test materials indicates that the appellant’s score on the technical component of the Incident Command: Non-Fire Incident scenario should be raised from 3 to 5.

ORDER

Therefore, it is ordered that appellant’s score for the technical component of the Incident Command: Non-Fire Incident scenario be raised from 3 to 5.

ClosingThis is the final administrative determination in this matter. Any further review should be pursued in a judicial forum.