Thomas Gander, CSC Docket No. 2023-2133

BackgroundThomas Gander appeals his score on the promotional examination for Battalion Fire Chief (PM3390C), Paterson. It is noted that the appellant passed the examination with a final average of 84.280 and ranks 19th on the eligible list.

The subject promotional examination was held on May 23, 2022, and 45 candidates passed. This two-part examination consisted of an integrated system of simulations designed to generate behavior similar to that required for success on the job. The first part consisted of multiple-choice items that measured specific work components identified and weighted by the job analysis. The second part consisted of three oral scenarios: Supervision, Administration and Incident Command. The examination was based on a comprehensive job analysis conducted by the Civil Service Commission (Commission), which identified the critical areas of the job. The weighting of the test components was derived from the job analysis data. It is noted that candidates were told the following prior to beginning their presentations for each scenario: “In responding to the questions, be as specific as possible. Do not assume or take for granted that general actions will contribute to your score.”

Each candidate in a given jurisdiction was scored by a team of three different Subject Matter Experts (SMEs), who were trained in current technical scoring procedures. Each of these SMEs were current or retired fire officers who held the title of Battalion Fire Chief (or Fire Officer 2) or higher. Candidates were also assessed by three New Jersey Civil Service Commission employees trained in oral communication assessment. As part of the scoring process, an SME observed and noted the responses of a candidate relative to the knowledge, skills and abilities (KSAs) that each exercise was designed to measure. An assessor also noted any weaknesses that detracted from the candidate’s overall oral communication ability. Each assessor then rated the candidate’s performance according to the rating standards and assigned the candidate a technical or oral communication score on that exercise.

In order to preserve the relative weighting of each of the components of the examination, the ratings for each portion were adjusted by a well-recognized statistical process known as “standardization.” Under this process, the ratings are standardized by converting the raw scores to z-scores, an expression of the deviation of the score from the mean score of the group in relation to the standard deviation of scores for the group. Each portion of the examination had a relative weight in its relation to the whole examination. Thus, the z-score for the multiple-choice portion was multiplied by a test weight of 36.53%, the oral technical scores were multiplied by a test weight of 53.91% and the oral communication scores were multiplied by a test weight of 9.56%. The weighted z-scores were summed and this became the overall final test score. This was weighted and added to the weighted seniority score. The result was standardized, then normalized, and rounded up to the third decimal place to arrive at a final average.

For the Supervision scenario, the appellant scored a 5 on the technical component and a 3 on the oral communication component. On the Administration scenario, the appellant scored a 5 on the technical component and a 3 on the oral communication component. Finally, on the Incident Command scenario, the appellant scored a 3 on the technical component and a 4 on the oral communication component.

Appellant's argumentThe appellant challenges his scores for the oral communication components of the Supervision and Administration scenarios and the technical component of the Incident Command scenario. As a result, the appellant’s test material and a listing of possible courses of action (PCAs) for the scenarios were reviewed.

For the oral communication component of the Supervision scenario, the assessor awarded the appellant a score of 3, based on a finding that he displayed a major weakness in word usage/grammar, as evidenced by the appellant’s use of filler words like “um” and “uh” 53 times during his response. Similarly, the appellant received a score of 3 for the oral communication component of the Administration scenario after the assessor found that the appellant displayed a major weakness in word usage/grammar, including using filler words like “um” and “uh” in excess of 50 times throughout his response and repeating phrases within sentences. Further, the assessor found that the appellant displayed a minor weakness in organization, as he appeared to lose his train of thought and struggled to find the right words to convey his points at times.

The technical component of the Incident Command scenario involves a response to a fire at a local auto parts store and auto repair shop. Question 1 asks what specific actions the candidate would take upon arriving at the scene. The prompt for Question 2 indicates that while crews are involved in extinguishment operation, an explosion occurs on Side C, emergency radio traffic has been transmitted by a fire fighter and that structural damage is now visible on Side C. Question 2 asks what specific actions the candidate should now take based upon this new information.

The assessor found that the appellant failed to perform the mandatory response of establishing command and missed a number of additional opportunities, including, in part, considering foam operations. Based upon the foregoing, the assessor awarded the appellant a score of 3, using the flex rule1.

On appeal, the appellant argues that because he received scores of 5 on the technical components of the Supervision and Administration scenarios, he should not have been penalized for his usage of filler words like “um.” As to the technical component of the Incident Command scenario, the appellant presents that he established command at a specified point in his presentation. He proffers that he further covered the subject PCA by calling for additional alarms, personnel, resources and government agencies. Moreover, he avers that he further demonstrated that he established command by controlling the scene throughout the entirety of the scenario as he established and terminated command, filled out National Fire Incident Reporting System (NFIRS) reports, and offered critical incident stress debriefing (CISD). Concerning the additional opportunity to consider foam operations, the appellant maintains that he covered this PCA at a specified point in his presentation by calling for a foam tender in case foam operations were needed.

CONCLUSION

In the instant matter, the appellant has failed to sustain his burden of proof with respect to his appeal of the oral communication scores he received for the Supervision and Administration scenarios. At the outset, the Commission emphasizes that the 2022 2nd Level Fire Supervisor Orientation Guide made clear to candidates that oral communication would be measured across all three scenarios and given its own independent score for each one. Thus, the appellant’s rating of 5 on the technical components for these scenarios does not provide a basis to raise his score on the oral components of these scenarios. Moreover, a review of the recordings of his presentations for these scenarios supports the assessors’ findings for each scenario.

1 Generally, candidates must identify all mandatory responses to receive, at minimum, a score of 3.

However, a score of 3 may also be achieved via the “flex rule,” where a candidate provides many additional responses, but does not give a mandatory response. However, a score higher than a 3 cannot be provided utilizing the flex rule.

Accordingly, the appellant’s score of 3 for the oral communication components of both the Supervision and Administration scenarios is affirmed.

As to the portion of the appellant’s appeal regarding the technical component of the Incident Command scenario, the Division of Test Development, Analytics and Administration (TDAA) agrees that the appellant should have been credited with the additional PCAs of considering foam operations and identifying materials which may be present at the incident scene. However, the record does not support a finding that the appellant should have been credited with the mandatory response of establishing command. Although the appellant indicated that he was setting up a command post, he did not communicate with sufficient clarity that he was the individual establishing command. It is noted that International Association of Fire Chiefs and National Fire Protection, Fundamentals of Fire Fighter Skills and Hazardous Materials Response 839 (4th ed. 2019) provides that an initial report to establish command should include the following information:

• A size-up report

• Command designation (and the location of the ICP, for larger incidents)

• The unit or individual who is establishing command

• An initial situation report

• The initial actions being taken

Additionally, the New Jersey Division of Fire Safety, Model Fire Incident Management Standard Operating Guides – Booklet 9 at 3 (Apr. 13, 2011) provides the following as an example of an effective report for the establishment of command:

“Dispatch from Engine 2.”

“Go ahead Engine 2.”

“Engine 2 is on location. We have a one story, single family dwelling (SFD) with nothing showing. We will be out investigating. Engine 2 is

Main Street Command.”

The appellant’s transmission effectively conveys the existence and location of the command post. However, setting up a command post is a distinct action from establishing that one is in command. The appellant’s statement did not indicate with sufficient clarity that the appellant or his unit was establishing incident command. Thus, because the appellant failed to state the mandatory action of establishing command with sufficient clarity, he was properly denied credit by the SME. Accordingly, the appellant’s score of 3, pursuant to the flex rule, remains correct.

Therefore, a thorough review of the appellant’s submissions and the test materials indicates that, except for the technical component of the Incident Command scenario, as indicated above, the decision below is amply supported by the record, and the appellant has failed to meet his burden of proof in this matter.

ORDER

Therefore, it is ordered that the appellant’s score for the technical component of the Incident Command scenario remain unchanged at 3, but that any appropriate agency records be revised to reflect the appellant’s identification of the above-noted PCAs.

ClosingThis is the final administrative determination in this matter. Any further review should be pursued in a judicial forum.