Andre Cruz, CSC Docket No. 2023-2025

BackgroundAndre Cruz appeals his score on the promotional examination for Battalion Fire Chief (PM3390C), Paterson. It is noted that the appellant passed the examination with a final average of 81.510 and ranks 34th on the eligible list.

The subject promotional examination was held on May 23, 2022, and 45 candidates passed. This two-part examination consisted of an integrated system of simulations designed to generate behavior similar to that required for success on the job. The first part consisted of multiple-choice items that measured specific work components identified and weighted by the job analysis. The second part consisted of three oral scenarios: Supervision, Administration and Incident Command. The examination was based on a comprehensive job analysis conducted by the Civil Service Commission (Commission), which identified the critical areas of the job. The weighting of the test components was derived from the job analysis data. It is noted that candidates were told the following prior to beginning their presentations for each scenario: “In responding to the questions, be as specific as possible. Do not assume or take for granted that general actions will contribute to your score.”

Each candidate in a given jurisdiction was scored by a team of three different Subject Matter Experts (SMEs), who were trained in current technical scoring procedures. Each of these SMEs were current or retired fire officers who held the title of Battalion Fire Chief (or Fire Officer 2) or higher. Candidates were also assessed by three New Jersey Civil Service Commission employees trained in oral communication assessment. As part of the scoring process, an SME observed and noted the responses of a candidate relative to the knowledge, skills and abilities (KSAs) that each exercise was designed to measure. An assessor also noted any weaknesses that detracted from the candidate’s overall oral communication ability. Each assessor then rated the candidate’s performance according to the rating standards and assigned the candidate a technical or oral communication score on that exercise.

In order to preserve the relative weighting of each of the components of the examination, the ratings for each portion were adjusted by a well-recognized statistical process known as “standardization.” Under this process, the ratings are standardized by converting the raw scores to z-scores, an expression of the deviation of the score from the mean score of the group in relation to the standard deviation of scores for the group. Each portion of the examination had a relative weight in its relation to the whole examination. Thus, the z-score for the multiple-choice portion was multiplied by a test weight of 36.53%, the oral technical scores were multiplied by a test weight of 53.91% and the oral communication scores were multiplied by a test weight of 9.56%. The weighted z-scores were summed and this became the overall final test score. This was weighted and added to the weighted seniority score. The result was standardized, then normalized, and rounded up to the third decimal place to arrive at a final average.

For the Supervision scenario, the appellant scored a 5 on the technical component and a 4 on the oral communication component. On the Administration scenario, the appellant scored a 3 on the technical component and a 3 on the oral communication component. Finally, on the Incident Command scenario, the appellant scored a 4 on the technical component and a 3 on the oral communication component.

Appellant's argumentThe appellant challenges his score for the oral communication component of the Incident Command scenario. As a result, the appellant’s test material for the scenario was reviewed.

With the oral communication component of the Incident Command scenario, the assessor indicated that the appellant displayed a major weakness in inflection/rate/volume, as evidenced by continuous brief pauses, breaks between words that hampered the flow of speech and a loud volume throughout the presentation. Accordingly, the assessor awarded the appellant a score of 3 for the oral communication component of the subject scenario. On appeal, the appellant states that on the date of the examination he was brought to a conference room to give his presentation and that an audio recording device was placed directly in front of him while a video camera was placed 10 to 15 feet away from him. The appellant suggests that the difference in the distance between these two recording devices was a factor in the assessor’s findings and he asserts that his volume was appropriate, particularly in light of the need to project to the camera 10 to 15 feet away. He also proffers that the room he was brought to differed from the arrangement for many other candidates, who were taken to basement rooms. With regard to pausing, the appellant notes that the 2022 2nd Level Fire Supervisor Orientation Guide states that “[p]ausing occasionally to review notes is expected and will not be penalized.”

CONCLUSION

A review of the appellant’s Incident Command presentation supports the assessor’s determination that the appellant displayed a major weakness in inflection/rate/volume. Even assuming, arguendo, that the configuration of the room and/or the placement of the recording devices contributed to the determination that he spoke at a high volume, the other issues cited by the assessor still support a finding that the appellant displayed a major weakness in inflection/rate/volume. In particular, his repeated mid-sentence pauses such as “[s]tretch a 2.5 hose line . . . through the alpha side . . door . . . in the . . . repair shop,” "[t]hey'll search off the line . . . thermal imaging . . . camera . . . They're searching for life and fire" and "[t]hey'll be . . . in the auto parts . . . and the repair shop. Primary searches," significantly detracted from the quality of his presentation. Further, while it is true that the 2022 2nd Level Fire Supervisor Orientation Guide did state that “[p]ausing occasionally to review notes is expected and will not be penalized,” the pauses at issue were more than occasional in frequency and, at times, did not appear to relate to the appellant’s review of his notes. Accordingly, a thorough review of the appellant’s submissions and the test materials indicates that the decision below is amply supported by the record, and the appellant has failed to meet his burden of proof in this matter.

ORDER

Therefore, it is ordered that this appeal be denied.

ClosingThis is the final administrative determination in this matter. Any further review should be pursued in a judicial forum.