Anthony Rinaldi, CSC Docket No. 2023-1852

BackgroundAnthony Rinaldi, Jr. appeals his score on the promotional examination for Battalion Fire Chief (PM4317C), Bordentown Township Fire District 1. It is noted that the appellant failed the subject examination.

The subject promotional examination was held on May 25, 2022, and one candidate passed. This two-part examination consisted of an integrated system of simulations designed to generate behavior similar to that required for success on the job. The first part consisted of multiple-choice items that measured specific work components identified and weighted by the job analysis. The second part consisted of three oral scenarios: Supervision, Administration and Incident Command. The examination was based on a comprehensive job analysis conducted by the Civil Service Commission (Commission), which identified the critical areas of the job. The weighting of the test components was derived from the job analysis data. It is noted that candidates were told the following prior to beginning their presentations for each scenario: “In responding to the questions, be as specific as possible. Do not assume or take for granted that general actions will contribute to your score.”

Each candidate in a given jurisdiction was scored by a team of three different Subject Matter Experts (SMEs), who were trained in current technical scoring procedures. Each of these SMEs were current or retired fire officers who held the title of Battalion Fire Chief (or Fire Officer 2) or higher. Candidates were also assessed by three Commission employees trained in oral communication assessment. As part of the scoring process, an SME observed and noted the responses of a candidate relative to the knowledge, skills and abilities (KSAs) that each exercise was designed to measure. An assessor also noted any weaknesses that detracted from the candidate’s overall oral communication ability. Each assessor then rated the candidate’s performance according to the rating standards and assigned the candidate a technical or oral communication score on that exercise.

In order to preserve the relative weighting of each of the components of the examination, the ratings for each portion were adjusted by a well-recognized statistical process known as “standardization.” Under this process, the ratings are standardized by converting the raw scores to z-scores, an expression of the deviation of the score from the mean score of the group in relation to the standard deviation of scores for the group. Each portion of the examination had a relative weight in its relation to the whole examination. Thus, the z-score for the multiple-choice portion was multiplied by a test weight of 36.53%, the oral technical scores were multiplied by a test weight of 53.91% and the oral communication scores were multiplied by a test weight of 9.56%. The weighted z-scores were summed and this became the overall final test score. This was weighted and added to the weighted seniority score. The result was standardized, then normalized, and rounded up to the third decimal place to arrive at a final average.

Each oral examination question, and overall oral communication, was rated on a five-point scale, with 5 as the optimal response, 4 as a more than acceptable passing response, 3 as a minimally acceptable passing response, 2 as a less than acceptable response, and 1 as a much less than acceptable response.

On the Supervision scenario, the appellant scored a 5 on the technical component and a 3 on the oral communication component. On the Administration scenario, the appellant scored a 1 on the technical component and a 3 on the oral communication component. Finally, on the Incident Command scenario, the appellant scored a 2 on the technical component and a 3 on the oral communication component.

Appellant's argumentOn appeal, the appellant challenges his scores on the oral communication components of the Administration and Incident Command scenarios and on the technical component of the Incident Command scenario.

On the oral communication component of the Administration scenario, the assessor found that the appellant displayed a major weakness in nonverbal communication “due to eye contact and writing while speaking.” The assessor indicated that the appellant also displayed a minor weakness in organization by using nearly two minutes of response time to continue his preparation. Based upon the foregoing, the assessor awarded the appellant a score of 3 for oral communication. On appeal, the appellant argues that because his “fail[ure] to make sufficient eye contact during his response” was cited as a minor weakness on the Supervision scenario, the similar note by the assessor on the Administration scenario means that his issue with eye contact should have similarly been rated as a minor weakness.

In reply, at the outset, the appellant does not appear to deny that he displayed weaknesses in nonverbal communication or organization. Rather, he contends that his weakness in nonverbal communication should have been characterized as minor, rather than major. Critically, even if the Commission were to agree with that contention, the appellant’s oral communication score for this scenario would remain unchanged at 3. Beyond this, the appellant’s claim that the assessor’s comments on this scenario and the Supervision scenario were substantially similar is without merit. Here, the assessor’s statement that the appellant displayed a “major weakness on nonverbal communication due to eye contact and writing while speaking” on the Administration scenario was sufficient to distinguish the appellant’s rating from that of the assessor on the Supervision scenario, who stated that the appellant “displayed a minor weakness in nonverbal [sic] by failing to make sufficient eye contact during his response” (emphasis added). Moreover, a review of the appellant’s presentations for these two scenarios demonstrates that while the appellant looked down at and read from his notes for a substantial portion of both scenarios, the appellant made eye contact with a modestly greater consistency during his Supervision scenario presentation. Even if it could be said that the appellant’s eye contact should have been considered equally weak on both scenarios, it would more accurately support a conclusion that the assessor for the Supervision scenario should have found that the appellant displayed a major weakness in nonverbal communication. Accordingly, the appellant has failed to sustain his burden of proof and his score of 3 for the oral communication component of the Administration scenario is affirmed.

On the oral communication component of the Incident Command scenario, the assessor awarded the appellant a score of 3 based upon findings that he displayed a major weakness in nonverbal communication and a minor weakness in organization. Specifically, with regard to nonverbal communication, the assessor indicated that the appellant failed to make and maintain consistent eye contact with the camera while speaking. As to organization, the assessor cited the appellant's pausing between and in the middle of sentences throughout his presentation. On appeal, the appellant makes similar arguments as previously presented.

In reply, as with the Administration scenario, the appellant does not appear to deny that he displayed weaknesses in nonverbal communication or organization. Rather, he contends that his weakness in nonverbal communication should have been characterized as minor, rather than major. As with the Administration scenario, even if the Commission were to agree with that contention, the appellant’s oral communication score for the Incident Command scenario would remain unchanged at 3. Further, as with the Administration scenario, the appellant’s claim that the assessor’s comments on this scenario and the Supervision scenario were substantially similar is without merit. Here, the assessor’s statement that the appellant displayed a “major nonverbal weakness as evidenced by the [appellant’s] failure to make and maintain consistent eye contact while speaking” on the Incident Command scenario was sufficient to distinguish the appellant’s rating from that of the assessor on the Supervision scenario, who stated that the appellant “displayed a minor weakness in nonverbal [sic] by failing to make sufficient eye contact during his response” (emphasis added). Moreover, a review of the appellant’s presentations for these two scenarios demonstrates that while the appellant looked down at and read from his notes for a substantial portion of both scenarios, the appellant made eye contact with a modestly greater consistency during his Supervision scenario presentation. Even if it could be said that the appellant’s eye contact should have been considered equally weak on both scenarios, it would more accurately support a conclusion that the assessor for the Supervision scenario should have found that the appellant displayed a major weakness in nonverbal communication. Therefore, the Commission finds that the appellant has failed to sustain his burden of proof and his score of 3 for the oral communication component of the Incident Command scenario is affirmed.

With regard to the technical component scoring, the Incident Command scenario involves a response to a report of fire at a local movie theater, part of which is in the process of a renovation. Question 1 asks what specific actions the candidate would take upon arriving at the scene. The prompt for Question 2 states that the party wall separating two theaters collapses during firefighting operations, trapping two firefighters. Question 2 then asks what specific actions the candidate should now take based on this new information.

The SME found that the appellant failed to perform the mandatory actions of ensuring a proper transfer of command in response to Question 1 and conducting a Personnel Accountability Report (PAR) in response to Question 2. In addition, the SME indicated that the appellant missed several additional opportunities, including the opportunity to check for horizontal extension into other theaters in response to Question 1. Based upon the foregoing, the assessor awarded the appellant a score of 2 on the technical component of the scenario. On appeal, the appellant argues that he covered the two mandatory responses at issue at specified points during his presentation.

In reply, a review of the appellant’s Incident Command presentation demonstrates that he was properly denied credit for the mandatory response of ensuring proper transfer of command. In this regard, it is noted that the appellant stated that '"Battalion 1 is gonna assume command. I have 26 State Route 5 command. Command post is going to be on Side A in the parking lot." Since the appellant was accepting the transfer of command, rather than establishing command, it was necessary for him to include relevant details, such as indicating that he would have a face-to-face meeting with the incident commander he was relieving or ensuring a transfer of information via the radio as part of that process. Since the appellant failed to do so, he was properly denied credit for the subject mandatory response. As such, even assuming, arguendo, that the appellant should have been credited with the mandatory response of conducting a PAR in response to Question 2, based upon the number of mandatory and additional responses he otherwise failed to identify, his score of 2 would remain unchanged. Accordingly, his score of 2 on the technical component of the Incident Command scenario is affirmed.

CONCLUSION

A thorough review of the appellant’s submissions and the test materials indicates that the decision below is amply supported by the record, and the appellant has failed to meet his burden of proof in this matter.

ORDER

Therefore, it is ordered that this appeal be denied.

ClosingThis is the final administrative determination in this matter. Any further review should be pursued in a judicial forum.