Stephen Muni, CSC Docket No. 2023-1973

BackgroundStephen Muni, Jr. appeals his score on the promotional examination for Fire Officer 2 (PM3398C), Trenton. It is noted that the appellant passed the examination with a final average of 86.130 and ranks eighth on the eligible list.

The subject promotional examination was held on April 26, 2022, and 15 candidates passed. This two-part examination consisted of an integrated system of simulations designed to generate behavior similar to that required for success on the job. The first part consisted of multiple-choice items that measured specific work components identified and weighted by the job analysis. The second part consisted of three oral scenarios: Supervision, Administration and Incident Command. The examination was based on a comprehensive job analysis conducted by the Civil Service Commission (Commission), which identified the critical areas of the job. The weighting of the test components was derived from the job analysis data. It is noted that candidates were told the following prior to beginning their presentations for each scenario: “In responding to the questions, be as specific as possible. Do not assume or take for granted that general actions will contribute to your score.”

Each candidate in a given jurisdiction was scored by a team of three different Subject Matter Experts (SMEs), who were trained in current technical scoring procedures. Each of these SMEs were current or retired fire officers who held the title of Battalion Fire Chief (or Fire Officer 2) or higher. Candidates were also assessed by three Commission employees trained in oral communication assessment. As part of the scoring process, an SME observed and noted the responses of a candidate relative to the knowledge, skills and abilities (KSAs) that each exercise was designed to measure. An assessor also noted any weaknesses that detracted from the candidate’s overall oral communication ability. Each assessor then rated the candidate’s performance according to the rating standards and assigned the candidate a technical or oral communication score on that exercise.

In order to preserve the relative weighting of each of the components of the examination, the ratings for each portion were adjusted by a well-recognized statistical process known as “standardization.” Under this process, the ratings are standardized by converting the raw scores to z-scores, an expression of the deviation of the score from the mean score of the group in relation to the standard deviation of scores for the group. Each portion of the examination had a relative weight in its relation to the whole examination. Thus, the z-score for the multiple-choice portion was multiplied by a test weight of 36.53%, the oral technical scores were multiplied by a test weight of 53.91% and the oral communication scores were multiplied by a test weight of 9.56%. The weighted z-scores were summed and this became the overall final test score. This was weighted and added to the weighted seniority score. The result was standardized, then normalized, and rounded up to the third decimal place to arrive at a final average.

Each oral examination question, and overall oral communication, was rated on a five-point scale, with 5 as the optimal response, 4 as a more than acceptable passing response, 3 as a minimally acceptable passing response, 2 as a less than acceptable response, and 1 as a much less than acceptable response.

On the Supervision scenario, the appellant scored a 5 on the technical component and a 5 on the oral communication component. On the Administration scenario, the appellant scored a 5 on the technical component and a 4 on the oral communication component. Finally, on the Incident Command scenario, the appellant scored a 5 on the technical component and a 4 on the oral communication component.

Appellant's argumentThe appellant challenges his scores for the oral communication components of Administration and Incident Command scenarios. As a result, the appellant’s test material was reviewed.

On the Administration scenario, the assessor found that the appellant displayed a minor weakness in word usage/grammar by repeating the word “both” unnecessarily, saying “mittee” before correcting it to “committee meeting” and using improper grammar at specified points during his presentation. Based upon the foregoing, the assessor awarded the appellant an oral communication score of 4 for this scenario. On appeal, the appellant argues that correcting himself enhanced his credibility by showing he was paying attention to his audience and committed to presenting accurate and professional information. He also maintains that it showed that he knew the correct terminology and was able to communicate effectively using the correct language. Moreover, the appellant proffers that by continuing with his presentation without skipping a beat, he showed he was confident, well-prepared and able to adapt to unexpected situations with ease. The appellant explains that he used the phrase “make sure there is [sic] no budget restrictions” during his presentation to emphasize the importance of the issue and to help ensure his team had the resources needed to do its job effectively. Finally, he argues that “taking away points for minor word mispronunciations or repeating a word when the overall presentation and questions were answered correctly and efficiently does not seem like the correct scoring.”

In reply, a review of the appellant’s Administration oral scenario presentation confirms that he displayed a minor weakness in word usage/grammar, particularly within the first two minutes of his presentation. A rating of 4 on oral communication correlates to a finding that a candidate’s oral communication was “more than acceptable,” while a score of 5 corresponds to an “optimal” response. As noted by the assessor, the appellant repeated the word “both” unnecessarily several times within the first two minutes of his presentation, said “mittee” before pivoting to “committee” and used improper grammar such as “we will make sure that there is [sic] no budget restrictions. . .” The appellant used improper grammar in other points during his presentation, including stating “I’m also gonna get [sic] permits division involved” approximately one minute and 20 seconds into his presentation and “we’re gonna review [sic] incident plan” around the two-minute mark1. It was thus reasonable for the assessor to find that the appellant’s presentation was “more than acceptable” (i.e., awarded him a score of 4), rather than “optimal” (i.e., a score of 5).

On the Incident Command scenario, the assessor found that the appellant displayed a minor weakness in word usage/grammar by excessively using “gonna” during his presentation. As a result, the assessor awarded the appellant an oral communication of 4 for this scenario. On appeal, the appellant argues that the use of colloquial language like “gonna” can enhance communication on the fireground in several ways. Specifically, he presents that it is common to use it in spoken English because it is language that a team would be familiar with, and that it would make it easier for them to follow and understand instructions as opposed to the phrase “I’m going to.” He further contends that the use of “gonna” improves efficiency by speeding up communications, which is critical in emergency situations. Finally, he avers that the use of “gonna” can help build rapport with a team by breaking down barriers and creating a more open and collaborative environment. Based upon the foregoing, the appellant argues that his use of “gonna” did not impede his ability to quickly and efficiently address the Incident Command scenario.

In reply, as indicated previously, the appellant’s rating of 4 on oral communication for the Incident Command scenario correlates to a finding that his response was “more than acceptable.” The appellant does not dispute that he uttered “gonna” at multiple points during his Incident Command presentation and a review of his presentation on appeal confirms that he did so. Notwithstanding the appellant’s arguments to the contrary, it was thus reasonable for the assessor to find that the appellant displayed a minor weakness in oral communication for the subject scenario and rated the appellant’s presentation as “more than acceptable” (i.e., awarded him a score of 4), rather than “optimal” (i.e., a score of 5).

1 While the appellant’s correcting of “mittee” to “committee” does not excuse the error for purposes of examination scoring, the Commission notes that he did not correct these other grammatical missteps.

CONCLUSION

A thorough review of the appellant’s submissions and the test materials indicates that the decision below is amply supported by the record, and the appellant has failed to meet his burden of proof in this matter.

ORDER

Therefore, it is ordered that this appeal be denied.

ClosingThis is the final administrative determination in this matter. Any further review should be pursued in a judicial forum.