Wayne Daniel, CSC Docket No. 2023-2015

BackgroundWayne Daniel, Jr. appeals his score on the oral portion of the promotional examination for Battalion Fire Chief (PM3388C), Newark.

The subject promotional examination was held on May 23, 2022, and 39 candidates passed. This two-part examination consisted of an integrated system of simulations designed to generate behavior similar to that required for success on the job. The first part consisted of multiple-choice items that measured specific work components identified and weighted by the job analysis. The second part consisted of three oral scenarios: Supervision, Administration and Incident Command. The examination was based on a comprehensive job analysis conducted by the Civil Service Commission (Commission), which identified the critical areas of the job. The weighting of the test components was derived from the job analysis data. It is noted that candidates were told the following prior to beginning their presentations for each scenario: “In responding to the questions, be as specific as possible. Do not assume or take for granted that general actions will contribute to your score.”

Each candidate in a given jurisdiction was scored by a team of three different Subject Matter Experts (SMEs), who were trained in current technical scoring procedures. Each of these SMEs were current or retired fire officers who held the title of Battalion Fire Chief (or Fire Officer 2) or higher. Candidates were also assessed by three Commission employees trained in oral communication assessment. As part of the scoring process, an SME observed and noted the responses of a candidate relative to the knowledge, skills and abilities (KSAs) that each exercise was designed to measure. An assessor also noted any weaknesses that detracted from the candidate’s overall oral communication ability. Each assessor then rated the candidate’s performance according to the rating standards and assigned the candidate a technical or oral communication score on that exercise.

In order to preserve the relative weighting of each of the components of the examination, the ratings for each portion were adjusted by a well-recognized statistical process known as “standardization.” Under this process, the ratings are standardized by converting the raw scores to z-scores, an expression of the deviation of the score from the mean score of the group in relation to the standard deviation of scores for the group. Each portion of the examination had a relative weight in its relation to the whole examination. Thus, the z-score for the multiple-choice portion was multiplied by a test weight of 36.53%, the oral technical scores were multiplied by a test weight of 53.91% and the oral communication scores were multiplied by a test weight of 9.56%. The weighted z-scores were summed and this became the overall final test score. This was weighted and added to the weighted seniority score. The result was standardized, then normalized, and rounded up to the third decimal place to arrive at a final average.

On the Supervision scenario, the appellant scored a 5 on the technical component and a 5 on the oral communication component. On the Administration scenario, the appellant scored a 2 on the technical component and a 4 on the oral communication component. Finally, on the Incident Command scenario, the appellant scored a 3 on the technical component and a 3 on the oral communication component.

Appellant's argumentThe appellant challenges his scores for the technical components of the Administration and Incident Command scenarios. The appellant also requests a review of his seniority score. As a result, the appellant’s test material and a listing of possible courses of action (PCAs) for the scenarios were reviewed.

The Administration scenario involves the candidate investigating an incident between Fire Fighter Hernandez and a Police Officer which culminated in the arrest of Fire Fighter Hernandez at the scene of a car accident where the candidate was serving as the incident commander. Question 1 asks what specific steps the candidate would take to investigate the incident between Fire Fighter Hernandez and the Police Officer. Question 2 states that the candidate has learned that Fire Fighter Hernandez is considering filing a civil suit and asks the candidate what actions should be taken concerning Fire Fighter Hernandez based upon this new information.

On the Administration scenario, the SME indicated that the appellant missed a significant number of PCAs, including opportunities to review the National Fire Incident Reporting System (NFIRS) in response to Question 1 and to inform the union and inform the Fire Chief about the new information regarding Fire Fighter Hernandez’s potential civil suit in response to Question 2. Based upon the foregoing, the SME awarded the appellant a score of 2. On appeal, the appellant argues that he identified each of the three above-noted PCAs.

In reply, a review of the appellant’s Administration scenario presentation fails to demonstrate that he should have been credited with the PCAs at issue. As noted above, candidates were told the following prior to beginning their presentations for each scenario: “In responding to the questions, be as specific as possible. Do not assume or take for granted that general actions will contribute to your score.” A substantial portion of his presentation focused on general changes to policies and procedures. The appellant’s response only contained a cursory discussion of how he would find out about what transpired during the incident and merely made a vague reference to conducting an investigation after learning that Fire Fighter Hernandez was contemplating a civil suit. He did not detail specific investigatory steps and he gave no direct statement about reviewing NFIRS in response to Question 1. With regard to keeping the Fire Chief informed, the appellant provided a final written report to the Fire Chief in response to Question 1 and was appropriately awarded credit for that PCA. However, he did not specifically indicate that he would inform the union or the Fire Chief after learning that Fire Fighter Hernandez was contemplating filing a civil suit. Therefore, he was appropriately denied credit for these distinct PCAs. Accordingly, the appellant’s score of 2 for the technical component of the Administration scenario is sustained.

With regard to the technical component of the Incident Command scenario, the scenario involves a response to a fire at a local auto parts store and auto repair shop. Question 1 asks what specific actions the candidate would take upon arriving at the scene. The prompt for Question 2 indicates that while crews are involved in extinguishment operations, an explosion occurs on Side C, emergency radio traffic is transmitted by a fire fighter and structural damage is now visible on Side C. Question 2 asks what specific actions the candidate should now take based upon this new information.

On the technical component of the Incident Command scenario, the assessor awarded the appellant a score of 3 using the “flex rule”1. The appellant’s score was based upon his failure to identify the mandatory response of ordering an emergency evacuation in response to Question 2 and his failure to identify a number of additional responses, including, in part, sounding an evacuation signal in response to Question 2. On appeal, the appellant contends that the actions he stated during his

1 Generally, candidates must identify all mandatory responses to receive, at minimum, a score of 3.

However, a score of 3 may also be achieved via the “flex rule,” where a candidate provides many additional responses, but does not give a mandatory response. However, a score higher than a 3 cannot be provided utilizing the flex rule.

presentation demonstrate that he conducted an evacuation and ensured the safety of all personnel involved. Specifically, he presents that immediately after the explosion, he called for an evacuation of exposures, shifted operations from offensive to defensive mode, focused on the safety of the individuals involved in the incident and ordered a Personnel Accountability Report.

In reply, a review of the appellant’s Incident Command presentation fails to demonstrate that he should have been credited with the mandatory response of ordering an emergency evacuation or the additional response of sounding an evacuation signal. As noted above, candidates were instructed that “[i]n responding to the questions, be as specific as possible. Do not assume or take for granted that general actions will contribute to your score.” The appellant indicated that after learning of the explosion he would shift from offensive to defensive operations. However, he did not specifically indicate that he would order an emergency evacuation or sound evacuation tones. Accordingly, his statements were too general to award him credit for the subject PCAs. Therefore, the appellant’s score of 3, pursuant to the flex rule, for the Incident Command scenario is sustained.

As to seniority, the Division of Test Development, Analytics and Administration has advised that the appellant has not yet taken the multiple-choice portion of the subject examination. After the appellant takes a make-up of the multiple-choice portion of the subject examination, he will receive a notice with his final score. Upon receipt of a scoring notice, he may request review of his score in accordance with N.J.A.C. 4A:4-6.4.

CONCLUSION

A thorough review of the appellant’s submissions and the test materials indicates that the decision below is amply supported by the record, and the appellant has failed to meet his burden of proof in this matter.

ORDER

Therefore, it is ordered that this appeal be denied.

ClosingThis is the final administrative determination in this matter. Any further review should be pursued in a judicial forum.