Gregory Ferrante, CSC Docket No. 2023-2050

BackgroundGregory Ferrante appeals his score on the promotional examination for Battalion Fire Chief (PM3380C), Hoboken. It is noted that the appellant passed the examination with a final average of 85.260 and ranks tenth on the eligible list.

The subject promotional examination was held on May 23, 2022, and 14 candidates passed. This two-part examination consisted of an integrated system of simulations designed to generate behavior similar to that required for success on the job. The first part consisted of multiple-choice items that measured specific work components identified and weighted by the job analysis. The second part consisted of three oral scenarios: Supervision, Administration and Incident Command. The examination was based on a comprehensive job analysis conducted by the Civil Service Commission (Commission), which identified the critical areas of the job. The weighting of the test components was derived from the job analysis data. It is noted that candidates were told the following prior to beginning their presentations for each scenario: “In responding to the questions, be as specific as possible. Do not assume or take for granted that general actions will contribute to your score.”

Each candidate in a given jurisdiction was scored by a team of three different Subject Matter Experts (SMEs), who were trained in current technical scoring procedures. Each of these SMEs were current or retired fire officers who held the title of Battalion Fire Chief (or Fire Officer 2) or higher. Candidates were also assessed by three Commission employees trained in oral communication assessment. As part of the scoring process, an SME observed and noted the responses of a candidate relative to the knowledge, skills and abilities (KSAs) that each exercise was designed to measure. An assessor also noted any weaknesses that detracted from the candidate’s overall oral communication ability. Each assessor then rated the candidate’s performance according to the rating standards and assigned the candidate a technical or oral communication score on that exercise.

In order to preserve the relative weighting of each of the components of the examination, the ratings for each portion were adjusted by a well-recognized statistical process known as “standardization.” Under this process, the ratings are standardized by converting the raw scores to z-scores, an expression of the deviation of the score from the mean score of the group in relation to the standard deviation of scores for the group. Each portion of the examination had a relative weight in its relation to the whole examination. Thus, the z-score for the multiple-choice portion was multiplied by a test weight of 36.53%, the oral technical scores were multiplied by a test weight of 53.91% and the oral communication scores were multiplied by a test weight of 9.56%. The weighted z-scores were summed and this became the overall final test score. This was weighted and added to the weighted seniority score. The result was standardized, then normalized, and rounded up to the third decimal place to arrive at a final average.

Each oral examination question, and overall oral communication, was rated on a five-point scale, with 5 as the optimal response, 4 as a more than acceptable passing response, 3 as a minimally acceptable passing response, 2 as a less than acceptable response, and 1 as a much less than acceptable response.

For the Supervision scenario, the appellant scored a 5 on the technical component and a 5 on the oral communication component. On the Administration scenario, the appellant scored a 3 on the technical component and a 5 on the oral communication component. Finally, on the Incident Command scenario, the appellant scored a 4 on the technical component and a 5 on the oral communication component.

Appellant's argumentThe appellant challenges his scores for the technical components of the Administration and Incident Command scenarios. As a result, the appellant’s test material and a listing of possible courses of action (PCAs) for the scenarios were reviewed.

The Administration scenario involves the candidate investigating an incident between Fire Fighter Hernandez and a Police Officer which culminated in the arrest of Fire Fighter Hernandez at the scene of a car accident where the candidate was serving as the incident commander. Question 1 asks what specific steps the candidate would take to investigate the incident between Fire Fighter Hernandez and the Police Officer. Question 2 states that the candidate has learned that Fire Fighter Hernandez is considering filing a civil suit and asks the candidate what actions should be taken concerning Fire Fighter Hernadez based upon this new information.

For the Administration scenario, the SME indicated that the appellant missed a number of PCAs, including the opportunities to review the National Fire Incident Reporting System (NFIRS) and to inform the union. Based upon the foregoing, the SME awarded the appellant a score of 3. On appeal, with regard to reviewing NFIRS, the appellant presents that “[a]lthough NFIRS was not outright stated, it [was] inferred” that he would obtain the report during his initial investigation. As far as informing the union, the appellant argues that his response demonstrates that the union was kept informed and involved in all meetings and policy development.

In reply, a review of the appellant’s Administration scenario presentation reveals that the appellant should not have been awarded a passing score for the technical component. As presented above, a score of 3 is a minimally acceptable passing response and candidates were told the following prior to beginning their presentations for each scenario: “In responding to the questions, be as specific as possible. Do not assume or take for granted that general actions will contribute to your score.” The appellant “inferr[ing]” that he would review NFIRS falls short of this mandate. More problematic for the appellant is that a review of his Administration scenario presentation on appeal reveals that he provided only a cursory discussion of the need to perform an investigation in his response to Question 1, which asked specifically about the steps that he would take to investigate the incident between Fire Fighter Hernandez and the Police Officer. The bulk of his presentation focused on assembling a committee that would evaluate and implement changes to policies, procedures and training. It was devoid of any substantive discussion of how the investigation would gather facts about the underlying incident itself or how he might interact with Fire Fighter Hernandez, the arresting Police Officer or other witnesses. Further, the appellant’s response did nothing to specifically indicate how he would respond to the information contained in Question 2. Based upon the minimal number of PCAs the appellant successfully identified, his response was much less than acceptable. As such, his score for the technical component of the Administration scenario must be decreased to 1.

The Incident Command scenario involves a response to a fire at a local auto parts store and auto repair shop. Question 1 asks what specific actions the candidate would take upon arriving at the scene. The prompt for Question 2 indicates that while crews are involved in extinguishment operations, an explosion occurs on Side C, emergency radio traffic is transmitted by a fire fighter and structural damage is now visible on Side C. Question 2 asks what specific actions the candidate should now take based upon this new information.

For the technical component of the Incident Command scenario, the assessor awarded the appellant a score of 4 based upon a finding that the appellant missed a number of additional opportunities, including, in part, the opportunity to sound evacuation tones. On appeal, the appellant argues that with his switch to defensive operations and his other actions, it was not necessary for him to sound evacuation tones.

In reply, the events presented in Question 2 demonstrate a clear need to order an emergency evacuation. The appellant did order an emergency evacuation during his response. However, his statements did not cover the separate action of sounding evacuation tones. N.J.A.C. 5:75-2.7(a) states that “[w]hen the risk posed to firefighters is so great that an incident commander or his or her designee must order an evacuation from a hazardous area, a uniform procedure and accompanying audible signal recognizable by all personnel operating on an incident scene shall be established, adopted and utilized by all fire departments” (emphasis added). Thus, under this fact pattern it was imperative for the appellant to specifically state that he was sounding evacuation tones. Based upon the foregoing and the instruction to candidates to “be as specific as possible. Do not assume or take for granted that general actions will contribute to your score,” his switch to defensive operations and his reference to other actions were insufficient to cover the subject PCA. Moreover, it is noted that although the SME awarded the appellant credit for the mandatory response of establishing command, a review of the appellant’s demonstrates that this was erroneous. In this regard, while the appellant announced his arrival, radioed dispatch and stated the location of the command post, his actions did not show that he was establishing command. As a result, the appellant’s score shall be reduced from 4 to 2 on the technical component of the Incident Command scenario.

Finally, since the Commission has determined that the appellant failed to achieve passing scores on the technical components of the Administration and Incident Command scenarios, it must also find that the appellant failed the subject examination. In this regard, it is the Commission’s duty, on appeal, to review and correct any identified errors that may have been made, even if to a candidate’s detriment. Accordingly, the appellant’s name shall be removed from the subject eligible list1.

ORDER

Therefore, it is ordered that this appeal be denied. It is further ordered that the appellant’s score on the technical component of the Administration scenario be reduced from 3 to 1 and that his score on the technical component of the Incident Command scenario be reduced from 4 to 2. It is further ordered that, since the appellant failed the subject examination based upon the foregoing scoring changes, that the appellant’s name be removed from the Battalion Fire Chief (PM3385C), Hoboken eligible list.

1 In so doing, the Commission stresses that no vested or other rights are accorded by an administrative error. See Cipriano v. Department of Civil Service, 151 N.J. Super. 86 (App. Div. 1977); O’Malley v. Department of Energy, 109 N.J. 309 (1987); HIP of New Jersey v. New Jersey Department of Banking and Insurance, 309 N.J. Super. 538 (App. Div. 1998).

ClosingThis is the final administrative determination in this matter. Any further review should be pursued in a judicial forum.