Robert Tierney, CSC Docket No. 2023-1922

BackgroundRobert Tierney appeals his score on the promotional examination for Battalion Fire Chief (PM3390C), Paterson. It is noted that the appellant passed the examination with a final average of 82.930 and ranks 28th on the eligible list.

The subject promotional examination was held on May 23, 2022, and 45 candidates passed. This two-part examination consisted of an integrated system of simulations designed to generate behavior similar to that required for success on the job. The first part consisted of multiple-choice items that measured specific work components identified and weighted by the job analysis. The second part consisted of three oral scenarios: Supervision, Administration and Incident Command. The examination was based on a comprehensive job analysis conducted by the Civil Service Commission (Commission), which identified the critical areas of the job. The weighting of the test components was derived from the job analysis data. It is noted that candidates were told the following prior to beginning their presentations for each scenario: “In responding to the questions, be as specific as possible. Do not assume or take for granted that general actions will contribute to your score.”

Each candidate in a given jurisdiction was scored by a team of three different Subject Matter Experts (SMEs), who were trained in current technical scoring procedures. Each of these SMEs were current or retired fire officers who held the title of Battalion Fire Chief (or Fire Officer 2) or higher. Candidates were also assessed by three New Jersey Civil Service Commission employees trained in oral communication assessment. As part of the scoring process, an SME observed and noted the responses of a candidate relative to the knowledge, skills and abilities (KSAs) that each exercise was designed to measure. An assessor also noted any weaknesses that detracted from the candidate’s overall oral communication ability. Each assessor then rated the candidate’s performance according to the rating standards and assigned the candidate a technical or oral communication score on that exercise.

In order to preserve the relative weighting of each of the components of the examination, the ratings for each portion were adjusted by a well-recognized statistical process known as “standardization.” Under this process, the ratings are standardized by converting the raw scores to z-scores, an expression of the deviation of the score from the mean score of the group in relation to the standard deviation of scores for the group. Each portion of the examination had a relative weight in its relation to the whole examination. Thus, the z-score for the multiple-choice portion was multiplied by a test weight of 36.53%, the oral technical scores were multiplied by a test weight of 53.91% and the oral communication scores were multiplied by a test weight of 9.56%. The weighted z-scores were summed and this became the overall final test score. This was weighted and added to the weighted seniority score. The result was standardized, then normalized, and rounded up to the third decimal place to arrive at a final average.

For the Supervision scenario, the appellant scored a 4 on the technical component and a 5 on the oral communication component. On the Administration scenario, the appellant scored a 4 on the technical component and a 5 on the oral communication component. Finally, on the Incident Command scenario, the appellant scored a 2 on the technical component and a 4 on the oral communication component.

Appellant's argumentThe appellant challenges his score for the technical component of the Incident Command scenario. As a result, the appellant’s test material and a listing of possible courses of action (PCAs) for the scenario was reviewed.

The Incident Command scenario involves a response to a fire at a local auto parts store and auto repair shop. Question 1 asks what specific actions the candidate would take upon arriving at the scene. The prompt for Question 2 indicates that while crews are involved in extinguishment operations, an explosion occurs on Side C, emergency radio traffic is transmitted by a fire fighter and structural damage is now visible on Side C. Question 2 asks what specific actions the candidate should now take based upon this new information.

The assessor awarded the appellant a score of 2 on the technical component based upon a determination that the appellant failed to perform the mandatory action of ordering an evacuation in response to Question 2 and missed a number of additional opportunities in response to Questions 1 and 2 including, in part, considering foam operations and sending an evacuation signal.

On appeal, the appellant argues that he should have been credited with the mandatory response of ordering an evacuation based upon his statement during his presentation that he was “going to immediately withdrawal [sic] all units to the exterior of the building and have a PAR conducted.” In support, he cites Vincent Dunn, Safety and Survival on the Fireground 408-09 (2nd ed. 2015) and John Norman, Fire Officer’s Handbook of Tactics 555 (5th ed. 2019) to argue that his call for a withdrawal was the appropriate course of action. Additionally, he notes that after withdrawing all units to the exterior, he called for an alarm which included a tower ladder for its master stream, a structural engineer with an additional safety officer to monitor for collapse, establishing a collapse zone, setting up in flanking positions and utilizing master streams. Moreover, he states that in light of the urgent radio message, he wanted to limit additional operational noise so that he could monitor the radio for additional information and progress reports to ensure firefighter safety. Finally, the appellant contends that the many additional responses he stated throughout his presentation merit the award of a higher score.

CONCLUSION

In the instant matter, the appellant has failed to sustain his burden of proof. The sources cited by the appellant do not support his contention that his call for a “withdrawal” constituted an evacuation order. Dunn, supra at 408-09, provides, in pertinent part:

There is a difference between withdrawing firefighters and calling for an emergency evacuation of firefighters. A withdrawal action is ordered when a rapidly increasing danger is anticipated. It is a proactive change of strategy an incident commander can order.

Common reasons for an emergency evacuation would be a terrorist bomb report, hazardous material discovery, report of collapse, and rapidly spreading, uncontrollable fire. When the incident commander orders an emergency exit evacuation, unlike a withdrawal, fire department tools and hoselines are left behind to speed up evacuation of personnel and a roll call or head count must be conducted after the evacuation to determine any missing firefighters.

Norman, supra at 555, draws a similar distinction, noting that “[a]n orderly withdrawal allows time to locate all of the members and ensure that everyone brings out their equipment. In an emergency evacuation, it’s drop your tools and run!”

Given the sequence of events here—an explosion, an emergency call and visible structural damage—it is abundantly clear that an urgent evacuation was the appropriate response and that calling for a withdrawal was insufficient. Moreover, the additional actions cited by the appellant are not substitutes for specifically ordering an evacuation. Therefore, the appellant’s score of 2 for the technical component of the Incident Command scenario is approoriate.

Accordingly, a thorough review of the appellant’s submissions and the test materials indicates that the decision below is amply supported by the record, and the appellant has failed to meet his burden of proof in this matter.

ORDER

Therefore, it is ordered that this appeal be denied.

ClosingThis is the final administrative determination in this matter. Any further review should be pursued in a judicial forum.