Stephen Kohler, CSC Docket No. 2023-2279

BackgroundStephen Kohler appeals his score on the oral portion of the promotional examination for Fire Captain (PM4449C), Atlantic City. It is noted that the appellant passed the examination with a final average of 87.800 and ranks 19th on the eligible list.

This two-part examination consisted of a written multiple-choice portion and an oral portion. Candidates were required to pass the written portion of the examination, and then were ranked on their performance on both portions of the examination. The test was worth 80 percent of the final score and seniority was worth the remaining 20 percent. Of the test weights, 35.90% of the score was the written multiple-choice portion, 22.04% was the technical score for the evolving exercise, 7.45% was the supervision score for the evolving exercise, 5.71% was the oral communication score for the evolving exercise, 23.20% was the technical score for the arriving exercise, 5.71% was the oral communication score for the arriving exercise.

The oral portion of the Fire Captain examination consisted of two scenarios: a fire scene simulation with questions designed to measure the knowledge of safe rescue tactics and procedures to safeguard citizens, supervision of fire fighters and the ability to assess fire conditions and hazards in an evolving incident on the fireground (Evolving Scenario); and a fire scene simulation designed to measure the knowledge of safe rescue tactics and procedures to safeguard citizens, supervision of firefighters and the ability to plan strategies and tactics based upon a building’s structure and condition (Arriving Scenario). Knowledge of supervision was measured by a question in the Evolving Scenario, and was scored for that scenario. For the Evolving Scenario, candidates were provided with a 15-minute preparation period, and candidates had 10 minutes to respond. For the Arriving Scenario, a five-minute preparation period was given, and candidates had 10 minutes to respond.

The candidates’ responses were scored on technical knowledge and oral communication ability. Prior to the administration of the exam, a panel of Subject Matter Experts (SMEs) determined the scoring criteria, using generally approved fire command practices, firefighting practices, and reference materials. Scoring decisions were based on SME-approved possible courses of action (PCAs) including those actions that must be taken to resolve the situation as presented. Only those oral responses that depicted relevant behaviors that were observable and could be quantified were assessed in the scoring process.

Candidates were rated on a five-point scale, with 5 as the optimal response, 4 as a more than acceptable passing response, 3 as a minimally acceptable passing response, 2 as a less than acceptable response, and 1 as a much less than acceptable response. For each of the scenes, and for oral communication, the requirements for each score were defined.

On the Evolving Scenario, the appellant scored a 3 on the technical component, a 3 on the supervision component, and a 3 on the oral communication component. On the Arriving Scenario, the appellant scored a 5 on the technical component and a 5 on the oral communication component.

Appellant's argumentThe appellant challenges his score for the technical and supervision components of the Evolving Scenario.

The Evolving Scenario involves a report of a fire at an abandoned school used for storage, with the candidate being the first-level supervisor of the first arriving ladder company. Battalion 3 is on scene first and the incident commander is reporting heavy fire from the second floor at the A/D corner of the building and orders the candidate to conduct a primary search, as he is getting reports of possible squatters inside of the vacant property. Question 1 asks, the candidate, as the supervisor of Ladder 5, to describe, in detail, what orders they would give their crew to carry out the assignment from the incident commander. The prompt for Question 2 states that while conducting primary search operations on the second floor, the candidate and their crew notice a structure member beginning to compromise from the interior on Side D. Question 2 then asks the candidate what actions they should take as a result.

The SME awarded the appellant a technical score of 3, pursuant to the “flex rule,”1 based upon a determination that the appellant failed to order a primary search on the second floor in response to Question 1 and missed the opportunity in response to Question 2 to instruct the crew to leave their equipment when evacuating. On appeal, the appellant argues that he addressed ordering a primary search by stating at a specified time that he “acknowledged the order and relayed the order to [his] crew,” which, per the prompt to Question 1, was an order from the Incident Commander to conduct a primary search. Regarding the PCA at issue from Question 2, the appellant expresses confusion regarding the SME’s statement, arguing that in a scenario with an abandoned building such as this, no piece of equipment should be left behind in the scenario because firefighters could easily encounter other obstacles on their way out of the building. In particular, the appellant contends that it was imperative for firefighters exiting the fire building in the scenario to have forcible entry tools like a Halligan bar at the ready.

The prompt for the supervision component of the Evolving Scenario provides that once outside of the building, the candidate orders their company to assist engine companies with back-up lines for defensive operations. It then states that while they perform this task, the candidate notices one of their firefighters stretching the back- up line into the collapse zone. The prompt then asks what actions the candidate should take on scene and back at the firehouse.

The SME awarded the appellant a score of 3 on the supervision component of the Evolving Scenario based upon a finding that the appellant failed to identify several actions, including, in part, opportunities to check the firefighter’s records and to monitor the firefighters’ progress. On appeal, the appellant argues that he covered this by stating that he “talked with the firefighter’s previous supervisor to see if anything like this had happened before and, if so, what was done about it.” The appellant maintains that he covered monitoring the firefighters’ progress by stating that he would “follow[ ] up with the firefighter the next tour and also provide[ ] mentoring with a senior firefighter” and that he would “h[o]ld training evaluations with the entire company.”

CONCLUSION

In the instant matter, as noted above, candidates were told the following prior to beginning their presentations for each scenario: “In responding to the questions, be as specific as possible. Do not assume or take for granted that general actions will contribute to your score.” Here, the appellant was appropriately denied credit for the mandatory response of ordering a primary search on the second floor in response to

1 Generally, candidates must identify all mandatory responses to receive, at minimum, a score of 3.

However, a score of 3 may also be achieved via the “flex rule,” where a candidate provides many additional responses, but does not give a mandatory response. However, a score higher than a 3 cannot be provided utilizing the flex rule.

Question 1 on the Evolving Scenario, as he merely gave a general statement about acknowledging the Incident Commander’s order and relaying it to his crew without specifying that the primary search would be conducted on the second floor of the fire building. Regarding the PCA of instructing the crew to leave their equipment, Vincent Dunn, Safety and Survival on the Fireground 408-09 (2nd ed. 2015) provides, in pertinent part:

A withdrawal action is ordered when a rapidly increasing danger is anticipated. It is a proactive change of strategy an incident commander can order.

Common reasons for an emergency evacuation would be a terrorist bomb report, hazardous material discovery, report of collapse, and rapidly spreading, uncontrollable fire. When the incident commander orders an emergency exit evacuation, unlike a withdrawal, fire department tools and hoselines are left behind to speed up evacuation of personnel and a roll call or head count must be conducted after the evacuation to determine any missing firefighters.

John Norman, Fire Officer’s Handbook of Tactics 555 (5th ed. 2019) draws a similar distinction, noting that “[a]n orderly withdrawal allows time to locate all of the members and ensure that everyone brings out their equipment. In an emergency evacuation, it’s drop your tools and run!” The appellant’s argument is inconsistent with this principle regarding evacuations. Accordingly, the appellant’s Evolving Scenario technical component score of 3, pursuant to the flex rule, is affirmed.

As to the supervision component of the Evolving Scenario, the appellant talking with the firefighter’s previous supervisor would only partially cover the information contemplated by the PCA of “[c]heck firefighter’s records (e.g., training, personnel),” as it would not necessarily reveal what specific penalty, if any, the appellant may have been subjected to with any prior disciplinary actions. Further, the statement cited by the appellant did not indicate that he would review the firefighter’s training records, which would be critical in evaluating the appropriate remedial actions that might be taken. As such, the appellant was properly denied credit for the PCA of checking the firefighter’s records. As to the PCA of monitoring the firefighters’ progress, the statements cited by the appellant were sufficient to award him credit for the distinct PCA of providing any necessary re-training, but were not sufficiently specific to cover this PCA. Finally, upon review of the appellant’s Evolving Scenario on appeal, the Division of Test Development, Analytics and Administration (TDAA) has determined that the appellant should have been credited with the PCA of advising of the consequences of freelancing/not following orders/operating within the collapse zone and that based upon the award of this additional credit, the appellant’s supervision component score should be raised from 3 to 4. The Civil Service Commission agrees with TDAA’s assessment.

ORDER

Therefore, it is ordered that this appeal be granted in part and that the appellant’s Evolving Scenario supervision component score be raised from 3 to 4 with retroactive effect.

ClosingThis is the final administrative determination in this matter. Any further review should be pursued in a judicial forum.