James Clancy, CSC Docket No. 2024-1071

BackgroundJames Clancy, Jr. appeals his score on the oral portion of the promotional examination for Fire Captain (PM5306D), Cinnaminson. It is noted that the appellant failed the subject examination.

This two-part examination consisted of a written multiple-choice portion and an oral portion. Candidates were required to pass the written portion of the examination, and then were ranked on their performance on both portions of the examination. The test was worth 70 percent of the final score and seniority was worth the remaining 30 percent. Of the test weights, 35.26% of the score was the written multiple-choice portion, 22.04% was the technical score for the evolving exercise, 2.79% was the oral communication score for the evolving exercise, 13.56% was the technical score for the administration exercise, 2.79% was the oral communication score for the administration exercise, 22.04% was the technical score for the arriving exercise, 2.79% was the oral communication score for the arriving exercise.

The oral portion of the second-level Fire Captain examination consisted of three scenarios: a fire scenario simulation with questions designed to measure knowledge and abilities in assessing risk (Evolving); a simulation designed to measure technical knowledge and abilities in administrative duties (Administration); and a fire scenario simulation designed to measure technical knowledge and abilities in strategy and attack plan and hazmat (Arriving). For the Evolving and Administration scenarios, candidates were provided with a 25-minute preparation period for both, and candidates had 10 minutes to respond to each. For the Arriving scenario, a five-minute preparation period was given and candidates had 10 minutes to respond.

The candidates’ responses were scored on technical knowledge and oral communication ability. Prior to the administration of the exam, a panel of Subject Matter Experts (SMEs) determined the scoring criteria, using generally approved fire command practices, firefighting practices, and reference materials. Scoring decisions were based on SME-approved possible courses of action (PCAs) including those actions that must be taken to resolve the situation as presented. For a performance to be acceptable, other than for oral communication, a candidate needed to present the mandatory courses of action for that scenario. Only those oral responses that depicted relevant behaviors that were observable and could be quantified were assessed in the scoring process. Scores were then converted to standardized scores.

Candidates were rated on a five-point scale, with 5 as the optimal response, 4 as a more than acceptable passing response, 3 as a minimally acceptable passing response, 2 as a less than acceptable response, and 1 as a much less than acceptable response. For each of the scenes, and for oral communication, the requirements for each score were defined.

On the Evolving Scenario, the appellant scored a 2 on the technical component and a 4 on the oral communication component. On the Administration Scenario, the appellant scored a 3 on the technical component and a 5 on the oral communication component. On the Arriving Scenario, the appellant scored a 2 on the technical component and a 4 on the oral communication component.

Appellant's argumentThe appellant challenges his score on the technical component of the Arriving Scenario. As a result, the appellant’s test material, video, and a listing of PCAs for the scenario were reviewed.

The Arriving Scenario involved the response to a fire at a two-story, duplex, wood-framed, residential property with a common cockloft where the candidate, a second-level Fire Captain and the company officer of Engine 3, will be the highest- ranking officer on scene. Upon arrival, the candidate sees smoke coming from the second-floor windows on Side A. The police department has yet to arrive and a large crowd has gathered. There are some residents near the door yelling to see if anyone is inside. There are multiple cars parked in front of the residence. The prompt asks the candidates what their concerns are when sizing up this incident and what specific actions should they take to fully address this incident.

The SME awarded the appellant a score of 2 on the technical component of the Arriving Scenario, based upon a finding that the appellant failed to identify multiple mandatory and additional responses, including, in part, failing to mention life safety, fire spread/attached exposures, and the opportunity to request a utility company. On appeal, the appellant argues that he should have been credited with mentioning fire spread based upon statements that an engine company from the additional requested alarm would be assigned to the Side D exposure and that an engine crew in the exposure would report if they found any fire spread or victims.

BackgroundIn reply, upon review of the appellant’s appeal the Division of Test Development, Analytics and Administration (TDAA) has determined that the appellant should have been awarded credit for the mandatory PCA of identifying life safety as a concern during his size-up and the additional response of requesting the Red Cross for any displaced residents. However, TDAA also advises that the credit the appellant received for the mandatory response of ordering vertical and horizontal ventilation must be stricken, as a review of his presentation on appeal demonstrates that while he directly addressed vertical ventilation, he failed to give a sufficiently clear statement indicating that he would also perform horizontal ventilation. Accordingly, based upon the foregoing, TDAA presents that the appellant’s rating of 2 for the technical component of the Arriving Scenario shall remain unchanged. The Civil Service Commission agrees with TDAA’s assessment.

CONCLUSION

A thorough review of the appellant’s submissions and the test materials indicates that the decision below is amply supported by the record and, except as indicated above, the appellant has failed to meet his burden of proof in this matter.

ORDER

Therefore, it is ordered that this appeal be granted in part and that appropriate agency records be revised to reflect the above-noted adjustments to the appellant’s scoring records for the technical component of the Arriving Scenario, but that the appellant’s overall score for this component remain unchanged at 2.

ClosingThis is the final administrative determination in this matter. Any further review should be pursued in a judicial forum.