Brian Klein, CSC Docket No. 2019-2461

BackgroundBrian Klein appeals his score for the oral portion of the promotional examination for Fire Lieutenant (PM1093V), Union Township. It is noted that the appellant passed the subject examination with a final average of 85.880 and ranks tenth on the resultant eligible list.

The two-part examination consisted of a written multiple-choice portion and an oral portion. Candidates were required to pass the written portion of the examination, and then were ranked on their performance on both portions of the examination. The test was worth 80 percent of the final score and seniority was worth the remaining 20 percent. Of the test weights, 31.35% of the score was the written multiple-choice portion, 22.49% was the technical score for the evolving exercise, 7.53% was the supervision score for the evolving exercise, 4.28% was the oral communication score for the evolving exercise, 19.23% was the technical score for the arriving exercise, 7.53% was the supervision score for the arriving exercise and 7.59% was the oral communication score for the arriving exercise.

The oral portion of the Fire Lieutenant examination consisted of two scenarios: a salvage and overhaul scene simulation with questions designed to measure the knowledge of salvage and overhaul operations, supervision of fire fighters and the ability to assess building conditions and hazards in an evolving incident on the fireground (evolving); and a multi-vehicle collision scene simulation designed to measure the knowledge of safe rescue tactics and procedures to safeguard citizens, supervision of firefighters and the ability to plan strategies and tactics based upon the accident scene (arriving). Knowledge of supervision was measured by questions in both scenarios, and was scored for each. For the evolving scenario, candidates were provided with a 15-minute preparation period, and candidates had 10 minutes to respond. For the arriving scenario, a five-minute preparation period was given, and candidates had 10 minutes to respond.

The candidates’ responses were scored on technical knowledge and oral communication ability. Prior to the administration of the exam, a panel of Subject Matter Experts (SMEs) determined the scoring criteria, using generally approved fire command practices, firefighting practices, and reference materials. Scoring decisions were based on SME-approved possible courses of action (PCAs) including those actions that must be taken to resolve the situation as presented. Only those oral responses that depicted relevant behaviors that were observable and could be quantified were assessed in the scoring process.

Candidates were rated on a five-point scale, with 5 as the optimal response, 4 as a more than acceptable passing response, 3 as a minimally acceptable passing response, 2 as a less-than-acceptable, and 1 as a much-less-than acceptable response. For each of the scenes, and for oral communication, the requirements for each score were defined.

For the evolving scenario, the appellant scored a 4 for the technical component, 4 for the supervision component, and a 5 for the oral communication component. For the arriving scenario, the appellant scored a 3 for the technical component, a 4 for the supervision component, and a 5 for the oral communication component. The appellant challenges his score for the technical component of the arriving scenario. As a result, the appellant’s test material, video, and a listing of PCAs for the scenario were reviewed.

The arriving scenario involved a report of a collision of three vehicles. The technical component consisted of two questions. Question 1 directed candidates to perform their initial reports to the camera as they would upon the arrival at the incident and to use proper radio protocols. Question 2 asked candidates what specific actions should be taken after giving their initial reports.

For the arriving scenario, the assessor assigned a score of 3, using the “flex rule,” and indicated that with Question 1 the appellant missed opportunities to communicate upon arrival that Ladder 1 was on location at the scene and preparing for offensive operations (e.g., victim care). With regard to Question 2, the assessor found that the appellant failed to specifically mention performing a 360 size-up to evaluate hazards/victims, which was a mandatory response, and that he missed the opportunity to appoint a safety officer. They used the flex rule to assign a score of 3. On appeal, the appellant avers that his response clearly conveyed that he as performing a 360 size-up, as he identified numerous hazards, including stating that he would perform diking and damming for potential liquids around storm drains; stated where he would position apparatuses based on location conditions, particularly the placement of an apparatus at each end of the street in order to block traffic from entering; notified the Department of Transportation about other hazards they could address in coordination with police and fire department personnel; communicated the number of vehicles, the involvement of a Battalion Chief in the accident, and the status of other drivers to the dispatcher. The appellant further maintains that he addressed the appointment of the safety officer by stating in his presentation that “[t]he Safety Officer will ensure fireground safety; which includes observing personnel actions, any movement of the vehicles or other vehicles and personnel wearing proper safety vests according to SOPs.”

Regarding the flex rule, mandatory responses are responses that are requirements for a performance to be acceptable (a score of 3). Sometimes, a candidate states many additional responses but does not give a mandatory response. The flex rule was designed to allow the SMEs to assign a score of 3 to candidates who fail to give a mandatory response, but who provide may additional responses. However, the SMEs cannot provide a score higher than a 3 in those cases. All mandatory responses must be given in order for a performance to be acceptable, whether there is one mandatory response or five of them. It is not assumed that candidates receive a score of 5 which is then lowered for lack of responses. Performances that include mandatory responses get a score of 3, and those without mandatory responses get a score of 1 or 2, unless the flex rule is use. Additional responses only increase a score from 3 to 4 or from 3 to 5.

In reply, at the end of every scenario and prior to the questions, instructions state, “[i]n responding to the questions, make sure that your actions directly relate to the scenario. Do not assume or take for granted that general actions will contribute to your score.” The SMEs expected that the candidate would “[s]pecifically mention[ ] that they would perform a 360 size-up to evaluate hazards/victims” after giving an initial report and that the candidate would state that they were appointing a safety officer. The appellant received credit for requesting law enforcement for eliminating ignition sources, describing appropriate apparatus/equipment placement, stating that there was a motor vehicle collision with three cars involved, and indicating that there were multiple victims with multiple injuries, which are different actions. Although the appellant identified various conditions on scene and addressed the actions he would take after giving his initial report, this is not the same as specifically stating that he would perform a 360 size-up to evaluate hazards/victims. Similarly, the SMEs expected that the candidate would appoint a safety officer. The appellant’s statement that “[t]he Safety Officer will ensure fireground safety; which includes observing personnel actions, any movement of the vehicles or other vehicles and personnel wearing proper safety vests according to SOPs” is not the same as saying he would appoint a safety officer. Rather, the appellant’s passing reference to the safety officer’s duties, in context, merely has that safety officer on-scene without the appellant’s involvement. The appellant missed a mandatory response, as well as the additional responses listed by the assessor, and his score of 3 for this component, using the flex rule, is correct.

CONCLUSION

A thorough review of the appellant’s submissions and the test materials indicates that the decision below is amply supported by the record, and the appellant has failed to meet his burden of proof in this matter.

ORDER

Therefore, it is ordered that this appeal be denied.

ClosingThis is the final administrative determination in this matter. Any further review should be pursued in a judicial forum.