Jarnel Williams, CSC Docket No. 2019-2172

BackgroundJarnel Williams appeals his score for the oral portion of the promotional examination for Fire Captain (PM1038V), Hillside. It is noted that the appellant failed the subject examination.

This two-part examination consisted of a written multiple-choice portion and an oral portion. Candidates were required to pass the written portion of the examination, and then were ranked on their performance on both portions of the examination. The test was worth 80 percent of the final score and seniority was worth the remaining 20 percent. Of the test weights, 31.35% of the score was the written multiple-choice portion, 22.49% was the technical score for the evolving exercise, 7.53% was the supervision score for the evolving exercise, 4.28% was the oral communication score for the evolving exercise, 19.23% was the technical score for the arriving exercise, 7.53% was the supervision score for the arriving exercise, and 7.59% was the oral communication score for the arriving exercise.

The oral portion of the Fire Captain examination consisted of two scenarios: a fire scene simulation with questions designed to measure the knowledge of safe rescue tactics and procedures to safeguard citizens, supervision of fire fighters and the ability to assess fire conditions and hazards in an evolving incident on the fireground (evolving); and a fire scene simulation designed to measure the knowledge of safe rescue tactics and procedures to safeguard citizens, supervision of firefighters and the ability to plan strategies and tactics based upon a building’s structure and condition (arriving). Knowledge of supervision was measured by questions in both scenarios, and was scored for each. For the evolving scenario, candidates were provided with a 15-minute preparation period, and candidates had 10 minutes to respond. For the arriving scenario, a five-minute preparation period was given, and candidates had 10 minutes to respond.

The candidates’ responses were scored on technical knowledge and oral communication ability. Prior to the administration of the exam, a panel of Subject Matter Experts (SMEs) determined the scoring criteria, using generally approved fire command practices, firefighting practices, and reference materials. Scoring decisions were based on SME-approved possible courses of action (PCAs) including those actions that must be taken to resolve the situation as presented. Only those oral responses that depicted relevant behaviors that were observable and could be quantified were assessed in the scoring process.

Candidates were rated on a five-point scale, with 5 as the optimal response, 4 as a more than acceptable passing response, 3 as a minimally acceptable passing response, 2 as a less than acceptable response, and 1 as a much less than acceptable response. For each of the scenes, and for oral communication, the requirements for each score were defined.

For the evolving scenario, the appellant scored a 3 for the technical component, a 2 for the supervision component, and a 2 for the oral communication component. For the arriving scenario, the appellant scored a 2 for the technical component, a 4 for the supervision component, and a 2 for the oral communication component. The appellant challenges his scores for the technical and oral communication components of the evolving scenario, and for the technical and supervision components of the arriving scenario. As a result, the appellant’s test material, video, and a listing of PCAs for the scenarios were reviewed.

The evolving scenario involved a fire in the computer lab of a high school. For the technical component, the assessor noted that the candidate failed to ensure that proper Personal Protective Equipment (PPE) was worn by the entire crew. It was also noted that he missed the opportunity to assist in evacuation. These were PCAs for question 1, which asked candidates to describe in detail the orders they would give their crew to carry out their assignment from the Incident Commander (IC). The assessor used the “flex” rule to assign a score of 3. On appeal, the appellant states that he said he would assist victims and additional personnel out of the building, and transfer their care to EMS. He states that he also notified command about the radio and finished the exercise by waiting for further orders at Rehab.

Regarding the flex rule, mandatory responses are responses that are requirements for a performance to be acceptable (a score of 3). Sometimes, a candidate states many additional responses but does not give a mandatory response. The flex rule was designed to allow the SMEs to assign a score of 3 to candidates who fail to give a mandatory response but who provide many additional responses. However, the SMEs cannot provide a score higher than a 3 in those cases. All mandatory responses must be given in order for a performance to be acceptable, whether there is one mandatory response or five of them. It is not assumed that candidates receive a score of 5 which is then lowered for lack of responses. Performances that include mandatory responses get a score of 3, and those without mandatory responses get a score of 1 or 2. Additional responses only increase a score from 3 to 4 or from 3 to 5.

A review of the appellant’s presentation indicates that the appellant did not ensure that proper PPE was worn by his entire crew. As this was a mandatory response, the appellant cannot receive a score higher than a 3. Nevertheless, the appellant did not state that he would assist in the evacuation. The fire building was a three-story high school measuring 260 feet by 195 feet. In these circumstances, the SMEs determined that an additional action would be to assist in the evacuation. Additionally, at the end of every scenario and prior to the questions, instructions state, “In responding to the questions, make sure your actions directly relate to the scenario. Do not assume or take for granted that general actions will contribute to your score.” Thus, candidates were required to articulate that they were assisting in the evacuation, and not leave it to the assessor to assume based on other actions.

In his response, the appellant stated, “As we go along, we’ll probably come across two unconscious victims. Notifying command that we have come across two unconscious victims and will start affecting a rescue immediately. As effective rescue we’ll call for additional crews, give a PAR, and also let command know our location and what we may need, which will probably be a stokes basket, some ropes and oxygen tanks. Also, notifying command that we need EMS outside as soon as we get outside. We’ll be going back using the same means of egress that we came in to, utilizing safety procedures and be cautious, making sure that the crew and also that the victims are getting outside to, to the EMS safely.” Responses are taken in context, and there is nothing in this passage that suggests that the appellant is assisting an evacuation. Rather, he is rescuing the two unconscious victims mentioned in question 2. The appellant missed the actions listed by the assessor, including a mandatory response, and his score of 3 for the technical component is correct.

For oral communication, the appellant received a score of 2. The assessor indicated major weaknesses in organization and inflection/rate/volume. For organization, the assessor indicated that the appellant failed to present ideas and logical fashion, and did not indicate that he was returning to a topic for question. For inflection/rate/volume, the assessor indicated that the appellant spoke at an inappropriate rate, had long pauses, and stumbled over his words. On appeal, the appellant indicated that there were two times where he indicated he was going back to a topic, and that he spoke clearly and loud enough to be heard.

In reply, a weakness in organization is defined as failing to present ideas in a logical fashion, failing to state a topic and provide supporting arguments, giving actions of order, and not indicating returning to a topic for question. A weakness in inflection/rate/volume is defined as failing to speak at an appropriate rate (long pauses/too fast/stumbling), failing to maintain appropriate pitch and volume, or improperly using pitch to convey meaning or emphasis.

The evolving scenario had three questions. A review of the appellant’s presentation indicates that the appellant gave one response, and did not indicate that he was transitioning from the first question to the second question. Rather, he incorporated his response to the second question within his response to the first question. After answering the third question, the appellant indicated he was going to respond again to the first question. Without referring to question 2, the appellant gave a few more actions regarding the victims mentioned in this question. After this, the appellant paused, and resumed discussing the victims, stating that he would let command know that he was outside with the victims prior to turning patient care over to EMS. The appellant then responded with more actions for question 3. After another pause, the appellant responded again to question 1. He was giving a list of tools when he paused midsentence for 18 seconds, then he provided more actions for question 2. He then gave an action for question 1. The appellant did not provide a summary or conclusion to his presentation. Rather, he would provide an action, pause, provide another action, pause, and did so until he simply did not provide another action after a pause and time ran out. He also repeated information already given. While the appellant may have twice indicated that he was answering another question, the presentation had a major weakness in organization.

The inflection/rate/volume weakness was indicative of the many long pauses in the presentation. The presentation was not continuous and fluid, but was delivered in a choppy manner after the appellant’s initial response to the questions. For example, after responding to the third question, the appellant paused in the presentation for 17 seconds while he checked his watch and looked at his notes. He paused in the presentation at other times for 22 seconds, 8 seconds, 18 seconds, 6 seconds, 15 seconds, 13 seconds, all prior to the two-minute warning. He paused again for 14 seconds, 18 seconds, 10 seconds, and 36 seconds before time ran out. The appellant’s score for this component is correct.

The arriving scenario involved a train/pedestrian collision. For the technical component, the assessor indicated that the appellant failed to relay that the pedestrian was struck by a train during the initial report, which was a mandatory response to question 1. He also indicated that he failed to confirm the status of the pedestrian, i.e., recover the body, which was a mandatory response to question 2. Lastly, he indicated that the appellant missed the opportunity to appoint a safety officer, which was another response for question 2. On appeal, the appellant stated that he spoke about the recovery of the body and he reported everything he saw, which included the body.

In reply, credit cannot be given for information that is implied or assumed. Question 1 asked for an initial report to be given upon arrival at the incident, and candidates were instructed to use proper radio protocols. At no time did the appellant state that there was a pedestrian struck by a train. After calling for various resources, the first mention of the pedestrian was when the appellant stated, “We’ll call an EMS company along with any left-over units to come recover the body.” The appellant did not properly respond to question 1, but simply responded to the situation. He did not state that he reported that he saw a body in his initial report, much less indicate that a pedestrian had been struck by a train. Recovering the body is not the same as either action noted by the assessor. After notifying the family to identify the body, the appellant gave a long pause, then stated, “After it has been established by EMS and ALS that the person is indeed actually dead.” While this is not a complete sentence, the appellant confirmed the status of the pedestrian, albeit very late in his handling of the scene, when he was to recover the body. While the appellant should receive credit for this response, this does not elevate his score above a 2. The appellant missed the other mandatory response, and he did not provide enough additional actions to warrant a higher score.

The supervision question for the arriving scenario involved a member of the appellant’s crew who has gone missing and is found giving an interview to a local TV crew. The assessor noted that the candidate missed the opportunity to recommend disciplinary action for the member. On appeal, the appellant states that he initiated disciplinary action.

In reply, a review of the file indicates that the appellant stated, “Once I get the firefighter to the side, and in front of the crew, I’d let him know that freelancing is unacceptable and not tolerable, and will be handled accordingly,” and “We will see if this is his first time, or his second time, and we’ll take procedures from there. Documenting everything, and letting him know that at any time he can appeal what goes on.” Once again, credit cannot be given for information that is and implied or assumed. “Handling accordingly” and “tak[ing] procedures from there,” is not initiation of disciplinary action. Candidates were required to articulate their knowledge in a way that was clear and unambiguous, and obscure references do not warrant credit. The appellant missed the action noted by the assessor, and his score of 4 for this component is correct.

CONCLUSION

A thorough review of the appellant’s submissions and the test materials indicates that the decision below is amply supported by the record, and the appellant has failed to meet his burden of proof in this matter.

ORDER

Therefore, it is ordered that this appeal be denied.

ClosingThis is the final administrative determination in this matter. Any further review should be pursued in a judicial forum.