Ray Rosa, CSC Docket No. 2020-1396

BackgroundRay Rosa appeals his score on the promotional examination for Battalion Fire Chief (PM2163W), Plainfield. It is noted that the appellant failed the examination.

This two-part examination consisted of an integrated system of simulations designed to generate behavior similar to that required for success on the job. The first part consisted of multiple-choice items that measured specific work components identified and weighted by the job analysis. The second part consisted of three oral scenarios; a Supervision, Administration and Incident Command scenario. The examination was based on a comprehensive job analysis conducted by the Civil Service Commission, which identified the critical areas of the job. The weighting of the test components was derived from the job analysis data.

For the oral portion, candidates had 60 minutes to prepare for all three scenarios and had 10 minutes per scenario to present their response. For all three oral exercises, the candidate was to assume the role of a Battalion Fire Chief. Candidates were scored based on the content of their response (technical) and the how well they presented their response (oral communication). These components were scored on a scale of 1 to 5 with 1 being the lowest rating and 5 being the highest rating.

Each candidate in a given jurisdiction was scored by a team of three different Subject Matter Experts (SMEs), who were trained in current technical and oral communication scoring procedures. Each SME is a current or retired fire officer who held the title of Battalion Fire Chief (or Fire Officer 2) or higher. As part of the scoring process, an SME observed and noted the responses of a candidate relative to the knowledge, skills and abilities (KSAs) that each exercise was designed to measure. An SME also noted any weaknesses that detracted from the candidates overall oral communication ability. The SME then rated the candidate’s performance according to the rating standards and assigned the candidate a technical or oral communication score on that exercise.

In order to preserve the relative weighting of each of the components of the examination, the ratings for each portion were adjusted by a well-recognized statistical process known as “standardization.” Under this process, the ratings are standardized by converting the raw scores to z-scores, an expression of the deviation of the score from the mean score of the group in relation to the standard deviation of scores for the group. Each portion of the examination had a relative weight in its relation to the whole examination. Thus, the z-score for the multiple-choice portion was multiplied by a test weight of 36.53%, the oral technical scores were multiplied by a test weight of 53.91% and the oral communication scores were multiplied by a test weight of 9.56%. The weighted z-scores were summed and this became the overall final test score. This was weighted and added to the weighted seniority score. The result was standardized, then normalized, and rounded up to the third decimal place to arrive at a final average.

For the technical and oral communication components of the Supervision, Administration and Incident Command scenarios, the appellant received scores of 3, 2, 1 and 4, 3, 3, respectively.

Appellant's argumentThe appellant challenges his score for the technical component of the Incident Command scenario. As a result, the appellant’s test material and a listing of possible courses of action (PCAs) for the scenario were reviewed.

CONCLUSION

The Incident Command scenario involved a fire at a medical facility which is a chemotherapy center. Question 1 asked for specific actions upon arrival at the scene. Question 2 indicated that during fireground operations, one of the firefighters falls into an intravenous (IV) stand with IV bags containing doxorubicin, a potent chemotherapy drug, soaking himself with the fluid, and he sustained a head injury. It asked for specific actions to be taken based on this new information.

For this scenario, the SME noted that the appellant failed to identify materials (e.g., MSDS, stock, inventory, etc.), which was a mandatory response to question 1. It was also indicated that the appellant failed to remove, triage, treat and transport exposed personnel and to conduct emergency gross decontamination, which were mandatory responses to question 2. Lastly, it was noted that he missed the opportunity to activate the Rapid Intervention Crew (RIC), which was an additional response to question 2. On appeal, the appellant states that he identified the substance, established an EMS branch for triage, treatment and transport and tracking for victims and families, and decontaminated all personnel.

In reply, at the outset, it is noted that if the candidate fails to identify two or more mandatory PCAs, a score of 1 is warranted. In his response to question 1, the appellant called for EMS for triage, treatment and transport. Yet, when one of the firefighters fell into an IV stand with IV bags containing doxorubicin, and soaked himself with the fluid, and sustained a head injury, the appellant did not conduct emergency gross decontamination with a handline for this individual, nor did he remove, triage, treat and transport exposed personnel. Having EMS on scene, for whatever reason, is a separate additional response for question 1. It was mandatory in question 2 that the candidate remove, triage, treat and transport exposed personnel, and the appellant did not take this action.

The appellant began responding to the question after the two-minute warning. Towards the end of his response to question 1, he stated, “Decon individuals if needed.” This is not sufficient to establish that the appellant would specifically conduct emergency gross decontamination with a handline on the affected firefighter. In fact, “decon” is a very general action, while emergency gross decontamination of the firefighter involves evacuating the patient (firefighter) from the high-risk area, removing his clothing, and performing a one-minute quick head-to-toe rinse with water. Also, the appellant may not know that doxorubicin is extremely acidic, or that accidental contact with the skin or eyes should be treated immediately by copious washing with water, or soap and water, or sodium bicarbonate solution, but “if needed” does not address the situation at hand. The scenario indicated that doxorubicin is a potent chemotherapy drug, and the SMEs determined that emergency gross decontamination was necessary for someone with a head wound and who had been soaked in it. It is noted that credit is not given for information that is implied or assumed.

In response to question 2, the appellant stated that if it was a Hazmat situation, he would try to get the location from the manager of the building. This response is so vague that it is unclear what the appellant is attempting to convey. He then stated he would manage the site of the event, and said, “Ah, know what is the possible relay, ah location of the individual to, to and safe recourse to, to, to, gather the individual.” This sentence makes no sense. The appellant then stated, “Um, identify the problem. Know what the, the chemical is and the hazards, and hazard risk evaluation of it.” In this case, a chemotherapy center has many different chemicals, not just doxorubicin. As such, candidates were to identify what may be in the building using MSDS sheets, bills of lading, or inventories in response to question 1. The appellant’s response is too late, refers only to the one substance encountered by the firefighter, and does not convey how the appellant intends to identify the chemical. He then states, “Proper PPE to deal with the, the rescue of the individual. And um have a Hazmat team notified to the team, notified to the team if need be to ah, ah handle the, um, the, the spill. Um, have the um, decon all members, um, terminate command and be free.” The appellant then repeats information already given. Again, he does not specifically perform emergency gross decontamination of the firefighter. The appellant’s score of 1 for this component is correct.

A thorough review of appellant’s submissions and the test materials indicates that the decision below is amply supported by the record, and the appellant has failed to meet his burden of proof in this matter.

ORDER

Therefore, it is ordered that this appeal be denied.

ClosingThis is the final administrative determination in this matter. Any further review should be pursued in a judicial forum.