John Cleary, CSC Docket No. 2020-1247

BackgroundJohn Cleary appeals his score on the promotional examination for Battalion Fire Chief (PM2146W), Bayonne. It is noted that the appellant passed the examination with a final average of 85.170 and ranks seventh on the eligible list.

This two-part examination consisted of an integrated system of simulations designed to generate behavior similar to that required for success on the job. The first part consisted of multiple-choice items that measured specific work components identified and weighted by the job analysis. The second part consisted of three oral scenarios; a Supervision, Administration and Incident Command scenario. The examination was based on a comprehensive job analysis conducted by the Civil Service Commission, which identified the critical areas of the job. The weighting of the test components was derived from the job analysis data.

For the oral portion, candidates had 60 minutes to prepare for all three scenarios and had 10 minutes per scenario to present their response. For all three oral exercises, the candidate was to assume the role of a Battalion Fire Chief. Candidates were scored based on the content of their response (technical) and the how well they presented their response (oral communication). These components were scored on a scale of 1 to 5 with 1 being the lowest rating and 5 being the highest rating.

Each candidate in a given jurisdiction was scored by a team of three different Subject Matter Experts (SMEs), who were trained in current technical and oral communication scoring procedures. Each SME is a current or retired fire officer who held the title of Battalion Fire Chief (or Fire Officer 2) or higher. As part of the scoring process, an SME observed and noted the responses of a candidate relative to the knowledge, skills and abilities (KSAs) that each exercise was designed to measure. An SME also noted any weaknesses that detracted from the candidates overall oral communication ability. The SME then rated the candidate’s performance according to the rating standards and assigned the candidate a technical or oral communication score on that exercise.

In order to preserve the relative weighting of each of the components of the examination, the ratings for each portion were adjusted by a well-recognized statistical process known as “standardization.” Under this process, the ratings are standardized by converting the raw scores to z-scores, an expression of the deviation of the score from the mean score of the group in relation to the standard deviation of scores for the group. Each portion of the examination had a relative weight in its relation to the whole examination. Thus, the z-score for the multiple-choice portion was multiplied by a test weight of 36.53%, the oral technical scores were multiplied by a test weight of 53.91% and the oral communication scores were multiplied by a test weight of 9.56%. The weighted z-scores were summed and this became the overall final test score. This was weighted and added to the weighted seniority score. The result was standardized, then normalized, and rounded up to the third decimal place to arrive at a final average.

For the technical and oral communication components of the Supervision, Administration and Incident Command scenarios, the appellant received scores of 5, 5, and 1, and 4, 4, and 5, respectively.

Appellant's argumentThe appellant challenges his score for the technical component for the Incident Command scenario. As a result, the appellant’s test material and a listing of possible courses of action (PCAs) for the scenario were reviewed.

CONCLUSION

The Incident Command scenario involved a fire at a medical facility which is a chemotherapy center. Question 1 asked for specific actions upon arrival at the scene. Question 2 indicated that during fireground operations, one of the firefighters falls into an intravenous (IV) stand with IV bags containing doxorubicin, a potent chemotherapy drug, soaking himself with the fluid, and he sustained a head injury. It asked for specific actions to be taken based on this new information.

For this scenario, the SME noted that the appellant failed to identify materials (e.g., MSDS sheets, bill of lading, stock, inventory, etc.), which was a mandatory response to question 1. It was also indicated that he failed to conduct emergency gross decontamination with a hoseline, which was a mandatory response to question 2. It was also indicated that the appellant missed the opportunities to set up a collapse zone, which was an additional response to question 1, and to activate the

Rapid Intervention Crew (RIC). On appeal, the appellant states that he called for a Hazmat team and assigned a Hazmat safety officer and Hazmat group supervisor. He states that, in response to question 2, he had the Hazmat team identify the material using MSDS sheets, Chemtrec and any technical data sheets supplied by the facility. Then he monitored the air and established isolation zones. Rather than activate a RIC, the appellant states that he chose to remove the firefighter with members of the Hazmat team in Level A suits, and brought him to treatment. As to collapse zones, he said that he would address the need for defensive operations while accessing his progress reports.

In reply, the SMEs determined that, as this was a chemotherapy center, the Incident Commander (IC) should determine what materials are in the building. That would be covered by requesting the MSDS sheets, bills of lading, stock, or inventories. The appellant did not take this action in response to question 1. Requesting Hazmat was another response for which the appellant received credit. However, this action is not the same as identifying materials that were in the building. In question 2, the appellant had the Hazmat response team determine what doxorubicin is with SDS and Chemtrec charts, and charts from the facility book. However, this ignores all the other chemotherapy drugs that would be in a cancer treatment center.

Also in question 1, the appellant considered going defensive if all was not going according to plan. This is not the same as setting up collapse zones, and credit is not given for information that is implied or assumed.

Question 2 referred to doxorubicin, and for this question the appellant stated, “At this time, ah a company is reporting that a firefighter has been, is fallen into an IV bag and is contaminated with a head injury. At this time, I will notify the Hazmat response team to report to the incident. I will ident.., they will identify the product and they will go in with Level A suits. They will isolate the area. They will establish ah, isolation zones, hot, cold, warm. They will remove all firefighters from the area, determine if it is safe to go in, um, public protective actions. They will evacuate all firefighters from the scene. They will determine the amount of, determine what the product is with SDS and Chemtrec charts and charts from the facility, the facility um, book. They will ensure the proper level PPE, Level A, to remove this firefighter. They will meter the area, rescue and remove the firefighter, for treatment triage transportation. They will … (pause) Okay, I also will address the rescue in here. We will immediately issue a Mayday. Um, I will clear all radio communication, establish command and tactical channels for the rescue. I will establish additional, ah, separate for the Fireground operations command and tactical channel. I will assign battalion three as interior rescue supervisor at this time. Redirect him. Um, I will also establish a supervisor for the rescue as well as continue firefighting operations. So firefighting on separate channel, rescue on separate channel. Have the member ah, give a mayday, the LUNAR acronym. Call for utilities to be secured.”

The appellant established a RIC in response to question 1, but did not activate it in response to question 2. Instead, he notified the Hazmat response team to “report to the incident.” The RIC includes members who are specially trained to rescue firefighters at all working fires or other situations which place members in a hazardous area. The Hazmat response team are responsible for directly managing and controlling Hazmat problems. As such, the Hazmat response team is not a suitable substitute for the RIC in rescuing a downed firefighter in a working fire situation. Redirecting a Battalion Fire Chief and assigning him as interior rescue supervisor is not the same. Thus, the appellant missed this additional response. He received credit for removing, triage, treatment and transport for the exposed personnel. Nevertheless, he did not activate the RIC, an additional response, or conduct emergency gross decontamination with a handline, which was a mandatory response, in response to question 2. As the appellant missed two mandatory responses, his score of 1 for this component is correct.

A thorough review of appellant’s submissions and the test materials indicates that the decision below is amply supported by the record, and the appellant has failed to meet his burden of proof in this matter.

ORDER

Therefore, it is ordered that this appeal be denied.

ClosingThis is the final administrative determination in this matter. Any further review should be pursued in a judicial forum.