John Meixedo, CSC Docket No. 2020-1290

BackgroundJohn Meixedo appeals his score on the promotional examination for Battalion Fire Chief (PM2157W), Newark. It is noted that the appellant failed the examination.

This two-part examination consisted of an integrated system of simulations designed to generate behavior similar to that required for success on the job. The first part consisted of multiple-choice items that measured specific work components identified and weighted by the job analysis. The second part consisted of three oral scenarios; a Supervision, Administration and Incident Command scenario. The examination was based on a comprehensive job analysis conducted by the Civil Service Commission, which identified the critical areas of the job. The weighting of the test components was derived from the job analysis data.

For the oral portion, candidates had 60 minutes to prepare for all three scenarios and had 10 minutes per scenario to present their response. For all three oral exercises, the candidate was to assume the role of a Battalion Fire Chief. Candidates were scored based on the content of their response (technical) and the how well they presented their response (oral communication). These components were scored on a scale of 1 to 5 with 1 being the lowest rating and 5 being the highest rating.

Each candidate in a given jurisdiction was scored by a team of three different Subject Matter Experts (SMEs), who were trained in current technical and oral communication scoring procedures. Each SME is a current or retired fire officer who held the title of Battalion Fire Chief (or Fire Officer 2) or higher. As part of the scoring process, an SME observed and noted the responses of a candidate relative to the knowledge, skills and abilities (KSAs) that each exercise was designed to measure. An SME also noted any weaknesses that detracted from the candidates overall oral communication ability. The SME then rated the candidate’s performance according to the rating standards and assigned the candidate a technical or oral communication score on that exercise.

In order to preserve the relative weighting of each of the components of the examination, the ratings for each portion were adjusted by a well-recognized statistical process known as “standardization.” Under this process, the ratings are standardized by converting the raw scores to z-scores, an expression of the deviation of the score from the mean score of the group in relation to the standard deviation of scores for the group. Each portion of the examination had a relative weight in its relation to the whole examination. Thus, the z-score for the multiple-choice portion was multiplied by a test weight of 36.53%, the oral technical scores were multiplied by a test weight of 53.91% and the oral communication scores were multiplied by a test weight of 9.56%. The weighted z-scores were summed and this became the overall final test score. This was weighted and added to the weighted seniority score. The result was standardized, then normalized, and rounded up to the third decimal place to arrive at a final average.

For the technical and oral communication components of the Supervision, Administration and Incident Command scenarios, the appellant received scores of 3, 1 and 2 and 5, 3 and 5, respectively.

Appellant's argumentThe appellant challenges his scores for the technical and oral communication components for the Administration scenario, and for the technical component of the Incident Command scenario. As a result, the appellant’s test material and a listing of possible courses of action (PCAs) for the scenario were reviewed.

CONCLUSION

In the administration scenario, the Deputy Fire Chief assigns the candidate to develop and be ready to implement a preplan for emergencies in places of worship in the first due response area, with an emphasis on risk management. Question 1 asked for actions to be taken to bring the first due response area incident action plans for places of worship in line with the Deputy Fire Chief’s assignment. Question 2 asked for specific information to be included in the pre-plans to effectively cover the arsonist threat.

For this scenario, the SME indicated that the appellant missed the opportunities to confirm the budget through the Deputy Fire Chief and delegate tasks to the members of the development team. These were actions to be taken in response to question 1. It was also noted that he did not give the responses of identifying exposures or outline building exposures, as responses to question 2. It is noted that he second response should have read “outline building construction.” On appeal, the appellant states that the scenario did not mention exposures or building construction, and places of worship can be anywhere on a street and have any construction. He argues that for question 1, he reached out to other organizations for information to be included in his educational program.

In reply, a review of the appellant’s presentation indicates that he reached out to other churches himself, and did not delegate the task. After doing so, he assembled a “board” and presented them with information to develop an educational course for the whole department. He then spoke mainly of developing and giving an educational course as his main action to bring his area incident action plans for places of worship in line with the assignment. In doing so, he failed to respond to question 2, providing no information at all that should be included in the pre-plans to cover the arsonist threat, which made them ineffective. The whole point of question 2 was that the candidate should identify his places of worship and lists specific information that would be needed for each. He missed many responses to question 1, including those listed by the assessor, and all responses to question 2. As his pre-plan contained no information, the appellant failed to implement a preplan for emergencies in places of worship in the first due response area. His score of 1 for this component is correct.

As to the oral communication component of this scenario, the assessor noted weaknesses in brevity and specificity. For brevity, the assessor indicated that his three-minute response did not address the scenario. For specificity, he noted that the actions were too general and failed to convey how the actions were to be carried out. On appeal, the appellant argues that he addressed the concerns as much as possible, explained all necessary actions and did not pause or divert from the task. He claims to have carried out the tasks to complete the program successfully.

In reply, a weakness in brevity is defined as a response so brief that it precludes adequately addressing the scenario. A weakness in specificity is defined as providing actions that are too general and failing to convey how those actions are carried out. The appellant’s presentation had both weaknesses. The appellant did not address question 2 at all, and his response to question 1 was sparse. The assignment was to develop and be ready to implement a preplan for emergencies in places of worship in the first due response area, with an emphasis on risk management. The appellant spoke only on developing an educational course. He assembled a team, but gave them no tasks. He did not identify any places of worship in his first due area, and gave no specific information to be included in the pre-plan regarding any places of worship. His score reflects at least two weaknesses in the presentation and will not be changed.

The Incident Command scenario involved a fire at a medical facility which is a chemotherapy center. Question 1 asked for specific actions upon arrival at the scene. Question 2 indicated that during fireground operations, one of the firefighters falls into an intravenous (IV) stand with IV bags containing doxorubicin, a potent chemotherapy drug, soaking himself with the fluid, and he sustained a head injury. It asked for specific actions to be taken based on this new information.

For this scenario, the SME noted that the appellant failed to identify materials (e.g., MSDS sheets, bill of lading, stock, inventory, etc.), which was a mandatory response to question 1. It was also indicated that he missed the opportunities to request Hazmat, and to set up a collapse zone, which were additional responses to question 1. On appeal, the appellant states that he informed all personnel to be aware of the chemicals and oils stored on site, and he was aware of the contents of the building. The appellant maintains that doxorubicin is not a hazardous material, but that he conducted a gross decontamination of all personnel. He states that he mentioned a possible collapse on three occasions, by having a charged line on the trusses, removing patients, and requesting a building engineer check the integrity of the structure.

In reply, the SMEs determined that, as this was a chemotherapy center, the Incident Commander (IC) should determine what materials are in the building. That would be covered by requesting the MSDS sheets, bills of lading, stock, or inventories. The appellant did not take this action in response to question 1. The appellant stated, “Upon arrival, or just before arrival, I will perform a mental size up of this location from pre-fire planning, knowing my district.” It is noted that credit cannot be given for information that is implied or assumed. This response does not indicate that he was aware of the contents of the building. The appellant also stated, “I’m also going to have a separate engine company bring in another water supply and get inside that garage, and keep the fire at bay. The garage is going to be evacuated. We cannot let the fire get into the garage because of possible Hazmat situation, because of all the oils that are in there.” This response is not the same as identifying materials, nor is it even what the appellant stated on appeal, that he informed all personnel to be aware of the chemicals and oils stored on site. It is merely a statement that oils in the garage were a possible Hazmat situation and he could not let fire get into the garage. This does not make sense, as there was no garage in the medical building. The fire was in the waiting room and through the front of the facility. The appellant spoke of this garage on multiple occasions, which does not address the specifics of this scenario, which does not describe a garage or have one in the pictures. Even so, oil in a garage is not the same as chemotherapy drugs.

The SMEs disagree with the appellant in that they found that doxorubicin was a hazardous material. Doxorubicin is so toxic that there is a limit to the amount that can be given in a lifetime, and it is one of the most powerful chemotherapy drugs ever invented. Regardless of Doxorubicin, there were many other chemotherapy drugs present in this facility, and the appellant is simply in error if he believes that chemotherapy drugs are not hazardous materials. He did not request a Hazmat team in response to question 1. His entire presentation has been reviewed, and the appellant did not mention setting up a collapse zone, and credit is not awarded for information that is implied or assumed. If the appellant was expecting a collapse, he did not prepare for it by setting up a collapse zone. Rather, he was concerned about keeping water on the roofline. The appellant missed a mandatory response, and his score of 2 is correct.

A thorough review of appellant’s submissions and the test materials indicates that the decision below is amply supported by the record, and the appellant has failed to meet his burden of proof in this matter.

ORDER

Therefore, it is ordered that this appeal be denied.

ClosingThis is the final administrative determination in this matter. Any further review should be pursued in a judicial forum.