Gregory Meehan, CSC Docket No. 2020-1260

BackgroundGregory Meehan appeals his score on the promotional examination for Battalion Fire Chief (PM2157W), Newark. It is noted that the appellant failed the examination.

This two-part examination consisted of an integrated system of simulations designed to generate behavior similar to that required for success on the job. The first part consisted of multiple-choice items that measured specific work components identified and weighted by the job analysis. The second part consisted of three oral scenarios; a Supervision, Administration and Incident Command scenario. The examination was based on a comprehensive job analysis conducted by the Civil Service Commission, which identified the critical areas of the job. The weighting of the test components was derived from the job analysis data.

For the oral portion, candidates had 60 minutes to prepare for all three scenarios and had 10 minutes per scenario to present their response. For all three oral exercises, the candidate was to assume the role of a Battalion Fire Chief. Candidates were scored based on the content of their response (technical) and the how well they presented their response (oral communication). These components were scored on a scale of 1 to 5 with 1 being the lowest rating and 5 being the highest rating.

Each candidate in a given jurisdiction was scored by a team of three different Subject Matter Experts (SMEs), who were trained in current technical and oral communication scoring procedures. Each SME is a current or retired fire officer who held the title of Battalion Fire Chief (or Fire Officer 2) or higher. As part of the scoring process, an SME observed and noted the responses of a candidate relative to the knowledge, skills and abilities (KSAs) that each exercise was designed to measure. An SME also noted any weaknesses that detracted from the candidates overall oral communication ability. The SME then rated the candidate’s performance according to the rating standards and assigned the candidate a technical or oral communication score on that exercise.

In order to preserve the relative weighting of each of the components of the examination, the ratings for each portion were adjusted by a well-recognized statistical process known as “standardization.” Under this process, the ratings are standardized by converting the raw scores to z-scores, an expression of the deviation of the score from the mean score of the group in relation to the standard deviation of scores for the group. Each portion of the examination had a relative weight in its relation to the whole examination. Thus, the z-score for the multiple-choice portion was multiplied by a test weight of 36.53%, the oral technical scores were multiplied by a test weight of 53.91% and the oral communication scores were multiplied by a test weight of 9.56%. The weighted z-scores were summed and this became the overall final test score. This was weighted and added to the weighted seniority score. The result was standardized, then normalized, and rounded up to the third decimal place to arrive at a final average.

For the technical and oral communication components of the Supervision, Administration and Incident Command scenarios, the appellant received scores of 4, 2, 1 and 4, 4, 5, respectively.

Appellant's argumentThe appellant challenges his scores for the technical components for each scenario. As a result, the appellant’s test material and a listing of possible courses of action (PCAs) for the scenarios were reviewed.

CONCLUSION

The Supervision scenario involved the use of social media where the Captain of Ladder 3 commented on a post about a recent incident to which Ladder 3 had responded. He gave out too much information, including posting pictures while on scene, and stating there was a fatality before the information had been released to the media and next of kin. Question 1 asked for specific actions to be taken now and back at the firehouse. Question 2 indicated that while the candidate is investigating the comments, he looks further into the Captain’s personal page and finds political posts regarding the local mayoral election. He is a supporter of the candidate who did not win, and he attempted to gain support for his political view as an employee of the town as a firefighter. Question 2 asked for specific actions to be taken based on this new information.

The SME indicated that the appellant missed the opportunity to document all actions taken, an action for question 2. On appeal, the appellant argues that he documented what took place between himself and the Captain, and sent it to the Chief.

In reply, the assessor was mistaken when he noted that the appellant missed the opportunity to document all actions taken, as he also gave the appellant credit for that action. The appellant did not receive credit in question 2 for explaining to the Captain how this (social media posts against the town’s current mayor) looks bad on the fire department, and he did not order the Captain to follow department SOPs regarding social media. Also, in question 1, the appellant did not have the Captain submit a written statement himself, or instruct him to remove comments and photos from the post. The appellant took the action listed by the assessor, but missed other actions which would enhance his score. As such, his score of 4 is correct.

In the administration scenario, the Deputy Fire Chief assigns the candidate to develop and be ready to implement a pre-plan for emergencies in places of worship in the first due response area, with an emphasis on risk management. Question 1 asked for actions to be taken to bring the first due response area incident action plans for places of worship in line with the Deputy Fire Chief’s assignment. Question 2 asked for specific information to be included in the pre-plans to effectively cover the arsonist threat.

For this scenario, the SME indicated that the appellant missed the opportunities to delegate tasks to the members of the development team, to outline the building construction, and to identify occupancy. The first is an action to be taken in response to question 1, and the second and third are actions to be taken in response to question 2. On appeal, the appellant states that he had bilingual members translate, religious heads speak to congregations, and members of the community collect information.

In reply, a review of the appellant’s presentation indicates that he concentrated his response to answering question 1, and did not respond at all to question 2. The appellant did not provide the noted two responses for question 2, and the responses that he gave in question 1 were not appropriate for question 2. A review of his presentation indicates that the appellant wanted members with dialects and those who speak multiple languages on his team. However, he did not delegate any actions to his members. He explained why he wanted these members on the team, but gave them no tasks to complete. It is noted that credit cannot be given for information that is implied or assumed. Additionally, he gave no specific information to be included in the pre-plan. As he did not respond to question 2, his presentation does not warrant a score higher than a 2.

The Incident Command scenario involved a fire at a medical facility which is a chemotherapy center. Question 1 asked for specific actions upon arrival at the scene. Question 2 indicated that during fireground operations, one of the firefighters falls into an intravenous (IV) stand with IV bags containing doxorubicin, a potent chemotherapy drug, soaking himself with the fluid, and he sustained a head injury. It asked for specific actions to be taken based on this new information.

For this scenario, the SME noted that the appellant failed to identify materials (e.g., MSDS sheets, stock, inventory, etc.), or to monitor or protect the truss roof, which were mandatory responses to question 1. It was also indicated that the appellant missed the opportunities to set up the collapse zone or to ensure shelter of displaced patients who did not require EMS treatment/transport, which were additional responses to question 1. On appeal, the appellant states that his Hazmat technical officer would be checking the database for doxorubicin. He also states that the scenario did not mention a truss roof, but was a one-story frame building.

In reply, at the outset, it is noted that if the candidate fails to identify two or more mandatory PCAs, a score of 1 is warranted. In his response to question 1, the appellant requested Hazmat, but he did not identify the materials in cancer treatment center. A chemotherapy center has many different chemicals, not just doxorubicin. As such, candidates were expected to identify what may be in the building using MSDS sheets, bills of lading, stock or inventories in response to question 1. The appellant’s response refers only to the one substance encountered by the firefighter, and ignores all other possible chemicals in the center. As to monitoring/protecting the truss roof, the scenario shows a picture of the building and an Incident Commander should be able to identify a truss roof from looking at a fire building. The appellant missed two mandatory responses, and other actions, and his score of 1 will not be changed.

A thorough review of appellant’s submissions and the test materials indicates that the decision below is amply supported by the record, and the appellant has failed to meet his burden of proof in this matter.

ORDER

Therefore, it is ordered that this appeal be denied.

ClosingThis is the final administrative determination in this matter. Any further review should be pursued in a judicial forum.