Edward Wilcher, CSC Docket No. 2020-1175

BackgroundEdward Wilcher appeals his score on the promotional examination for Battalion Fire Chief (PM2157W), Newark. It is noted that the appellant passed the examination with a final average of 74.440 and ranks 22nd on the eligible list.

This two-part examination consisted of an integrated system of simulations designed to generate behavior similar to that required for success on the job. The first part consisted of multiple-choice items that measured specific work components identified and weighted by the job analysis. The second part consisted of three oral scenarios; a Supervision, Administration and Incident Command scenario. The examination was based on a comprehensive job analysis conducted by the Civil Service Commission, which identified the critical areas of the job. The weighting of the test components was derived from the job analysis data.

For the oral portion, candidates had 60 minutes to prepare for all three scenarios and had 10 minutes per scenario to present their response. For all three oral exercises, the candidate was to assume the role of a Battalion Fire Chief. Candidates were scored based on the content of their response (technical) and the how well they presented their response (oral communication). These components were scored on a scale of 1 to 5 with 1 being the lowest rating and 5 being the highest rating.

Each candidate in a given jurisdiction was scored by a team of three different Subject Matter Experts (SMEs), who were trained in current technical and oral communication scoring procedures. Each SME is a current or retired fire officer who held the title of Battalion Fire Chief (or Fire Officer 2) or higher. As part of the scoring process, an SME observed and noted the responses of a candidate relative to the knowledge, skills and abilities (KSAs) that each exercise was designed to measure. An SME also noted any weaknesses that detracted from the candidates overall oral communication ability. The SME then rated the candidate’s performance according to the rating standards and assigned the candidate a technical or oral communication score on that exercise.

In order to preserve the relative weighting of each of the components of the examination, the ratings for each portion were adjusted by a well-recognized statistical process known as “standardization.” Under this process, the ratings are standardized by converting the raw scores to z-scores, an expression of the deviation of the score from the mean score of the group in relation to the standard deviation of scores for the group. Each portion of the examination had a relative weight in its relation to the whole examination. Thus, the z-score for the multiple-choice portion was multiplied by a test weight of 36.53%, the oral technical scores were multiplied by a test weight of 53.91% and the oral communication scores were multiplied by a test weight of 9.56%. The weighted z-scores were summed and this became the overall final test score. This was weighted and added to the weighted seniority score. The result was standardized, then normalized, and rounded up to the third decimal place to arrive at a final average.

For the technical and oral communication components of the Supervision, Administration and Incident Command scenarios, the appellant received scores of 4, 3, 1 and 5, 3, 3, respectively.

Appellant's argumentThe appellant challenges his scores for the technical components for each scenario, and the oral communication components of the Administration and Incident Command scenarios. As a result, the appellant’s test material and a listing of possible courses of action (PCAs) for the scenarios were reviewed.

CONCLUSION

The Supervision scenario involved the use of social media where the Captain of Ladder 3 commented on a post about a recent incident to which Ladder 3 had responded. He gave out too much information, including posting pictures while on scene, and stating there was a fatality before the information had been released to the media and next of kin. Question 1 asked for specific actions to be taken now and back at the firehouse. Question 2 indicated that while the candidate is investigating the comments, he looks further into the Captain’s personal page and finds political posts regarding the local mayoral election. He is a supporter of the candidate who did not win, and he attempted to gain support for his political view as an employee of the town as a firefighter. Question 2 asked for specific actions to be taken based on this new information.

The SME indicated that the appellant missed the opportunity to ensure the legal department has been contacted. This was a response to question 1. On appeal, the appellant argues that he contacted the Employee Assistance Program (EAP).

In reply, the EAP is not the legal department. The appellant missed the action listed by the assessor and his score of 4 is correct.

In the administration scenario, the Deputy Fire Chief assigns the candidate to develop and be ready to implement a pre-plan for emergencies in places of worship in the first due response area, with an emphasis on risk management. Question 1 asked for actions to be taken to bring the first due response area incident action plans for places of worship in line with the Deputy Fire Chief’s assignment. Question 2 asked for specific information to be included in the pre-plans to effectively cover the arsonist threat.

For this scenario, the SME indicated that the appellant missed the opportunities to research literature related to arson and outline the building construction. The first it is an action to be taken in response to question 1, and the second is an action to be taken in response to question 2. On appeal, the appellant states that he mentioned police and arson, churches, planning charts and graphs and information updates.

In reply, a review of the appellant’s presentation indicates that he gave a generic response to question 1, and did not respond to question 2. At the end of his presentation, he indicated that his single response was for both questions. However, he did not provide two responses, and the responses that he gave were not appropriate for question 2. He did not research literature related to arson in response to question 1. He did not outline the building construction or give any specific information that should be included in the pre-plans to effectively cover the arsonist threat, other than prepare a response plan. The appellant stated, “Some of my committee will assist the Police, EMS, community leaders, government officials, City Council members and the media. Data from all, I will be getting data from all um, committees throughout the investigations. All files will be noted. We will fact find analysis. We’ll continue to analyze information through our program. Content and location. Development of the program. Constant monitoring, and evaluation.” The instructions in the scenario tell candidates to be as specific as possible and not to assume or take for granted that general actions will contribute to a score. Nevertheless, the appellant’s score was replete with general actions. He did not specify researching literature related to arson, and the passage above does not establish that he did.

The appellant did not mention churches in detail as he states in his appeal. Also, mentioning planning charts and graphs, and information updates, is not the same as identifying that building construction should be included in pre-plans. The appellant’s presentation lacked the actions as noted by the assessor and his presentation does not warrant a higher score for this component.

As to oral communication, the appellant received a score of 3 and the assessor noted a major weakness in the area of brevity. Specifically, he states that the appellant’s response was so brief that it precluded him from adequately addressing the scenario. The appellant argues that he felt confident on the subject matter and with a clear understanding which resulted in very direct responses.

In reply, a weakness in word brevity is found when the candidate’s response was so brief that it precluded him from adequately addressing the scenario. A review of the appellant’s presentation indicates that his presentation had this weakness. The appellant did not respond to question 2. If he thought he had, then he did not understand the question, as he did not provide specific information to be included in the pre-plans to effectively cover the arsonist threat. He also spoke in phrases at times, rather than in complete sentences. The appellant’s score for this component will not be changed.

The Incident Command scenario involved a fire at a medical facility which is a chemotherapy center. Question 1 asked for specific actions upon arrival at the scene. Question 2 indicated that during fireground operations, one of the firefighters falls into an intravenous (IV) stand with IV bags containing doxorubicin, a potent chemotherapy drug, soaking himself with the fluid, and he sustained a head injury. It asked for specific actions to be taken based on this new information.

For this scenario, the SME noted that the appellant failed to monitor or protect the truss roof, which was a mandatory response to question 1. It was also indicated that he failed to conduct emergency gross decontamination with a hoseline, which was a mandatory response to question 2. It was also indicated that the appellant missed the opportunities to ensure monitoring of air, and to communicate with a building representative, which were additional responses to question 1. On appeal, the appellant states that he mentioned wood frame and collapse information, had a Decon unit, had a Hazmat team, mentioned air supply time, and communicated with dispatch and the building engineer.

In his appeal, the appellant gives a two-minute time frame from which to glean an appropriate response of monitoring or protecting the truss roof. During that time, the appellant stated, “All members will understand the building construction. They will understand the weaknesses, the strengths, collapse potential and the (unintelligible) potential of the one-story frame house. And they will also understand the proper techniques and procedures to deploy on the scene according to the construction.” This response does not directly address the question, which asked for specific actions to take upon arriving at the scene. Further, the appellant does not explain how all members will understand the topics that he points out. Next, this was not a one-story frame house, but was a one-story wood-frame medical facility. Lastly, there is no mention of monitoring of protecting the truss roof in this response.

In the remainder of the two-minute timeframe when the appellant states that he gave the mandatory response, he obtains water supplies, stretches hose lines, conducts a size-up, performs searches, removes victims, calls for resources, notifies hospitals, expands the Incident Command System, and protects exposures. It is very unclear how the appellant finds that he monitored or protected the truss roof with this response. Again, credit is not given for information that is implied or assumed. The appellant failed to monitor or protect the truss roof as noted by the assessor. Also, even though a firefighter had soaked himself in doxorubicin, the appellant did not conduct emergency gross decontamination with a hoseline. He had a Decon unit, and a Hazmat unit, but this is not the same. An Incident Commander would be remiss to not address the specific details of the scenario by conducting an emergency gross decontamination with a hoseline in response to question 2.

Additionally, the scenario indicated that there is a thick black smoke emanating from the roof of a chemotherapy center. The action of “monitoring the air” was monitoring the air for hazardous and toxic substances. It was not monitoring air cylinders to ensure an adequate air supply, as there is no reason for that action given the facts of the scenario. The appellant did not ensure monitoring of the air. Likewise, candidates were expected to communicate with a building representative, in response to question 1. In response to question 2, after acknowledging the Mayday, and taking many actions, including repeating actions, the appellant was giving a list of resources when he stated, “Progress reports will be done. A building engineer so for any unknown um rooms, areas will um spread throughout this fire. We will have hot, warm and cold zones, and we will post uphill and upwind the chemicals and conditions um during this scene.” The grammar of the sentence regarding the building engineer is such that a building engineer is somehow related to unknown rooms and areas that spread throughout the fire. A candidate must speak in coherent sentences in order to receive credit for an action, and the appellant did not indicate that he would communicate with a building representative. The appellant missed at least two mandatory actions, and thus his score cannot be higher than a 1 regardless of the additional actions. His score for this component is correct.

For the oral communication component of the Incident Command scenario, the assessor noted minor weaknesses in organization, and inflection, rate and volume. Specifically, it was noted that the appellant consistently made random statements without supporting arguments during his responses, and he spoke in a slow/low rate with no use of pitch to convey meaning for a presentation. On appeal, the appellant argues that he was focused on speaking his actions and conveying information directly and thoroughly.

In reply, a weakness in organization is defined as failing to present ideas in a logical fashion, state a topic, and provide support and arguments, or the candidate consists only gives actions out of order orders not indicate he is returning to a topic or question. A weakness in inflection, rate and volume is failing to speak at an appropriate rate (long pauses/too fast/stumbles), fails to maintain appropriate pitch and volume, and improperly uses pitch to convey meaning or emphasis. A review of the appellant’s presentation indicates that it contained both weaknesses. At the start of the presentation the appellant was speaking very low so that it came across as mumbling, and, at times, the presentation needed to be reviewed in order to discern his words. Later, his actions were unrelated one to another, such as establishing zones and establishing a command post uphill and upwind in the same sentence, as indicated in the passage above. In his response to question 2, the appellant repeated many actions he had already taken without indicating that he was summarizing information rather than responding to question 2, which resulted in a fragmented presentation. The appellant’s score of 3 for this component will not be changed.

A thorough review of appellant’s submissions and the test materials indicates that the decision below is amply supported by the record, and the appellant has failed to meet his burden of proof in this matter.

ORDER

Therefore, it is ordered that this appeal be denied.

ClosingThis is the final administrative determination in this matter. Any further review should be pursued in a judicial forum.