Steven Spurr, CSC Docket No. 2020-1178

BackgroundSteven Spurr appeals his score on the promotional examination for Battalion Fire Chief (PM2169W), Union Township. It is noted that the appellant failed the examination.

This two-part examination consisted of an integrated system of simulations designed to generate behavior similar to that required for success on the job. The first part consisted of multiple-choice items that measured specific work components identified and weighted by the job analysis. The second part consisted of three oral scenarios; a Supervision, Administration and Incident Command scenario. The examination was based on a comprehensive job analysis conducted by the Civil Service Commission, which identified the critical areas of the job. The weighting of the test components was derived from the job analysis data.

For the oral portion, candidates had 60 minutes to prepare for all three scenarios and had 10 minutes per scenario to present their response. For all three oral exercises, the candidate was to assume the role of a Battalion Fire Chief. Candidates were scored based on the content of their response (technical) and the how well they presented their response (oral communication). These components were scored on a scale of 1 to 5 with 1 being the lowest rating and 5 being the highest rating.

Each candidate in a given jurisdiction was scored by a team of three different Subject Matter Experts (SMEs), who were trained in current technical and oral communication scoring procedures. Each SME is a current or retired fire officer who held the title of Battalion Fire Chief (or Fire Officer 2) or higher. As part of the scoring process, an SME observed and noted the responses of a candidate relative to the knowledge, skills and abilities (KSAs) that each exercise was designed to measure. An SME also noted any weaknesses that detracted from the candidates overall oral communication ability. The SME then rated the candidate’s performance according to the rating standards and assigned the candidate a technical or oral communication score on that exercise.

In order to preserve the relative weighting of each of the components of the examination, the ratings for each portion were adjusted by a well-recognized statistical process known as “standardization.” Under this process, the ratings are standardized by converting the raw scores to z-scores, an expression of the deviation of the score from the mean score of the group in relation to the standard deviation of scores for the group. Each portion of the examination had a relative weight in its relation to the whole examination. Thus, the z-score for the multiple-choice portion was multiplied by a test weight of 36.53%, the oral technical scores were multiplied by a test weight of 53.91% and the oral communication scores were multiplied by a test weight of 9.56%. The weighted z-scores were summed and this became the overall final test score. This was weighted and added to the weighted seniority score. The result was standardized, then normalized, and rounded up to the third decimal place to arrive at a final average.

For the technical and oral communication components of the Supervision, Administration and Incident Command scenarios, the appellant received scores of 2, 2, 1 and 4, 3, 5, respectively.

Appellant's argumentThe appellant challenges his scores for the technical component for each scenario, and the oral communication component of the Administration scenario. As a result, the appellant’s test material and a listing of possible courses of action (PCAs) for the scenarios were reviewed.

CONCLUSION

The Supervision scenario involved a crew who abandoned their apparatus and attended a sporting event for free when they should have been staged outside of the event. Later that evening, Engine 7’s crew is late to respond to a call where the candidate arrived one minute earlier than they did, and the homeowner is upset. This question asks for specific actions to be taken now and the back at the firehouse.

The SME indicated that the appellant missed the opportunities to keep the Chief informed of possible City Hall threats/contact, to review applicable SOGs and SOPs, and to inform them of the right to union representation. On appeal, the appellant argues that he did not have to inform them of the right to union representation, as this is covered by “Weingarten rights,” and that the supervisor has no obligation to inform a subordinate that he is entitled to union representation. Also, he maintains that the subordinates may not have been in a union.

In response, a review of the appellant’s audiotape and related examination material indicates that the appellant missed the actions listed by the assessors, and his argument on appeal does not support that he took those actions. Instead, the appellant argues that he should not have had to take one of the actions. Eventually, a member is going to violate a policy or procedure, and there must be a disciplinary system in place that has been clearly communicated to all members. Disciplinary measures may have been negotiated as part of a labor contract, and may consist of disciplinary steps. While the Weingarten rights do not require the employer to ask the employee if she/he wants a representative, the SMEs found that this is a good practice in maintaining employee morale and good labor relations. The Commission is not persuaded by the appellant’s argument that he should not have to inform a subordinate that he is entitled to union representation. The appellant missed the three actions noted by the SME, and his score of 2 will not be changed.

In the administration scenario, a drug addiction facility is being located in an established a residential area in the candidate’s first due response area, and the citizens are upset. A credible threat has been made regarding the ribbon-cutting ceremony to be held in two weeks regarding a protest at the ceremony and a firebombing on new construction. The candidate is to develop an incident action plan for the ceremony and address fire protection for this building. Question 1 asked for steps to take to evaluate and address the community fire protection regarding the dedication ribbon-cutting ceremony and beyond at the new Crawford Drug Addiction Facility. Question 2 asked for specific information that should be included in this incident plan to effectively cover the threats made on social media.

For this scenario, the SME indicated that the appellant missed the opportunities to prepare an evacuation plan for civilians/crowd control, to identify exposures, and to research the possibility of a secondary attack. These were actions to be taken in response to question 2. On appeal, the appellant states that he ordered crowd control. He also states “No PCA’s were noted as missing. Technical score was a 2. Appeal – No PCA’s missed. Minimum score should have been a 3.”

In reply, it is unclear what the appellant is referring to when he states that the SME did not note any PCAs as missing. The SME clearly wrote three actions. Not all scenarios have mandatory responses, and this is one of them. The appellant missed many actions to provide additional action in response to question 1, and all three actions noted by the assessor for question 2. As to preparing an evacuation plan for civilians/crowd control, this action was in regard to covering the threat of a sizable protest and a fire bombing on the new construction. The appellant did not respond to each question separately, but provided one continuous reply. At one point, the appellant stated, “Um, area containment dur, during it, you know, setting up barriers to, to keep the crowd small but again, well not small, but contained that if something does happen it’s easier for us to control. But again, that would fall mostly on law enforcement.” In this context, the appellant is not looking out for the welfare of the people in the crowd, such as having an evacuation plan for civilians or crowd control, but is containing them. The appellant has not indicated a plan to evacuate civilians, and his control of the crowd was area containment using barriers. A holistic view of the appellant’s presentation indicates that his score of 2 for this component is correct.

As to oral communication, the appellant received a score of 3 and the assessor noted a major weakness in the area of word usage/grammar. Specifically, he states that the appellant used “ahs” “ums” and “you know” throughout his presentation. The appellant argues that this score was not consistent with the score for oral communication for the supervisory scenario although the comment was the same.

In reply, final scores are derived by examining behavior throughout the entire exercise. Assessors are not concerned with feelings, opinions, or inferences, just what candidates did during the exam. Behaviors can be identified, and the score on one component does not determine the score on another component. If that were the case, it would not be necessary to delineate, define and score different components. The oral communication score is characterized by the effective expression of ideas in individual situations (including organization, gestures, and nonverbal communication), and adjusting language or terminology to intended audiences. Since behaviors are different in each presentation, it is possible to score differently, depending on the actual performance. For the supervision scenario, the SME noted a minor weakness in word usage/grammar, while for the administration scenario, the SME noted a major weakness in word usage/grammar. In any event, the appellant would have failed even if he scored 5s in oral communication for both performances as he did not have an average of 2.5 in the technical portions.

A weakness in word usage/grammar is found when the candidate mispronounces words, uses sentences that are grammatically incorrect, repeats words and phrases, or uses inappropriate words. A review of the appellant’s presentation indicates that his presentation has this weakness. For example, the appellant stated, “Um, whether it being a um, a website department website, um or even Facebook to let people know that the fire department is out there. Um, not directing to a specific um, organization but that what we do as far as fire prevention um, as far as you know, city um edu.. or, I’m sorry, ah fire education, things like that.” This passage has numerous distracting verbal mannerisms such as “um” and “you know,” is grammatically incorrect, repeats words, and is difficult to follow. At another point, the appellant stated, “So I would have them sta…, one engine, stage outside um the area ah, in full protective gear ready to respond, and and like I said, you know, remote area. So this way the public doesn’t perceive a threat. It doesn’t create any angst, among them.” In this passage, the appellant does not complete a word, and uses “um,” “ah,” “like I said” and “you know.” The appellant repeats the word “and,” and the phrase “remote area” is not used appropriately in the sentence. At another point, the appellant stated, “Um, a unified command post at the incident. Updates. Info…information sharing.” Here, the appellant was not speaking in complete sentences, but was reading bullet points from his notes. The appellant’s presentation had the weakness noted by the assessor and his score of 3 will not be changed.

The Incident Command scenario involved a fire at a powder metallurgy facility. Question 1 asked for specific actions upon arrival at the scene. Question 2 indicated that during fireground operations, there is an explosion inside the facility and a Mayday is being broadcast. It asked for specific actions to be taken based on this new information.

For this scenario, the SME noted that the appellant failed to identify materials; and failed to monitor/protect the truss roof. These were mandatory responses to question 1. It was also indicated that the appellant missed the opportunities to set up a collapse zone and to request a rehab unit, which were additional responses to question 1. On appeal, the appellant states that the materials were identified in the scenario, which he repeated, and he called on-site experts and Hazmat to respond. He states that the scenario did not indicate truss roof construction, and that he requested EMS to set up a rehab.

In reply, the scenario indicated that upon arrival, the candidates saw smoke emanating from one of the loading bay doors, and a manager indicated that a fire started in the loading area. While the scenario did not specify that it was a truss roof, something must hold up the spans. Given that it was built in 1967, it is not heavy timber construction. While it is clearly not a bowstring truss, it could be a steel bar or web truss. Next, the appellant consulted his Hazmat guidebooks, and therefore identified the materials. The assessor listed this action as a mandatory action, when it was an additional action. Similarly, the assessor listed setting up a collapse zone as an additional action when it was a mandatory action, and the appellant missed that response. Although these actions were inadvertently switched, the appellant did not take the action of setting up a collapse zone. He also failed to expand the Incident Command System (ICS), and protect the storage area by hoseline or removal of the drums.

Lastly, requesting EMS was a separate response from requesting a rehab unit. The appellant requested EMS, and then had them set up a rehab area. As such, while the appellant did not request a rehab unit, he had a rehab area set up by EMS. Regardless, this is an additional action, not a mandatory action. Requesting Hazmat, and requesting EMS, were separate responses for which the appellant was credited. Although the appellant acknowledged the Mayday in response to question 2, he failed to remove the downed firefighter. The appellant missed other additional actions, such as switching fire operations to another frequency in question 2, and checking for extension into the other buildings in question 1. A review of the presentation indicates that the appellant’s score of 1 is correct.

A thorough review of appellant’s submissions and the test materials indicates that the decision below is amply supported by the record, and the appellant has failed to meet his burden of proof in this matter.

ORDER

Therefore, it is ordered that this appeal be denied.

ClosingThis is the final administrative determination in this matter. Any further review should be pursued in a judicial forum.