Robert Fanholz Jr., CSC Docket No. 2020-1370

BackgroundRobert Fanholz Jr. appeals his score on the promotional examination for Battalion Fire Chief (PM2144W), Asbury Park. It is noted that the appellant failed the examination.

This two-part examination consisted of an integrated system of simulations designed to generate behavior similar to that required for success on the job. The first part consisted of multiple-choice items that measured specific work components identified and weighted by the job analysis. The second part consisted of three oral scenarios; a Supervision, Administration and Incident Command scenario. The examination was based on a comprehensive job analysis conducted by the Civil Service Commission, which identified the critical areas of the job. The weighting of the test components was derived from the job analysis data.

For the oral portion, candidates had 60 minutes to prepare for all three scenarios and had 10 minutes per scenario to present their response. For all three oral exercises, the candidate was to assume the role of a Battalion Fire Chief. Candidates were scored based on the content of their response (technical) and the how well they presented their response (oral communication). These components were scored on a scale of 1 to 5 with 1 being the lowest rating and 5 being the highest rating.

Each candidate in a given jurisdiction was scored by a team of three different Subject Matter Experts (SMEs), who were trained in current technical and oral communication scoring procedures. Each SME is a current or retired fire officer who held the title of Battalion Fire Chief (or Fire Officer 2) or higher. As part of the scoring process, an SME observed and noted the responses of a candidate relative to the knowledge, skills and abilities (KSAs) that each exercise was designed to measure. An SME also noted any weaknesses that detracted from the candidates overall oral communication ability. The SME then rated the candidate’s performance according to the rating standards and assigned the candidate a technical or oral communication score on that exercise.

In order to preserve the relative weighting of each of the components of the examination, the ratings for each portion were adjusted by a well-recognized statistical process known as “standardization.” Under this process, the ratings are standardized by converting the raw scores to z-scores, an expression of the deviation of the score from the mean score of the group in relation to the standard deviation of scores for the group. Each portion of the examination had a relative weight in its relation to the whole examination. Thus, the z-score for the multiple-choice portion was multiplied by a test weight of 36.53%, the oral technical scores were multiplied by a test weight of 53.91% and the oral communication scores were multiplied by a test weight of 9.56%. The weighted z-scores were summed and this became the overall final test score. This was weighted and added to the weighted seniority score. The result was standardized, then normalized, and rounded up to the third decimal place to arrive at a final average.

For the technical and oral communication components of the Supervision, Administration and Incident Command scenarios, the appellant received scores of 3, 1, 1 and 2, 3, 5, respectively.

Appellant's argumentThe appellant challenges his scores for the technical component for the Administration and Incident Command scenarios, and for the oral communication component of the Supervision scenario. As a result, the appellant’s test material and a listing of possible courses of action (PCAs) for the scenarios were reviewed.

CONCLUSION

The Supervision scenario involved a crew who abandoned their apparatus and attended a sporting event for free when they should have been staged outside of the event. Later that evening, Engine 7’s crew is late to respond to a call where the candidate arrived one minute earlier than they did, and the homeowner is upset. This question asks for specific actions to be taken now and the back at the firehouse.

As to oral communication, the appellant received a score of 2 and the assessor noted major weaknesses in the areas of brevity and specificity, and a minor weakness in nonverbal communication. Specifically, for brevity, he states that the response was so brief that it precluded him from adequately addressing the scenario (approximately one minute). For specificity, the assessor noted that the appellant’s actions were too general, and he failed to convey how those actions were to be carried out. For example, he did not indicate why he was investigating or provide the details of the investigation. For nonverbal communication, the assessor noted that the appellant’s gestures were distracting as he held up the booklet during his response. On appeal, the appellant only contends the weakness in specificity. He states that he had multiple meetings with the firefighters and the officer involved, and it should have been clear why he was investigating as the scenario indicated the infraction. He states that he interviewed other battalion chiefs and inquired if the crew had previous problems following direct orders.

In reply, a weakness in word brevity is found when the candidate’s response was so brief that it precluded him from adequately addressing the scenario. A weakness in specificity is found when actions are too general, and the candidate fails to convey how those actions are carried out. A weakness in nonverbal communication is when a candidate uses gestures ineffectively, thereby causing confusion or distractions, and failing to maintain eye contact when speaking.

A review of the appellant’s presentation indicates that he responded to the scenario in a minute and 45 seconds, thereby missing opportunities to provide further information. However, weaknesses in specificity and nonverbal communication were not found. The appellant ignored the issue of the citizen’s threat to contact city hall, but this is covered by a weakness in brevity. The appellant provided enough actions for a score of 3 on the technical component, and the actions he did not take are a weakness in brevity, not specificity. The appellant held the booklet in front of him, however, this was not a distraction. As such, the appellant’s score for this component should have been a 3, which is acceptable, and which contains one major weakness. In any event, the appellant would have failed even if he scored 5s in oral communication for all performances as he did not have an average of 2.5 in the technical portions. As the appellant would still fail the examination, a score change would not result in a passing score.

In the administration scenario, a drug addiction facility is being located in an established a residential area in the candidate’s first due response area, and the citizens are upset. A credible threat has been made regarding the ribbon-cutting ceremony to be held in two weeks regarding a protest at the ceremony and a firebombing on new construction. The candidate is to develop an incident action plan for the ceremony and address fire protection for this building. Question 1 asked for steps to take to evaluate and address the community fire protection regarding the dedication ribbon-cutting ceremony and beyond at the new Crawford Drug Addiction Facility. Question 2 asked for specific information that should be included in this incident plan to effectively cover the threats made on social media.

For this scenario, the SME indicated that the appellant missed the opportunities to have the Deputy Fire Chief approve the plan in question 1, and to identify exposures, determine sprinkler system operability, and to identify hydrant locations, for question 2. On appeal, the appellant states that since the scenario did not state who owned the drug treatment facility, it should fall under the authority having jurisdiction, and he did not need the Deputy Fire Chief’s approval of the plan. He states that he assigned a division group supervisor to each floor of the facility to facilitate fire operations which include the use of the sprinkler system.

In reply, the scenario indicated that the Deputy Fire Chief has given the candidate the assignment of developing an incident action plan, and addressing the fire protection at this building. The appellant is not stating that he obtained the Deputy Fire Chief’s approval of the plan, but that he did not have to do so since the scenario did not state the owner of the facility. This reasoning is simply flawed. If a supervisor asks for the development an incident action plan for a building, the subordinate should get the supervisor’s approval of the incident action plan that was created. The appellant thanked the Deputy Fire Chief for the opportunity to rectify the situation, and did not indicate that he had no jurisdiction for this assignment. He did not keep the Deputy Fire Chief informed of his progress or ask him to approve the plan.

The appellant assigned a division group supervisor to each floor of the facility, and was credited with implementing his plan. However, this action is not one of those listed by the assessor. The appellant did not respond to each question separately, but provided one continuous reply, which included a few responses to question 1 and no responses to question 2. The appellant’s arguments on appeal are in response to question 1, while the assessor comments are specific information that should be included in the incident action plan. He did not mention identifying exposures, determining sprinkler system operability, or identifying hydrant locations in his presentation. The appellant did not respond to each question separately, but provided one continuous reply which was sparse for question 1 and unresponsive to question 2. He missed the actions noted by the SME, and his is score of 1 for this component is correct.

The Incident Command scenario involved a fire at a powder metallurgy facility. Question 1 asked for specific actions upon arrival at the scene. Question 2 indicated that during fireground operations, there is an explosion inside the facility and a Mayday is being broadcast. It asked for specific actions to be taken based on this new information.

For this scenario, the SME noted that the appellant failed to set up a collapse zone; failed to monitor air quality; and failed to set up cold and warm and hot zones prior to the explosion. These were mandatory responses to question 1. It was also indicated that the appellant failed to remove the downed firefighter, which was a mandatory response to question 2. On appeal, the appellant states that he ordered a Hazmat team to monitor the air, and set up zones, and that he had the RIC rescue the firefighter and ALS and BLS treat the firefighter.

In reply, the appellant stated, “I would request utilities, police, EMS, ALS, a rehab unit, a RIT team, an arson investigator, fire official, and a Hazmat team to stand by if needed.” The appellant received credit for requesting a Hazmat and establishing a RIC in question 1, however, this response is simply not specific enough to warrant credit for the mandatory responses noted by the assessor. He did not order the Hazmat unit to monitor the air, or set up zones, and he did not set up a collapse zone in response to question 1. For question 2, he activated the RIC, but he did not state that they should remove the downed firefighter. He called ALS group supervisor for treatment of firefighters, but he did not indicate that the firefighters were removed, and credit is not given for information that is implied or assumed. The appellant missed four mandatory responses, and his score of 1 for this component is correct.

A thorough review of appellant’s submissions and the test materials indicates that the decision below is amply supported by the record, and the appellant has failed to meet his burden of proof in this matter.

ORDER

Therefore, it is ordered that this appeal be denied.

ClosingThis is the final administrative determination in this matter. Any further review should be pursued in a judicial forum.