Jose Pujols, CSC Docket No. 2020-1176

BackgroundJose Pujols appeals his score on the promotional examination for Fire Officer 2 (PM2158W), North Hudson Fire and Rescue Service. It is noted that the appellant passed the examination with a final average of 82.420 and ranked seventh on the eligible list.

This two-part examination consisted of an integrated system of simulations designed to generate behavior similar to that required for success on the job. The first part consisted of multiple-choice items that measured specific work components identified and weighted by the job analysis. The second part consisted of three oral scenarios; a Supervision, Administration and Incident Command scenario. The examination was based on a comprehensive job analysis conducted by the Civil Service Commission, which identified the critical areas of the job. The weighting of the test components was derived from the job analysis data.

For the oral portion, candidates had 60 minutes to prepare for all three scenarios and had 10 minutes per scenario to present their response. For all three oral exercises, the candidate was to assume the role of a Battalion Fire Chief. Candidates were scored based on the content of their response (technical) and the how well they presented their response (oral communication). These components were scored on a scale of 1 to 5 with 1 being the lowest rating and 5 being the highest rating.

Each candidate in a given jurisdiction was scored by a team of three different Subject Matter Experts (SMEs), who were trained in current technical and oral communication scoring procedures. Each SME is a current or retired fire officer who held the title of Battalion Fire Chief (or Fire Officer 2) or higher. As part of the scoring process, an SME observed and noted the responses of a candidate relative to the knowledge, skills and abilities (KSAs) that each exercise was designed to measure. An SME also noted any weaknesses that detracted from the candidates overall oral communication ability. The SME then rated the candidate’s performance according to the rating standards and assigned the candidate a technical or oral communication score on that exercise.

In order to preserve the relative weighting of each of the components of the examination, the ratings for each portion were adjusted by a well-recognized statistical process known as “standardization.” Under this process, the ratings are standardized by converting the raw scores to z-scores, an expression of the deviation of the score from the mean score of the group in relation to the standard deviation of scores for the group. Each portion of the examination had a relative weight in its relation to the whole examination. Thus, the z-score for the multiple-choice portion was multiplied by a test weight of 36.53%, the oral technical scores were multiplied by a test weight of 53.91% and the oral communication scores were multiplied by a test weight of 9.56%. The weighted z-scores were summed and this became the overall final test score. This was weighted and added to the weighted seniority score. The result was standardized, then normalized, and rounded up to the third decimal place to arrive at a final average.

For the technical and oral communication components of the Supervision, Administration and Incident Command scenarios, the appellant received scores of 5, 4, 1 and 5, 4, 5, respectively.

Appellant's argumentThe appellant challenges his score for the technical component of the Incident Command scenario. As a result, the appellant’s test material and a listing of possible courses of action (PCAs) for the scenario were reviewed.

CONCLUSION

The Incident Command scenario involved a fire at a powder metallurgy facility. Question 1 asked for specific actions upon arrival at the scene. Question 2 indicated that during fireground operations, there is an explosion inside the facility and a mayday is being broadcast. It asked for specific actions to be taken based on this new information.

For this scenario, the SME noted that the appellant failed to set up a collapse zone prior to the explosion; failed to set up cold and warm and hot zones; and failed to protect the storage area, such as using the hoseline and removing drums of powdered metals. These were mandatory responses to question 1. It was also indicated that the appellant missed the opportunity to ensure that command is uphill and upwind, which was an additional response to question 1. On appeal, the appellant states that he set up a command post on side a, which was upwind, protected the loading dock, augmented the system in the loading dock, and use the outside lines to protect the loading dock area and prevent a BLEVE (Boiling Liquid Expanding Vapor Explosion) and ensure that barrels are not knocked over. He states that after the explosion he established collapse zones and set up a transit to monitor collapse hazards.

In reply, the scenario indicated that upon arrival, the candidates saw smoke emanating from one of the loading bay doors, and a manager indicated that a fire started in the loading area. It is noted that the loading dock is on side A, and is separate from the storage area, which was on side C. Side C faces small wooded area beyond which was a trailer storage area containing five 55-gallon drums full of powdered metals used in the metallurgy process, such as lead, cobalt, and magnesium. The scenario indicated that the storage area was at the rear of the facility and is not yet affected.

After taking many actions, near the end of his response to question 1 the appellant stated, “I’m also going to have lines go to protect exposures, make sure that all auxiliary appliances have been attached to and are working properly. Once the fire has been knocked down, we’re gonna get rehab for the members due to the heat, switch out members, have my staging officer give me fresh crews. Have the safety officer review the building. Find out what’s going on. Make sure the fire has been put out. When we have firefighters on the outside of this building as well, with two and a half’s to make sure the fire doesn’t spread out. Make sure we move any occupants and you ah, not knock over the um, the 55-gallon drums. If there’s a BLEVE involved, if there’s liquids involved, if anything is improper then make sure we ah continue, contain, maybe use a fog stream to suppress that as well.” In this passage, the appellant was talking about the exposures, but there were many, and he did not specify the storage area. He then referred to the protecting the fire building. Although he referenced the 55-gallon drums, he did so in the context of the fire building, and did not specify that his crew was in the storage area. Also, he insured that crews did not knock the drums over, but he did not order them to remove them from the area. Lastly, as the loading dock area was a separate area from the storage facility, the appellant cannot receive credit for protecting the storage area by taking actions at the loading dock.

Next, the appellant was required to set up cold and warm and hot zones in response to question 1, that is, before an explosion. A review of the appellant’s presentation indicates that he did so after the explosion, as he explains in his appeal. It is not acceptable to set up collapse zones after an explosion when, given the facts of the scenario, the Incident Commander should be aware that explosives are near fire. He clearly did not set up collapse zones before the explosion, as noted by the assessor.

Finally, the instructions in the scenario tell candidates to be as specific as possible and not to assume or take for granted that general actions will contribute to a score.

This was a formal examination setting, and candidates were required to specifically state the actions that they would take in response to the questions. The appellant established a command post on side A, and he received credit for establishing a command post for this response. However, he did not state that he was doing so because it was uphill or upwind. It may have been the most convenient place. It cannot be deduced why the appellant established a command post on side A, and he is required to state his reasoning in order to get credit. The appellant missed three mandatory responses, as well as the additional response, and his score of 1 for this component is correct.

A thorough review of appellant’s submissions and the test materials indicates that the decision below is amply supported by the record, and the appellant has failed to meet his burden of proof in this matter.

ORDER

Therefore, it is ordered that this appeal be denied.

ClosingThis is the final administrative determination in this matter. Any further review should be pursued in a judicial forum.