Joseph Matullo, CSC Docket No. 2023-1926

BackgroundJoseph Matullo appeals his score on the promotional examination for Deputy Fire Chief (PM5172C), West Orange. It is noted that the appellant did not achieve a passing score on the oral portion of the subject examination.

The subject promotional examination was held on April 23, 2022, and six candidates passed. This was an oral examination designed to generate behaviors similar to those required for success in a job. The examination consisted of four scenario-based oral exercises. Each exercise was developed to simulate tasks and assess the knowledge, skills and abilities (KSAs) important to job performance. These exercises covered four topic areas: 1) Incident Command: Non-fire Incident, 2) Supervision, 3) Administration, and 4) Incident Command: Fire Incident. The test was worth 70 percent of the final score and seniority was worth the remaining 30 percent. The various portions of the test were weighted as follows: technical score for the Incident Command: Non-Fire Incident, 24.42%; oral communication score for the Incident Command: Non-Fire Incident, 3.155%; technical score for the Supervision scenario, 14.17%; oral communication score for the Supervision scenario, 3.155%; technical score for the Administration scenario, 11.81%; oral communication score for the Administration scenario, 3.155%; technical score for the Incident Command: Fire Incident, 36.98%; and oral communication score for the Incident Command: Fire Incident, 3.155%.

The candidates’ responses were scored on technical knowledge and oral communication ability. Prior to the administration of the exam, a panel of Subject Matter Experts (SMEs) determined the scoring criteria, using generally approved fire command practices, fire fighting practices, and reference materials. Scoring decisions were based on SME-approved possible courses of action (PCAs) including those actions that must be taken to resolve the situation as presented. For a performance to be acceptable in the technical component for some scenarios, a candidate needed to present the mandatory courses of action for that scenario. Only those oral responses that depicted relevant behaviors that were observable and could be quantified were assessed in the scoring process.

This examination was given using the chain oral testing process, and candidates were given 10 minutes to respond to each question. Candidate responses to each question were rated on a five-point scale (1 to 5) from no response through optimum according to determinations made by the SMEs. Oral communication for each question was also rated on the five-point scale. This five-point scale includes 5 as the optimal response, 4 as a more than acceptable passing response, 3 as a minimally acceptable passing response, 2 as a less than acceptable response, and 1 as a much less than acceptable response. It is noted that candidates were told the following prior to beginning their presentations for each scenario: “In responding to the questions, be as specific as possible. Do not assume or take for granted that general actions will contribute to your score.”

For the Incident Command: Non-Fire Incident scenario, the appellant scored a 2 on the technical component and a 5 on the oral communication component. For the Supervision scenario, the appellant scored a 3 on the technical component and a 3 on the oral component. On the Administration scenario, the appellant scored a 2 on the technical component and a 5 on the oral component. Finally, on the Incident Command: Fire Incident scenario, the appellant scored a 2 on the technical component and a 5 on the oral component.

Appellant's argumentThe appellant challenges his scores for the technical components of the Incident Command: Non-Fire Incident, Supervision, Administration and Incident Command: Fire Incident scenarios. As a result, the appellant’s test material, video recording and the list of PCAs for the scenarios were reviewed.

The Incident Command: Non-Fire Incident involves the response to a car submerged in a pool at a residence. The prompt asks what actions the candidate would take in response to the incident.

For the technical component of the Incident Command: Non-Fire Incident, the assessor found that the appellant failed to complete the mandatory response of performing a 360-degree size-up and several additional opportunities, including assigning a crew to stabilize the vehicle and assigning a Public Information Officer (PIO). On appeal, the appellant argues that he should have been credited with the PCAs of performing a 360-degree size-up, stabilizing the vehicle and calling for a PIO.

Specifically, he maintains that by indicating that he would perform a size-up and describing the entire area, he covered performing a 360-degree size-up. As to stabilizing the vehicle, he proffers that by having the first engine company enter the water and shut off the vehicle, he signaled that he was beginning the stabilization process and the elimination of hazards. He also avers that calling for a special unit to respond with specialized tools and equipment further covered that PCA. In terms of calling for a PIO, the appellant presents that it is commonplace in New Jersey for police departments to represent PIOs. As such, he maintains that by assigning the situation to police who often double as PIOs, he should have been awarded credit for identifying this PCA.

In reply, a review of the appellant’s presentation for the Incident Command: Non-Fire Incident demonstrates that he was properly denied credit for the PCAs at issue. As noted above, candidates were told the following prior to beginning their presentations for each scenario: “In responding to the questions, be as specific as possible. Do not assume or take for granted that general actions will contribute to your score.” Regarding the PCA of performing a 360-degree size-up, the appellant stated, in relevant part, that “When I arrive on scene, I will perform a size-up” and then described the scene. Since a “size-up” without further elaboration could refer to part of the scene (e.g., just the car on scene), as opposed to the entirety of it, his statement was too general to award credit for performing a 360-degree size-up. As to the PCA of stabilizing the vehicle, the statements cited by the appellant are too vague to demonstrate that he would perform all actions necessary to stabilize the vehicle involved in the incident. Similarly, the mere action of delegating tasks to Police Officers does not specifically convey that the appellant was assigning a PIO. Accordingly, based upon the foregoing, the appellant’s score of 2 on the technical component of the Incident Command: Non-Fire Incident is proper.

The Supervision scenario involves the issue of a subordinate, Battalion Fire Chief (BFC) Smith, being late in turning in his reports to the candidate, while two others holding the same rank have not had the same issue. Question 1 asks what initial and specific steps the candidate should take to address this issue. Question 2 presents that third parties have informed the candidate that BFC Smith has been delayed or slow in responding to emergency incidents and that this may be contributing to his delays in providing reports to the candidate. It adds that some also suspect something outside of the fire department is happening with BFC Smith. Question 2 then asks, based on this new information, what actions the candidate should take.

For the technical component of the Supervision scenario, the SME found that the appellant missed opportunities to check the dispatch logs for timelines of BFC Smith’s responses and to document all interviews and findings. Accordingly, the SME awarded the appellant a score of 3 for this component. On appeal, the appellant maintains that he should have been credited with the PCAs of checking the dispatch logs and documenting all interviews and findings. Specifically, he cites statements that he would check previous reports of BFC Smith in support of his contention that he should have been credited for the PCA of checking dispatch logs and his statement that he would write a memo and forward it up the chain of command to support his claim that he should have been credited with the PCA of documenting all interviews and findings.

In reply, a review of the appellant’s presentation for the Supervision scenario reveals that he stated that he would write a memo and forward it up the chain of command. Again, candidates were told the following prior to beginning their presentations for each scenario: “In responding to the questions, be as specific as possible. Do not assume or take for granted that general actions will contribute to your score.” Here, since the appellant did not specify that his memo would document all interviews and findings, he was appropriately denied credit for the subject PCA. Furthermore, a review of the appellant’s presentation fails to demonstrate that he covered the PCA of checking the dispatch logs. Accordingly, his score of 3 on the technical component of the Supervision scenario is appropriate.

The Administration scenario presents that the Fire Chief has tasked the candidate with preparing for a parade on Halloween evening in coordination with the police department, including ensuring safety and code compliance efforts. Question 1 asks what course of action the candidate should take to complete their task. Question 2 asks how the candidate would handle requests from some department members to be included in the parade and leave requests from others requesting off for the Halloween holiday.

For the technical component of the Administration scenario, the SME found that the appellant missed opportunities to submit a final plan for the Fire Chief’s approval, documenting all meetings and setting timelines. Accordingly, the SME awarded him a score of 2. On appeal, the appellant maintains that he should have been credited with the PCAs of submitting plans to the Fire Chief and setting timelines. Specifically, he contends that his statement about working with the Fire Official covered the PCA of submitting a final plan for the Fire Chief’s approval because the Fire Official’s involvement would produce a final plan for the Fire Chief. He further maintains that his discussion of forming a planning committee conveyed that he would set timelines.

In reply, a review of the appellant’s presentation for the Administration scenario demonstrates that he was properly denied credit for the PCAs of submitting a final plan to the Fire Chief and setting timelines, as the statements he cites from his presentation for this scenario do not directly indicate that he would perform the subject PCAs. Accordingly, his score of 2 for the technical component of this scenario was correct.

The Incident Command: Fire Incident involves a fire at a one-story recreation center that consists, in part, of attached classrooms on Side C utilizing steel bar joists. The Fire Department Connection (FDC) is on Side C. Question 1 asks what actions the candidate should take to fully address the incident. Question 2 provides that during the incident someone busts out of a window from one of the Side C classrooms and a recreation center counselor yells that there are still kids in there. It also states that crews are reporting that water is not flowing from the sprinkler heads. Question 2 then asks what actions should be taken based on this new information.

For the technical component of the Incident Command: Fire Incident, the assessor found that the appellant failed to perform the mandatory action of performing a multi-sided walk-around view/360-degree size-up and checking the pressure being fed to the FDC, and the additional response of requesting the Health Department. On appeal, the appellant argues that he should have been credited for performing these actions. Specifically, with regard to a 360-degree size-up, he proffers that while he did not specifically mention “360” with the size-up, because he made clear he was performing a size-up of the entire area, he should have been awarded credit. As to the mandatory response of checking FDC pressure in response to Question 2, he asserts that since he supplied the FDC with adequate water at standard pressure, he should have received credit. Finally, he maintains that because he called for a Liaison Officer to work with outside agencies, it covered contacting the Health Department and allowed him, as the Incident Commander, to focus on tactics and strategies.

In reply, a review of the appellant’s presentation does not demonstrate that he should have been credited with any of the foregoing actions. Specifically, regarding the mandatory PCA of performing a 360-degree size-up, the appellant indicated that he was performing a “size-up,” without specifying that it was a 360-degree size-up and gave an initial report to dispatch which described conditions on scene from the perspective of Side A, but failed to demonstrate that he was mobile and checked each side of the incident scene for changing conditions, as is required for a 360-degree size- up. As such, he was properly denied credit for this mandatory response. As to the mandatory response of checking the FDC pressure in his response to Question 2, his discussion of the FDC came in the context of establishing a water supply in response to Question 1. Critically, his response to Question 2 does not indicate that he would check the FDC pressure after learning of the lack of water flow from the sprinkler heads on Side C. The Commission observes that recognized experts in the field have noted the importance of trying to get a nonfunctioning sprinkler system to work. For example:

If we find a fire in a building where the sprinklers are not operating, we will have to stretch handlines and conduct a manual attack, but we should also make every effort to get the sprinklers into action as well.

Immediately begin supplying the FDC and monitor the effect that this effort achieves, if any.

John Norman, Fire Officer’s Handbook of Tactics 133 (5th ed. 2019).

Finally, concerning the PCA of requesting the Health Department, naming a Liaison Officer is far too general of an action to credit the appellant with identifying that action. Here, even assuming, arguendo, that the Liaison Officer would be the individual contacting the Health Department, it was still imperative that the appellant specifically identify the need for this action to be performed as part of the response to the incident. Since the appellant failed to do so, he was properly denied credit for this PCA. Accordingly, based upon the foregoing, the appellant’s score of 2 for the technical component of the Incident Command: Fire Incident scenario is proper.

ORDER

Therefore, it is ordered that this appeal be denied.

ClosingThis is the final administrative determination in this matter. Any further review should be pursued in a judicial forum.