Melvin Jumper, CSC Docket No. 2019-2249

Melvin Jumper (PM2519W), Ewing; Anthony Buono, Gregory Goncalves, Jacqueline Molina and Gabriel Rivera, represented by David Beckett, Esq., Racheda Allen, Ashley Arce, Tracie Ashford, Jorge Astuquilca, Michael Barreiro, Queen Bates, Ashley Bishop, Sabrina Brison, Shamayne Brown, Virgilio Cardona, Samouri Clegg, Larry Collins, Carlos Colon, Crystal Corbett, Melissa Corchado , Larry Davis, Paulette Dent, Sherri Dillard, Jamillah Ellis Damaris Febus, Javier Figueroa, Adolfo Furtado, Edgardo Gonzalez (2019-2417), Edgardo Gonzalez (2019-2449), Nicole Goodwin, Tonya Goulbourne, Darnell Graham, Natasha Green, Leonardo Guzman, Nassim Hamami, Alex Haralam, Tarik Haynes, Gilbert Hernandez, Antoinette Holland, Eleazar Irizarry, Jennifer Jeffra, Andy Jimenez, Maria Lebron, Melody Linton, Horacio Lorenzo, Ricardo Macieira, Elizabeth Malave, Peter Malave, Maria Malave-Mitti, Virginia Marrero, Douglas Marshall, Shukirra Marshall, Claudia Martinez, Brian McAdams, Sharice McClees, Lamar Melvin, Michael Noel, Jaret Perez, Elvin Polanco, Richard Ramdas, Shamar Reddick, Isabel Reyes, Teasla Reynolds, Tara Rich, Norman Richardson, Jesus Rivas, Arnaldo Rivera, Orlando Rivera, Veronica Rivera, Luz Romero, Jose Rosa, Osbaldo Rosa, Rafael Rosa, Rockean Sanders, Andy Santana, Luan Serrano, Karama Thomas, Taibu Thomas, Pedro Torres, Steven Vazquez, Emerson Verano, Anthony Wade, Victor Williams, Careem Yarborough and Latoya Young-Dunlap (PM2540W), Newark; Jeffrey Gennari (PM2541W), North Bergen; Phillip Calicchio, Priscilla Caraballo, Tyseme Holmes, Frank Lusk, Wilson Lazu, Richard M'Causland, Mahmoud Rabboh and Luis Torres

(PM2544W), Paterson; and Michael Wallace (PM2559W), Winslow; appeal the validity of the subject examination.1

Joseph Cevallos, Salvatore Cordi and Jonathan Donker (PM2514W), Bloomfield; Marquis Brock, Juan Cosme, Mohamed Diabate, Angel Pared, Isabel Reyes and Valeria Sanchez-Bermudez, represented by Steven A. Varano, Esq., Quayshaun Brooks, Dennis Colon, Santos Duran and Emily Santiago (PM2540W), Newark; Pedro Borrero, Jose Castellanos, Felipe Diaz, Juan Garcia, Sebastian Gomez, Robert Klein, Victor Lora, Lillian Sanchez, Gabriele Spallacci, Novar Vidal, represented by Steven A. Varano, Esq., Anthony Castronova, William Herrmann and Salvatore Marotta (PM2544), Paterson; appeal the omission of the last 10 items from scoring.

BackgroundThese appeals have been consolidated due to common issues presented by the appellants.

By way of background, the subject two-part examination, which was administered on February 23, 2019, consisted of a video-based portion, items 1 through 21, and a multiple-choice portion, items 22 through 85. It is noted that for the video portion, candidates were provided with 35 minutes. However, it is further noted that candidates were instructed, “As the video progresses, questions will be presented for you to answer in the time provided. The questions will be clearly indicated as they appear on the screen and will be read aloud by the narrator on the video . . . Answer each question in the time provided. During the response time, the video will display a number which corresponds to the amount of response time remaining. If you have completed your response before the time for the question has expired, you may place your pencil on the desk and sit quietly while you wait for the scenario to continue.” Candidates were provided with one hour and 40 minutes to complete the multiple-choice portion. As noted in the 2018-2019 Police Sergeant Orientation Guide (Orientation Guide), which was available on the Civil Service Commission’s (Commission) website, the examination content was based on the most recent job analysis verification which includes descriptions of the duties performed by incumbents and identifies the knowledge, skill and abilities (KSAs) that are necessary to perform the duties of a Police Sergeant. As part of this verification process, information about the job was gathered through interviews and surveys of on-the-job activities of incumbent Police Sergeants throughout the State. As a result of this process, critical KSAs were identified and considered for inclusion on the exam. The test was worth 80 percent of the final average and seniority was worth the remaining 20 percent.

Candidates for the subject test were provided with two separate opportunities for review: 1) candidates were permitted to review keyed test booklets (test booklet review); and 2) candidates were permitted to inspect their answer sheets and compare them to the keyed answer sheet (answer sheet review). See N.J.A.C. 4A:4-6.4(a). It is noted that test booklet reviews were held on March 6, 7, 8 and 13, 2019. Appeals resulting from the test booklet reviews were addressed in In the Matter of Gordon Harvey, et al., County Police Sergeant and Police Sergeant (CSC, decided September 10, 2019).2 Subsequently, on November 13, 2019, the subject lists were issued.3 Answer sheet reviews were scheduled from January 7 through 10, 2020. Candidates were provided with the opportunity to review their answer sheet, the scoring key and an individualized scoring report. The copy of the scoring key provided at review did not provide responses for items 76 through 85. As a result, candidates were not able to compare their responses on their answer sheet to the key for these items. Furthermore, it was at this time that candidates first became aware that the last 10 items were omitted from scoring. Candidates who inquired as to why the items were omitted were informed that the items were removed after a statistical analysis was conducted on how the items performed for the entire population.

1 It is noted that appeals regarding the test administration issues on the date of the examination will be addressed by the Commission in a separate decision at a subsequent meeting.

Appellant's argumentWith respect to test validity, appellants argue that adequate time was not provided to complete the test.4 In this regard, appellants present that “the fact patterns for case law [based questions] were extremely long extending to a page and a half to merely answer one question.” Appellants assert that the subject examination did not test their knowledge, skill and abilities but rather, tested their reading speed. In his submission dated March 15, 2019, Beckett specifically presents, in part, that the subject test “should be the second promotional examination that has been given under [the Second Amended Consent] Decree5 and so this examination is subject to paragraph 80 of the Second Amended Consent Decree and its myriad requirements.” In this regard, Beckett contends that the subject exam “appears to

Background2 It is noted that subsequent to the decision in In the Matter of Gordon Harvey, supra, but prior to issuing the subject lists, TDA determined to omit the last 10 items from scoring.

3 The resultant eligible lists were made available on the Commission’s website and individual scoring notices were mailed to candidates. It is further noted that neither the website nor the scoring notices informed candidates of the omission of the last 10 items.

4 It is noted that the Commission reviewed each and every appeal. However, given that many of the appeals were identical, or nearly identical, and/or presented the same or similar issues, each individual appellant’s claims are not recounted herein.

5 It is noted that in January 2010, the United States Department of Justice (USDOJ) filed a complaint against the State of New Jersey and the Civil Service Commission (Commission), alleging that the selection process utilized by the State to test and appoint candidates to the Police Sergeant title between 2000 and 2008 had a disparate impact on African-American and Hispanic candidates in violation of Title VII of the Civil Rights Act of 1964, 42 U.S.C. §2000e et seq., as amended. During the pendency of this litigation, the parties agreed to the terms of a settlement which was formulated into a Consent Decree which the Court approved and entered as final on June 12, 2012. The terms of the Consent Decree provided, in pertinent part, that the State, in consultation with USDOJ, develop a new Police Sergeant examination and scoring process.

fall far short of their [sic] of the Second Amended Consent Decree. The test was not a proper measuring of skills and experienced [sic] needed to be a Sergeant. The time allotted for written answers was unilaterally restricted and there was no prior notice of these substantial alterations in the time that would be allotted.6 The test became a race to finish as opposed to a test that would actually determine someone’s skills and experience to be a Sergeant.” Beckett indicates that “we estimate that up to 50% of the candidates from the Newark and Paterson Police Departments were either unable to finish or had to blindly answer to finish. These are two of the jurisdictions that were at the heart of the Second Amended Consent Decree . . . All of this speaks strongly of a test that is neither appropriate nor a fair measurement of candidates. The appellants strongly question whether there was any pre-testing or pre-review of this examination by the Civil Service Commission or United States experts to properly evaluate how long an average test taker would need to complete the exam.” In addition, Beckett requests “all information that was provided under Article [sic] 80, when it was provided to the United States experts, in what form and what and when the responses were.” Beckett also requests that he be provided with “data from prior examinations showing how many officers in Paterson and Newark Police Departments completed that earlier examination.” In support of the appeal, Beckett submits additional documentation including a certification from Gabriel Rivera dated September 19, 2019, in which Rivera indicates, in part, that he has taken prior Police Sergeant examinations and the subject test “was significantly different than the prior exam, which resulted for the first time in me being unable to finish a significant part of the test;” “you could not use any of the time allotted to the video for the written questions even though you had more than enough time for that part;” “we had less time to answer the written questions and yet that narratives for the written questions were longer than the prior exam with fewer questions per narrative;” “in the prior exam held in 2015-20167 there was a video format part with a second written part . . . so it is not the format;” and a certification from Gregory Goncalves dated December 2, 2019 in which Goncalves maintains, in part, that “the combination of longer narratives, fewer questions per narrative, less time being allotted in total for answering written questions, and no ability to use leftover video part time, resulted in a test that was unfair and biased. This combination created a biased test favoring native English speakers.”

Regarding the omission of the last 10 items, appellants inquire as to why these items were omitted and that “this decision was never publicized, nor was any pertaining information sent via email or regular mail.” They maintain that they should not be “penalized for actually finishing the test and for being able to complete

6 Beckett maintains that “past written examinations allocated close to two hours and thirty minutes for written questions alone. The number of written questions in this current examination was only marginally reduced, and each test question, especially in those where legal issues were posed, took far longer than prior exam questions had ever taken, thus further reducing the allotted time.”

7 It is noted that the prior examination for Police Sergeant (PM5107M), Newark, was administered on

June 1, 2013.

all 85 questions in the time that was allotted.” Varano argues that “the elimination of the last ten (10) questions in the scoring of the results is completely arbitrary and adversely impacts the examinees that completed the exam in the allotted time. Moreover, it is contrary to the express instructions set forth in the CSC Multiple Choice Exam Orientation Guide . . . and the 2018-2019 Police Sergeant Orientation Guide . . . In the[se g]uides, examinees are repeatedly advised as to the importance of time management and to provide answers to each question.” Varano maintains that the candidates “studied for the exam and relied upon the instructions in the [g]uides when taking the exam. Specifically, [they] intentionally spent less time on questions [they] found more difficult so that [they] would have enough time to complete the entire exam, including the last ten questions. The elimination of the last ten questions in computing the scores penalizes examinees . . . who followed the CSC’s instructions provided in the [g]uides and ensured they managed their time properly to complete all the questions of the exam.” Varano further argues that “it is unclear at this time how the remainder of the test was scored and the weight given to each question. Specifically, it is unknown whether any other unanswered questions were graded as wrong answers or were eliminated as the last ten questions. If examinees skipped certain sets of questions, but completed the questions near the end, then the elimination of the questions near the end, would once again penalize examinees that followed the CSC instructions.” In addition, Varano requests the following: “1. A copy of the answer sheets . . . ; 2. A copy of the answer keys; 3. The criteria used to score the tests.”

CONCLUSION

In the present matter, it is noted that the February 23, 2019 administration of the Police Sergeant examination was the fourth administration subsequent to the Second Amended Consent Decree. Specifically, the first administration occurred on June 1, 2013; the second on January 16, 2016; the third on October 28, 2017; and the fourth on February 23, 2019. Thus, while the PM2540W examination is the second Police Sergeant examination administered to Newark,8 it is the fourth administration of the Police Sergeant examination overall. As such, Beckett’s assertion that the subject test is the second administration is incorrect. Accordingly, paragraph 80 does not govern the administration of the PM2540W test. Moreover, even if Beckett’s claim that the PM2540W exam was the second administration were correct, Paragraph 81 of the Second Amended Consent Decree provides:

Due to the importance of test security, along with concerns about the proprietary nature of test development materials, all correspondence and communications between and among the Parties and their test developers, consultants and/or experts in connection with the performance of the obligations set forth in Paragraphs 77 through 80 shall be held confidential and shall not be disclosed to any third party in the absence of a court order compelling such disclosure.

8 As noted previously, the Police Sergeant (PM5107M), Newark examination was the first administration pursuant to the Second Amended Consent decree.

Similarly, Beckett’s request for information from the PM5107M examination cannot be granted pursuant to the provisions of Paragraph 81.

It is noted that in all four testing cycles, each examination consisted of a video- based portion and a multiple choice portion, which was clearly described in the respective Orientation Guides, in the “Exam Information” section, for each exam administration. It is further noted that the Division of Test Development and Analytics (TDA) was contacted regarding this matter and indicated that during the test development process, it determined that sufficient time had been allocated to complete the multiple-choice portion of the test.9 In this regard, although Beckett does not specify what he means by “pre-testing or pre-review,” TDA indicated that Subject Matter Experts (SMEs) were asked to read and answer each item without the key being provided. SMEs then provide a difficulty rating for each item. In addition to importance and relevance ratings, difficulty ratings were considered in an attempt to balance, where possible, the overall difficulty level of the exams. Moreover, the length of examination time for all four examination cycles was based upon input from the SMEs.

After each administration in all four cycles, TDA conducted several analyses which included an analysis of the individual items in the examination, as well as an analysis of the performance of subtests and of the entire test. As a result, TDA determined that as the subject test progressed, the number of candidates who did not provide responses to items increased. In addition, as part of the analyses noted above, TDA conducted an adverse impact analysis.10 TDA’s review found that the last ten

9 Although Beckett maintains, as noted above, that “past written examinations allocated close to two hours and thirty minutes for written questions alone,” a review of the record finds that the PM5107M test allocated one hour and 35 minutes for the multiple choice section and, as noted previously, the subject test allocated one hour and 40 minutes.

10 The U.S. Equal Employment Opportunity Commission (EEOC) defines adverse impact as “a substantially different rate of selection in hiring, promotion, or other employment decision which works to the disadvantage of members of a race, sex, or ethnic group.” See 29 CFR §1607.16. In this regard, Title VII of the Civil Rights Act of 1964 (see 42 U.S.C. §2000e et seq.) prohibits the use of discriminatory employment tests and selection procedures. Furthermore, Title VII prohibits the use of neutral test or selection procedures that have the effect of disproportionately excluding individuals based on race, color, religion, sex or national origin. In 1978, the EEOC adopted the Uniform Guidelines on Employee Selection Procedure under Title VII (see 29 CFR Part 1607), which provides guidance to employers on how to determine whether their tests and selection procedures are lawful. See https://www.eeoc.gov/laws/guidance/employment-test-and-selection-procedures. One of the measures of adverse impact utilized by the Uniform Guidelines is the Four-Fifths Rule which is “… a selection rate for any race, sex or ethnic group which is less than four-fifths (4/5ths) or eighty percent (80%) of the selection rate for the group with the highest rate as a substantially different rate of selection.” See Questions and Answers to Clarify and Provide a Common Interpretation of the Uniform questions presented evidence of adverse impact in that there was a disparity between the performance of minority candidates as compared to the performance of non- minority candidates. As a result, TDA determined to omit the last ten items from scoring. Furthermore, TDA determined that the omission of these 10 items did not render the subject test invalid in that sufficient KSAs were tested in the remainder of the test.

With regard to the scoring of the test and Varano’s requests for additional information, as noted previously, candidates were provided with the opportunity to review their answer sheets, the scoring key and an individualized scoring report. The scoring report provided candidates with a “step-by-step explanation of how [a candidate’s] final test score was calculated.” Furthermore, it is noted that authorized representatives, which include attorneys, union representatives or employers, may accompany candidates during the review process. Nevertheless, for informational purposes, it is noted that for each item on both the multiple-choice and video portion of the exam there was only one correct answer, and each correct answer was worth 1 point. Incorrect or omitted items were not awarded any credit. However, as thoroughly explained in the scoring report, several steps are necessary to calculate a candidate’s final average in order “to account for the relative importance of the knowledge and ability areas tested and the different scoring/rating scales used throughout the test. The scoring process described here is typical for this type of testing procedure and conforms to professional standards developed by experts in this field. It is also used routinely for public safety promotional examinations administered throughout the country.”

Accordingly, the appellants have failed to meet their burden of proof in this matter.

Guidelines on Employee Selection Procedures, 44 FR 11998 (March 2, 1979). See also, https://www.eeoc.gov/laws/guidance/questions-and-answers-clarify-and-provide-common- interpretation-uniform-guidelines.

ORDER

Therefore, it is ordered that these appeals be denied.

ClosingThis is the final administrative determination in this matter. Any further review should be pursued in a judicial forum.