Loading...
HomeMy WebLinkAbout1989-1968.Palatino, Ragos & Patterson.90-12-05 -- -- . . -- ONTARIO EMPLOYES DE LA COURONNE f CROWN EMPLOYEES Dt: L 'ONTARIO 1111 GRIEVANCE COMMISSION DE SETTLEMENT ~ REGLEMENT BOARD DES GRIEFS 180 DUNDAS STREET WEST SUITE 2100 TORONTO, ONTARIO MSG lZ8 TELEPHONE/TELEPHONE (416) 326-1388 180, RUE DUNDAS OUEST BURE;AU 2100 TORONTO (ONTARIO) M5G IZ8 FACSIMilE /TElECOPit (4/6) 326-1396 1968/89, 19,70/89 1972/89 IN THB MATTBR OF AN ARBITRATION Onder THB CROWN EMPLOYEES COLLECTIVE BARGAINING ACT Before THE GRIEVANCE SETTLEMENT BOARD BETWEEN OPSEU (palatino/Ragos/Patterson) Grievor - and - The Crown in Right of Ontario (Ministry of Health) Employer BEFORE: W Kaplan Vice-Chairperson J c. Laniel Member I. Cowan Member FOR THE P. Chapman GRIEVOR Counsel Ryder, Whitaker, Wright & Chapman Barristers & Solicitors FOR THE 0 McKeown EMPLOYER Counsel Hicks Morley Hamilton stewart Storie Barristers & Solicitors HEARING. November 5, 6, 7, 1990 r 1 i I I 2 Introduction Tes Palatino, Cecilia Ragos, and stephen Patterson ( lithe grievors") work at the Ministry of Health in Toronto Ragos and Patterson economists; palatino . systems officer The are 1S a grievors were unsuccessful candidates in a job competition held by the Ministry of Health (lithe empl oyer II ) for two Economist 4 positions, and by grievances dated November 17, 1989 they sought to be placed in the positions for which they had unsuccessfully competed. At the commencement of the hearing counsel for the union advised the Board that the only remedy the union was seeking was an order directing the employer to re-run the competition. The evidence presented accordingly focused on the job competition process. The two successful candidates, Moe Watkin and Derry O'Mahoney, were present throughout the hearing. Both Mr. Watkin and Mr. O'Mahoney were invited to participate in the hearing and both made submissions to the Board The Facts The two positions in question, Economic Consultant, Institutional Information and Economic Consultant, Community Information were posted on June 9, 1989 with a closing date of June 30, 1990 Although two different positions were to be filled they were treated as one position for the purpose of running the competition. All three grievers submitted their applications on time and in due course were invited to attend interviews Before reviewing the evidence in detail it is helpful to briefly set out l .. ~~ ..- (~ I ~ " 3 the process that took place. After the jobs were posted a selection committee was formed A questionnaire for the two positions was prepared and seventy applications were reviewed. Interviews were conducted in late September 1989. Nine persons were scheduled for interviews, including the grievors and the two successful candidates. Points were assigned, by a consensus marking system, for correct answers to various questions. All the interviewees within ten percent of the top scorer were placed on a short list. Only one of the grievors, Tes Palatino, made that list Reference checks were conducted on the six persons on the short list and additional points assigned as a result of that process. The difference in scores between the high and low scorer was not great, and the successful candidates were selected on the basis of seniority as provided for in Article 4 3 of the Collective Agreement. That Article states: In filling a vacancy, the Employer shall give i primary consideration to qualifications and I ability to perform the required duties. I Where qualifications and ability are I relatively equal, length of continuous service shall be a consideration In brief, it was the union position that the competition should be voided because the selection of the successful candidates was made on the basis of an incomplete record, in that the selection process adopted did not result in the necessary and relevant . ,-- '\ 4 information about the experience and abilities of the candidates being brought to the attention of the selection panel. The union also raised concerns about the method of scoring adopted by the panel, as well as the composition of the panel and the manner in which references were checked. The employer took the position that the competition was run in good faith and run well, and even if there were some minor flaws in the process, those flaws did not and could not have affected the result. More specifically, employer counsel submi tted that the focus on the interview was appropriate, that the questions asked were relevant in light of the requirements of the two positions and that the answers provided the selection panel with the information necessary to judge and assess all of the candidates, The two successful candidates argued that the process was fair and should not be interfered with. Before reviewing the evidence, it is helpful to briefly describe the organizational structure of this branch of the Ministry of Health. There are two different components of this branch Institutional Information and Community Information As previously noted, one of the economist jobs in question was to be in Institutional Information, and the other job was in community Information. At the time of the grievance, the Manager of Institutional Information was Mr Watt. Reporting to Mi:' Watt was Maris Gailitis Under Mr Gailitis was one of the vacant positions and under the vacant position was Mr. Patterson, one of I ~ -- ~ '"'" 5 the grievors On the Community Information side of the Branch, Mr Glen Simpson was the Manager Reporting to Mr Simpson was the Coordinator, Mr. Boyd Gill. The other vacancy for which the competition was being held was under Mr Gill and Ms. Ragos, one of the grievors, appeared on the organizational chart under this vacancy. Mr Watt and Mr. Simpson reported to the Director of the User support Branch, i.e. , the branch under discussion. Ms Palatino worked in the Management Systems Branch, which is another branch within the division. All three grievors gave evidence on behalf of the union. Tes Palatino, who made the short list, testified first. Ms Palatino testified generally about her educational and employment history Suffice it to say that she began work with the employer on January 27, 1986. Ms Palatino's position is Business Analyst, and, as earlier noted, she is in the systems officer classification. She has undertaken a variety of responsible tasks in this position, and for a period of one year acted in a position one level higher than her own. Along with the other two grievors, Ms palatino applied for the two Economist positions In due course she was notified about an interview, and arrived approximately ten minutes before the interview was scheduled to begin. Ms Palatino was given a copy of the position specification and she reviewed it prior to the interview Obviously, she only had enough time to give it a -~ ~ 6 cursory review. other than the posting, which contained a very short description of the two positions, Ms. palatino had no other information about the duties and responsibilities of the two positions, except, of course, what she was able to read in the period prior to the commencement of her interview. The selection panel consisted of Mr. Simpson, Mr. David Coughlin, the Manager of Application Services in the Management Systems Branch and Mr. Gailitis Ms. Palatino knew all three men Mr. Coughlin was above Ms Palatino in the Management Systems Branch, but her immediate supervisor was David Slapack. Mr Slapack reported to Mr Coughlin. One of the successful candidates, Mr Watkin, also reported to Mr. Slapack. Ms. Palatino testified that Mr. Simpson began by asking questions, and that Mr Coughlin and Mr. Gailitis also asked questions. Some of the questions were quite long, and Ms Palatino requested clarification. She was advised, however, that while the questions could not be clarified, they could be repeated. Introduced into evidence was a copy of the "Applicant Evaluation criteria." This 26-page document is divided into four sections. The first section contains questions relating to applicants' education and work experience Twenty-five points are available under this section. The second section contains questions relating to knowledge and skills. Fifty-five points are available under this section. The third section contains ~ - ,!' 7 questions relating to candidates' communication abilities Ten points are available here. The final section has as its objective the assessment of candidates' personal and interpersonal characteristics Fifteen points are available here. The total number of points possible in the interview is 105 In her evidence, Ms. palatino testified that she was never questioned about some of the matters for which points were assigned Ms. Palatino also objected to the way in which communication skills were evaluated, principally to the evaluation of written skills having been based solely on the covering letter and resume submitted as her application for the positions in question. Ms. Palatino testified that when the questions were concluded she was given the opportunity to ask the selection panel members some questions She was later asked to provide the names of two references, and she was required to sign a form consenting to those persons being contacted In November 1989 Ms Palatino was notified of the results of the competition and filed a grievance It is useful to set out all of the candidates' results and to briefly describe the process that was followed after all of the interviews were completed I - ) 8 The results: Insofar as the job competition process is concerned, the work of the selection committee proceeded as follows After the tabulation of the individual interview results, the selection committee decided to obtain references from all the candidates who scored within 10% of the top scor ing applicant. Reference checks were accordingly obtained on the candidates Watkin through Palatino. I For the purpose of the reference checks, each of the six was given twenty points. Points were deducted for factors considered significant by the selection committee in its assessment of the applicant's capability to perform the duties, functions and responsibilities of the two positions. -- I ~ 9 As a result of these reference checks the following scores were obtained. Mae Watkin 20 Brad Graham 18 Robert Pearse 16 Peter Marshall 20 Derry O'Mahoney 17 Tes Palatino 20 An overall score could be calculated and they were as follows Moe Watkin 93 5 Brad Graham 88 5 I Robert Pearse 86.5 \ Peter Marshall 89 0 Derry O/Mahoney 84 5 Tes Palatino 86.0 The seniority of the candidates and the two grievors not on the short list is as follows' Moe Watkin December 10, 1980 Brad Graham December 1, 1987 Robert Pearse External Peter Marshall January 23, 1989 Derry O/Mahoney September 22, 1980 Tes palatino January 27, 1986 Cecilia Ragos May 13, 1968 Stephen Patterson August 2, 1977 Mr. Watkin and Mr. a/Mahoney had, by a significant margin, the longest seniority of the candidates on the short list, and were offered the positions in question In addition to the matters noted above, Ms. Palatino testified that she was concerned about the fact that the three members of the selection panel were in-house, and she also expressed concern about whether one of the panel members would evaluate her fairly - 10 Ms Palatino is also of the view that there was something wrong with the way the questions were scored because she only received 3 points for communication abilities, and the other candidates received 6 points. other concerns related to the absence of a person from Human Resources on the selection committee, and the fact that she saw some of the candidates speaking to selection committee members outside of the interview room. In cross-examination Ms. palatino was carefully questioned about whether or not any of the members of the selection committee were biased against her, and no evidence was provided to support her earlier claim. Ms Palatino also testified that her preparation for the interview had consisted of asking some economists in the Branch about their work, and that she also read up on economics. This interview was the first Ms Palatino had had since joining this employer. Ms. Palatino's earlier statement that she had not been asked all the questions on the Applicant Evaluation criteria was clarified and it appears that she was in fact asked all of the questions. Ms Palatino agreed with counsel for the employer that the questions she was asked were relevant to the position in question, although some of the questions did not appear as relevant to her as others Ms. palatino was also cross-examined about her resume and the fact that it was somewhat out of date However, she was adamant in her evidence that she did not intend this resume and covering letter to be considered as samples of her written work for the purpose of evaluating her written - I 11 communication skills Ms. Palatino maintained that it was unfair I not to provide clarification of questions particularly where a i given applicant, because of background, does not understand a word or expression Mr. Watkin also asked Ms. palatino several questions, and she testified that the selection committee that hired her had consisted of three persons, including Mr Watkin, and that all three of these persons came from the same branch, the Management Systems Branch. In re-examination, Ms. Palatino stated that the information contained on her resume would not give a full picture of her skills, abilities and accomplishments. Ms Ragos testified next. She testified generally about her education and experience. Ms Ragos has worked at the employer in progressively senior positions since 1968 Just prior to the competition, Ms. Ragos reported to Mr. Gill, and just prior to that she reported to Mr. Gailitis, and did so for approximately three to four years. Ms. Ragas testified that as an insider she knew all about the posted positions and so preparation was not necessary. As was the case with Ms. Palatino, Ms Ragos was interviewed in september 1989 Approximately twenty minutes prior to the interview taking place Ms Ragos was given copies of the position specifications, which she then reviewed. Ms. Ragos was advised at I - 12 the start of the interview that the questions would be read to her, but in that fairness to all the candidates, they would not be clarified They would, however, be repeated. Ms Ragos testified that she was given an opportunity to answer all of the questions, and that her answers were short and to the point She also testified that other than the preliminary questions about her education and experience, she was not asked further questions with respect to either. Ms. Ragos indicated that she was not asked any follow-up questions in response to the answers she provided. Ms Ragos was not asked to provide the panel with any written work and did not do so Had she been asked to do so she would have provided examples of reports she had written. Ms Ragos was asked to provide references, one of whom was to be her immediate supervisor Ms Ragos consented to Mr. Gill and Mr Gailitis being contacted as her references When the results of the competition were announced Ms. Ragos was upset because she felt that the interview had figured too significantly in the process, and that no value had been placed on her experience and job performance. In cross-examination Ms. Ragos testified to enjoying a very good working relationship with Mr. Simpson, and she indicated that Mr Simpson would be aware of many of her activities and accomplishments, not to mention her educational experience Mr - I 13 Gail i tis was also familiar with this grievor's work Ms. Ragos testified that this job interview was the first she had attended in quite some time for she was satisfied with her job and was not actively searching for a new position. Ms. Ragos elaborated on her preparation for the position and told the Board about some of the studies and other materials she consulted as part of that preparation. Counsel for the employer drew Ms Rago's attention to a number of questions in the Applicant Evaluation Criteria which would have enabled her, in the process of answering questions, to reflect and comment on her background and experience. Ms Ragos indicated that she might not have taken full advantage of the opportunities to discuss these matters, in that she assumed that the selection committee would be familiar with her experience and qualifications. Ms. Ragos did not realize that she had to go over her background and qualifications in response to questions on point. Ms Ragos expressed concern that her score was particularly low in the knowledge and skills/personal- interpersonal characteristics category, and testified that had references been obtained and had her personnel file been consulted, she would have scored higher in these two categories and higher overall in the result. Her seniority may have then resulted in one of the two positions being offered to her Mr Patterson was the third and final witness for the union He I ~- 14 too testified about his background and experience At the time of the competition Mr. Patterson reported to Mr Gailitis, and one month previous to that Mr. Patterson had reported to Mr Gill, although his dealings wi th Mr Gill had been rather limited in that he was receiving instructions from Mr Simpson. Mr. Patterson was interviewed in September 1989 He expressed some surprise about the composition of the selection committee. He testified that he thought Mr Watt should have been on the committee given the senior nature of the positions in question, and he questioned Mr Coughlin's presence on the committee given that Mr. Coughlin was in systems and the two positions were economist positions. Mr. Patterson testified that he was given a copy of the position specification approximately fifteen minutes prior to the interview, but that this did not make much difference since the position specification he was given was virtually identical to his own position specification Mr Patterson was accordingly familiar with what it contained. Mr Patterson's interview was virtually identical in format to that of Ms. palatino and Ms. Ragos. Mr. Patterson testified that had he been asked he would have submitted written material, and that he did not know that his covering letter and resume were to be used to evaluate his written communication skills. At the end of the interview Mr. Patterson was asked to provide the name of his immediate supervisor as a reference. I - 15 Mr Patterson testified that he and Ms Ragos were already performing the same job as the one they were competing for, and he told the Board that he believed that promotion from the Economist 3 position to the Economist 4 position was not based on a different set of skills and abilities, but was a reward for doing a good job. In his view, he and Ms. Ragos were already incumbents and he testified that the consensus system of scoring adopted by the selection committee was unfair because one member of the selection committee could persuade another as to an appropriate result Moreover, in his view, the selection committee could not obtain quantitative evidence for the evaluation of written communication skills on the basis of a resume and cover letter. In cross-examination Mr. Patterson testified to his generally good working relationship with Mr Simpson Mr. Patterson also testified to having some experience in attending job interviews, but he pointed out that he liked his job and was not actively seeking a new one. To prepare for his September interview, Mr Patterson read over his own job specification, and made some personal notes Mr Patterson was questioned about his application and about the fact that it was handwritten and had a spelling mistake in it He testified that he hand-wrote his letter of application because that had more feeling to it and that the spelling mistake merely indicated that he was human and made mistakes. Mr. Patterson was also asked about a number of -- 16 questions raised during the interview and his answers to them, as well as about their relevance to the position he was applying for In general he agreed that these questions were relevant Mr. Patterson also indicated that his interview performance may have been affected because he was put in a bad mood just prior to the interview beginning when he read the position specification for the Economist 4 position and found out it was similar, if not virtually identical, to his position specification as an Economist 3. Mr. Patterson told the Board that he had been concerned for some time that he was not appropriately classified. Mr Simpson testified on behalf of the employer. He testified about his responsibilities and he described in some detail how the selection committee was established and how it went about its work Mr simpson told the Board that the two positions in question became vacant as a result of a reorganization within the Branch It is not necessary for the purposes of this decision to discuss this reorganization. Mr. Simpson became the chairperson of the selection committee because he is the senior person from the Institutional Information and Community Information areas. Moreover, Mr simpson had experience with postings and selections, having served on selection committees in the past. Mr Simpson first asked Mr. Watt to sit on the committee, but Mr. Watt did not feel qualified to serve because he was an Operations Engineer and he I I - I . I 17 felt that he did not have the necessary economics background for the task. Mr Watt deferred to Mr. Gailitis, who agreed to join the committee. Mr. Simpson then contacted someone from Human Resources serving the division to be the third member of the committee. This person was not, however, able to serve because of work commitments. Mr. Simpson then contacted Mr. Barry McKie, who was Manager of the Toronto Production Systems, to see if he could serve. Mr. McKie, who occupied a comparable position to Mr. Simpson and Mr. Watt, was not available Mr Simpson then contacted Mr Coughl in to see if he was free Mr. Coughlin was free and agreed to join the committee Mr. Gailitis was contacted around June 1989, and he and Mr Simpson set about reviewing the numerous applications received in order to prepare an interview list. Mr Simpson, Mr Gailitis and later Mr Coughlin all agreed as to the marking of the interview results They agreed to adopt a consensus marking system. The drafting of the questions in the Applicant Evaluation criteria and the assigning of scores to each of the four parts was done by Mr. Gill, who has a master's degree in economics. Mr Simpson and Mr Gailitis had some minor input into this process. Mr. Coughlin reviewed the questions relating to the development of information systems. Mr Simpson described the consensus marking procedure as follows Each member of the selection committee made notes during the " -~ 18 interview process. At the conclusion of the interview, each member reviewed his notes. A discussion then took place with each member discussing the answers to each question, and comparing the answers to the model answers earlier prepared A score for the question would then be discussed and a consensus reached. In his evidence, Mr Simpson testified that no member of the selection committee dominated the discussion, but rather each took turns leading the discussion and that the process was a salutary one in that one member might be able to draw the other members' attention to something that had been missed, thereby raising a candidate's score Mr Simpson also told the Board that answers of previous candidates would be reviewed in this process in order to ensure the general application of the same standards and criteria. Mr. Simpson testified that the candidates were evaluated strictly on their responses to the questions posed during the interview with respect to reference checks, Mr Simpson testified that the selection committee decided to check the references of any applicant who received a score on the interview within ten percent of the top score. This process was described above. Mr Simpson also described the actual interview process, and his evidence did not significantly differ from that provided by the grievors. All the candidates were given a copy of the position specification immediately prior to the start of the interview All the candidates were asked to provide the names of references, ~ ~ " . 19 including their immediate supervisor, at the beginning of the interviews. Mr. Simpson told the Board that all the candidates were advised that clarification would not be provided but that questions would be repeated, and he described how each of the selection committee members asked different questions at different times Some candidates who had only worked for a particular supervisor for a short period of time were allowed to give the names of an earlier supervisor who had a greater familiarity with their work. Mr Simpson told the Board that the candidates were only evaluated on the answers they gave in the interviews. After all the candidates were interviewed the scores for each one was tabulated. The references of the top six were then checked and a final score reached. The selection committee, determining that all six were within an acceptable range, then examined the seniority of the top six. Mr. Watkin and Mr O/Mahoney were identified as having the longest seniority, and Mr Simpson then examined their personnel files to see if there were lIconunents or factors" that would affect offering them the positions There were not and the selection committee awarded the two positions to Mr Watkin and Mr O'Mahoney. Mr. simpson testified at some length about the grievors' interviews, and those of the two successful candidates His notes of those interviews, taken on the Applicant Evaluation - - . 20 criteria, were introduced as a series of exhibits. It is not necessary to rehearse Mr Simpson's evidence on this matter in any detail. Suffice it to say that he generally impressed the Board with his reasoned and credible explanations about why the questions were given certain values and why the candidates were marked in the way they were. While none of the members of the Board are economists, and while many of the questions asked and answers given are well beyond the expertise of this panel of the Board, it was clear that there was a rational basis to the questions. The questions related to the jobs being filled, and by and large the grievors agreed that this was so, although they objected to the process and felt that their knowledge and abilities should have been assessed on more than just their answers to these questions. There was no example, for instance, of a complete answer not receiving full marks. There were numerous examples of candidates giving correct answers other than the model answers and still receiving credit for them There were also examples of the grievors receiving higher marks for some of the questions than the successful candidates, and the reverse is also true. By and large, Mr. Simpson's evidence indicated the professional manner in which the interviews were conducted. We do, however, have some concerns about the process and we will return to these concerns later in this award In cross-examination, Mr. Simpson testified that he considered all of the candidates qualified for the position and that is why l - - " 21 they were selected for interviews. He also testified that he was aware that both he and Mr. Gailitis knew that employees who reported to them had applied for the positions, and they knew this because of their review of all the applications Mr Coughlin did not know who had applied because he did not become a member of the selection committee until just prior to it beginning its interviewing work. Mr Simpson also elaborated on his evidence about the consensus scoring system. He described the process as one involving discussion, as one in which all three members of the committee were equal participants, and as a process involving a great deal of consensus, as the three committee members were generally within a point or a point-and- a-half in their assessment of the various candidates. Generally what disagreements arose did so as a result of different recollections of what the candidate had said These disagreements could and would be resolved by reference to notes and by further discussion among the members of the committee The fact that Mr. simpson and Mr Gailitis were economists allowed them to bring their expertise to bear on the process, while Mr Coughlin had a good knowledge of systems and could share his expertise in that area In cross-examination Mr. Simpson also testified that he was the person who checked references of the candidates who made the short list, and he did so by phoning the persons identified by those candidates as their supervisors. An example of the , - 22 reference form was introduced into evidence Mr Simpson testified that obtaining references accomplished two purposes First, it enabled the selection committee to cross-reference answers given in the interview with information provided by the references. Second, it allowed the selection committee to obtain information on those candidates, for example, those from outside the public service, who were not personally known to them. The reference results were not, however, used in any way to adjust the scores given on the interview Mr. Simpson was also questioned as to whether it was appropriate for Mr Watkin to identify Mr. Coughlin as his immediate supervisor when his immediate supervisor was, in fact, Mr. Slapack. It was suggested to Mr. simpson that the fact that Mr. Coughlin was a member 0 f the selection committee was another reason why Mr Coughlin was unsuitable for this purpose. Mr. Simpson testified, however, that Mr Slapack was not familiar with Mr Watkin's work, and that other candidates were allowed to name persons other than their immediate supervisor when this was the case Moreover, even though Mr Coughlin was not able to answer all of the questions on the reference form about Mr. Watkin, he was, in Mr Simpson's view, sufficiently familiar with this candidate to serve as an appropriate reference Mr. Coughlin testified next He has worked at the employer since May 1987. This was the first job competition that Mr Coughlin has participated in. In his evidence, Mr Coughlin described how - I I - ~ 23 Mr Simpson telephoned him and invited him to participate on the selection committee in August 1989. Mr. Coughlin's evidence as to the interview and selection process did not differ in any significant way from that of Mr. Simpson. Mr. Coughlin testified that in terms of scoring the candidates he felt that his views were respected and that no single member of the selection committee dominated the consensus scoring process. Mr Coughlin reiterated Mr. Simpson's evidence that the candidates were only evaluated on what they said during the interview process, and not on the basis of selection committee members' knowledge of the different candidates' skills and abilities. Mr. Gailitis was the third and final witness for the employer He has been working for the employer for fifteen years At the time of the gr ievance Mr. Patterson was reporting to him Mr Gailitis has served on a number of selection committees, and he described how he came to serve on this committee. His evidence on this point, as well as on the work of the committee, I corroborated the accounts earlier provided to the Board by Mr. Simpson and Mr. Coughlin Mr Gailitis told the Board that the system of consensus scoring was adopted in part as the result of habit in that it had been used successfully on previous job competitions He also told the Board that consensus scoring keeps people honest by eliminating "highballing or lowballing," insofar as it required selection committee members to justify the scores they proposed to give for each of the questions Mr - ~~ 24 Gailitis testified that while he had a sense of the strengths and weaknesses of some of the people who applied in the competition, this sense did not result in any positive or negative predisposition to any of the candidates prior to the start of the interview process. In cross-examination Mr Gailitis indicated that he knew Mr. Patterson and Ms Ragas. Mr. Gailitis told the Board that this knowledge did not affect the grading process in that the selection committee attempted to be as objective as possible in grading the responses. In response to a question from union counsel, Mr Gailitis said that he thought that Mr Patterson had the qualifications for the posted position but that he did not think that Mr. Patterson had prepared well for the interview Moreover, it was Mr. Gailitis's evidence that Mr Patterson later indicated to him that this was in fact the case, that is to say, Mr. Patterson had not prepared The evidence having concluded, the case proceeded to argument Union Arqument Ms chapman expressed two major concerns with the competition process and a host of minor concerns The first major union concern was the heavy emphasis placed by the employer on the interview as the means of gathering information about the candidates, and the second major concern was with respect to the I ~ . c_ I I 25 method of consensus scoring. 1- The Interview By placing virtually an exclusive emphasis on the interview, counsel argued, the employer failed to take into account the objective record of the candidate's prior experience, qualifications and abilities. In counsel's view, the jurisprudence of this Board is clear that in assessing candidates in a job competition, the employer must make an effort to collect information about candidates other than through the interview process alone: that is to say, the employer must conduct reference checks with immediate supervisors, and the employer must review personnel files and performance appraisals Counsel did not deny that interviews play an important part in job competitions, but she argued that their limitations are well recognized To be sure they can assist in measuring skill, ability and qualifications. They also measure interview performance. Counsel argued that many qualities cannot be adequately assessed as a result of interviews, such as interpersonal skills, for instance. Or how candidates conduct themselves in the work place, to give another example. In counsel's submission interviews assist in gathering information, but to obtain a complete and objective picture of a candidate's skill, ability and knowledge a comprehens i ve look at the entire work record is required And in this case that was not done, as -~ 26 only six candidates had their references checked, and the personnel files of only two candidates, the successful ones, were reviewed. Counsel argued that the problem of relying on interviews in this case was exacerbated because members of the selection committee scrupulously avoided bringing any of their personal knowledge of the candidates' skill and ability to the interview process Counsel also submitted that the references checks that were conducted did not result in the necessary relevant additional information being brought to the attention of the selection committee, because these reference checks were negative, not positive in nature. The six candidates on the short list began the reference check with a perfect score Points were deducted for information considered undesirable. Accordingly, counsel argued, the skill, ability and knowledge of even the six candidates on the short list were not effectively assessed. Accordingly, counsel argued, the scores provided the selection committee with very little relevant information about the skill, ability and knowledge of the different candidates. Moreover, these reference checks did not provide much in the way of comparative information for the purpose of making relative assessments of the different candidates. Needless to say, those candidates not on the short list 'never had their references checked and so no reference information was gathered about them. - .. 27 Counsel expressed a collateral concern in connection with the reference checks, namely who was contacted. It was, in her view, arguable whether Mr. Coughlin should have served as a reference for Mr Watkin when Mr. Watkin's supervisor was Mr Slapack, and counsel pointed out the apparent gaps in his knowledge of Mr Watkin as evidenced by his inability to answer some of the questions on the reference form. Counsel also pointed out that in the same way the reference check was a negative check, so too was the review of the two successful candidates' corporate files In conducting this review Mr. simpson was not looking for information to complement the assessment of skill, ability and knowledge obtained as a result of the interview, but rather he was looking for factors that might negatively impact on offering the position to the two highest-ranking candidates. Union counsel discussed at some length whether additional information, i e , information obtained as a result of reference checks on all of the candidates and information obtained through the review of every candidate's personnel file and performance appraisal, would have made a difference Counsel argued that it would have made a difference, and she cited the case of Mr. Patterson as an example on point Counsel argued that Mr. Patterson might reasonably have expected that his past record of service would have been taken into account by the selection committee in assessing his ability for I - 28 the Economist 4 position. Counsel noted that Mr Gailitis considered Mr Patterson qualified for the position, and argued that his low score was not reflective of insufficient skill, ability and knowledge, but rather it indicated poor interview performance Maybe, counsel suggested, Mr. Patterson was having a bad day. Maybe Mr. Patterson had not attended an interview for some time and had become rusty. Counsel's point was that Mr. Patterson's performance on the interview should not be determinative of his ability to perform the position in question Union counsel also turned to the evaluation af communication skills in the interview process. In counsel's view, such an evaluation could best be obtained by a discussion with a candidate's immediate supervisor, not by an evaluation during a short and highly stressful interview process. Ms Palatino scored particularly low in communication skills, and while she was not the most senior of the applicants by a long shot, a few extra points might have been enough to push her over the top so that her relative lack of seniority would not have come into play. In this respect it is worth noting that at the end of the competition, Ms palatino scored higher than one of the successful candidates Had she scored substantially higher she could have argued that she should have been awarded the job on the basis of her superior standing. Counsel also expressed concern with the evaluation of written skills being based on the covering letters and resumes submitted for the positions In I I , -- I 29 counsel~s view, neither a covering letter nor a resume could tell I the selection committee anything about a candidate's ability to write reports and communicate effectively in writing. Moreover, this was another example of how checking references could have provided the selection committee with information about the different candidates' communication abilities. The same could be said with respect to making an assessment of the different candidates' interpersonal skills. Counsel agreed that the interview process was an important means of testing a candidate's skill, ability and knowledge, but she argued that it should not be relied upon exclusively. Just because a candidate was not able in an interview to answer a question did not mean that he or she did not know the answer By obtaining information from a variety of sources the selection committee would have been better able to make a determination with respect to skill, ability and knowledge Performance appraisals, for example, might provide exactly this sort of information. References would also assist But for these references to be useful ~ counsel argued, they must be positive, not negative. And, counsel suggested, if positive references had been conducted in this case, that is to say where points were given for positive qualities, instead of being deducted for negati ve ones, the final scores might have been very different They might, for example, have been so different as to put Ms Ragas and Mr. Patterson in the top ten percent, where their . -- 3D seniority would have given them the edge, all other factors being relatively equal. Counsel drew the Board's attention to some case law on point. In Marek (Samuels) 414/83, a job competition case, the decision to select the successful candidate was made solely on the basis of the application form and the interview. The Board in that case said: It is hard for this Board to understand how this could occur, in view of the repeated direction this Board has given on the need to consult personnel files and candidates' supervisors, particularly when one of the candidates is known to the interviewers - see, for example, MacLellan and DeGrandis, 506/81, 507/81, 690/81 and 691/81, wherein the jurisprudence is summarized at pages 25 and 26' The jurisprudence of this Board has established various criteria by which to judge a selection process: 1 Candidates must be evaluated on all the relevant qualifications for the job as set out in the Position Specification. 2 The various methods used to assess the candidates should address these relevant qualifications insofar as is possible. For example, interview questions and evaluation forms should cover all the qualifications 3 Irrelevant factors should not be considered. 4 All the members of a selection committee should review the personnel files of all the applicants 5 The applicants' supervisors . - 31 should be asked for their evaluations of the applicants 6. Information should be accumulated in a systematic way concerning all the applicants In Leslie, 126/79, the primary basis on which this Board ordered a new selection process was the fact that the interviewers knew one of the candidates, and had relied on the interviews alone, without any recourse to the supervisors of other candidates Nor in Leslie, had the interviewers referred to the grievor's personnel file or performance appraisals In our view, this conduct alone fatally flaws the selection process undertaken by the interview panel here (at 4-5). The Board in Marek went on to consider the questions posed in the interview process and whether or not they could elicit the necessary information The Board found that they could not, and it concluded that for "a host of reasons" the selection process was inadequate (at 8). In Poole (Samuels) 2508/87, the Board was again called upon to consider a job posting grievance In this case, five persons were selected for interviews At the interviews, a series of questions were asked of each candidate to elicit information concerning the candidates' qualifications and experience And then the candidates were scored on these answers without any regard to the information on the application forms, or information which might have been found in personnel files, or information from the applicants' supervisors at the Hospital. Apparently it is Ministry policy to base its decision entirely on the scores at the interview If this is the Ministry's policy, then it is absolutely incomprehensible to us why it should be so And this Board has explained in detail many times what is required to fulfil the requirements of Article 4.3 The jurisprudence is of long-standing ..[cites -- 32 Marek].... There must be a full gathering of information concerning the qualifications and ability of the applicants. It is simply not satisfactory to consciously ignore information as was done here (At 2-4, emphasis not ours.) Counsel also referred the Board to the Skagen & Glemnitz (Springate) 1934/87, 1936/87 award, a case in which the union sought an order requiring the employer to re-run a job competition In this case, a job applicant was selected solely on the basis that she scored highest in the interviews. The Board found that: Had the grievors' past experience and Mr Anderson's views concerning the manner in which they had carried out their duties been weighed along with the interview results, it is possible that the final result might have been different. Further, while Mr Anderson had not forwarded any material for inclusion in the grievors' corporate personnel files, it is possible that over the years one or more of his predecessors had done so Any such material should have been taken into account (at 13). Likewise, union counsel argued, the employer in this case should have taken this additional information into account II Consensus Scoring The second maj or union concern related to the use of consensus scoring Counsel expressed the view that notwithstanding the evidence heard on this point, the very real possibility exists that consensus scoring may unduly influence selection committee members and thereby affect the outcome. Consensus scoring might also result, counsel suggested, in "horsetrading" among committee I ~ . 33 members. Counsel conceded that there was no evidence of either of these possibilities occurring, but pointed out that such evidence is hard to obtain Moreover, counsel suggested that union concerns about consensus scoring were exacerbated when there were also concerns about the composition of the panel, such as in this case. Union counsel did not make any allegations of bias or of improper behaviour, but suggested that the absence of an objective committee member who had no knowledge of any of the applicants was a matter of some concern, as was the lack of any written record of how each committee member scored the various candidates. In Bent (Fisher) 1733/86, the Board considered the matter of consensus scoring It said The term "consensus scoring" means that the three members of the selection panel discussed and ultimately agreed on a score for each individual question. This Board finds that the method of consensus scoring is inappropriate and should not be continued by this or any other Ministry in the selection process. The purpose of having three individuals on a selection committee is presumably so that each person can bring their own input into the decision-making process. A consensus method of scoring means that the parties must agree on the scoring for each question and that raises the spectre of either a majority of two overruling the third person each time or a dominant member of the panel exercising his superior position or knowledge over the other members Furthermore, it leads to the possibility that there would be "horse-trading" between selection committee members. For example, a committee member could agree to a certain scoring on one question in exchange for another committee member changing his position on another question There is in this Board's opinion, no advantage at all in consensus scoring over individual scoring An issue was raised with respect to an advantage in consensus scoring in that it allows a member of the selection committee lacking technical . I - - 34 knowledge, like a Human Resources person, to get the benefit of the superior technical knowledge of the other panel members which would not be available in an individual scoring system However, even in an individual scoring system, the selection committee members could quite properly discuss their individual points of view and ask questions of each other, however when it comes down to the actual scoring, each member of the selection committee must put down his own opinion without the necessity of reaching a consensus Therefore, this Board would envisage a free and lively discussion among the selection committee members before individual scoring was exercised but ultimately, the member must score the candidate as he deems fit and not so as to please the other members of the selection committee If this open approach is taken it would allow the less technically competent member of the selection committee to hear the views of his colleagues but still exercise his own judgment when it comes to scoring. Obviously, when a comparison is made between the scores of individual claimants it is appropriate and indeed proper to average the scores of the selection committee so that a proper comparison can be made. This, however, does not in any way infringe on the right of the individual member on the selection committee to express his views, it just means when you compare it to other applicant's scores, the effect of a minority position will be diluted which is perfectly proper. Therefore, this Board finds that the use by the selection committee of consensus scoring system was a defect (at 1-3) In the Bent case the employer also only contacted the supervisor of the successful candidate for a reference check in spite of numerous previous Grievance Settlement Board decisions that say it is an essential part of the selection process to do a reference check on the candidates. This Ministry, however, seems to adopt the practice that they are only verifying their decision by checking with the supervisor of the successful candidate presumably to see if the reference check bears out their opinion. This again emphasizes the slavish devotion that the Ministry seems to have with respect to interview scores and its failure to understand that an interview is only part of the selection process, another part being a reference check of all the candidates who have at least obtained interviews (at 4) . The presence of defects is not sufficient, however, for a ~ i -- .. 35 grievance to be sustained. As the Bent case makes clear, the union must establish that the defects would have affected the outcome. In Bent the Board found that they did not and so dismissed the grievance. In this case, counsel argued that they might have affected the outcome. The final case raised by union counsel was the Eadie (Devlin) 766/88, decision. In this case the Board reviewed three decisions including the Bent case, which commented on consensus scoring and made the following observation In our view, the impropriety of consensus scoring is not as clear as suggested, particularly in the Bent award and, in future, the Board may wish to re-examine this method of scoring to determine whether it can satisfy the requirements of the Collective Agreement. In light of the jurisprudence which has developed to date, however, and for the sake of consistency within the Board, we find the selection process in this case was defective because the members of the selection panel did not independently score the oral interview We also find that the selection panel acted improperly in failing to consider and review the Grievor's performance appraisals (at 13) In Eadie the Board went on to consider what effect the consensus scoring might have had, and determining that the grievor was qualified for the position in question, the Board found that there was a possibility that the outcome of the oral interview was affected by the method of scoring "In view of the conclusion reached by the Board concerning consensus scoring, the only appropriate remedy is to rerun the job competition" (at 14) . -- 36 III Other Problems In addition to the union's two major concerns relating to the reliance on the interview results and to consensus scoring, counsel briefly reviewed other problems which she argued may have affected the outcome in this case Counsel argued that the composition of the selection committee, and in particular the absence of any outsider, raised concerns, particularly from the point of view of the perception of objectivity The limited opportunity to review the position specification may also, counsel argued, have affected the outcome. So too might have the refusal to provide clarification with respect to some of the questions. Counsel submitted that there was an undue and unnecessary emphasis on oral communication skills and this emphasis detracted from true testing of skill, ability and knowledge The evaluation of the covering letter and written resume for the written part of the interview was, counsel argued, not a good way to go about evaluating written skills. Counsel asked the Board to order that the competition be rerun, but that it be limited to those persons who had applied and participated in the first instance. Counsel requested that a new selection committee be struck, and that the members of this committee be directed to review the personnel files and performance appraisals of all the candidates. In addition, counsel submitted that this committee should be directed to secure a reference from at least one supervisor familiar with the I -- , 37 work of each candidate. Counsel requested that the committee be directed to retain all records of the scoring of the interviews, and that it also be directed not to engage in consensus scoring Counsel also argued that the new committee should be instructed to design and conduct the new selection process in a manner that would not favour the incumbents for the experience they gained since the original composition. Finally, counsel asked the Board to retain jurisdiction over the implementation of this award Emplover Araument Employer counsel began his submissions with a consideration of the onus of proof in this case. Counsel argued, and this Board ~ accepts, the proposition that it is up to the union to show that there were flaws in the competition and that these flaws went to the result. Put another way, merely pointing out the flaws is not determinative; the union must also demonstrate that those flaws may have affected the outcome of the job competition In counsel's view, the union has not met its burden of proof. Counsel argued that no selection process is perfect. A job competition is by nature a human endeavour, one in which there are bound to be mistakes. Counsel conceded there may have been flaws in this case, but that these flaws did not affect the result. Counsel illustrated this contention by identifying a number of the purported flaws and discussing their impact . 1 - 38 Counsel pointed out that the union identified a flaw in the method of checking references Counsel submitted that there was no flaw in this process and that even if Ms. Ragas and Mr Patterson had obtained full marks I this would not have been sufficient to put them on the short list. With respect to the fact that references from immediate supervisors were only obtained for the six top candidates, counsel argued that this was perfectly appropriate because these candidates had been found to be relatively equal In addition, these references elicited appropriate information about the knowledge and abilities of the candidates necessary far the positions in question. Counsel suggested that the union argument on this point was somewhat at cross-purposes in the sense that the union was criticizing the members of the selection committee for failing to bring their own knowledge of the different candidates' skill, ability and knowledge to bear in the interview assessment, while at the same time arguing that the composition of the selection committee was somehow improper With respect to the composition of the selection committee, counsel argued that its composition was not flawed The absence of a member of Human Resources did not mean that the committee was not objective in going about its work In addition, the fact that the three members of the committee were all managers in the same division was neither here nor there. Not only was this situation not irregular, it was normal procedure within the - ';' 39 ontario Public Service. Counsel also pointed out that some concerns were expressed by Mr. Patterson about Mr Coughlin's presence on the committee in that he was not trained as an economist, yet the union was arguing that there should have been a Human Resource person on the committee who would know even less about an economist's work than Mr Coughlin. As was the case with respect to references and composition, counsel argued tha t the union case was at cross-purposes Counsel also argued that the criticism of consensus scoring was not well placed given the fact that there was no evidence of bias in the reaching of consensus results. There was no evidence whatsoever, in counsel's submission, of any "horsetrading" or of any improper activity in the marking process. All the evidence was to the opposite effect, and supported the employer's claim that the marking was fair, through and uniformly applied In this respect, counsel pointed out that the grievors had generally agreed that the questions asked d.ur ing the interview were relevant to the positions in question Counsel sought to distinguish those cases that concluded that consensus scoring was inappropriate by suggesting that these cases were specific to their facts Counsel argued, moreover, that there must be some evidence of abuse by consensus scoring for the Board to rule that it is inappropriate per se In the absence of such abuse, or of any evidence pointing to such abuse, I --" I 40 or of any evidence suggesting that had some other method of I scor ing been adopted the resul t would have been different, the union could not, employer counsel submitted, describe the consensus scoring system as a fatal flaw. Indeed, counsel went on to make an affirmative argument, based on the evidence presented in this case, for consensus scoring. Such scoring forced the committee members to be as accurate and fair as possible in that it required the members to defend their suggested scores for each question and it brought out information that one of the members might otherwise have missed Counsel submitted that there was not a single piece of evidence suggesting anything other than that this selection committee acted fairly and in good faith. Counsel drew the Board's attention to a number of cases illustrating these points. In Renton & Ross (Roberts) 1577/84, 1578/84, among the issues considered was whether the selection process adopted by the employer was so flawed so as to require a new competition As in the instant case, in Renton & Ross a system of consensus scoring was employed Moreover, the selection committee did not review the personnel files of the grievors, nor did they contact the grievor's supervisors in making an assessment of their qualifications and abilities to perform the position in question Notwithstanding these deficiencies, the grievances were dismissed. It is important, however, to point out that the deficiencies with respect to not ~ I -; I 41 reviewing personnel files and contacting supervisors were ameliorated as a result of two of the three members of the selection committee possessing considerable background knowledge of the grievors' skills and abilities, and this information was communicated to the other committee member. The Board found: In light of this evidence, which was not disputed, it must be concluded that any prejudice which might have arisen as a result of the deficiencies was mitigated by the briefing process . There was nothing in the evidence to indicate that these briefings were anything but objective. On the evidence, neither of these persons had any reason to be dishonest, biased or actuated by malice or ill will. Certainly, in the interests of fairness, it would have been preferable for members of the panel to have obtained this background information from the more usual sources, but it cannot be said in the circumstances that these flaws in procedure were sufficiently serious to have impaired the ability of the panel properly to assess the relative qualifications and abilities of the candidates (at 18) Employer counsel also pointed out that the questions on the Applicant Evaluation criteria provided candidates with an opportunity to bring their skill, ability and knowledge, including past performance, to the attention of the selection committee in that a number of these questions required reference to these factors. Counsel also drew the Board's attention to the Woods (Watters) 2253/87 decision. In this case the Board describes the job competition as having "followed the normal coursen (at 2) That normal course included the members of the selection committee apparently all coming from the same division of the same ministry In addition, the selection committee in this case did ... , 42 not review personnel files or contact supervisors with respect to this issue the Board said: There is no doubt in our minds that generally competition panels should resort to personnel files and candidate's supervisors in coming to a decision as to the relative merits of those applicants interviewed. Numerous panels of this board have imposed this requirement in order to assure that a full assessment is made as to the applicant's qualifications and ability to perform the required job duties. Such a course of action will also ensure that a candidate will not be prejudiced in terms of their right to claim a benefit arising from greater seniority. An employer who fails to engage in this type of process and who relies exclusively on the interview to determine success within the competition is taking a substantial risk that this board may ultimately set aside the competition should its result be contested (at 10-11). The Board in Woods then went on to consider whether this was a case where the result of the competition should be set aside. After considerable reflection, we have concluded that we should not so order in this case We have not been persuaded that a review of the grievor's personnel file or a more meaningful consultation with supervisors would have resulted in her being successful in this competition. There was no evidence before us to suggest that such information would have materially augmented the information that was already before the panel (at 11) Counsel argued that it was incumbent upon the union in the instant case to adduce evidence that the additional information would have made a difference. Since no such evidence was adduced, the inference should be drawn that such evidence does not exist. The final case referred to by the employer is the Burrows (Ratushny) 426/86 decision Counsel argued that this case demonstrates the absence of stare decisis at the GSB insofar as I , - .. 43 consensus scoring is concerned In this case it was argued, inter alia, that the selection process was unfair because too great a reliance was placed on the interview results, and the panel members did not individually rate the answers of the I candidates The Board in this of the view that case was interview results should not form the sole basis for assessing the qualifications of candidates "The personnel files should be reviewed and considered and supervisors should be\consultedll (at 10) . In this case, the Board found that this was done. The Board in Burrows also considered whether or not the consensus scoring system adopted by the selection committee in this case was unfair The Grievor took the position that each of the panel members did not record an individual score for each of the questions. It was argued that this defeated the very purpose of having more than one person on the panel. However, the evidence of one of the panelists was that each panelist did form an independent rating in relation to each question During the meeting following the interview, each panelist, in turn, would express his rating for a particular question Discussion would follow and a consensus would be reached as to what score should be assigned for that question and answer. In these circumstances, we are not satisfied that the method of rating answers was unfair (at 11-12) For these and other reasons the grievance in Burrows was dismissed. Counsel concluded by stating the job competition in the instant case was done fairly and by urging the Board to dismiss the three grievances. I ~ 44 Incumbents' Aroument Mr. Watkin argued that the grievances should be dismissed In his view none of the candidates received special treatment There was, in his view, no evidence of bias in the process The documents introduced into evidence supported the employer's claim that the process was fair. Mr. O'Mahoney made the point that greater reliance on references from supervisors would not have made much difference to his application and that of Ms. Ragos and Mr. Patterson because their work was largely unsupervised. Mr O'Mahoney suggested that this situation should be contrasted with the cases relied upon by the union where supervision was an important part of the job and where supervisors would as a result have something to say. Mr. O'Mahoney also suggested that there were a large number of highly qualified candidates interested in the two positions. One way of distinguishing between them was a test of their skill, ability and knowledge In his view, the test in this case was fair. Decision In our view this grievance must succeed In Ouinn (Prichard) 9/78, the Board said: the employer must employ a process of decision-making designed to consider the relative qualifications and ability of the candidate in a competition which will ensure that sufficient relevant information is adduced before the decision-makers in order that they may make their comparisons in the confidence that they are able to thoroughly and properly compare the qualifications and abilities of the competing applicants . . In order to fulfill these obligations, the employer l .~ - "':l 45 must design and utilize a selection process in job competitions that is consistent with the purposes of the selection process. Thus, under this collective agreement, the process must be designed to elicit in a systematic manner sUfficiently comprehensive I information about each applicant relevant to the qualifications and ability required to perform the job I in order that a fair and reasonable assessment of the relative strengths of the candidates can be undertaken and the final selection made. I I We find that the Applicant Evaluation criteria used by the I employer did provide the employer with significant information about the candidates and about their skills, abilities and I knowledge. We also find, however, that the employer in this case I relied too heavily on the interview results. The authorities are I extremely clear that the employer must not rely solely on interviews in job posting cases. At the very least the employer must also conduct reference checks of all candidates with I immediate supervisors and review all applicants' personnel files I Some reference checks were done in this case, but only for those I applicants who made the short list. The review of personnel I files was limited to the two successful candidates, and it was not directed at determining their qualifications for the I positions in question, but at discovering whether there was anything in the records that affected their suitability for those positions As was noted in the Woods case, if the employer relies solely on the interview for making an assessment of candidates it does so at its peril In this case there is no distinguishing factor that would permit J 46 us to say that the manner in which the interview was conducted allowed the information that could normally be obtained through reference checks and review of personnel files to be brought to the selection committee's attention. Indeed, the evidence was clear that the selection committee members scrupulously avoided bringing to bear any of their own knowledge and experience of the candidates to the evaluation of their interview responses. And so it was that the candidates were evaluated almost entirely on the basis of their interview performance. Turning to the interview process, it is hard for the Board to understand why all the candidates for the position in question were not given copies of the position specification well in advance of the interviews Assisting the candidates in preparing for the interviews by giving them as much information as possible about the positions in question seems essential to us and hardly an administrative burden to the employer. It is also hard to understand why the candidates were not given copies of the questions they were being asked along with an indication of the marks assigned to each question, minus of course the model responses. In this way they could have made a judgement as to how much detail was required in answering each question They could have seen what emphasis the employer was placing on the different aspects of the interview. A considerable number of the questions asked were extremely long and involved. Even the best prepared candidates would have had difficulty in remembering all I - I -; 47 the points to be covered as they were called upon to give a response These observations aside, we find the reliance on the interview as the means of evaluating the different candidates a major flaw in the running of this competition and one not corrected, as in Renton & Ross, by the members of the selection committee bringing to bear their knowledge of the abilities of the different candidates. We should point out, however, that such a "correction" would not, in any event, have persuaded us in this case for we are of the view that the gathering of information demands that references be obtained on all candidates and that the personnel files of all candidates, where available, be reviewed. These are basic requirements, established by panel after panel of this Board, and not only do we agree with them, but we find no basis in the present case to depart from them In reaching this finding we would like, however, to make it clear that we have found no evidence whatsoever of any bad faith or improper activity by the members of the selection committee, as was suggested by one of the grievors Indeed, all of the evidence is to the opposite effect. The evidence is clear that the members of this committee acted fairly and properly We reject categorically any suggestion of bias by them for there was not a hint of such conduct. Quite the contrary. We were impressed by their principled diligence and fairness in their assessment of the candidates' in the interviews The one caveat to this is in the evaluation of the candidates written work based -<>- 48 on their covering letters and resumes. We do not see how such written work could possibly be considered relevant evidence of an ability to communicate in writing at the Economist 4 level. Having found a major flaw in the running of this competition, the next question to be addressed is whether or not this flaw would have made a difference. In our view the results of the interview may not have been any different. To be fair, the Board was far from impressed with the evidence of Ms. palatino and Mr Patterson about their preparation for the interview. To find, however, that the result of the interview may not have changed does not necessarily mean that the grievance should be dismissed There is every possibility that additional information, gathered through reference checks and through the review of the candidates' personnel files and performance appraisals may have resulted in a higher overall assessment of the different candidates. It may also have resulted in a lower overall assessment. There is really no way of knowing, because despite the repeated directions of this Board, the employer did not conduct reference checks of all the candidates and did not review the personnel files of all the candidates On this basis alone we are granting the grievance A few additional observations are, however, in order. In our view the references that were obtained did not provide the Board with the kind of information necessary to make a full ~ ~- '7 49 evaluation of candidates. The evidence in this case is clear All the candidates started out with twenty points. Points were then deducted II for factors considered significant by the Selection Board in its assessment of the applicant's capability to perform the duties/functions/responsibilities of the two positions II with respect, we do not think that this is an adequate method of obtaining references. In our view, references must be designed so as to elicit both positive and negative information about different candidates' skills, ability and knowledge. Then this information must be brought meaningfully to the attention of the selection committee so that it can fully evaluate a particular applicant and also allow a comparative assessment of the different candidates to be made The fact, as was the case here, that the employer considered all of the candidates qualified for the position does not lessen the obligation, in our view, for a comprehensive reference check to be conducted. Employer counsel submitted that it was incumbent upon the union to demonstrate how reference checks and review of performance ) appraisals could have affected the result. With respect we disagree The employer has carriage of the job competition process It also has access to supervisors and personnel records It hardly behooves the employer, notwithstanding clear direction from this Board to secure reference checks and personnel information in job competitions, to say that it is up to the I ~ 50 union to supply such information at arbitration and demonstrate the effect it would have had. We are of the view that it was up to the employer to initially consider this information as part of the job competition process It chose not to do so. In our view this evidence might have made a difference, but there is no way of knowing so until this evidence is gathered In Skagen & Glemnitz, for example, the Board said: It is quite possible that had the selection committee considered supervisors' evaluations as well as candidates' personnel files, the result of the competition may have been the same. There is sufficient doubt as to whether this would have been the case, however, that we are satisfied that the selection . . . should be set aside (at 13-14). Other cases have reached comparable results. This is to be contrasted with those cases where the Board has found that additional evidence would not have made any difference. (See for example, Woods at 11ff.) How might this additional evidence have made a difference? For Ms. Palatino to have succeeded in the competition she would have had to score much higher than any of the other candidates given Article 4.3 of the Collective Agreement As already noted, reference information and review of personnel records and performance appraisals may have affected the selection committee's overall assessment of Ms Palatino Possibly, she might have scored very highly and the employer may not then have awarded the position to relatively equal candidates on the basis of seniority. It is possible that this additional information I ~ -- I I .~ 51 I would have established MS. palatino as the "superior" candidate. Turning to the matter of seniority, Ms Ragos and Mr Patterson are more senior than Ms. Palatino and the two successful candidates All that Ms. Ragas and Mr. Patterson would have had to establish was relative equality. Given that the employer considered these candidates to be qualified, and employer witnesses testified as much, it is not beyond the realm of possibility that additional information might have improved their scores In this regard, we note that Ms. Rages carne within six points of inclusion en the short list. In short, it is fair to say, based on the evidence which we heard, that all the candidates considered for the two Economist 4 positions were competitive, and it is reasonable to draw from this evidence the inference that additional evidence about them may have affected their overall evaluation results, had the interview been complemented by reference checks and a review of their personnel files and performance appraisals In our view, this is the burden the union must meet. This is the burden the union has met. We would like to make some concluding observations with respect to consensus scoring. The general practice of this Board is to disapprove of consensus scoring. In this case, however, it is worth taking note that this method of scoring was done fairly and to the general benefit of the different candidates The fact that the results of each interview were not tallied until all the I .:L . 52 interviews were completed, as well as the fact that the members of the selection committee made a sustained effort to compare the responses of the different candidates to the different questions, illustrates to us the conscientious manner in which the selection committee went about its work We agree with the Board in Eadie that "the impropriety of consensus scoring is not as clear" as has been suggested. In this case we find no impropriety on this , .. basis, but as already noted our decision to order a rerun is based on other factors. We also do not see anything wrong with the membership of a selection committee being comprised of persons all of whom work in the same division. In a case of this kind, where candidates are competing for a highly skilled position requiring a great deal of knowledge and expertise it makes some sense for the members of the selection committee to be persons with the knowledge and ability to evaluate the candidates appropriately. The problem in this case was not consensus scoring, or the fact that the committee members came from the same division, but that the selection committee did not conduct reference checks and review the personnel files of all the interviewed candidates in addition to the interviews in order to systematically obtain all relevant information about the different applicants By not conducting reference checks and by not reviewing personnel and other records the selection committee denied itself potentially valuable sources of information in its determination of the best J, - ..; 53 candidates for the position in question, as well as a source of information that this Board has mandated in decision after decision in job competitions Very simply, the selection process in this case does not comply with the requirements of this Board, and we find no reasons in the facts why those requirements should not be applied. We therefore order: 1. That the competition be rerun. 2. That the new competition be limited to those persons who participated in this competition 3 That a new selection committee be struck and that no member of the selection committee in this case be appointed to it 4. That references from each candidate's supervisor or some other person familiar with each candidate's work be obtained and be considered by the selection committee. That such references be designed so as to elicit information about each candidate's skill, ability, and knowledge as well as suitability for the positions in question. 5 That each member of the selection committee review each candidate's personnel file and performance appraisals for the purposes of determining their suitability and relative standing for the positions in question 6 That the selection committee retain all written records with respect to the rerunning of this competition 7. That the competition be designed in such a way so as to not favour the current incumbents. 8. That if any of the grievors prove successful in this competition, they are to be compensated from the st~rting date of the positions in question. We retain jurisdiction over the implementation of this award -- I : - ! 54 Dated at ottawa this 5th day of December 1990. William Kaplan Vi~hairperson 111 DISSENT" (D1ssent attached) I. Cowan Member ~ I I I to -~ c '1 1968/89, 1970/89, 1972/89 DISSENT PALATINO/RAGOIS/PATTERSON I have carefully studied the decision of the majority and have concluded, with respect, that I am unable to concur with their decision. In this case it is clear from the evidence, and uncontradicited, that an exhaustive effort was made by management to formulate interview questions which were directly relevant to the responsibilities of the vacant positions and which provided each candiate full opportunity to demonstrate past experience and skills in all required areas. No limit was placed on the time required by the candidate for reply. Model answers were prepared in advance as a guide for scoring by panel members and the value assigned to each area of questioning was appropriate. Although inappropriate makeup of the selection panel and potential bias of its numbers was mentioned no evidence was adduced to support these opinions. It is perhaps understandable for an unsuccessful candidate to excuse his/her failure by reference to these factors but to express a "feeling" that they may exist is simply not enough on which to conclude a flaw in the process exists. The fact that Communication Skills were scored based entirely on the written application and resume as well as conduct during the interview has been viewed as a flaw. Perhaps a request for submission of more formal written material would have provided a better basis for judgement of writing skills but I am satisfied that performance during the interview, particularly since all questions were directly related to the candidates area of expertise, was an adequate basis for judging oral communication skills. Since the maximum score for communication skills was 10 of a total of 105 I do not believe this to be a major flaw nor one which could have skewed the results. Another perceived flaw was allowing the candidates access to the position descriptions only briefly and immediately prior to their interview Since all three grievors were familiar with the area in which the vacancies existed they could have, had they wished to make the effort, questioned other employees more familiar with the responsibilities of the vacant positions. I do not view this as a flaw. with respect to reference checks, each candidate was asked to name two referees one of whom was to be a current or past supervisor having knowledge of the candidates work performance. The fact that these checks were conducted by the panel chairman rather than all three panelists does not diminish their value since notes were made and shared with other panelists. -'- , - 2 - In a more general view it is common knowledge that the process of candidate selection is more art than science and is not one which can be made a science by the imposition of rigid rules and proceedures For example it can hardly be said that the original screening of 80 applicants by referral only yo written applicati9ns will always result in the best candidates being chosen for interview. . Further, reliance on interviews with supervisors and \ persual of appraisals on file does not guarantee freedom from bias since it is sometimes more important to be familiar with the supervisor or the author of the appraisal than its contents. In the result I believe that this competition was conducted with meticulous care being taken to formulate and score questions carefully designed to elicit information directly related to the jobs advertised. I believe the system used for reference checks and file review were adequate and lastly it is my opinion that arbitrators should avoid the procedures for candidate selection and should instead concentrate cn the adequacy of the I procedure chosen and the integrity with which the process has been conducted bearing in mind that the objective is selection of the best candidate. I I would have dismissed the grievances. W-- I. Cowan Member