Loading...
HomeMy WebLinkAboutP-2011-3487.Mitta.13-06-27 DecisionPublic Service Grievance Board Suite 600 180 Dundas St. West Toronto, Ontario M5G 1Z8 Tel. (416) 326-1388 Fax (416) 326-1396 Commission des griefs de la fonction publique Bureau 600 180, rue Dundas Ouest Toronto (Ontario) M5G 1Z8 Tél. : (416) 326-1388 Téléc. : (416) 326-1396 P-2011-3487 IN THE MATTER OF AN ARBITRATION Under THE PUBLIC SERVICE ACT Before THE PUBLIC SERVICE GRIEVANCE BOARD BETWEEN Daniel Mitta Complainant - and - The Crown in Right of Ontario (Ministry of Municipal Affairs and Housing) Employer BEFORE Kathleen G. O’Neil Vice-Chair FOR THE COMPLAINANT Stephen Moreau Cavalluzzo Hayes Shilton McIntyre & Cornish LLP Counsel FOR THE EMPLOYER Susan Munn Ministry of Government Services Legal Services Branch Counsel HEARING February 8 & 25 and March 1, 2013 - 2 - Decision [1] This decision deals with the complaint of Daniel Mitta, Architect, in relation to a competition for the position of Coordinator of Code Development in the Building and Development Branch of the Ministry of Municipal Affairs and Housing. The complaint alleges that the selection process was seriously flawed, and should be re- run. The employer disagrees, maintaining that the process was fair and impartial, and that no term or condition of the complainant’s employment was breached. Factual context [2] The events leading most directly to this grievance began with the July 25, 2011 posting for a six-month position, Coordinator of Code Development in the Code Development Unit of the Building and Development Branch. The job was described as contributing to public safety by providing leadership and expertise in building code development. The complainant, Mr. Mitta, had been working in that branch for 11 years by then, had significant expertise in the development of the Building Code beyond his training as an architect, and had acted as manager for a period of time. He applied, as he saw the posting as a good development opportunity, which would allow him to contribute his knowledge in a fulfilling role. Three other employees of the Building and Development Branch, two engineers, and another architect, also applied and were interviewed. There were two other applicants, but one did not meet the minimum educational criterion of a university degree in Building Sciences, and the other withdrew from the competition. [3] Nancy Smith, the other architect among those interviewed, was the successful candidate. She started in the position on November 1, 2011 as a six-month secondment. It has since been extended, and at the time of the hearing of this matter in early 2012, Ms. Smith was still in the position. - 3 - [4] Before posting a position, policy requires the establishment of selection criteria. These are defined in a document entitled “Manager’s Guide to Staffing in the Ontario Public Service” as: - bona fide staffing requirements expressed in terms of qualifications – the knowledge, skills, abilities and competencies required to enter the position. They are necessary for the successful performance of work and reflect the relative importance of each aspect of the work to be performed. [5] Alek Antoniuk, Manager, Code Development and Advisory Unit, was the hiring manager and the employer’s witness. In consultation with Human Resources, he approved a position description, and set the selection criteria as follows: Technical Knowledge: Experience and knowledge 25% of theories, principles and practices of building science related to the design, operation and maintenance of buildings. Leadership and Project Management Skills: Ability 20% to interpret and apply management policies and directives related to financial and human resources. Plan, lead and conduct specialized field studies and provide project leadership to project staff. Analytical, Research and Judgment Skills: Ability 20% to analyse building performance levels and assess the implications of problems and emerging building safety issues in order to make recommendations for code development and interpretation. Judgement skills to organize, coordinate and supervise subordinates. Communication and Interpersonal Skills: 20% Establish effective working relationships with colleagues, government officials from various levels and representatives from the private sector. Provide effective consultation services and advice and develop correspondence, briefing notes and other materials. Thorough knowledge of the Ontario Building Code, 15% standards and related legislation (example: the Fire Code, plumbing regulations) to lead the provision of technical, research and consulting services for building code development, interpretation and issue resolution. [6] The job ad reads in language similar to these criteria, but they were not identified to the candidates as such in the ad, nor did the weighting percentages appear there. In order to evaluate the candidates’ ability to meet these criteria, the employer chose one screening and three rating methods. In an e-mail dated September 2, 2011, Mr. Antoniuk informed the applicants of the rating methods and their weighting within the overall evaluation score as follows: a project assignment and presentation worth - 4 - 30%; a Building Code Exam worth 30%, and an interview that would account for 40%. He testified that he had set criteria for many technical competitions during his career at the Ministry, and he usually set each method at about 1/3 with the interview rounded up, as the most important method. He enclosed the assignment, which required a written report concerning a building code issue in advance of the interview, and advised that a presentation of the report would precede the interview. The 30% weighting of the exam was also confirmed in a September 15 e-mail from Mr. Antoniuk – scheduling the two-hour exam, and indicating that James Douglas would be on the interview panel. As it turned out, Mr. Douglas was not able to attend and was later replaced. [7] By e-mail dated September 26, Mr. Antoniuk advised the candidates of a change in the weighting, in accordance with the percentages assigned to the selection criteria. That e-mail advised that the selection criteria had established 15% as the weighting for knowledge of the Building Code, which the exam was intended to test, so the weight for the exam could not be more than 15%. This then required a revision of the weighting for all the evaluation methods. The weighting of the exam was reduced from the 30% announced on September 2, to 15%, the project assignment and presentation were raised from 30% to 40% and the interview from 40% to 45%. Mr. Antoniuk’s email added that the exams had not been marked, so that he had no idea how the change in weighting would affect the overall evaluation score. This was the first time the selection criteria had been disclosed as such, less than 24 hrs before the first interview, and a couple of days before Mr. Mitta’s interview. [8] Mr. Antoniuk explained in his evidence that by September 26 when he had the interview questions ready, he checked to make sure it all fit in the matrix of the selection criteria, at which point he discovered his mistake and had to readjust the percentages, as he advised in the e-mail. - 5 - [9] The interview panel named in the September 26 e-mail was also different from what had been announced earlier. Mr. Douglas’ name no longer appeared as part of the panel with Mr. Antoniuk, being replaced by Larissa Hretchka, Manager Electrical Safety Unit from the Ministry of Consumer Services, and Krystyna Paterson, Manager, Fire Safety Standards from the Office of the Fire Marshal. Mr. Antoniuk said that when Mr. Douglas became unavailable, he looked for managers used to dealing with people with technical expertise. The complaint does not challenge the composition of the panel itself. [10] The September 26 e-mail also added the following caution to the four candidates, three of whom worked within Mr. Antoniuk’s own unit, and the fourth being from a unit which worked closely with his own: You will need to assume that the interview panel does not know anything about you. Therefore, it is important that you show how you are the ideal candidate for this position. [11] On the day of their interviews, each candidate gave a 15 minute presentation on the assignment, followed by the interview during which each candidate was asked pre- established questions as follows: 1. Based on your skills and experience, describe why you believe you are the best candidate for the position of Code Development Co-ordinator in the Building and Development Branch of the Ministry of Municipal Affairs and Housing. (10 points) 2. How do you keep current and informed about building codes and the design and construction industry? (5 points) 3. Give an example of where you held responsibility for leading or coordinating a high profile technical (or similar) project. Did the project meet its objectives? How did you contribute to its success? (10 points) 4. Team leaders have to deal with conflict situations. Describe a recent disagreement or conflict you personally had to handle. (5 points) 5. Describe a situation when you had to persuade someone recently to accept an idea or plan. (5 points) 6. In your professional judgement what are two emerging trends in building design or construction that may affect Building Codes? How could Codes respond to these trends? (10 points) - 6 - [12] Ms. Smith received the best overall score for the three evaluation methods, with a total of 84.3, while Mr. Mitta placed fourth with 65.9. Of the separate scores for the three methods, he placed first on the exam, with 38 out of 50 questions correct, whereas the others all had 36 out of 50 correct. Ms. Smith placed first in the interview and Mr. Mitta fourth. As for the assignment and presentation, she scored highest, with Mr. Mitta second. [13] Mr. Mitta had read the job ad and the job specification, and formed the impression that the position was highly technical, as almost all the listed duties included something about technical abilities. This impression was enhanced by the knowledge that Mr. Antoniuk had supported a mandatory requirement for formal education in building science, as expressed in an e-mail to the members of his department prior to the opening of the competition. This was further emphasized for Mr. Mitta by the fact that the manager’s rationale made reference to the possibility that effects of errors in the coordinator’s work could include loss of life. Given the small size of the rest of the unit, three people reporting to a manager, he did not see that a Coordinator would have managerial duties, or much reason to being doing conflict management. Although there was mention in the job duties of recommending new employees, assigning work and recommending performance pay, Mr. Mitta thought that in light of the reality of the unit, this would involve mostly day-to-day coordination work. He saw that there would be need for leading technical studies or projects, something he had done in the past. [14] Given Mr. Mitta’s view of the heavy technical emphasis in the job, he expected the evaluation process to emphasize this aspect as well. He prepared for the interview in this light, especially after the memo announcing the change above, which included a table setting out what percentage of the overall score related to each of the selection criteria, and listing a percentage associated with each of the three evaluation methods. For instance, it indicated that technical knowledge would be worth 25% of the overall evaluation, and that whole percentage was attributed to the interview. From this, Mr. Mitta concluded there would be significant technical content in the - 7 - interview questions, including scenarios concerning structural considerations and how to apply the building code and talk to people about it. He did not find this to be the case, and questioned the efficacy of the assignment and the interview questions to assess the candidates’ capacity in this regard, which he considered of the highest importance. He found the interview questions to emphasize skills needed for the manager’s job rather than a technical coordinator position. [15] On the day of the interview, Mr. Mitta’s presentation went fine, he said, and his second place score for that portion is consistent with that. However, when he was given the interview questions, he was quite surprised and disconcerted, as he did not think they reflected the job specifications. This had the result that he felt mislead by the previous communications, and was quite nervous during the interview, despite substantial preparation and decades of experience. He had not prepared for questions such as those dealing with difficult staff or conflict resolution. Communication skills had been attributed 20% of the overall evaluation, but the September 26 memo indicated 15% was assigned to the project assignment and presentation, and only 5% to the interview. This did not give Mr. Mitta the sense that “soft skills” would be much a part of the interview. As well, the description of the selection criterion related to communication skills did not include a reference to staff conflicts. Rather, the wording, set out above, emphasized effective working relationships for consultation services under the rubric of communication skills. [16] Mr. Mitta felt the exam was not an appropriate test for those who were involved in developing or drafting Code provisions, as it tested knowledge of the code, but not how to apply it. To do that, Mr. Mitta testified, the level of understanding has to be deeper, including knowledge related to subjects such as why sprinklers fail, or what causes fatalities in buildings. This level of knowledge would include understanding how concrete deteriorates over time or the variety of methods to deal with fire suppression and their advantages and disadvantages. In Mr. Mitta’s estimation neither the exam nor the interview came anywhere near testing that kind of knowledge. Mr. Mitta explained that the position in question requires someone able to appear as the Ministry’s technical representative at a hearing such as one - 8 - concerning the recent collapse of a parking garage in Elliott Lake, so that the evaluation should test for that before the person would be put in that position. [17] Nonetheless, for testing knowledge of the existing Ontario Building Code, it appeared that the exam met Mr. Mitta’s expectations. It asked for specific technical answers, such as what types and thickness of roof sheathings were required by the Code for a certain kind of building, or dealing with what loads, forces and effects needed to be considered in the design of structural members, to give just two examples drawn from the 50 questions. [18] For his part, as hiring manager, Mr. Antoniuk was of the view that technical knowledge was very important, and that the process did assess it. However, he was not looking primarily for a technical specialist in the coordinator position. He was looking for someone to help him as manager with a large project leading to the 2012 edition of the Building Code which required many amendments, and engaged many associated engineering and architectural issues and policy considerations. He saw the coordinator assisting in planning how the project was to be carried out within the parameters given by senior management, up to the deputy minister level, dealing with timing, resources issues and coordination with others such as the legal advisors, to make sure the regulation they drafted was aligned with policy as well as technically and legally sound. In these circumstances, he was looking for someone to draw on the knowledge of all of the technical experts and get them to work together to produce what the government needed to further develop the Building Code. This included reporting technical resource and other issues to his attention for action. [19] Mr. Antoniuk had held the coordinator job between 2003 and 2008 when he became manager, but it had remained vacant since. In 2008, he was managing 10-12 employees, but around 2010 the unit was reduced in size. At the time of the posting of the coordinator’s position, he was directly managing four advisors. - 9 - [20] When Mr. Mitta heard on October 28 that he had not been the successful candidate, he went to speak to Mr. Antoniuk. They met more formally about it on November 16, at which point Mr. Mitta elaborated on his view that the selection process was not fair, or sufficiently related to the job, and asked that it be re-run. Mr. Mitta testified that Mr. Antoniuk admitted he made a mistake on both occasions, but emphasized how much better the successful candidate had done in the interview. Further, he said that Mr. Antoniuk said he would consider re-running it, but indicated that it was only a six month position, a shorter period than it would take to re-run. Mr. Antoniuk’s recollection was that the “mistake” he acknowledges making was in the communication of the weighting of the criteria, but not in the overall evaluation process, since the selection criteria never changed and everyone had the same information in that respect. [21] Mr. Mitta also testified that his work had not changed since the coordinator has been in place. He remains responsible for large buildings, and considers that he is performing part of the job of the coordinator. For example, the job specification indicates that it is part of the coordinator’s job to oversee and participate in study groups to identify required additions to the Code. Nonetheless, for the most recent high profile committee of this sort, regarding glass falling off building balconies, it was Mr. Mitta rather than the Coordinator who went to the meetings as the technical representative of the Ministry, to make a presentation and have the necessary conversations with architects, engineers, installers and glass manufacturers. He was required to explain what the Code said about load and ability to sustain impact to people who know the intricacies. [22] Mr. Antoniuk did not view committee work as solely the coordinator’s work, testifying that both code advisors like Mr. Mitta and the coordinator would have responsibilities in respect of different committees. Mr. Antoniuk indicated that Mr. Mitta’s work with the expert committee on falling glass was to contribute his knowledge of the Code as a resource to this committee, which was not in a coordination role. Mr. Antoniuk indicated that he considers Mr. Mitta a great advisor, and acknowledged that he had been involved with many parts of the - 10 - Building Code, having been the lead on the development of substantial portions of it in recent years. [23] Further details of the facts related to the complainant’s challenges to the competition will be set out as necessary in consideration of the merits below. Excerpts - Policy [24] Relevant policy relied on include: Excerpts from the Public Service Commission Directives Staffing Policy 9. Recruitment and Selection This section covers activities designed to attract and hire people with appropriate qualifications to meet organizational goals. It is assumed that competitions will be used to staff vacancies. Exceptions to the competitive process should be supported by a documented business rationale. Mandatory Requirements 9.1. Recruitment activities must be based on a documented description of the duties, responsibilities and qualifications required of the work to be performed. 9.2. Employee selection must be based on a thorough assessment of a candidate’s ability to demonstrate the qualifications required for effective performance. For permanent work, candidates must be assessed using at least one screening method (e.g., review of resumes) and at least three rating methods (e.g., work samples, interviews, etc.). 9.3. Personnel screening checks must be completed in accordance with the Personnel Screening Checks Policy. An offer of employment can only be extended when the top candidate has been granted clearance through the screening check process. Selection Criteria 9.4. Selection criteria form the basis on which all candidates are screened and rated and must: • be developed from, and weighted in accordance with, the relative importance of the duties of the work to be performed • reflect the required qualifications for each vacancy to be staffed • be established at the outset and not changed during the selection process • reflect legal or mandatory requirements for credentials, where required. … 9.13. All screening and rating methods used must be valid, bias-free and reflect pre-determined selection criteria. - 11 - Reference Checking 9.15. Reference checks must be completed when their inclusion in the selection process could influence the outcome of the hiring decision. Submissions of the Parties [25] Counsel for the complainant invites a conclusion that the competition process was flawed in many respects, sufficiently so that the competition should be declared unfair and the results set aside. The case is not based on an allegation of actual bias, but contained what counsel referred to as some objective bias, as a result of the flawed process. As remedy, the complainant seeks an order that the competition be re-run subject to a number of terms, as well as damages. [26] It is the complainant’s position that the competition process failed to adequately assess the qualifications and experience of the candidates. In particular, in the complainant’s view, the interview did not properly test the candidates’ experience and qualifications relative to job requirements, a problem exacerbated by an excessive emphasis on the interview. Counsel argues that there is no question that if the interview process had been better that the outcome could have been different, and that Mr. Mitta could have won the job. Counsel submits that the complainant had exceptional experience and credentials, and that an interview did not do justice to credentials in an area like this. Further, no scores were given for resumes, which contained all the credentials; they appear to only have been used to screen for whether the basic educational credentials were present. Nor were references called to determine qualifications. [27] As well, it was said at the outset of the hearing that the employer created selection criteria that favoured one candidate over another, which was particularly problematic when all candidates were well known to the manager before the competition. It is the complainant’s position that the final selection criteria did not reflect the relative importance of the duties of the work to be performed and the required qualifications for the position. Moreover, it is the complainant’s view that the rating methods did - 12 - not reflect the pre-determined selection criteria as they were changed and the method used did not properly test for the criteria established for the job. Nor did they do a good or fair job of evaluating the candidates for the job, in the complainant’s estimation. Further, the announced weighting of the evaluation methods changed, creating confusion. [28] Counsel submitted that sections 9.13 and 9.4 of the Staffing Directive require the duties of a job to be described at the outset, so that the selection criteria and the evaluation methods will flow from them and be capable of testing for the skills actually required. He observed that, for example, if there was a physical skill involved, an interview would not likely be the appropriate method to test for such a skill. [29] It is the complainant’s position that it was clear in the evidence that the coordinator’s position required a high degree of technical competence, to be an expert advisor, with coordination aspects being secondary. This assertion is supported from various sources, starting with Mr. Antoniuk’s emphasis on the need for a high level of education, because of the life or death nature of the job in terms of protecting the public from harm from building failure, for example. Counsel submitted that the manager’s defense of a formal degree in building science was based on the fact that technical knowledge was required in the position to communicate with other people with a high degree of technical knowledge. In counsel’s submission, Mr. Antoniuk’s memo was written to let everyone know it is a highly technical position. Consistent with this, there is no mention of soft skills. [30] Further, Counsel invited the inference that the general structural situation in the Code Development Unit indicates that technical aspects of the job would far outweigh others. There is already a manager managing four advisors. With the coordinator’s position vacant, the manager had been doing some of the coordination. Since the successful candidate had been a code advisor, and was not replaced in that role, counsel submitted that she would still need to retain the advisor function. - 13 - Together with the manager’s emphasis on the need for formal technical education, this indicates there is still going to be a lot of specialist advisory work, which would itself indicate that the coordination work would be more secondary, in counsel’s view. Further, in counsel’s submission, the documents indicate that the position will be largely responsible for coordinating the development of the Code in respect of large buildings, indicating the work is that of a specialist on large buildings, which had been a focus of the unit, and Mr. Mitta in particular, for some time. Counsel emphasizes that the position description lists the very purpose of the position as being to plan with respect to large buildings and requires skills such as advanced working knowledge of all fields of building engineering or architecture of large buildings. This means the person being sought will be coordinating at a high level of technicality, doing a large building specialist advisor job with some of the coordinating duties downloaded. [31] Counsel acknowledges that there is wording in the documents such as the job ad relating to working with colleagues, but submits that they are not repeated in the weightings. From the applicants’ point of view there is no way for the candidates to know what is being sought except from what is communicated, which emphasizes technical knowledge at 25%. Counsel asserts that even within the portions that mention the “softer” skills, the vast majority of the words point to technical criteria and skills. For instance, a large part of the 20% for communication is technical communication about such things as field studies, and developing standards in working groups of experts and stakeholders rather than coordination. [32] Similarly, counsel submits the description of the selection criterion dealing with analytical and research skills relates mostly to analyzing building performance levels, rather than issues concerning people. In counsel’s submission, coordination is mainly contained in the 20% attributed to the selection criterion relating to leadership and project management. Counsel sees a small portion of some other criteria as related to the coordination duties, leaving 60 to 70% of the job for technical knowledge and the ability to develop and communicate that, dealing with other people in the process. - 14 - [33] As to the interview, and what was done and not done to assess the skills necessary to the job, counsel relies on case law such as Alderson, cited below, where a competition was overturned because the interview and testing did not permit any assessment of work experience and performance, holding the employer to a standard where skills are not pre-judged, and all the relevant qualifications are taken into account. Counsel notes remarks in the case law indicating that interview performance is not a good way to evaluate experience and qualifications, and that there are more extensive and reliable methods of obtaining information such as references and a review of past performance. In the instant case, there is no evidence that personnel files were reviewed or references checked except for the successful applicant, and the interview is fundamentally flawed, as in the Alderson case. Counsel submits that, since the flawed interview process did almost all the work in evaluating technical skills, the candidates did not receive credit for anything else in this respect. Counsel submits that there was only a very small difference between Mr. Mitta and the successful candidate going into the interview, and that with the flawed interview, the result should not be allowed to stand, as in the Vipari decision, cited below, as the flaws could have affected the result. [34] Counsel invites the conclusion that it was a very flawed interview, which did not measure the technical skills required for the job. Acknowledging that the successful candidate apparently presented very well in the interview, counsel submitted that was where she won the job, but it was not directed at the depth of knowledge required. Further, Mr. Mitta did not expect the type of questions he was met with because of all the other indications about the technical nature of the job in earlier communications. [35] In the alternative, counsel argued that even if the interview could have worked to assess the technical qualifications, given that all the members of the interview panel had technical knowledge themselves, the process could have been improved by the kind of screening of resumes for the level of qualification done in the Kuntz decision from the Grievance Settlement Board (GSB), cited below, dealing with a competition for a code advisor position. The questions in that case, more numerous, - 15 - and more specifically related to the application of the Building Code, were found to be acceptably related to the requirements of the job. Counsel queries why there were no such technical questions for the job of coordinating the work of advisors, people who have to answer the type of questions asked in the interviews for the code advisors in that case. [36] Although acknowledging that this is not a relative equality case, as in some of the precedents referred to in argument, and that the complainant is not seeking to have the successful candidate displaced, counsel drew attention to the facts in evidence about the successful candidate. According to her resume, she had been with the Ministry since 2008, eight years less that Mr. Mitta. There is nothing in the resume indicating a specialty regarding large buildings or compliance issues, before or during the time with the Ministry. Counsel notes that she completed an exam in this area, but contrasts it to the fact that Mr. Mitta teaches the course that leads to the exam. Further, her job as an advisor was to advise on accessibility issues, which engages a very small section of the Code, in counsel’s submission, while Mr. Mitta was responsible for advising on the whole of it and is a specialist on large buildings. [37] As to remedy, rather than seeking an order that the complainant be placed in the position, a re-run of the competition with a new selection panel and damages is requested. Conditions on the re-running of the competition are sought, such as ignoring the incumbent’s experience since 2011, when she obtained the position. Further conditions requested include that the selection panel would review all the personnel files and resumes and score the latter, and that the Board remain seized in case there are problems with the new competition. [38] The complainant also claims damages to compensate for the loss of the opportunity to fairly compete until a re-run, to be calculated as a percentage of the salary difference between the two positions. Counsel referred to various viable methods to calculate damages in circumstances of multiple competitors, recommending the approach of Arbitrator Sims in the Alberta Health Services decision cited below. - 16 - Other damages should be based on general principles of making the complainant whole as in the Roleau decision from the GSB, cited below, in counsel’s view. [39] Counsel for the complainant referred to the following case law from this Board: Vipari v. Ontario (Ministry of Community and Social Services) P/0033/94 (Lynk); D. McAdam v. Ontario (Ministry of Finance) P/0015/95,1996 CanLII 5655 (ON PSGB) (Leighton); D. Majkot/Ferguston/lmonti v. Ontario (Ministry of the Attorney General) P/0025/97,1999 CanLII 13879 (ON PSGB) (Leighton); C. Lacombe v. Ontario (Ministry of Colleges and Universities) P/0008/89, 1990 CanLIl 2587 (ON PSGB); (Willes), and from the GSB, Ontario Public Service Employees Union v. Ontario (Ministry of Transportation)- Grievance of Kuntz), GSB 0058/97, 2002 CanLlI 45766 (ON GSB); (Kirkwood); Association of Management, Administrative and Professional Crown Employees of Ontario (Alderson), GSB #2006-1007, 2008 CanLIl 70505 (ON GSB); and 2011 CanLII 78576(ON GSB); Ontario Public Service Employees Union v. Ontario (Ministry of Northern Development, Mines and Forestry)- Grievance of Boucher), GSB 2010-0571 , 2011 CanLlI 78576 (ON GSB) (Dissanayake); Ontario Public Service Employees Union v. Ontario (Ministry of the Environment)- Grievance of Roleau), GSB 0582/98, 2001 CanLlI 25759 (ON GSB); (Harris); Ontario Public Service Employees Union v. Ontario (Ministry of Community Safety and Correctional Services)- Grievance of Tardiel), GSB #2005- 1443, 2010 CanLlI 81917 (ON GSB) (Albertyn) as well as from the more general arbitral jurisprudence: re Civil Service Commission and Nova Scotia Government Employees Union, (1986) 25 L.A.C. (3d) 404 (Outhouse); re Co-Operative Regionale de Nipissing Sudbury Ltd. and Northern Ontario Joint Council (R.W.D.S.U.) (Wolfe), (1998) 77 L.A.C. (4th) 328 (Solomatenko); Re Municipality of Metropolitan Toronto and C.U.P.E., (1986) 26 L.A.C. (3d) 333 (Burkett); The Kingston Police Services Board v. Kingston City Police Association (Saunders), unreported decision of Arbitrator Starkman dated January 12, 2004; Windsor (City) and C.U.P.E., Loc. 543, (1992) 27 L.A.C. (4th) 349, 1992 CLB 12920, 27 C.L.A.S. 161 (Samuels); Grande Yellowhead Regional Division 35 and C.U.P.E., Local 1357 (Proulx), (2010), 197 L.A.C. (4th) 357, 2010 CLB 22424, 103 C.L.A.S. 106 - 17 - (Tettensor); Arbitration of Alberta Health Services v. Alberta Union of Provincial Employees (McGinnis, Hejnar, Wujcik), 2011 CanLll 94996 (AB GAA) (Sims); Alberta Health Services (Calgary Health Region) and the United Nurses of Alberta (Local 1) (Davies), 2011 CanLII 94977 (AB GAA) (Francis). [40] Turning to the employer’s argument, Counsel for the employer submitted that although the competition may not have been perfect in every respect, there were insufficient grounds to allow the grievance. She emphasized that there was no issue about the grievor’s actual technical skills or his work in his current job as a code advisor. The employer did not dispute that the staffing directive formed part of the complainant’s terms and conditions of employment, but maintains that they were not breached. [41] The employer’s submissions were organized around the following four issues: 1. Were the selection criteria developed from and reflective of the relative importance of the requirements of the job, as provided by section 9.4 of the staffing directives? 2. Were the assignment and interview questions related to the selection criteria as required by section 9.13 of the staffing directives? 3. Were the selection criteria changed throughout the process? 4. Even if the Board finds a violation of the staffing policy, is the violation of a magnitude sufficient that it is likely there could have been a different result without it, such that the competition should be re-run? [42] Employer counsel invites concentration on the documents and evidence of Mr. Antoniuk, who had held the coordinator job for five years in the past, to determine what the posted job required. This is to be preferred over the complainant’s interpretation of what the job should be, in the employer’s submission, as he never acted in the job, which had been vacant for some time. It is the employer’s view that the complainant mistakenly thought that the posted job was very similar to his own position, primarily a technical job. The employer’s position is that the evidence does not support a finding that the coordinator position is primarily technical. Rather, employer counsel highlights the words in the job specification and posting relating to - 18 - planning, organizing and supervising to indicate that the primary function of the job, as the title would suggest, was coordination. The employer was not looking for another specialist, but for a generalist with the “soft” skills required to be coordinator. Most centrally, in this respect, employer counsel submits that the case law establishes that the task of the arbitrator is not to second guess what the employer has determined the job should be. The issue is not whether it should be a coordinator position, but whether the applicants for the coordinator position were fairly evaluated, including whether what the employer decided to recruit for is reflected in the criteria used to select the successful candidate. [43] Acknowledging that large buildings are referred to throughout the materials describing the job, employer counsel submits that the technical knowledge and skills in this respect must be used and exercised in the context of a coordinator’s position and linked to the soft skills. It is not, as the complainant argued, a large building specialist which was required; it was a coordinator of code development, which very clearly requires both technical and soft skills. Therefore the selection criteria should accurately reflect that balance, in the employer’s view. [44] Counsel for the employer submits that the job ad relates back to the job specifications, not word for word, but clearly picking up the required team leadership, coordination, project management, analytical and judgment skills required to supervise subordinates. This theme is borne out in the evidence of Mr. Antoniuk as well, in the employer’s view, noting that the description of this job is about prioritization, work plans, communication among staff, and ensuring that various individuals produce the technical output within the timelines. In the employer’s submission, that is what separates the advisor from the coordinator position. [45] Counsel argues that the complainant had a misconception that because there were a relatively small number of employees in the department, the position would not primarily relate to coordinating. By contrast, Mr. Antoniuk’s evidence indicated that the work was suffering because no one was doing the coordinating work. Further, in - 19 - the employer’s view, the fact that there had been a vacancy for a number of years cannot indicate what work was required when it was posted, something not in the complainant’s knowledge. To the extent that the complainant thought some of the work he was still doing was properly work of the coordinator, employer counsel maintains that there is often overlap in roles, and that this is not the issue before this Board. [46] It is the employer’s position that the selection criteria were developed from and weighted in accordance with the job requirements, and that the screening and rating criteria reflected the pre-determined selection criteria, so that there was no flaw in either respect. Counsel invites a finding that overall, 40% of the evaluation was aimed at technical knowledge, and 60% at softer skills. In the employer’s submission, this reasonably aligns with the breakdown in the selection criteria and the requirements of the position. Further, submits the employer, given that the soft skills constituted a significant part of the job, it was reasonable for them to be a significant part of the evaluation process. In this regard, it is the employer’s position that it would be inappropriate to conduct a competition where such skills were ignored because it may be difficult to evaluate them. Rather, the employer submits that section 9.4 of the staffing policy requires the employer to evaluate them. The fact that it may be easier to evaluate technical skills cannot be a justification to make them a larger part of the evaluation criteria, in the employer’s view. [47] Moreover, counsel argues that the fact that Mr. Antoniuk felt a degree in building science was a necessary requirement, does not mean that was the main feature of the job. In the employer’s view, it simply means that it was one of the requirements. [48] The evaluation methods reflected the predetermined criteria, which can be seen from the questions asked and the type of answers that were being sought, in the employer’s submission. Counsel observes that the interview questions that Mr. Antoniuk said were intended to test technical and design experience and knowledge (1, 2 and 6) have scores which add up to 25 out of the total 45 points for the interview questions. Even if they were not as specific as Mr. Mitta thought was - 20 - appropriate, it is the employer’s position that they do not have to be very specific, or perfect. The standard is not whether there are better or different questions that could have been asked, in the employer’s view. Rather, the issue is whether they did or did not evaluate technical knowledge. The remainder of the questions (3, 4 and 5) are related to the “soft skills” related to the coordination duties, such as analytical/research/judgment and interpersonal skills, all relevant to the job in question, in counsel’s submission. [49] As to the complainant’s criticism of the assignment, as almost a formatting exercise, it is the employer’s view that this is misplaced, as the assignment required analysis of options, pros and cons of those options, and recommendations, all of which goes far beyond a formatting exercise. Counsel submits that these elements clearly engage requirements of the posted job such as analysis, judgment and research, as well as skills related to project management, because it requires a work plan to implement the analysis. Moreover, communication and interpersonal skills were evaluated from the presentation. Counsel submits that whether the assignment was hard or easy is irrelevant; the point is whether it requires the candidates to demonstrate the required skills. Further, given the variety of possible responses, there is no right or wrong answer, so that marking cannot be precise, including a permissible subjective element. [50] As to the disparity in marks between those of the successful candidate and the complainant’s, counsel observes that the argument was not that Mr. Mitta should have gotten a higher score on the questions in which he scored low; he conceded that he performed poorly, so that it is not at issue whether what was said in the interview was marked correctly. In any event, there is no evidence to justify a finding that his answers should be considered equal to those of the successful candidate. In the result, counsel invites a finding that section 9.13 was fulfilled. [51] In argument, counsel for the complainant conceded that there had been no change in the selection criteria themselves. In these circumstances, employer counsel argues - 21 - that the fact that there had been an administrative miscommunication about the weighting criteria cannot affect the validity of the process. [52] As to the provisions of section 9.2, counsel maintains that there is no evidence that this was permanent work; it was a temporary position which has been extended. Even if this work is considered permanent, the process included screening of the minimum qualifications, as well as three rating methods, the exam, the assignment and the interview. Counsel stresses that the policy does not require specific rating methods, such as reference checks, personnel file reviews or scoring of resumes. Given that the managers are given significant flexibility in this respect, it is the employer’s position that the fact that these particular methods were not used does not constitute a violation of the staffing policy. [53] Further, in the employer’s view, it was not, as suggested by complainant’s counsel, a breach of the policy when Mr. Antoniuk did not reply to Human Resources in terms of developing the interview questions or weighting methods in conjunction with them, or update them during the process, as neither is a mandatory requirement. [54] As to the complainant’s criticism of the emphasis on the interview as an evaluation method, counsel submits that an interview is a permissible evaluation method, and given the fact that the position was a coordinator’s position, an excellent way to test general knowledge and soft skills, which were included in the questions asked. [55] The employer’s primary position is that if there is no violation of the staffing policy, there is no violation of the complainant’s terms and conditions of employment, and therefore no basis on which to allow the grievance. In the alternative, if the Board finds flaws in the process, counsel argues that they are insufficient to justify a re-run. Firstly, all the candidates had the same information during the competition, and there is no evidence that anyone was disadvantaged in this respect. The fact that the employees in the department were informed about Mr. Antoniuk’s position on the educational requirement cannot make the process flawed, as the competition was not open yet. Moreover, the successful applicant had the same information as Mr. Mitta, - 22 - and the person who did not have the memo about the education requirement, because he worked in a different unit, performed better than Mr. Mitta overall, such that counsel submits that no conclusion can be drawn that this impacted the outcome of the competition. [56] As to the changes in the weighting, there is no evidence that it could have impacted the outcome, and thus does not warrant a re-run on this basis, especially as Mr. Mitta had the last interview, and therefore the most time to absorb the information, in the employer’s view. As to the complainant’s evidence that Mr. Antoniuk acknowledged a mistake, counsel maintains that this is not an admission that a re-run is appropriate. At most, he might have said he would consider it. [57] Further, it is the employer’s position that there is no proper basis for an award of damages, as there is insufficient evidence to find the grievor should have had the position. [58] Counsel relies on the following case law in support of the employer’s arguments: Kehoe v. Ontario (Ministry of Transportation and Communication) 914/82, decision dated August 12, 1982 (Jackson); Simlote v. Ontario (Ministry of of Transportation) P/0015/91 (Willes), Itenson v. Ontario (Ministry of Community and Social Services) P/0005/94 (Willes), and George v. Ontario (Ministry of Community and Social Services) P/0017/95 (Agarwal), and from the Grievance Settlement Board, Ontario Public Service Employees Union v. Ontario (Ministry of Ministry of Transportation)- Grievance of Bent), GSB 1733/86 (Fisher) and Ontario Public Service Employees Union v. Ontario (Ministry of Ministry of Transportation)- Grievance of Jobson), GSB #2008-1107 (Petryshen). [59] In reply, complainant’s counsel emphasized that the 60/40 split in weighting for the technical and coordination functions of the job was not announced at the outset, but on September 26, just before the interviews. In any event, he did not accept that analytical and communication skills were “soft” skills that could be separated from - 23 - the “technical”. In the end, counsel submitted that it was a question of weighting. Further, counsel submitted that Section 9.13 is very clear that the employer must rate and screen according to pre-determined selection criteria. [60] Counsel for the complainant argues that it is not enough to say that all the candidates had the same flawed information, because it affected people differently. The central point is that if the process was flawed, it should be remedied. [61] As to what is sufficient to order a re-run, rather than putting the complainant in the job, which is not what is requested, counsel maintains the test is not whether the complainant would have won, but whether he could have won, if the competition had not been flawed as it was. Counsel submitted that it is not “second guessing” the employer to analyze the questions to see if section 9.13 was satisfied; it is just the necessary analysis. The point is that the complainant’s position is that the interview questions did not reflect the duties of the position. Considerations and Conclusions Did the employer breach the terms and conditions of the complainant’s employment? [62] This Board is empowered by statute to enforce the terms and conditions of the complainant’s employment, those things which make up his contract of employment with the employer, the provincial government. As a professional architect, excluded from collective bargaining, the complainant is technically covered by an individual contract of employment. Nonetheless, in practice, the great majority of the terms are created by policies and procedures generally applicable to the public service. As noted, it was not disputed that the Staffing Directive relied on by the complainant constitutes part of the terms and conditions of his employment. Rather, it is the employer’s position that it was not breached. If there is no breach of the directives or some other term and condition of the complainant’s contract, the Board has no legal basis on which to award a remedy. - 24 - [63] The complainant’s argument is based on the Board’s authority to ensure competitions are conducted in a procedurally fair manner and that management exercises its discretion in this regard in good faith without bias in accordance with the applicable legislation and directives, as discussed in the text Ontario Public Service Employment and Labour Law by Hadwen, et al., referred to in his counsel’s submissions. No actual bias is alleged by the complainant, although counsel made reference in argument to the fact that procedural fairness and the merit principle are designed to make it possible for the diverse people of Ontario to compete on an equal footing, which was not always the case historically. As noted in Vipari, cited above, merit is considered to be the central principle governing recruitment and promotion. Commitment to the merit principle continues to be reflected in the policies here in evidence. For instance, the Appendix to the Staffing Directives, as well as the “Manager’s Guide to Staffing in the Ontario Public Service”, define “merit” as the selection of a candidate whose qualifications best meet employment- related selection criteria. The employer in no way disputes the merit principle, and maintains that the process successfully selected the person most suited to the job. [64] Procedural fairness is recognized in the Board’s jurisprudence as an important protection of the merit principle, and accordingly the Board exercises its jurisdiction to review disputed selection processes for flaws serious enough to impair the integrity of the selection decision. In Vipari, cited above, the Board found that, in order to overturn a competition, there would have to be a defect in the selection process important enough to raise a real possibility that compliance with the procedure might have brought about a different result. The onus of proof rests with the complainant in cases of this kind. The Board’s considerations in the review of competition decisions seek to strike an appropriate balance between the candidates’ right to fair treatment in the selection process and the employer’s right to establish a suitable process and criteria for selection. [65] I start with the question of whether there has been compliance with the staffing directives, a principal part of the complaint here in question. - 25 - Were the selection criteria developed from and reflective of the relative importance of the requirements, as provided by section 9.4 of the staffing directives? [66] Before getting into the details of whether the selection criteria reflected the relative importance of the job requirements, it is appropriate to acknowledge that the relative importance of the various requirements is greatly influenced by whether the job is considered first and foremost a technical job, with secondary coordination functions, as the complainant argues, or whether it is first and foremost a coordination job, with technical knowledge and skills an important, but secondary, aspect. The employer, who created the job, is of the view that it is the latter. In light of the case law cited above, it is my view that the test of such an issue is properly whether the employer’s view of the position was within the reasonable range. If so, the question is whether the selection criteria were reasonably based on the position as conceived by the employer who has the right to define positions, subject to any contractual limitations. [67] Turning to the position specification, it is not difficult to find support for both views of the essence of the position. The very purpose of the position, set out as the first element of the Job Specification, lends some support to each of the competing views of the job. It reads as follows: To plan, organize and supervise the daily operation of a unit administering and interpreting the Ontario Building Code with respect to new and renovated large buildings in order to ensure public safety in Ontario. The opening words of that purpose description: “To plan, organize and supervise the daily operation of a unit” support the idea that the reason for the existence of the position was the coordination function. However, the description of the function of the work unit itself: “a unit administering and interpreting the Ontario Building Code with respect to new and renovated large buildings in order to ensure public safety in Ontario” carries the weight of an emphasis on large buildings and public safety, which is consistent with the complainant’s theory of the actual essence of the job. [68] Moving through the duties and responsibilities set out in the Job Specification, one sees a similar split, persuading me that the job is quite thoroughly a hybrid, reasonably viewed from either of the parties’ perspectives. Major duties and - 26 - responsibilities such as providing expert technical advice, recommending new standards to ensure incorporation and application of current specialized knowledge on structural adequacy and conducting specialized field studies certainly have a technical emphasis. However, they are balanced by the duties to plan, organize and co-ordinate the daily operation of the unit to achieve the section’s objectives and ensure an appropriate level of service. This involves providing both technical and administrative supervision to a multi-disciplinary team, including recommending new employees, establishing work priorities, assigning and administering workload, reviewing staff reports, coordinating and reviewing staff responses to Coroner’s reports, as well as coordinating technical input for innovative materials and systems proposed by stakeholders. I accept without hesitation that the coordination duties in this context cannot reasonably be done without an adequate level of technical knowledge, but that point was not in dispute between the parties. The issue is whether the weighting and methods used adequately credited that aspect of the job. Given the above mixture of job duties, I find the employer’s view of the job to be squarely within the reasonable range. [69] The next necessary question is whether the selection criteria reflect the relative importance of the requirements of the position. Having found the employer’s view of the position a reasonable one, I approach this from that conception of the job, i.e. primarily a coordination role, but with a very significant technical knowledge requirement. The criteria, set out in full above, were weighted as follows: Technical Knowledge - 25%; Leadership and Project Management Skills - 20%; Analytical, Research and Judgment Skills - 20%; Communication and Interpersonal Skills - 20%; Knowledge of the Ontario Building Code and related legislation - 15% [70] Employer counsel characterized this weighting as 40% technical, comprising the first and last of the above headings, and 60% softer skills relevant to coordination, comprising the rest, and argued that this was reflective of the proportions of the two skill sets required. The evidence does not persuade me that this is an inaccurate or unreasonable representation of the job specifications. Moreover, the detailed - 27 - descriptions of the selection criteria suggest that the technical element is interwoven with the criteria which the parties sometimes referred to as softer skills, and thus, I do not see them as watertight compartments that can be evaluated precisely in isolation from each other. [71] I note that the issue of criteria being biased toward a particular candidate was not pursued in final argument, so I find it unnecessary to deal with that issue here. [72] The complainant does not take issue as much with the above weighting of the criteria themselves, as with the idea that the assessment method did not accurately reflect them. In the material submitted with his grievance, for example, he explains his view that, given the questions asked at the interview, the actual selection criteria weighting was: 0% for technical knowledge and research and judgment skills related to building performance levels, which were represented to be worth 45% in the chart communicated to the candidates, while 85% was given over, in practice, to communication skills, when it had been said to be worth 20%, with 15% accorded to the exam. This leads to the next issue : Were the rating methods valid and reflective of the selection criteria as required by section 9.13 of the staffing directives? [73] Section 9.13 of the staffing directive provides that: All screening and rating methods used must be valid, bias-free and reflect pre- determined selection criteria. The thrust of the complainant’s case in respect of the rating methods is twofold. First, and perhaps foremost, it is argued that they simply did not do the job of measuring the necessary technical qualifications, which I take, in respect of section 9.13, as an argument that they were not valid. Secondly, they did not actually, in content and practice, reflect the pre-determined selection criteria, in the complainant’s view, effectively producing a changed weighting of the selection criteria, with 0% measuring technical knowledge or research and judgment related to building performance, the foundation on which the public safety component of the job is seen to rest. Although counsel conceded that the actual selection criteria had - 28 - not been changed, the criticism that the evaluation methods were not adequately reflective of the pre-determined selection criteria remains. Essentially, this is an argument that, even if they were not changed on paper, they were nonetheless changed in effect, seen from the candidates’ point of view. This second point will be dealt with later in this decision, as it is somewhat separate from the issue relating to section 9.13 of the Staffing Directive. [74] As to the assignment component of the evaluation, the candidates were required to generate options for harmonizing portions of the Ontario Building Code with a model national code, including providing an analysis of the pros and cons of the options in order to make a recommendation, and then to establish a work plan for its implementation. The complainant thought the assignment did not get at technical knowledge, and in essence, Mr. Antoniuk agreed. As manager, he saw it as getting at research, analysis, communication and judgment skills, rather than the underlying technical knowledge of building science. Although the complainant did not think it required enough of any of these aspects, I tend to agree with employer counsel that this amounts to saying that he considered the question too easy. The essential requirement is that it measure areas relevant to the job, which I find that it did. In the absence of a specific contractual or technical standard against which to measure its validity, I am of the view that it would have to be quite a bit more elementary than it was, for me to find that it was not valid as a rating method. In any event, I find that the assignment may have been more difficult than it first appeared, since it did not provide a lot of structure to the applicants, but did in my view, give an opportunity to display relevant skills in the areas at which it was aimed. [75] The complainant also criticized the Building Code exam, as insufficient to get at the ability to apply the Code necessary to the job. Nonetheless, he did not object to the content of the exam for the purpose of measuring knowledge and ability to navigate the current Code. It is my view that, although not as extensive as Mr. Mitta would have found appropriate in the more rigorously technical evaluation process he would have designed, it was nonetheless valid for its stated purpose, and quite relevant to - 29 - the requirements of the job. The 15% attributed to it was not challenged in substance, but as to the change in the weighting, which will be dealt with below. [76] The majority of the criticism rests on the approach taken by the employer to the interview, the emphasis put on it, and the capacity of the questions asked to evaluate the necessary skills for the job, chiefly here, the technical knowledge component. In the complainant’s view, the emphasis on what counsel termed managerial type questions, such as how to resolve conflicts between employees, amounted to changing criteria, and using rating methods that do not reflect what the job was represented to be or the proper weighting. [77] It was not suggested that the questions asked were irrelevant to the job requirements. Rather, it was the nature of the rather open-ended questions, and whether the questions “got at” the requirements of the job, particularly in light of the importance of the technical aspect of the job. It was also not suggested that the complainant was not scored correctly, that he had provided information for which he was not given adequate credit, as in some of the case law referred to. It is overall a structural critique of the questions and their relation to the job requirements. [78] In looking at the questions, the Board agrees with complainant’s counsel that it is necessary to review the questions in order to resolve the present dispute. It has been said in earlier jurisprudence that it is not the role of this Board to judge which questions should or should not be asked at the interview (see for instance the George decision, cited above). However, even in light of that statement, the Board went on to deal with the allegations made by the grievor concerning the interview questions in that case. Where the allegation is that the procedural flaw resides partly in the failure of the interview questions to reflect the job requirements, it is necessary to review them to deal with that part of the complaint, in light of the requirements of the current staffing directive and the general obligation of procedural fairness. The level of review is discussed below. - 30 - [79] Mr. Antoniuk explained his view of the questions in his evidence. He explained that the first, second and sixth questions were intended to get at the candidates’ technical knowledge. He saw the first as a generic question where the candidates could explain how their skills and experience fit what was being looked for. In the expected answers that the selection panel worked from, under the headings of Technical Knowledge, the expected response says “comprehensive knowledge/experience in building design and construction, in order to interpret and provide expert information/advice to stakeholders”, which closely reflects the selection criteria set out above. Other headings roughly cover the other selection criteria. It is clear to me that the expected or ideal answer cut across all the requirements for the job. [80] The second question was intended to address technical knowledge as a supplement to the information gained in the screening process that each candidate held an engineering or architectural degree. Mr. Antoniuk testified that these degrees represented rigorous training, which could not be successfully completed without a significant technical knowledge base. The objective was to see how the candidates were maintaining that knowledge, assuring that knowledge acquisition had not stopped with the degree. The sixth question was also intended to test technical knowledge in terms of what a candidate had done since graduating, in respect of the ability to step back and synthesize from all the details as to what trends are emerging. This is important, said Mr. Antoniuk, because the Building Code is a forward looking document, so that those coordinating the work should not just be in a reactive mode. In his view, especially when giving advice to the government, it is necessary to understand the implications of technical matters, for example, thinking about where the technology was leading. I find that this question was relevant both to the assessment of technical skills and to the selection criterion related to analytical skills, articulated as including assessing the implications of emerging building safety issues. [81] As to why no questions were aimed at knowledge of specific technical areas, Mr. Antoniuk said that in the interview portion, the selection panel was more interested - 31 - in the broad technical knowledge, leadership, and communication skills, technical knowledge of the Ontario Building Code having already been covered off by the exam. The purpose of the position in his eyes, was to draw on the knowledge of all of the experts on staff to get them to work together to produce the advice the government needs, so that knowledge of a specific speciality was not what he was looking for. [82] The other questions were not primarily aimed at technical issues, general or specific, but centred on the skills needed to organize and coordinate technical work and personnel, which both sides accept has a large technical underpinning. Mr. Antoniuk explained that the technical qualifications provided by a degree were not enough for the job, that other skills were needed. As he said, he had met lots of engineers with good technical knowledge but atrocious communication skills. Of the remaining questions, the third question was aimed at evaluating general project management skill and leadership, as well as analytical skills, while the fourth question was mainly directed at communication and interpersonal skills, such as how the candidate had dealt with conflict. As for the fifth, it was intended to test a mix of leadership, judgment and communication skills, asking the candidate to discuss an experience of persuading someone to accept an idea or plan, how they would choose an approach, and show what was in it for the other person. [83] As to Mr. Mitta’s evidence that he did not find the panel helpful in asking follow-up questions, Mr. Antoniuk said he found Mr. Mitta’s responses precise and unambiguous, and thus did not feel there was anything on which he needed to follow up, something which was not challenged, and does not indicate anything problematic in my view. [84] I have carefully reviewed the questions and both sides’ evidence about them in light of the complainant’s well articulated rationale for expecting, as essential to an adequately rigorous selection process, a type of question quite different from those with which he was faced. The argument that the public safety component of the job - 32 - is so important that it is essential that the person chosen be technically competent was not disputed by the employer. Where the parties diverge on the issue of the sufficiency of the interview questions for the task they hoped to achieve – the hiring of the person best suited to the position – circles back to the difference in the view of the position dealt with above. At the risk of being repetitive, Mr. Antoniuk was looking for a technically competent person who could coordinate a team of specialists; he was not looking principally, for instance, for the person with the most technical knowledge, unless it was also accompanied by the project management and leadership skills particular to the coordination function, needed to organize, supervise and coordinate very talented professional staff and consultants. Mr. Mitta thought this emphasis was misplaced and missed an adequate evaluation of technical skills. [85] The question is then whether the interview, composed as it was of the above questions, was a valid assessment method. “Valid”, found in section 9.14 above as a necessary attribute of a rating method, is not a term defined in the policy documents in evidence. I accept that at a minimum, such a method must be capable of evaluating the skill in question, so that, as counsel for the complainant put it, if the skill is code development, you do not test for piano playing. [86] It is clear that Mr. Mitta felt technical knowledge was the paramount consideration and that the interview questions were not an adequate tool to serve the purpose of evaluating such knowledge. I have no difficulty understanding this position, nor appreciating the high professional standards that it represents in Mr. Mitta. Nonetheless, in the end, I do not find the employer’s view of the evaluative process unreasonable, and I am not persuaded that the interview questions are of a nature that supports a finding that they were not a valid method, or a breach of the staffing directives. They were not shown, overall, to lack the capacity to test for the mix of skills for which the employer was looking. And I have not been persuaded that the skills the employer was looking for were not relevant or reflective of the position specification. Counsel for the complainant articulated the criticism at one point in a way that suggested the employer was testing for the wrong thing, doing a surface - 33 - “beauty contest” of how slick the candidates could be in discussing management skills. I do not accept that characterization of the interview questions, as I find that they were capable of measuring what the employer was looking for in a job of the hybrid nature discussed above. Unlike the situation in the Roleau decision, cited above, where the panel’s questions were found not to have captured the selection criteria, I find the questions adequately represented the selection criteria. Like the grievor in Kehoe, cited above, the complainant sincerely believes that too much emphasis was put on the managerial side of things, and not enough on the technical element. There is no doubt that Mr. Mitta’s speciality in large buildings represented a skill set very relevant to the job, but the fact that the employer did not focus centrally on that skill set is not enough to find that the employer’s approach was unreasonable, in light of its view of the position. [87] It is readily apparent that the interview questions did not directly ask about technical scenarios as the complainant expected from his reading of the material provided in advance of the interview. Nonetheless, the questions provided a framework in which a candidate could put forward technical knowledge. The questions may not have “helped” the candidate provide the evidence to meet the weightings that had been provided in advance, but I do think they have to be judged in tandem with the fact that before the interview, the candidates had access to the job specification, the job ad and the weightings of both the selection criteria and the rating methods. In my view this gave the candidates enough information to know the scope of the job the employer had in mind, and to allow them to do their best to relate their skills and experience to those criteria in their answers to the questions posed. [88] To the extent that what the complainant seemed to be expecting was something like an oral exam on building science, I accept that an interview would not perhaps be the best vehicle. Given the variety of topics involved in building science, which deals with the principles of physics, chemistry, and the environment, and how they affect buildings in terms of controlling factors such as heat transfer, moisture and air movement, it would be very difficult for a short interview to function well as a test - 34 - of that. In these circumstances, I do not find it unreasonable that the employer relied on questions related to maintenance of the knowledge acquired in the professional degrees. Overall, I have not been persuaded that there was insufficient technical content in the interview questions, given that three of the six were specifically aimed at technical knowledge. The employer’s approach was different than the complainant expected. However, the initial screening had put the minimum requirement at the quite high level of knowledge of building science of a university degree in the area. Given this, I do not find, on the evidence before me, that it is more likely than not that the emphasis on maintaining knowledge and ability to spot trends was insufficient for the purpose of evaluating candidates for the job, as conceived by the employer. [89] Some of the complainant’s criticism was directed at the fact that the answers to the interview questions were not directly measurable. The case law on competitions in the public service has recognized that not all evaluation is capable of being completely objective, or broken down mathematically, as there are not always right or wrong answers to interview questions. Some subjectivity in evaluation is unavoidable, and not in and of itself a flaw. See for instance, OPSEU (Kai) and Ministry of Solicitor General and Correctional Services, GSB No. 3289/92 (Abramsky). [90] In general, the jurisprudence does not hold the employer to a standard of precision in the design of interview questions or their marking. As the Grievance Settlement Board said in the Bent decision cited above, comments that are equally applicable here: It is not the duty of the Board to conduct a microscopic dissection of each and every question put forward by a selection committee with respect to the degree of difficulty. As the Board found in that case that the disputed questions were related to the job in question, asking questions in rough proportion to the requirements of the job, and with respect to topics involved in the daily operation of the job, the grievor’s challenge to them was not successful. The Board held in that decision at p. 9: - 35 - Although it is important that the criteria used in the selection process relate to the requirements of the job as set out in the job specification it is not necessary for every single criteria of the job specification to be given the exact mathematical equivalent in the selection process. Rather it is sufficient if there is a rough correlation between the requirements of the job as set out in the job specification and of the criteria set forth by the selection committee. [91] This is the context which underlies my view that it is not necessary for each and every element of the evaluation process to directly correspond to wording in the job specification as well. For instance, the complainant was concerned that the question asking the candidates to describe a recent conflict or disagreement they had handled did not reflect the published requirements, as the job specification does not state a requirement to resolve conflicts. Even so, that interview question was relevant to aspects of the job specification, such as those referring to the coordination function, articulated in wording such as “Good supervisory and communication skills are required to establish effective working relationships with subordinates, colleagues and government officials” or “Good leadership and project managements skills and knowledge of ministry/OPS management policies and management policies and directives are required to lead and supervise section staff…”, as well as accountability for the effective daily operation of the unit’s programs. The position specification does not define everything that is included in those requirements, but in my view, the ability to handle conflict situations as team leader is reasonably related to such provisions, and therefore reasonably included in the interview questions. The evidence describes the work of the unit as involving a great variety of stakeholders, consultants, staff and issues. It appears to me that, in this context, the potential for conflict, or diverging views needing resolution, which might arise within a project managed by the coordinator, would be ever present. [92] The complainant also found the questions overly general, contrary to the suggestion in the Manager’s guide that they be specific. For instance that guide provides, at pg. 73: Questions should be directed to specific topics and not be overly general. A question such as, “Could you tell us something about yourself?” is overly general and is not specifically related to the selection criteria. “Describe how your skills in “X” relate to this position,” is more appropriate. Like other aspects of this dispute, the issue is reasonability - whether the questions were unreasonably general, so as not to be valid. Although the questions were open- - 36 - ended, and a candidate could work many things into the answers, I have not been persuaded that they offend the above guideline, as they are all directed at quite specific topics. [93] Overall, having carefully considered everything before me, I do not find that the rating methods, including the interview, were invalid or failed to reflect the staffing criteria. [94] The complainant’s case also includes the assertion that the interview was over- emphasized to the exclusion of other methods. Much has been written in the case law cited, and elsewhere, about the limitations of interviews, especially if they are the only evaluation method used. This is because there are many people who do not perform as well in the artificial situation of an interview as they do once in a job. Actual merit can be missed when there is too much emphasis on which candidates are most skilled at putting themselves forward at interviews. Nonetheless, the case law has not suggested that interviews should not be used, but that they should be complemented by other rating methods. [95] Reviews of personnel files, resumes and contacting references are other methods that the jurisprudence of the Grievance Settlement Board has repeatedly held are important parts of a thorough assessment of candidates. That jurisprudence has developed in the context of provisions of a collective agreement requiring that a position be given to a candidate with the greater seniority where qualifications are relatively equal, which puts a heightened emphasis on the comparison of all the skills, ability and qualifications of the candidates. However, the staffing directives also require a through assessment of a candidate’s ability to demonstrate the qualifications required for effective performance, as set out in section 9.2 above. Employer counsel correctly points out that the staffing directive only requires three evaluation methods, even for permanent positions. This Board’s jurisprudence has not been as specific about the review of personnel files, ranking of resumes and contacting references as that of the GSB. Nonetheless, there is no doubt that the assessment would have been more thorough if personnel files had been reviewed, - 37 - references had been contacted and resumes considered by all members of the panel. Both the complainant and the successful candidate reported to Mr. Antoniuk, who chaired the selection panel, and thus may have had some knowledge of their personnel files. However, this was not dealt with in evidence, and there is no reason to conclude that either of the other panel members had access to the files, or that there was any attempt to review them within the selection process. [96] As for resumes, they were reviewed to ascertain if the candidates met the minimum mandatory education requirement. As well, there is some indication in an August 17 e-mail from an Administrative Assistant in the Toronto Recruitment centre that it was intended that Mr. Antoniuk rank the resumes. However, there is no evidence that that was done, or that any part of the scores which were part of the decision making reflected such an activity. There is a document entitled “Resume Screening Sheet” in evidence which, provides a scale of points ranging from 0-3, with which one could rank a resume on the extent to which it met the mandatory requirements. The document bears a handwritten note which reads: “because the competition yielded such a small applicant pool, the manager interviewed all that met the mandatory requirement”. This was not expanded upon in evidence, but is perhaps an indication that the resumes would have been ranked if some candidates were not to have been offered an interview, as in some of the cases referred to in argument. [97] The absence of these elements of a more thorough process is a flaw in the process. However, it does not invalidate the rating methods used, in terms of the requirements of the staffing directive that they be valid and reflect the selection criteria. Were the selection criteria changed throughout the process? [98] I do not find that the selection criteria changed, so that there is no breach of the staffing directives in this regard. [99] What changed during the selection process was the information given to the candidates concerning the weighting of the various elements of the evaluation - 38 - methods, and the composition of the selection panel. Although it was surely preferable to correct the information about the relative weightings at the first opportunity, the last minute changes were distracting and disconcerting to Mr. Mitta, and perhaps to others. This was a very unfortunate aspect of the process, which indicated indecision on the part of the employer to Mr. Mitta, and served to distract from the credibility of the process. Although a flaw, it was not a breach of the staffing directive, as the selection criteria themselves were not changed, and the correction was to align the weighting of the rating methods with the weighting of the selection criteria. Nor do I find that it was tantamount to reducing the weighting of the technical aspect of the selection criteria to zero as suggested in the complainant’s material. For the reasons discussed earlier in this decision, I find that the evaluation of technical skills adequately reflected the selection criteria for the hybrid job the employer envisioned. [100] Mr. Mitta’s confidence in the process was further impacted when faced at the interview with questions that did not align with his expectations. It was not disputed that his performance in the interview was negatively affected by this situation, or that he has felt stressed by the situation since, given how unfair he felt the competition was. [101] Another way in which Mr. Mitta felt let down by the process related to his view that the materials described a very technical job, whereas the selection process did not treat the job in this way. I have discussed above the fact that he and the employer had two reasonably held, but quite different, views of the position. However, I do not find it to have been a flaw that the employer treated the job in the way it did, as the employer’s approach reasonably reflected the pre-determined selection criteria and job specifications. In these circumstances, given that the material on which the employer based its view was available to the candidates in advance, the fact that the grievor was negatively impacted by his reaction to the content of the rating methods, including the interview questions, is not enough to find that they were flawed in themselves. Although I accept that Mr. Mitta felt mislead by the sequence of events described above, I do not find that the information published misrepresented the - 39 - requirements of the job, or failed to inform the candidates of the importance of the coordination side of the posted job. Competitions are inherently stressful, particularly among such highly qualified candidates and close colleagues as those here involved, any of whom might have been able to perform well in the coordinator’s position. Nonetheless, the clash of expectations which occurred is not of such a nature that would constitute a breach of the staffing directive or the requirements of procedural fairness. Was the process sufficiently flawed that the competition should be re-run? [102] To summarize the above, I do not find that there were breaches of the Staffing Directive, although there were flaws in the process, in the sense of things that could have been done better in the disputed selection process. The question becomes whether, but for the flaws, there was a real possibility that the outcome might have been different, that the flaws somehow prevented the panel from assessing the relative qualifications and abilities of the candidates. After much reflection, I am not of the view that they were of such a nature. [103] I have considered the jurisprudence relied on by the complainant, such as Alderson where excessive reliance on an interview resulted in the failure to give credit to the grievor for ten years performance in the very position for which she was applying. The Alderson decision dealt with a very different factual situation than the one here under consistion, as there was no evidence that Mr. Mitta did not get credit for any experience or qualifications he should have, although I accept that there was no specific scoring of these elements. The instant case is much closer to the GSB’s decision in Nacyznski, Ontario Public Service Employees Union v. Ontario (Ministry of Economic Development and Trade), 2006 CanLII 17523 (ON GSB) referred to in the Jobson decision, cited above, where the grievor questioned the relevance, weighting and level of difficulty of many of the interview questions. In Nacyznski, the GSB held that an unsuccessful applicant who complains that a selection committee failed to gather appropriate sorts of information can be expected to put - 40 - before the Board any of the missing information which supports his or her claim to the job. It was found not to be sufficient merely to state that the information was improperly not assessed. It must be submitted and shown that it would have, or could have, made a difference. The GSB in that case went on to consider whether the evidence indicated information that, if it had been credited, could have made a difference in the result. Finding that the information available in fact supported the decision made by the employer, despite flaws in the process, the grievance was denied. Unlike the evidence in Vipari, cited above, where the evidence included information about what would have been disclosed if the flaws had not existed, such as a much more favourable reference if more than one reference had been checked, that is not the case here. [104] The materials filed by all the candidates interviewed were in evidence, including their resumes, and I have reviewed them. Counsel highlighted aspects of the evidence which could have favoured Mr. Mitta, such as his longer service in the Ministry and expertise in large buildings. However, there was no evidence to show, for instance, that he had more building science knowledge or more overall or project management experience than he was given credit for, or as compared to the successful candidate, or that scoring the resumes or reviewing the personnel files or contacting references would have disclosed such information. The evidence before me does not in itself provide a sufficient basis for such an inference, as both Mr. Mitta and the successful candidate were qualified architects with significant experience, and she had specific project management experience, which was not established to be inferior in weight as related to the job in question than Mr. Mitta’s. Without such information, it is difficult to make the crucial finding that a more extensive use of different evaluation methods, or more difficult or rigorous questions in any of the three evaluation methods, could have changed the outcome of the competition. The fact that Mr. Mitta had longer service in the Ministry and the unit in question is not as relevant in the context of excluded employees with no seniority provisions such as in many of the cases cited. - 41 - [105] In sum, then, the situation is that Mr. Mitta is a highly qualified and valued member of the Code Development Unit, who held a reasonable view of what the posted position would require. It differed considerably from the vision of the employer, which I have also found to be reasonable. The selection criteria and rating methods were designed and weighted to fit the employer’s image of the position, and I do not find that they breached the Staffing Directives or were fundamentally flawed. There were flaws in the process, as discussed above, but I do not find, on the evidence before me, that it is more likely than not that, but for their presence, the result would have been different. In these circumstances, the issue of remedy does not arise. [106] In the result, for the reasons set out above, the complaint is dismissed. Dated at Toronto, Ontario this 27th day of June 2013. Kathleen G. O’Neil, Vice-Chair