The URL can be used to link to this page

Home My WebLink About1989-0894.Ojo.93-01-12 · ~"': : '" ~, :, '~ ON~'AR~O EMPLOY~$OELA COURONNE '-'t ~C '"~': % '"' ': "' ' GRIEVANcECe°W~ ~,,,~o~E~s C,OMMISSION°E ~,o~~,~,o DE SETTLEMENT REGLEMENT BOARD DES GRIEFS 180 ~;~tNOA$ STFtEET WEST, SUITE 2 ~00, TORONTO, ONTARIO. MSG 1Z8 TELEPHONE~TELePHONE: ~416) 326- ~388 180, RUE DUNOAS OUEST, BUREAU 2100 TORONTO (ONTARIO], MSG ?Z8 FACSIMJLE/TEL~COPlE : . (416) 326- ~396 ' 894/89' ZN THE NATTER OF AN AItBZTRATZON Under 'TEE CROWN EMPLOYEES COLLECTIVE BARGAINING ACT Before THE GRIEVANCB SETTLEMENT BOARD ." " 'i ' · :' · . eri'evor .... The Crown in Right of' Ontario ' (Hinistry "of Transportat[oh) Employer BEFOI~: A. Barrett Vice- ers-on ' ;- _ F.. Seymour . ,, Member. ~:.. .... D. Montrose' Member Foa T~ ~ · J; Monger... . ~. .. UNION counsel - . -. Gowling,. Strathy & Henderson Barristers & SOlicitors'' FOR THE E. McKnight - .~ , . . EMPLOYER " staff Relations Advisor "' Ministry of Transportation. 'INCUMBENTS= D. Russell J. Lobo : HE]~RIN~, May 26, 1992 S~pte~ber 8, 1992 .... November 16, 1992 DECISION This grievance involves a job competition in March, 1989. There were two positions open for a job titled Junior Tester in the Systems Improvement Office. The competition was~.~:"~-~-~--,.,~m=..~, and the job is an entry level position classified as Systems Officer 1. The Junior Tester works as part of a team which tests the viability of new or changed computer software Dro~rams. Nine applicants were interviewed for the job, and the incumbent, Ms. Russe!!, was the clear front runner. The other incumbent, Ms. Lobo, obtained the second highest score in the competition, but she was'not so far ahead of the candidate who came third or Mr. Ojo who came fourth that she was considered to be substantially and demonstrably more qualified for the job than they. The employer judged these three candidates to be relatively e~ua! and thus~~"~o,~=~.~ ~o~ Ar~c!e~ 4.3 ~ · ap~!~an~ with the most the collective agreement, ~ranted the job to the ~ ~- ~ seniority: Ms. Lobo. Thus~ in this grievance the union bears the onus of provin~ that Mr. Ojo is a substantially and demonstrably superior candidate to Ms. Lobo if he is to be awarded the job. Alternatively, if the competition that was held was so flawed that we cannot determine _ ~and~a~e was, we are urged with any degree of certainty who the best - ~m ~ to order a rerun of the competition. The union initially took the position that Mr. Ojo was superior or relatively equal to Ms. Russell, who has less seniority than he, but at the end of the evidence the argument with respect to Ms. Russell was aband6ned and we. were asked:only, to weigh thelrelative merits.of~Mr..'Ojo and Ms. Lo.bo. .'~ ., . .Mr. Ojo is~ particular.l¥ aggrieved because .just- prio~r' to ~the -competition he had spent 13 months on secondment in the. very position he was competing for. The union saxs that Mr.:.i0jo's demonstrated ability to perform the work should.have been given much .greater'weight by interviewing panel than the results of test scores in a competition-that was ·fundamentally flawed anyway. His direct experience on the job combined with his many' years of experience-as a-user of one of the systems as a Vehicle and Licence Issuing Clerk shou!d..far, outweigh Ms. :Lobo's four years' experience as-.a tester -in-another. deD&rtment, on -' another system.. Union ~counsel .cites Nixo____D_n, GSB' ~2418187 {Fisher.), Poole,. #2508/87 (Samue!s)..and ~a!l/Powers, GSB-#8~6/89 .(Gorsky) for ~the principle, that.blind re!iance~on test scores, combined-,with--a refusal to consider readily available evi~ence, qf actual performance on the job, ~./ .'is.a .... da,,,ental flaw in .the ~ecision-makin~ process that warrants ~his Board overturning a selection decision. The Ministry responds that it did not ignore Mr. Ojo's experience on secondment. That is what got him an interview for! the .job, and the knowledge obtained on the §ob would help him answer the.competition questions which were directly job-related. The panel had before it Mr. Ojo's employee performance report while on secondment-which was compiled by Mr. Petersen~ the chairperson of. the interview panel. He was rated 3 "competent" and "a fair to good tester". In addition, personnel files 'were reviewed and reference checks made with direct supervisors, and that information affected the scoring. Mr. Ojo's direct supervisor during his secondment, Mr. Gibbs, was consu!tedf and he provided Mr. Ojo with a fairly lukewarm reference. His performance appraisals rated him as competent. In contrast, Ms. Lobo's past three performance appraisals rated her "excellent", and her immediate supervisor gave her a glowing reference. The Ministry says that~ as a matter of principle, in a Ministry- wide competition for an entry level position it would be unfair to ~ive a secondee any greater advantage in the competition than he already gains from having the hands-on knowledge and experience that will enable him to answer the test questions more effectively. Experience on the job in question was not a requirement set out in the job posting. If i% was, no one but secondees could ever qualify, and that is not fair either. Education and/or direct experience in the testing of automated business systems was the primary qualification for the job, as well as an ability to analyze and assess the impact of changes to a system in order to establish the test requirements. The ability to organize, meet deadlines and work independently were also considered. We cannot say, as in the Nixon and Poo!e cases {supra) ~ that the selection panel i~nored readily available information about Mr. Ojo's job performance in favour of blind adherence to test scores. There is no reason in this case why Mr. Ojo's 13 months' experience ~ the Systems Improvement Office should be given greater weight than Ms. Lobo'.s four years' testing experience in another department. In fact her more favourable LaDpraisa!s scored- her some~points,-~althou~h greater experience was not scored directty~ for her..or~any.other candidate.. The union attacks the competition ~rom several other directions as well. The -competition was.-..not run..flawles, stx~-~nd not 'all' of the' requirements and guidelines set out. i- the ~jurisprudgnce of this Board were .followed...The most seriou~.,.gbjegtion 're,ares to the. method of scoring ~the oral test. (There-.was a-written test as well ,which,was 'scored anonymous!y,b~al!.~hree-Dane%,~mbers, .and this, is not objected to.) The~ain.. objections'.to the 0ral-tes~..are that there Were no pre- ~ .determined writ. ten correqt answers..and, there was no clear marking scheme a~reed upon _in advance. Each of the -panel. members,~ was .to 'mark each' question 0ut. Rf %0. marks,~but where questions had subsections there was no _agreement~.~,.to how .the' answers to those questions, were ito...be weighted. Eor .!nstance,.-q~e question worth '!'0 marks had:.!4 subsections. to it, and only one of .the testers appeared to.,mark .each subsection seDarate!~. The other .t~o.gave a.g!oba! score for the entire question. .This issue was ~ ,, ~ . In d,sc,sse~ in Thiruma!ai~-.GSB #979/86 (S!one). _ that case,, due to a lack of prior consensus, the.different interviewers ~n .some instances ga~e widely different assessments .of t.he same answer,'. ranging from a low of four to a high of eight ou't. of !0, .and weighted their answers to sub-~uestio~s m~=~3,, That of ........... =. panel the board concluded that there was ample evidence upon which tO.conclude that the numerical scores did not represent a fair.objective basis for ~comDaring the candidates. · Mr. Petersen, the chair of this interview Dane!, testified as to how the oral test was devised and administered. All three panel members were systems experts. They decided that an anonymously-marked written test to assess the more technical aspects of the candidates'ab~t~es~ ~ would be the most fair and obfective manner of measurement. With respect to the oral test they constructed it together by referring to the job specificatien and making up questions under each category of qualifications. Each of four icate~ories was to be marked out of !0, but subsequently different weights would be assigned to those ~ark~ proportionate to the reGuirements of the position. The Dane! members discussed the ~uestions in some detail in advance and also discussed the type of answerz the Guestions were expected to elicit. Mr. Petersen said the panel did not prepare model answers to the ~uestions because they hav~ ~ had ~ems~_~ ~ the past with office security and the answer sheet has been ~to!en. Me said that all three panel members were syztems exoerts_ a.,_-~ knew full well as they were discussin~ the ~uestions wh~t the appropriate answers would be. Mr. Petersen thought it was fairer not to reGuire specific answers to specific Guestions in subsections, so as to ~l!ow the candidates to express themselves individually and d~=~*~, In other words, it did not matter whether you gave a ~..~...~..~~=~,,~ answer ~..~ response to subsection (a) or subsection (h) as long ms ~t was in there somewhere. At the interviews, each panel member took notes of the answer~ ~iven by the candidates and scored each candidate privately. Later, at meeting held for that purpose~ the panel members compared scores and answers~ and where there was a significant discrepancy amongst the panel ~r (' 6 members, the answers would be discussed in detail to see if one of the panel members wanted to change his/~e~ mind about the scope'given. At the end, the three scores were averaged for a;consensus score and that is ~he mark the candidate was'given'.. We no{e from~the score"she~ ~Hat there were no substantial ~scr~Dancies amongst ~t~e~ane! member~'in their marking. Where fractfons'of scores had be~n'grant~d, ~he Consensus score was 'rounded out, sometimes to the higher' 'who!e number and sometimes to the lower, by consensus. With respect to ~r.. ojo, all / individual scores of each Dane! member were close, ,-,q~ th~ bigges~. ~.s~repan~y in'any one categor7 being two marks between the.~igh and ~he loW. In each case, he was given the median scoff. MS.~' Lobe was rounded ': up '~ a fractional score t~ice, and Mr. Oj° ~':¥0~d~d down' twice. Ove~a~! there Were nO significant divergences of 'opinion amongst the pa~e! member~, a~d f~na~'~e~g~'n~ ~edu~ed~'the-i~act of d~scre~an~'es. At tb~'"c~nc!usion of ~'the marking of bo'~h the' Writt%n 'and b'ra'l tests, some of. the lowest scoring! candidates were e!im~nafed. F'g~ 'the remaining candidates, their o~ ~ ~ ~ ' ' p~so..ne~ files were 'revie'wed, as Mr. Petersen said, "to see if the test process was' valid". A very positive' or very..~=~=~,,=~.~ personnel file would result ~ an adjustment of the marks u~ or down by 2.5~. ~:'neutral ~ ~-betWeen pers6nne! assessment would leave the marks un~hang~d. Then'the pa~e! did.'reference' checks' At least one was done ~for e'a-lh candidate, with 6~escribed 'quest'ions to be asked of each referee. A~ain a 2.~% ..~ .......... was made for a very positive or very negative~'r~erence and .no. adjustment was made for'a neutral reference. 'In this aspect of the' assessment~, Ms. Lobe 'gain'ed 5~ on her'score and Mr. Ojo's score remained unchanged. The final scores were tallied out of !000 after the weighting mentioned earlier. Ms. Russell scored 831~ Ms. Lobo scored 680, and Mr. Ojo scored 589. The candidate who came third scored 640. Ms. Russell was way ahead of the rest and the first to be offered a job. Candidates 2, 3 and 4 were considered by management to be close enough that they were relatively equal. Seniority was considered and Ms. Lobo was offered the second vacancy. In reviewing the answer sheets and tally sheet of all of the panel members at the hearing, Mr. Petersen was sometimes unable to say why one answer had obtained more marks than another answer or why one panel member had scored a candidate higher~ or lower than another. Of course 'he was testifying three years after the ~TM~' it ~oo..pe~lon .and would be surprising if he were to remember with clarity everything that was thought and discussed at the time. Union counsel says that the employer bears an onus to be able to explain why one answer is better than =~*~ in ~,,s*~-;~ its decision and if it cannot, then the competition is ~-~=~,, flawed. We do not think that the lengthy passage of time between the competition and the hearing i~ this matter, with the attendant deleterious effect on witnesses' memorie~ must necessarily devolve to the benefit o~ the grievor. Mr. Petersen's ~.~~..~=~,~= about why each panel member came up with the score he/she did or how subsections of questions were scored individually does not lead to the conclusion that marks assigned were somehow arbitrary or subjective. Mr. Petersen had with"~im'his, notes which were made at the-time .of .the .ComPetition, but.-- observed that they were rather cryptic and did not reveal nuances which would have been present in his mind at the time of the competition, a!thOU~h no longer. H~~ was able to ·review the"questions~.with Junion counsel and-c0m~ent upon'the relative ~ual. it¥'of the 'answers?~iven'~in ~.os~ cases., 'althouGh he c~ndid!y testified: I c~n t 'really+ tell you today why one ~ers~n'Got one'more point than another" in'reSponse to particular 'q~stion. ~ ..... .ObvioUsly if the~'pane!:~had prepared model answers~to'~the, questSons· asked on~ the ora!~'ihterview," and had prescribed mark~ f6r individual~, answers to'subsections of~uest!ons., this. difficultY~ in being ~precise' about the sc~rin~ would not have arisen at the'hearinGY'~That~s'why this Board has· recommended as '~-Principle"of fairness that~there should.be pre-ordained answers to'~uestions-and pr~-ordained scores for questions and Darts of questions. However~ the resu!tin~ !ack'..~f-clarity three ' years later ~ not ~ TM ~ ' ' · ~ ...... o~ow.n~ that procedure does not.!ead':-us'-to'~ believe'that the competition wa's fundamentally flawed and that Mr. Ojo was~not given, a fair' andreasonable assessment of his qualifications at the time of the competition. We conclude this for several~reasons. ~ the · First, this, lack of pre~ordained"answers' and sub=scores is-' only majdr flaw in the ~ *~'~' ~ ~e co,,,pe~1o... Pre-screeninG was done:appropriately.- writteK test was'" an. appropriate ,test and. administered fairly. Substantial care was taken 'to frame and weight th'e questions to the qualifications for the. job.-Personnel files were searched and'reference che~ks were made for all .of the. finalists ~ an appropriate' fashion. co...p.e~..on . the scorin~ an aPPropriate decision was made'aS to which candidates were relatively equal; then seniority was appropriately cons idered. With respect to the flaw, we note that there was very little discrepancy amongst the marks awarded by each panel member for each candidate's answer. W~.ere there was a significant discrepancy in scoring · as between candidates, Mr. Petersen was able to explain why in each case. Thus the discrepancies that remain unexplained are insignificant enough that we find they would not have affected the final result of the competition. In the final scoring, Ms. Lob.o achieved 91 more marks than Mr. Ojo out of !000. Even if we gave Mr. Ojo the benefit of the doubt' on every questionable mark, he could not have achieved enough marks to lead to the conclusion that he was substantially and demonstrably more qualified for the job than Ms. Lobo. Even with !00 or more additional marks, he would still be relatively equal to Ms. Lobo and she would still get the job as the most senior candidate. Accordingly, the grievance must be dismissed. As an aside, we could not held wondering how it can be that management is afraid to draw uD pre-ordained answers to questions for fear they may be ~ ~ ~. . s~o~e.. A further foundation to this fear was provided ±n the testimony of Mr. Ojo who said that he was quite sure the questions were all written down in a book somewhere because his -^~-,,~ in the ~ce had provided him with a list of questions and rehearsed him in the answers prior to ~h~..e competition. He said that 90~ ~ the questions asked of him were ones that ~ m ....... .... a~ been provided ~ hiTM in advance. That i~ a rather startlin~ state of affairs. We expect that once the.employer seeks and finds a solution to the security Drob!em, it can adopt the more prudent policy of drawin~ up. model answers and a detailed scorin~ scheme in advance of compet'itions, Dated at.Toronto this 12t:h day of January, 1993'. A. Bart'eft, cha±r~>erson' ~.~ E. SeYmour, Member . D. Montrose, Member