Transcription
WatsonGlaser III (W-G III)Frequently Asked Questions
Frequently Asked QuestionsWhat does the Watson-Glaser measure?What is new in the W-G III?Watson-Glaser is a verbal ability test measuringcritical thinking ability—defined as the ability to lookat a situation and clearly understand it from multipleperspectives while separating facts from opinions andassumptions.Previous versions of the Watson-Glaser had fixedforms, meaning that all test-takers received the sameset of items.Over time, however, items can become over-exposed,which may be an issue when tests are used for highstakes purposes such as recruitment. With W-G III, itemsare drawn from a large bank, ensuring that the chancesof two test-takers receiving the same set of items isminimal, but the tests are nevertheless equivalent interms of difficulty.Watson-Glaser scores are used as reliable predictorsof how someone may perform in a number of jobs.Watson-Glaser measures an individual’s ability to analyze, interpret, and draw logical conclusionsfrom written information; recognize the difference between assumptionsand facts; evaluate the strength of arguments; and draw correct inferences. T he test is suitable for unsupervised (unproctored)administration because questions/items arerandomly selected from a large item bank foreach administration. I t is also suitable for online completion in asupervised environment, such as assessment ordevelopment centers. W -G III has a more sophisticated scoring system thanprevious versions. T here is now a large bank of business-relevant itemsappropriate for international use.Watson-Glaser II vs. Watson-Glaser IIIWatson-Glaser II forms D & EWatson-Glaser IIIOnline33Item-bankedr3Fixed form3r40 items (in 3 sub-test areas)33Suitable for unproctored testingr3Timedr3*3 (proctored testing only)3Suitable for development33Paper version33 (WG-II is equivalent)Suitable for recruitment*W e strongly recommend using the timed W-G III version for recruitment purposes. An untimed version of W-G III is also availablein the US only.888.298.6227 TalentLens.com2
Frequently Asked QuestionsIn what languages is W-G III available?Score Interpretation and ReportsThe W-G III test and profile reports are available infollowing languages:Profile Reports US English UK English Australian English Indian English French French Canadian Castilian Spanish US Spanish DutchAn overall percentile score is presented and alsoindividual percentile scores for the three subscales:What is happening to previousWatson-Glaser versions?Watson-Glaser Forms A, B, and C (UK), and Short (S) areno longer available in Pearson’s catalog. If you have usedthese versions and wish to continue using them pleasecontact Customer Service.W-G II Forms D and E are still available, but W-G III ismost appropriate if an unproctored/unsupervisedadministration is preferred.How are randomly generated tests equivalent interms of difficulty and item content?Each W-G III item is coded by difficulty on a finelyincremented scale. The tests may differ slightly indifficulty overall, but this is accounted for in the scoringsystem. The passages in the test are coded by contenttheme and categorized by area (Business, Science,Education, etc.). This ensures that each test is built withan equal number of items, difficulty levels, and contentthemes, and that each candidate receives an adequatenumber of business-related passages.Various other test rules control for factors such asword count, so reading requirements are similar acrosstest versions.The Number Correct, T-score, STANINE score, andSTEN score are reported in the Additional TechnicalInformation section of the candidate’s Profile Report.Use the percentile rank or one of the standardizedscores that takes into account item-difficulty differencesof administrations to compare scores. Candidate rawscores, called theta scores, are converted to percentilesand available in the platform dashboard.Why should raw scores not be used whencomparing W-G III results?For W-G III the total number of correct responses is NOTthe raw score that is used to create the percentile andstandardized scores. W-G III test takers are presentedwith varying item sets that differ slightly in difficulty, solittle can be inferred from traditional number correct.The scoring system takes into account the exactdifficulty level of the items each person has completedto calculate an estimate of their ability—a thetascore—which is converted to a percentile forinterpretation purposes.Interview ReportThis report enables interviewers to conduct a structured,critical thinking behavioral interview with samplequestions. It will only be available with W-G II forms Dand E until further notice.888.298.6227 TalentLens.com3
Frequently Asked QuestionsDevelopment ReportIn addition to these US norms, many country-specificoccupational norms are available.The Development Report aids in constructing acustom learning and development plan to enhance anindividual’s skills in the areas of critical thinking. Thisreport is available with W-G II and will be available withW-G III (timed and untimed) in 2018 in US English only.There are currently no global norms.Can I still use my own custom (bespoke)company norm?Yes, it may be possible to map custom norms to W-GIII for a fee. Please contact your local Customer Serviceteam for further information.Test Administration and CompletionHow long does it take to administer the test?W-G III is timed and respondents are allowed amaximum of 30 minutes to complete the test. Alongwith test instructions and untimed practice questions(which are not part of the 30 minutes) the total timeto administer may be up to 40 minutes. An untimedW-G III is available in the US but this is recommendedfor use in a development context or when reasonableaccommodations must be made during recruitment.W-G III, when timed, provides the type of standardizedtesting conditions that are appropriate when scores areused to make high stakes decisions.What norms will be available?Norms derived from previous versions of theWatson-Glaser will still be available. These havebeen statistically mapped to W-G III, so they are stillappropriate and equivalent. The following US normsare available. See Figure 2.Is there a paper version of the test?Yes, W-G II Forms D and E are equivalent to W-G IIIand available in paper in certain languages. Percentilesfrom these tests can be compared to W-G III percentiles(provided the same norm comparison group is used).Paper tests should be completed only undersupervised conditions.What should be done if I suspect an applicanthas cheated on the unsupervised W-G III?If you suspect that a candidate cheated on theassessment, you may retest the candidate in asupervised setting, using the online W-G III or W-G IIforms D or E paper tests. If the test was completedunsupervised for screening purposes, you shouldconsider re-testing the candidate at a later stage inthe process.Figure 2. US norms availableOccupational Norm Groups, USPosition Type/LevelNorm Groups, USEducational BackgroundNorm Groups, USAccountantExecutiveHigh school diploma (or GED)ConsultantDirector1–2 years of collegeEngineerManager3–4 years of collegeHuman Resource ProfessionalSupervisorBachelor’s degreeInformation Technology ProfessionalProfessional/Individual contributorGraduate degree(Master’s and Doctoral)Sales RepresentativeHourly/Entry levelManager in Manufacturing/Production888.298.6227 TalentLens.com4
Frequently Asked QuestionsCan I transfer assessment inventory from olderversions of Watson-Glaser to W-G III?Does 30 minutes provide sufficient time tocomplete the 40 Watson-Glaser III items?It is not possible to transfer inventory.Yes. Across both conditions, with and without thetime limit, all participants attempted all items withone exception—in the timed condition, 1 of the 137participants attempted 39 of the 40 items. This is strongevidence that the 30-minute time limit is sufficient. Inaddition, the vast majority of participants reported that30 minutes provided sufficient time to complete thetest to the best of their ability. In the untimed condition,99% of the respondents said they probably or definitelyhad enough time; in the timed condition 98% said theyprobably or definitely had enough time.Can W-G III be completed on a mobile phoneor tablet?We recommend that the W-G III be completed on adesktop or laptop computer with a dependable internetconnection.Can W-G III be linked to an Applicant TrackingSystem (ATS)?Generally the test can be linked, but contact us forfurther information.Timed vs Untimed:Equivalency Study ResultsHow does imposing a time limit affect WatsonGlaser scores and test-takers?We conducted a study to assess how providing a30-minute time limit affects Watson-Glaser test scoresand the candidates who complete the test. The WatsonGlaser III was administered twice to a group of 137online-survey participants, once with a 30-minutetime limit and once with unlimited time. The study wascounterbalanced with half taking the timed version firstand half taking the untimed version first. Results showedthat the 30-minute time limit had little effect on WatsonGlaser scores.Does the time limit affect the fairness of the test?No. The question here is whether the rank order ofcandidates’ scores changes when they complete thetest under timed versus untimed conditions. In otherwords, are some candidates disadvantaged by placinga time limit on the test? Watson-Glaser III theta scorescorrelated .73 across the timed and untimed conditions,which is not significantly different in magnitude thanthe reliability that would be expected from simplyadministering a test that is twice as long.Do candidate reactions differ across the timedversus untimed administration?Yes, somewhat. Some candidates reported feelingmore anxious or rushed when completing the WatsonGlaser under the timed condition. For example, 3%reported feeling rushed in the untimed condition and17% reported feeling rushed in the timed condition.Some TakeawaysBased on the study results, we expect scores on thetimed Watson-Glaser III to be comparable to scoreson other versions of the test (e.g., Watson-Glaser IIadministered with unlimited time). There will alwaysbe differences across test administrations, due todifferences in test content or variation in the testingconditions, but the Watson-Glaser is a reliable testand these differences will be, on average, small.(See the manual for details about correlations acrossforms and test–retest reliability.) We found no evidencethat imposing a time limit affects relationships acrosstest versions.Though some candidates responded negatively to timelimits, our study showed that this didn’t affect theirscores, and the vast majority still thought they hadsufficient time to do their best.Does the time limit change average test scores?No. Average theta scores did not differ significantlyacross the timed and untimed condition.888.298.6227 TalentLens.com5
Frequently Asked QuestionsFor more information or to order,please visit TalentLens.com888.298.6227 TalentLens.comCopyright 2018 Pearson Education, Inc. or its affiliate(s). All rights reserved.Warning: No part of this publication may be produced or transmitted in any form or by any means,electronic or mechanical, including photocopy, recording, or any information storage and retrievalsystem, without permission in writing from the copyright owner.Pearson, TalentLens, Watson-Glaser, and Watson-Glaser Critical Thinking Appraisal are trademarks, inthe US and/or other countries, of Pearson Education, Inc. or its affiliates. TALEN8266-16834 SR 11/18
Watson-Glaser is a verbal ability test measuring critical thinking ability—defined as the ability to look at a situation and clearly understand it from multiple perspectives while separating facts from opinions and assumptions. Watson-Glaser scores are used as reliable predictors of how someone may perform in a number of jobs. Watson-Glaser measures an individual’s ability to
