Transcription
CLEAR Annual Educational ConferenceDo Delivery Changes Change Test ResultsNew Orleans, LouisianaSept. 11-13, 2014Surprise! Do Test DeliveryChanges Change Test Results?Presenters Tadas Dabsys– Executive Vice President, PSI Services LLC Ruben Garcia– CEO, Innovative Examinations Ron Rodgers– Psychometrician, Continental Testing ServicesPSI Test Centers Serving chaumburg(2 locations)SkokieNilesAmer. Soc. Svcs-Uptown2800 W. Peterson, ChicagoGalesburgRincon Center3710 N. Kedzie, ChicagoPeoria &East PeoriaNormalChampaignPSI Test CenterSpringfieldFerguson,MOCollege of DuPage1. Bloomingdale2. Glen Ellyn3. Naperville4. WestmontPSI Test Center332 S. Michigan Ave.O’FallonChicago State University9501 S. King Drive, ChicagoMt. VernonSIUCarbondaleJolietEvansville,INPSI-operated sites Chicago SpringfieldOak ForestChicago Heights6 third-party sites(colleges) withonsite proctors21 InnovativeExams remotelyproctored sitesCouncil on Licensure, Enforcementand Regulation1
CLEAR Annual Educational ConferenceDo Delivery Changes Change Test ResultsNew Orleans, LouisianaSept. 11-13, 2014Registration and Scheduling Centralized PSI System Automated 24/7 or Live CSRs Transparent to CandidatesInnovative Exams Kiosk ProctoringInnovative Exams is a Leader inSecure Online ProctoringWe Providing Clients with TailoredRemote Proctoring SolutionsTesting Environment Overview Kiosks are installed in trusted sites– Community Colleges– Business Centers– Corporate Offices Sites do not proctor Self-service process to thecandidateCouncil on Licensure, Enforcementand Regulation2
CLEAR Annual Educational ConferenceDo Delivery Changes Change Test ResultsNew Orleans, LouisianaSept. 11-13, 2014Testing Kiosk Security Overview Physical LockdownKiosk “mode” Lockdown during examHard Drive EncryptionPrivacy FilterMultiple High-Definition WebCamerasID Scanning TechnologiesSecure.Kiosk.ProctoringProctoring and Exam Administration Proctoring Ratio 1 proctor per 10 ExamineesEnvironment Must be Clear Before Test is ReleasedExaminees Must Maintain Proper Posture and KeepBoth Hands on DesktopNo software to installDesktop Virtual LockdownExams Can be Paused or Ended In Real-timeMatch photo against ID or Single Sign-onIdentity Management - KioskCouncil on Licensure, Enforcementand Regulation3
CLEAR Annual Educational ConferenceDo Delivery Changes Change Test ResultsNew Orleans, LouisianaSept. 11-13, 2014Personal Care ProfessionsCosmetology-English PerformanceJuly 1, 2012Dec. 31, 2013NumberAverage ScorePassFailPass 81%10Personal Care ProfessionsCosmetology-Spanish PerformanceJuly 1, 2012Dec. 31, 2013NumberAverage ScorePassFailPass ed10779.6832478%Totals25479.61915975%11Personal Care ProfessionsNail Technicians PerformanceJuly 1, 2012Dec. 31, 2013NumberAverage ScorePassFailPass red47481.035511975%Totals88679.861024669%12Council on Licensure, Enforcementand Regulation4
CLEAR Annual Educational ConferenceDo Delivery Changes Change Test ResultsNew Orleans, LouisianaSept. 11-13, 2014Emergency Medical ServicesTrauma Nurse Specialists (TNS)NumberAverage ScorePassFailPass RatePaper ExamsJan-Sept 201317383.91324176%2013-14 CBTExams29183.32127973%13Emergency Medical ServicesTrauma Nurse Specialists (TNS)Oct 1, 2013 July 31, 2014NumberAverage ScorePassFailPass d17782.91255271%Totals29183.32127973%14Emergency Medical ServicesEmergency Medical Technicians (EMTs)NumberAverage ScorePassFailPass RatePaper Exams2010-201314,06775.710,9413,12678%2013-14 CBTExams1,84270.299285054%15Council on Licensure, Enforcementand Regulation5
CLEAR Annual Educational ConferenceDo Delivery Changes Change Test ResultsNew Orleans, LouisianaSept. 11-13, 2014Emergency Medical ServicesEmergency Medical Technicians (EMTs)October 2013thru July 2014 On-site Proctored Remotely ProctoredOct-Mar Apr-July Oct-Mar Apr-JulyNumber337278630597Average 221Pass Rate48%53%49%63%16Emergency Medical ServicesParamedicNumberAverage ScorePassFailPass RatePaper Exams2009-20135,30277.74,34296082%2013-14 CBTExams73667.431442243%17Emergency Medical ServicesParamedicsOctober 2013thru July 2014 On-site Proctored Remotely ProctoredOct-Mar Apr-July Oct-Mar Apr-JulyNumber13296234274Average ss Rate33%50%36%51%18Council on Licensure, Enforcementand Regulation6
CLEAR Annual Educational ConferenceDo Delivery Changes Change Test ResultsNew Orleans, LouisianaSept. 11-13, 2014Emergency Medical ServicesEmergency Medical Technicians (EMTs)Apr - July 2014875 CandidatesShare of TestAverage ScorePassFailPass RateNew QsEdited QsSame 19Emergency Medical ServicesParamedicsApr - July 2014370 CandidatesShare of TestAverage ScorePassFailPass RateNew QsEdited QsSame 0Emergency Medical ServicesEmergency Medical Technicians (EMTs)First attemptSecond attemptThree or more attemptsTotalsPaper 2013CBT 2014NP Rate1,960 81%25348%11352%2,326 76%NP-Rate1,212 76%41029%156 13%1,056 59%21Council on Licensure, Enforcementand Regulation7
CLEAR Annual Educational ConferenceDo Delivery Changes Change Test ResultsNew Orleans, LouisianaSept. 11-13, 2014Emergency Medical ServicesEmergency Medical Technicians (EMTs)Programs with 10 or more 2014 CBT attempts90-100% Pass80-89% Pass70-79% Pass60-69% Pass50-59% PassBelow 50% PassFirstSecond3 Totals8911835300163130011233465131322Emergency Medical ServicesParamedicsPaper 2013First attemptSecond attemptThree or more attemptsTotalsN6106151722CBT 2014P Rate90%41%29%84%NP-Rate286 76%210 33%112 20%608 51%23Emergency Medical ServicesParamedicsPrograms with 10 or more 2014 CBT attempts90-100% Pass80-89% Pass70-79% Pass60-69% Pass50-59% PassBelow 50% PassFirstSecond3 Totals92234300205131000013303241124Council on Licensure, Enforcementand Regulation8
CLEAR Annual Educational ConferenceDo Delivery Changes Change Test ResultsNew Orleans, LouisianaSept. 11-13, 2014Selecting Remotely Proctored SitesDocument each remote site Provide detailed directions to helpcandidates find the test site Visit and/or document with pictures thatthe kiosk location is suitable for testing Who may be in the room during testing? How will candidates sign in and bedirected to the testing location?25Working with RemotelyProctored SitesKiosk Location Matters Place kiosks to minimize disruptions Learning Centers can present manysources of unforeseen distractions “Normal activity” may presentproblems for some candidates Isolated rooms or spaces work best26Candidates May Be CreativeUse Video/Audio Evidence to Verify Candidates may complain of “distractions” thatare undetected on audio & video recordings ofexams Many candidates do not realize that theirclaims can be investigated with video/audioevidence Respond fairly when candidate complaints areverified by video evidence27Council on Licensure, Enforcementand Regulation9
CLEAR Annual Educational ConferenceDo Delivery Changes Change Test ResultsNew Orleans, LouisianaSept. 11-13, 2014How Video Evidence Can HelpKiosk placement was part of the problem28How Video Evidence Can HelpCandidate used phone while testing 15 texting events verified in video record Candidate looked over shoulder to determinewhether she was being “watched” before texting Texting occurred during sections on which sheperformed poorly on two previous attempts Her “passing score” was voided and neverreleased29How Video Evidence Can HelpCandidate’s attention was not on test30Council on Licensure, Enforcementand Regulation10
CLEAR Annual Educational ConferenceDo Delivery Changes Change Test ResultsNew Orleans, LouisianaSept. 11-13, 2014Monitoring & Managing Remote SitesOther Remote Proctoring Lessons Monitor sites constantly – they may move akiosk or change conditions unexpectedly Some locations may need to close if conditionsare no longer supportive of candidate testing Remind candidates that they may use noisecancelling headphones during testing Pay attention to candidate complaintsSpeaker Contact InformationTadas Dabsys, Executive Vice PresidentPSI Services LLC, Burbank, CAEmailtadas@psionline.comPhone800-357-1595, ext. 7203Ruben Garcia, CEOInnovative Examinations, Chicago, r. Ron Rodgers, Director of Measurement ServicesContinental Testing Services, Inc. (CTS), LaGrange, IL& President, Employment Research Institute, Wilmette, ncil on Licensure, Enforcementand Regulation11
Cosmetology-English Performance Personal Care Professions 11 July 1, 2012-Dec. 31, 2013 On-site Proctored Remotely Proctored Totals Number 147 107 254 Average Score 79.6 79.6 79.6 Pass 108 83 191 Fail 35 24 59 Pass Rate 73% 78% 75% Cosmetology-Spanish Performance Person
