Transcription
Interview Engineering:The science of predictiveand fair hiring
Hello! I’mLusen MendelDirector of Developer Relations at Karatlus@karat.comlinkedin.com/in/lus
PPb) Negation ㄱPPQd) Disjunction P v QQc1) Conditional P QPQc2) Conditional P Q,alternative notation
12345You sourcecandidatesInterviewEngineersconduct livetechnicalinterviewsYou bringcandidatesonsite andmake hiresInterviewingInfrastructurepowers yourhiring processwith data andinsightYour processgets smarter andmore predictiveover time
ClientcandidatesKarat InterviewEngineersClient on-siteFast trackSolidNot ready
Interviewersget betterwith practicePercent of Overall 26-150151-175# of interviews conducted per interviewer176-200
How many engineering hoursdo you spend per hire?
Average number ofengineering hours per hire75
How many onsites per offer?
Selective onsite to offer ratio23:1?
Simplistic onsite to offer ratio23:1?
Average onsite to offer ratio5:1?
Desired onsite to offer ratioFast trackSolidLearning1:12:14:1
Today’s agendaData Insight #1: Funnel metricsData Insight #2How to make predictive hiring decisionsHow to ask relevant questionsQuestions
Percent of Overall ErrorsInterviewersget betterwith 5126-150151-175# of interviews conducted per interviewer176-200
Clap if you don’t know howoften interviewer error occursat your company
Clap if you can’t tell the differencebetween a candidate that chokes and acandidate doesn’t meet the bar?
Redos to the rescue!
Not ready
Not readyAdvance
RedoNot readyAdvance
RedoNot readyAdvance
Initialcandidatesource24:1RedoNot readyAdvance
Initialcandidatesource24:1Redo28:1Not readyRedocandidatesAdvance
Redos relieve candidate pressureInitialcandidatesource5.4 daysRedo5 daysNot readyRedocandidatesAdvance
Today’s agendaData Insight #1: Funnel metricsData Insight #2How to make predictive hiring decisionsHow to ask relevant questionsQuestions
Today’s agendaHow to make predictive hiring decisions1. What are false positives/negatives?2. What to watch out for3. How to choose competencies
ExampleJob Title:DevOps EngineerJob Description:Proficient in Chef
ExampleHigh DevOps skillsLow DevOps skills
ExampleHighly connectedprofessionalnetworkFirst in professionalnetwork
Today’s agendaHow to make predictive hiring decisions1. What are false positives/negatives?2. What to watch out for3. How to choose competencies
What to watch out forTrue skill of thecandidateLow skillHigh skillSkill assessment:Danger is false negativeBehavioral interview:Danger is false positive
Today’s agendaHow to make predictive hiring decisions1. What are false positives/negatives?2. What to watch out for3. How to choose competencies
Above and beyondperformerSuccessfulafter onboarding(False negative)Not successfulafter onboarding(True negative)
Job description:What you’ll do on the job:- responsibilities behavior results processes and working styles Hiring requirements:- certifications experiences knowledge and skills
PromotionskillsonnrLeaLearn onthe jobCurrentskillsjobethHiring requirements:- certifications - experiences - knowledge and skills -Things that can be learned on thejob don’t impact hiring decision-Things candidates need to comein with do impact hiring decision
Today’s agendaHow to make predictive hiring decisions1. What are false positives/negatives?2. What to watch out for3. How to choose competencies
Question calibration(Scientific process, wonky graphs)
1234Identifycompetencyand signalsTest and refinequestion usingcalibratedcandidatesPublish questionfor real candidateinterviewsMonitor questionperformance:calibration, retire
Actual scoreTrue oracle score
Actual scoreTrue oracle score
Actual scoreTrue oracle score
Actual scoreHiring barTrue oracle score
Number of candidatesHiring barScore
ScoreNumber of candidates
Summary of questionbest practices
Best practices for questions- List the competencies being assessedWhat is the objective/signal of thequestion? What is noise?(i.e. context, split)- Remove noise (i.e. context, noise)- Assess one thing at a time- Create template/checklist for question guide:- Common approaches- Common pitfalls- Test cases- Hint progression & impact-Beware candidate choice in multi-partquestions
Best practices for questions- List the competencies being assessedWhat is the objective/signal of thequestion? What is noise?(i.e. context, split)- Remove noise (i.e. context, noise)- Assess one thing at a time- Create template/checklist for question guide:- Common approaches- Common pitfalls- Test cases- Hint progression & impact-Beware candidate choice in multi-partquestions
Best practices for questions- List the competencies being assessedWhat is the objective/signal of thequestion? What is noise?(i.e. context, split)- Remove noise (i.e. context, noise)- Assess one thing at a time- Create template/checklist for question guide:- Common approaches- Common pitfalls- Test cases- Hint progression & impact-Beware candidate choice in multi-partquestions
Best practices for questions- List the competencies being assessedWhat is the objective/signal of thequestion? What is noise?(i.e. context, split)- Remove noise (i.e. context, noise)- Assess one thing at a time- Create template/checklist for question guide:- Common approaches- Common pitfalls- Test cases- Hint progression & impact-Beware candidate choice in multi-partquestions
Best practices for questions- List the competencies being assessedWhat is the objective/signal of thequestion? What is noise?(i.e. context, split)- Remove noise (i.e. context, noise)- Assess one thing at a time- Create template/checklist for question guide:- Common approaches- Common pitfalls- Test cases- Hint progression & impact-Beware candidate choice in multi-partquestions
Thank you.Any questions?Free workshops:Predictive hiringQuestion creationInterview communicationStructured write-ups 2019 Karat. All rights reserved
Best practices for questions - List the competencies being assessed What is the objective/signal of the question? What is noise? (i.e. context, split)
