Transcription
TECHNICAL REPORTValidity of the Hogan Personality Inventory,Hogan Development Survey, and theMotives, Values, Preferences Inventory forDeveloping Leaders using The EssentialLeader modelDocumentation of Evidence forCompetency ValidationNovember 2019CONFIDENTIAL: This document is intended for use internal to The Aware Leader andshould not be shared with any third party. 2019 Hogan Assessment Systems, Inc. all rights reserved.CONFIDENTIAL1
RESEARCH SUMMARYThis report summarizes the competency research conducted for The Aware Leader’scompetency model: “The Essential Leader”. The Aware Leader collaborated with Hogan tovalidate the Hogan Personality Inventory (HPI), Hogan Development Survey (HDS), andMotives, Values, Preferences Inventory (MVPI), to predict an individual’s potential to performthe competency-based requirements associated with successful Leader performance basedon The Essential Leader model.The report details the methods used to (a) identify the accumulation of validity evidence and(b) provide rationale for the HPI, HDS, and MVPI scale selection for each competency.Hogan researchers conducted a review of the model to understand the key competencies andaligned those competencies to the Hogan Competency Model (HCM). Next, employing asynthetic/job component validation process, we searched the Hogan archive to locatecriterion measures of HCM competencies aligned with the client’s competencies. This process(a) identified valid predictors of those job components based on the results of archivalvalidation studies, (b) aggregated correlations across multiple studies for eachcomponent/competency, and (c) applied the results to the same components/competenciesrequired for Leaders. Hogan then used these results and content validity/expert judgment todevelop the final scale recommendations used to predict each competency.The remainder of this document describes (a) the research process, (b) the recommendedscoring guidelines, and (c) the estimated impact of using the assessments to develop Leaders.We organized this document in the following sections: Introduction – project overview Competency Identification – establishing critical competencies Synthetic/Job Component Validity – research on jobs with similar components Recommendations – scale selection, application, and implementationFor more detailed information about the processes detailed in the following pages, pleaseconsult the Hogan Competency Research Methodology (HCRM) annex. The annex containsnon-company specific details regarding all research steps available to clients. 2019 Hogan Assessment Systems, Inc. all rights reserved.2
CONTENTSRESEARCH SUMMARY21 INTRODUCTION71.1Problem and Setting . 71.2User, Location(s), and Dates of Study . 72 COMPETENCY IDENTIFICATION82.1Job Analysis . 82.2Competency Alignment . 83 SYNTHETIC/JOB COMPONENT VALIDITY163.1Validity of the HPI for Predicting Job Performance . 163.2Validity of the HDS for Predicting Job Performance . 203.3Validity of the MVPI for Predicting Job Performance . 244 RECOMMENDATIONS254.1Scale Selection . 254.2The Aware Leader Competency Model Mapped to the HPI, HDS, & MVPI . 264.3Adverse Impact. . 334.4Uses and Applications . 334.5Accuracy and Completeness. 33REFERENCES34ANNEX: Hogan Competency Research Methodology (HCRM)35Annex Summary . 35A1. WHAT TO MEASURE AND WHY36A1.1 The Hogan Competency Model . 36A1.1.1 Competency Definitions . 37A1.1.2 Competitor and Academic Competency Models . 37 2019 Hogan Assessment Systems, Inc. all rights reserved.3
A1.1.3 Competency Mapping . 37A1.1.4 Domain Model. 41A1.2 Personality Measurement and Prediction . 43A1.2.1 Approach and Rationale . 43A1.2.2 Measuring Personality . 44A1.2.3 Personality as a Predictor of Important Outcomes . 45A1.2.4 Advantages of Using Personality Assessments . 46A1.3 Assessments . 47A1.3.1 The Hogan Personality Inventory . 47A1.3.2 The Hogan Development Survey . 52A1.3.3 The Motives, Values, Preferences Inventory . 56A2. COMPETENCY IDENTIFICATION58A2.1 Job Analysis . 58A2.1.1 Job Analysis Survey . 58A2.1.1a Performance Improvement Characteristics . 58A2.1.1b Derailment Characteristics Questionnaire . 61A2.1.1c Motivational Improvement Characteristics . 63A2.1.1d Competency Evaluation Tool . 65A3. SYNTHETIC/JOB COMPONENT VALIDITY69A3.1 Synthetic Validity . 69A3.1.1 Gathering Synthetic Validity Evidence . 70A4. RECOMMENDATIONS72A4.1 Scale Selection. 72A4.2 Competency Scoring . 72A4.2.1 Profiles . 72 2019 Hogan Assessment Systems, Inc. all rights reserved.4
A4.2.2 Algorithms . 73A4.2.2a Moderate Scale Scores . 74A4.2.2b Custom Norms . 74A4.2.3 Similarity Across Methods . 74A4.3 Adverse Impact . 74A4.4 Uses and Applications . 76A4.5 Accuracy and Completeness . 77ANNEX REFERENCES78Appendix A: Academic and Company Competency Models95Appendix B: Hogan Competency Model97Appendix C: Crosswalk between the 56 Item CET, the 62 Item CET, andConfigure100 2019 Hogan Assessment Systems, Inc. all rights reserved.5
TABLES & FIGURESTable 1 The Aware Leader and Hogan Competency Alignment . 9Table 2 HPI Correlations with Critical Competencies . 17Table 3 HDS Correlations with Critical Competencies. 21Table 4 Overview of Hogan Scale Usage . 27Table 5 The Aware Leader Competency Model Mapped to the HPI, HDS, & MVPI . 28Table A1 Crosswalk Between HCM Versions . 39Table A2 Domain Model of Job Performance, Example Competencies, and PersonalityMeasures . 42Table A3 Correlations between HPI Scales and other FFM Assessments . 50Figure A1 Relationships between FFM Inventories and the HPI Scales . 51Table A4 PIC Items . 60Table A5 HPI and PIC Scale Definitions . 61Table A6 DCQ Items . 62Table A7 HDS and DCQ Scale Definitions . 63Table A8 MIC Items . 64Table A9 MVPI and MIC Scale Definitions . 65Table A10 CET Items. 67 2019 Hogan Assessment Systems, Inc. all rights reserved.6
1 INTRODUCTION1.1 Problem and SettingThe Aware Leader development process is critical for developing talented Leaders who willcontribute to the long-term success of their respective companies. The complexities ofsuccession planning and the dynamic job market warrant continuous evaluation andimprovement of Leader development processes.A review of alternative development techniques prompted The Aware Leader to conclude thatan assessment of personality characteristics could enhance the current procedures used todevelop Leaders at any company. The Aware Leader contacted Hogan and initiated researchto construct a measurement platform for The Aware Leader’s “The Essential Leader”competencies. By aligning the Aware Leader competency model to the Hogan CompetencyModel (HCM), Hogan can leverage the Hogan Personality Inventory (R. Hogan & J. Hogan,2007; hereafter “HPI”), the Hogan Development Survey (R. Hogan & J. Hogan, 2009; hereafter“HDS”), and the Motives, Values, Preferences Inventory (J. Hogan & R. Hogan, 2010; hereafter“MVPI”) to create custom scale recommendations. For more information on the developmentof the HCM and how Hogan uses personality to predict performance based on competencies,please refer to HCRM chapter A1. If the inventory scales demonstrated validity, The AwareLeader planned to use the assessments to help develop Leaders.Our research conforms to standards outlined in the Uniform Guidelines on Employee SelectionProcedures (Equal Employment Opportunity Commission, 1978; hereafter “UniformGuidelines”), The Principles for the Validation and Use of Personnel Selection Procedures(Society for Industrial and Organizational Psychology, 2003; hereafter “Principles”), and theStandards for Educational and Psychological Testing (American Educational ResearchAssociation, 2014; hereafter “Standards”). In areas where the Uniform Guidelines, Principlesand/or Standards proved vague or inapplicable, the research approach relies on the broaderscientific/professional literature for guidance.1.2 User, Location(s), and Dates of StudyThe Aware Leader is an independent consulting firm founded by Richard Metheny (The AwareLeader, 2019). Hogan conducted research described in this report between September 2019and November 2019. Although most work occurred online or over the phone, participatingindividuals were located:Hogan Assessment Systems11 S. GreenwoodTulsa, OK 74120The Aware Leader638 Turner AvenueDallas, TX 75208For additional information regarding (a) the foundation, (b) rationale behind the stepsdescribed in this report, or (c) assessments used in this study, please consult the HCRM. 2019 Hogan Assessment Systems, Inc. all rights reserved.7
2 COMPETENCY IDENTIFICATIONThe first step in competency modeling is to identify the critical components of a job using jobanalysis. Many clients conduct a job analysis internally and develop a model of competenciescritical for success within a role or their organization. For clients who have conducted theirown job analysis and created their own model, Hogan will not complete a job analysis and willmove to competency alignment. If they have not identified critical competencies, Hogan canconduct a job analysis to identify critical competencies.2.1 Job AnalysisIn the present study, Hogan did not conduct a job analysis because conversations betweenHogan and The Aware Leader led to a decision to focus on the critical competencies identifiedby The Aware Leader’s internal research. For more information on Hogan’s approach to jobanalysis, please consult HCRM section A2.1.2.2 Competency AlignmentAs part of the development of competency-based guidelines, Hogan aligned The Aware Leadercompetencies with the HCM (Hogan Assessment Systems, 2016). To align the twocompetency models, clear competency definitions were necessary. For this project, we reliedon information provided by The Aware Leader to define each competency and illustrate thetypes of work styles and behaviors associated with each component. The Hogan researchteam then used their expert knowledge of competencies and job performance to align theAware Leader competencies (and competency components) with Hogan competencies. Formore information on this process, see HCRM section A2.3. Table 1 displays the alignment ofthe Hogan and The Aware Leader competency models. 2019 Hogan Assessment Systems, Inc. all rights reserved.8
Table 1 The Aware Leader and Hogan Competency AlignmentThe Aware LeaderThe Aware Leader CompetencyCompetencyDefinitionHogan CompetencyHogan CompetencyDefinitionThinking CriticallyDecides with speed andconviction, has mental agilityDemonstrates strategicforesight and an orientation tothe preferred futureSees the enterprise as a seriesof integrated and interlockingprocessesGenerates original thoughts andis resourceful in finding ways toimprove thingsDecision MakingUses sound judgment tomake timely and effectivedecisions.DisplayingConfidenceProjects poise and selfassurance whencompleting work tasks.AnticipatingProblemsForecasts and detectserrors, gaps, andpotential flaws.Driving StrategyDirects effort to achievelong-term businessobjectives.Business InsightApplies businessknowledge to achieveorganizational goals andobjectives.Driving ChangeChampions new methods,systems, and processesto improve performance.Driving InnovationStimulates creative ideasand perspectives that addvalue.AnticipatingProblemsForecasts and detectserrors, gaps, andpotential flaws.VerbalCommunicationExpresses ideas andopinions effectively inspoken conversations.Comes to well-reasonedconclusions andsolutions, testing them againstrelevant criteria and standardsSolving ProblemsIdentifies solutions givenavailable information.Thinks and acts ethically andmorallyIntegrityActs honestly inaccordance with moral orethical principles.Sets high standards of executionincorporating mission into dailyactivitiesQuality FocusStrives to meet qualitystandards and producequality work productsRaises vital questions andproblems, formulating themclearly and preciselyUnderstanding Impact 2019 Hogan Assessment Systems, Inc. all rights reserved.9
Table 1 The Aware Leader and Hogan Competency Alignment (Continued)The Aware LeaderThe Aware Leader CompetencyHoganCompetencyDefinitionCompetencyHogan CompetencyDefinitionKnows potential impacts andconsequences of decisionmaking in situations bothinternally and externallyTaking SmartRisksEvaluates tradeoffsbetween potential costs andbenefits and actsaccordingly.Shows patience and respondscalmly to stressful or tryingsituationsHandling StressManages pressure withoutgetting upset, moody, oranxious.Understands and manages thepower and politics inherent inany organizationPolitical SavvyRecognizes, interprets, andworks within the politicalenvironment of anorganization.Is in touch with emotions andeffects on behavior to betterunderstand how to influencepeople to achieve outcomesInfluencingOthersPersuades others to helpachieve organizationalgoals and objectives.Driving ChangeChampions new methods,systems, and processes toimprove performance.Industry InsightApplies knowledge ofindustry trends andoutlooks to achieveorganizational goals andobjectives.Leveraging WorkSkillsApplies technology and jobrelevant abilities tocomplete work tasks.Presenting toOthersConveys ideas andinformation to groups.Financial InsightApplies financial knowledgeto achieve organizationalgoals and objectives.Industry InsightApplies knowledge ofindustry trends andoutlooks to achieveorganizational goals andobjectives.Leveraging KnowledgeDemonstrates continuousimprovement mindset andknows the industryLeverages functional knowledge,technical skills and expertise andshares with othersOptimizes financial acumen tomake business decisionsStays up-to-date with industryspecific content knowledge,rapidly assimilating and usingnew knowledge 2019 Hogan Assessment Systems, Inc. all rights reserved.10
Table 1 The Aware Leader and Hogan Competency Alignment (Continued)The AwareThe Aware Leader yHogan CompetencyDefinitionVerbalCommunicationExpresses ideas andopinions effectively inspoken conversations.ProcessingInformationGathers, organizes, andanalyzes diverse sources ofinformation.Makes evidence-based decisionsDecision MakingUses sound judgment tomake timely and effectivedecisions.Draws on personal andprofessional strengths as well asareas for personal developmentto build capacityProfessionalismActs in accordance with jobrelated values, principles,and standards.Self ManagementDemonstrates appropriatemotivation, attitude, andself-control.Managing ConflictResolves hostilities anddisagreements betweenothers.Self ManagementDemonstrates appropriatemotivation, attitude, andself-control.Decision MakingUses sound judgment tomake timely and effectivedecisions.Solving ProblemsIdentifies solutions givenavailable information.Self-DevelopmentActively acquires newknowledge and skills toremain current with and/orgrow beyond jobrequirements.Handling StressManages pressure withoutgetting upset, moody, oranxious.OvercomingObstaclesPursues goals andstrategies despitediscouragement oropposition.Effectively seeks and sharesinformationLeading SelfIs aware of judgments, emotionsand physical sensations resultingin greater clarity and resolve indetermining the best options –especially in situations of conflictand change.Knows one's own method ofdecision making and problemsolvingUnderstands own strengths andlimitations, values selfknowledge, and seeks feedbackManages the quality of theirenergy, regardless of theexternal pressures they’re facing 2019 Hogan Assessment Systems, Inc. all rights reserved.11
Table 1 The Aware Leader and Hogan Competency Alignment (Continued)The AwareThe Aware Leader yHogan CompetencyDefinitionIs responsible and accountable foractions and reactionsAccountabilityAccepts responsibilityfor one’s actionsregardless ofoutcomes.Clearly communicates expectationsVerbalCommunicationExpresses ideas andopinions effectively inspoken conversations.Team BuildingAssembles productivegroups based uponrequired skills, goalsand tasks.Challenges other people appropriatelyand rejects mediocrityDrivingPerformanceProvides guidanceand feedback tomaximizeperformance ofindividuals and/orgroups.Constructively supports and managesdisagreementsManaging ConflictResolves hostilitiesand disagreementsbetween others.Entrust work to others, utilizingindividual and team strengths toachieve goalsDelegatingAssigns work to othersbased on tasks, skills,and workloads.DrivingPerformanceProvides guidanceand feedback tomaximizeperformance ofindividuals and/orgroups.Leading OthersDemonstrates generalleadership ability andeffectiveness.Engages with impact motivating,persuading and exciting othersInspiring OthersMotivates others toaccomplishorganizational goals.Cuts through the noise with authentic,credible self-promotion that helpsothers as well as themselvesDisplayingConfidenceProjects poise andself-assurance whencompleting worktasks.Directing PeopleBuilds and leads teams and developsfuture leadersInstructs, guides, and oversees theperformance of staffInfluencingOthers 2019 Hogan Assessment Systems, Inc. all rights reserved.12
Table 1 The Aware Leader and Hogan Competency Alignment (Continued)The AwareThe Aware Leader yCuts through the noise with authentic,credible self-promotion that helpsothers as well as themselvesServes as a change agent, assistsothers in understanding neededchanges, reasons for change and thechange processEstablishes and models standards thatfosters exceptional quality andcontinuous improvementArticulates ideas clearly and organizesideas effectivelyCommunicates organizational mission,vision, objectives and prioritiesHogan CompetencyDefinitionDevelopingPeopleProvides support,coaching, training,and career directionto others.IntegrityActs honestly inaccordance withmoral or ethicalprinciples.Driving ChangeChampions newmethods, systems,and processes toimprove performance.Influencing OthersPersuades others tohelp achieveorganizational goalsand objectives.Quality FocusStrives to meet qualitystandards andproduce quality workproducts.DrivingPerformanceProvides guidanceand feedback tomaximizeperformance ofindividuals and/orgroups.VerbalCommunicationExpresses ideas andopinions effectively inspoken conversations.ProcessingInformationGathers, organizes,and analyzes diversesources ofinformation.VerbalCommunicationExpresses ideas andopinions effectively inspoken conversations.Presenting toOthersConveys ideas andinformation to groups.Taking InitiativeTakes action withoutneeding directionfrom others.Initiating ActionTakes action without being promptedand takes personal responsibility 2019 Hogan Assessment Systems, Inc. all rights reserved.13
Table 1 The Aware Leader and Hogan Competency Alignment (Continued)The AwareThe Aware Leader yHogan CompetencyDefinitionLeads a transformation/change agendaDriving ChangeChampions newmethods, systems,and processes toimprove performance.Executes the visionDriving StrategyDirects effort toachieve long-termbusiness objectives.Working HardConsistently strives tocomplete tasks andassignments at work.Self ManagementDemonstratesappropriatemotivation, attitude,and self-control.EngagementDemonstrates loyaltyand commitmentthrough enthusiasmand extra effort.Driving for ResultsAccomplishes goals,completes tasks, andachieves results.Decision MakingUses sound judgmentto make timely andeffective decisions.FlexibilityChanges direction asappropriate based onnew ideas,approaches, andstrategies.Works collegially, recognizesimportance of relationships, inclusivity,and diversityLeveragingDiversityRespects and valuesindividual differencesto obtain a desiredeffect or result.Maintains broad internal and externalnetworks of business relationshipsNetworkingBuilds and maintainsa system of strategicbusiness connections.Interacts capably with a wide variety ofstakeholdersLeveraging PeopleSkillsGets along well withothers, is tactful, andbehaves appropriatelyin social situations.Has high level of energy and motivationto sustain performance over timeTurns ideas into actions that result ingetting things implementedShows good judgment in strategies,tactics, and people decisions, reversingcourse quickly when neededRelatingSuccessfully 2019 Hogan Assessment Systems, Inc. all rights reserved.14
Table 1 The Aware Leader and Hogan Competency Alignment (Continued)The AwareThe Aware Leader yBuilds trusting, collaborativerelationshipsShows empathy towards others andconsiders other feelingsActively listens, offers full attentionwhen others speak, and is seen as acoach and mentor 2019 Hogan Assessment Systems, Inc. all rights reserved.Hogan llaborativerelationships tofacilitate current andfuture objectives.Caring aboutPeopleDisplays sensitivitytowards the attitudes,feelings, orcircumstances ofothers.Listening toOthersListens and restatesthe ideas andopinions of others toimprove mutualunderstanding.DevelopingPeopleProvides support,coaching, training,and career directionto others.15
3 SYNTHETIC/JOB COMPONENT VALIDITYThe following section presents synthetic/job component validity. The Hogan archive containsinformation from over 1,000 research studies conducted from 1981 to the present andprovides a means to identify the best predictor(s) of each competency in the HCM. Lemming,Nei, & Foster (2016) mapped each of the criteria from over 375 criterion-related validitystudies in the Hogan archive onto the Hogan competencies and conducted a meta-analysisfor each Hogan scale-by-competency relationship. These meta-analyses provide stableestimates of the relationships between the Hogan scales and the critical competencies forLeaders as defined by The Aware Leader. For more information on Hogan’s approach, pleaseconsult the HCRM section A3.3.1 Validity of the HPI for Predicting Job PerformanceTable 2 contains operational validities (corrected for sampling error, unreliability in thecriterion measure, and range restriction; see HCRM A3.3.1 for more information) of therelationships between the seven HPI scales and the critical competencies for Leaders asdefined by The Aware Leader. Each competency has empirical support for several scales: Thinking Critically: Adjustment, Ambition, and Learning Approach Understanding Impact: Adjustment, Ambition, Sociability, Interpersonal Sensitivity, andPrudence Leveraging Knowledge: Adjustment, Ambition, and Learning Approach Leading Self: Adjustment, Ambition, Interpersonal Sensitivity, and Prudence Directing People: Adjustment, Ambition and Interpersonal Sensitivity Influencing Others: Adjustment, Ambition and Interpersonal Sensitivity Initiating Action: Adjustment and Ambition Relating Successfully: Adjustment and Interpersonal Sensitivity 2019 Hogan Assessment Systems, Inc. all rights reserved.16
Table 2 HPI Correlations with Critical 02.06*.02.07*Thinking .09*.10*Demonstrates strategicforesight 05.04-.02.05Sees the enterprise .03.09*525,305.11*.17*.04.03.05*.08*.05Thinks and acts 637,136.16*-.01-.06.11*.16*-.04.00Sets high standards 293,048.13*.07-.09*.09*.16-.03.05Knows potential impacts 182,545.10*.36*.12*.14*.00.11*.10*Shows patience and Understands andmanages Is in touch cides with speed Generates originalthoughts Raises vital questions Comes to well-reasoned Understanding ImpactLeveraging KnowledgeDemonstrates continuousimprovement mindset Leverages functionalknowledgeOptimizes financialacumen Stays up-to-date with Effectively seeks and Makes evidence-baseddecisions Leading SelfDraws on personal Is aware of Knows one’s own Understands ownstrengths Manages the quality 178.11*.23*.01.08.09*.03.04 2019 Hogan Assessment Systems, Inc. all rights reserved.17
Table 2 HPI Correlations with Critical Competencies (Continued)KNADJAMBCompetencyIs responsible and Directing PeopleClearly communicatesexpectationsBuilds and leads Challenges other people Constructively supportsand Entrust work to Instructs, guides, and 04.13*716,999993572Influencing OthersEngages with impact Cuts throu
Validity of the Hogan Personality Inventory, Hogan Development Survey, and the for Developing Leaders using The Essential Leader model Documentation of Evidence for Competency Validation November 2019 TECHNICAL REPORT CONFIDENTIAL: This docu
