Transcription
SAP White PaperPerformance ManagementTotal Workforce Performance Management 2017 SAP SE or an SAP affiliate company. All rights reserved.Using talent calibration to effectively manage the realitythat all employees are valuable but some employees aremore valuable than others1 / 43
Table of Contents4Introduction6Part I. Methods for Assessing Employee Contributions9Part II. Defining Calibration12 Part III. Calibration Design37 ConclusionLauren Pytel and Steven Hunt2 / 43 2017 SAP SE or an SAP affiliate company. All rights reserved.
Total Workforce Performance ManagementThis paper discusses research studyingmethods for total workforce performancemanagement. The term “total workforce”is used to emphasize performancemanagement processes designedto guide decisions regarding themanagement of groups of employees.This can be contrasted to methodsdesigned to manage and develop theperformance of individual employeesin isolation. While aspects of totalworkforce and individual performancemanagement overlap, certain elementsof total workforce management areinherently distinct from methods usedfor individual performance management.Foremost is the need to manageindividual differences in performancefound across different employeesworking in the same group. That isthe focus of this paper.3 / 43 2017 SAP SE or an SAP affiliate company. All rights reserved.
Total Workforce Performance ManagementHow to fairly and accurately assessemployees using calibrationAll employees do not perform at the same level1. Every employee contributes different levelsof value to the organization due to differences inproductivity, skills, potential, or any number ofother job-relevant factors2. These differencesare not minor. Contributions made by high performing employees can be several times greater than contributions made by solid or“average” employees3. And companies thatmanage, develop and invest in employees considering their relative contributions significantlyoutperform companies that treat employees asthough they all provide equal value4. Recognizing that some employees are more valuablethan others isn’t just good for companies; it isalso good for employees. The use of consistent,transparent methods to assess and rewardemployee contributions is a key factor affectingemployees’ perceptions of justice, fairness,and equity5.on how their contributions should be evaluated.Many people also find it uncomfortable to havetheir performance compared against theirpeers. Employees, particularly those at the “lower end” of the performance distribution, may experience considerable stress from anassessment process that compares their contributions with their coworkers8. But it is possibleto develop an effective process for assessingemployee contributions. Research on employeejustice has shown that most employees, eventhose who may be struggling in their roles, canaccept assessment results as being fair provided they understand how they were assessed,believe the process was accurate and consistently applied, and that the results were delivered in a sensitive manner5. The challenge is thisrequires companies to have an accurate and fairmethod to assess employee contributions—something far easier said than done.Managing differences in employee contributions is critical to maximizing company performance. But accurately assessing differences inemployee contributions is one of the mostsensitive and difficult areas of human capitalmanagement6, 7. People know not everyone contributes equally. But people don’t always agreeGroup-based assessment methods, referred toas “calibration sessions”, provide a promisingavenue for assessing and managing differencesin employee contributions. We define calibrationsessions broadly as “meetings where organizational stakeholder discuss the relative contributions of employees.” Calibration does not imply4 / 43 2017 SAP SE or an SAP affiliate company. All rights reserved.
Total Workforce Performance Managementforced ranking or any other specific assessment technique. Despite the growing popularityof calibration, little research has studied the effectiveness of different calibration methods.This paper addresses this issue with a mixtureof qualitative research conducted with SAPSuccessFactors customers that use our solutions for different types of calibration, and psychological research of performance appraisalsand group decision-making. We provide evidence for the advantages of calibration, discusscritical design questions for creating calibrationprocesses, and offer suggestions on usingcalibration to accurately and fairly assessemployees. We also provide examples showinghow technology is enabling companies to implement calibration in more effective ways.This paper is not about technology, but manymethods discussed in this paper could not beimplemented in a scalable and sustainablefashion without modern, cloud-based technology solutions.The paper is divided into three sections:1. M ethods for assessing employeecontributions. An overview of the four mostcommon methods used to assess employeecontributions, and a discussion of whycalibration sessions are often the mosteffective of the four.2. D efining calibration. A review of whatcalibration involves and what makes it aneffective method for assessing differencesin employee contributions.3. Calibration design. A discussion of criticaldesign questions that must be addressed tobuild an effective calibration process thatreflects the specific needs of your organization.The goal of this paper is to provide informationcompanies can use to implement group basedcalibration processes that fairly and effectivelyaddress the reality that all employees are valuable but some are more valuable than others.5 / 43 2017 SAP SE or an SAP affiliate company. All rights reserved.
Total Workforce Performance ManagementPart I. Methods for Assessing EmployeeContributionsAll companies use some method of assessing employee contributions . As long as company leaderswant to know “who are the high performers”, companies will evaluate the relative contributions ofemployees. However, the methods used to assessemployee contributions vary widely from one organization to the next. Some companies have noformal method to assess performance. Thesecompanies rely on unstructured, intuitive opinionsand hushed conversations about which employees are more valuable than others. We do not recommend the use of such poorly defined methodssince they are likely to be less accurate, less fair,and less useful than more standardized methods.Granted, unstructured performance assessmentmethods might be sufficient for very small companies where everyone knows everyone else (e.g.,fewer than 50 employees). But they are unlikely towork in larger companies where there are considerable legal and financial risks associated withhaving no defined method to measure employeeperformance10. Virtually all larger companies havesome established processes to assess employeecontributions . These processes typically usesome combination of the following four methodsto define and measure the value employees provide to the organization:1. Contractually defined criteria. This involvesdefining employee value based on criteria such asjob tenure, certification testing, or educational degrees. This method is widely used in unionizedworkforces, but is not considered an effectivemethod for accurately measuring true employeeperformance. For example, the ending of the “honeymoon phase” associated with longer tenuretends lead to a decrease in organizational commitment and performance levels11, 12, 13, 14.2. Objectively measured criteria. This methodworks for jobs where there are direct links between individual employee actions and clearlyand easily measurable work outcomes. Examplesinclude jobs that measure performance based onindividual sales revenue, piece rate productivity,or absence of product defects. A challenge of using objective performance criteria is that thisform of performance criteria is not available orparticularly relevant for many jobs. Most objective performance criteria are also influenced byfactors that are outside the control of individualemployees. For example, how much sales revenue an employee generates depends largely onthe sales territory they are assigned and thenumber and type of customers it contains.6 / 43 2017 SAP SE or an SAP affiliate company. All rights reserved.
Total Workforce Performance Managementcontributions. Third, the act of collecting managerratings is often viewed as a highly burdensomeand administrative task.Figure 1: Technology illustration for objectivemeasured performance criteria. This screen illustrates how objective criteria can be incorporatedinto goal management technology. In this example an employee’s performance level is automatically calculated based on attainment of specificfinancial measures.3. Individual manager evaluations. This typicallyinvolves managers rating employee performancebased on an annual, quarterly or monthlyschedule. Historically, this has been one of themost common methods used to measure employee contributions. But this method suffers from significant limitations. First, managers may beaffected by biases that impact the accuracy oftheir evaluations. Second, managers have a limited perspective into the contributions of their employees15. That is, managers can only evaluateemployees based on their view of the employee’sperformance and this view may not accuratelyrepresent the full nature of the employee’sFigure 2: Technology illustration for individualmanager evaluations. This screen shot illustratesone of the many ways that manager ratings canbe collected through the use of performancemanagement technology.4. Group calibration sessions. This methodinvolves groups of organizational stakeholderscollectively meeting to discuss and evaluate employee contributions. These sessions are sometimes called “talent review meetings”. We believethat the use of calibration sessions, when done appropriately, is generally superior to the prior threemethods of performance assessment. One reasonis because data from contractually defined criteria,objectively measured criteria, and individual manager evaluations can all be incorporated into calibration sessions.7 / 43 2017 SAP SE or an SAP affiliate company. All rights reserved.
Total Workforce Performance ManagementThere are several additional reasons why calibration sessions are likely to be more fair, accurateand efficient than other assessment methods: Fair: When calibration processes are transparently communicated, employees are less likely tofeel as though their fate is in the hands of a singlemanager who may or may not have an accurateperception of their value. Including multiplestakeholders in the assessment helps ensureemployees’ full range of contributions across thecompany is being accurately and fairlyconsidered. Accurate: According to group decision-makingresearch, groups can act as a “check & balance”system against individual biases. Groups withmembers of diverse backgrounds and perspectives have also been shown to make higherquality decisions than individuals16. Efficient: If done right, calibration need not bea drain on organizational time and resources.Calibration sessions can provide leadershipwith extremely valuable conversations aboutbusiness and talent within the organization in arelatively short amount of time.We believe calibration tends to be more effectivethan other methods of performance assessmentfor most jobs. But calibration does not, and as wewill argue, should not look the same across everyorganization or every job. There are differentmethods of conducting calibration, different purposes it may serve, and other organizational factors that may affect its value and usefulness. Thepurpose of this paper is not to reveal the one andonly way calibration must be performed, but tosupport an organization’s use of calibration in athoughtful and strategic way.Figure 3: Technology illustration for calibrationtalent review. This is one of several screen shotsprovided that show how calibration methods canbe supported through technology. In this example, a manager or group of managers are able tosimultaneously review and compare the performance of multiple employees on a range ofdifferent performance dimensions and jobrelevant characteristics.8 / 43 2017 SAP SE or an SAP affiliate company. All rights reserved.
Total Workforce Performance ManagementPart II. Defining CalibrationAccording to Merriam Webster’s dictionary, theword “calibrate” means to measure against astandard. In the context of talent management,“calibration” refers to methods used to ensurethat evaluations of employee contributions arebased on a common and appropriate set of standards. We use the word “contribution” intentionally because calibration is used in a variety ofways. Perhaps the most common use of calibration is to assess differences in employee performance in their current role. Calibration is alsofrequently used to assess differences in employee potential to move into future roles. It can alsobe used to assess the criticality of an individualemployee’s skills and capabilities to the operations of the company. For example, determiningif certain employees possess crucial technicalknowledge that could not be easily replaced ifthey left the company. Calibration can also beused to determine how to allocate compensation, job opportunities, or development resources across employees in ways that will maximizethe return on investment for the organization.Calibration creates an “equal playing field” by providing managers with reference points, commonstandards and shared criteria against which tomake judgments about employees. Rather thanrelying on the subjective opinions and attitudes ofa single manager, calibration brings in the opinions of multiple people with different viewpointsand perspectives to ensure that employees areevaluated based on their actual behaviors, skillsand accomplishments. This is particularly important for reducing idiosyncratic biases foundacross managers and reducing subconscious biases that can negatively impact women, ethnicminorities and other historically under-represented groups17, 18.Calibration also addresses one of the most common problems associated with assessmentsmade by individual managers: the tendency ofmanagers to categorize all employees as beingroughly the same. Left on their own, many managers seek to avoid confrontation, hurt feelings,etc. by placing most or all their employees at thesame general level. They may also categorize themajority of employees in a way that implies theircontributions are all “above average”, even thoughby definition this is not possible. A fundamentalpart of effective calibration is using group-baseddecision making methods to challenge and helpmanagers differentiate between outstanding, solid and below average contributors. Some common methods for doing this include:9 / 43 2017 SAP SE or an SAP affiliate company. All rights reserved.
Total Workforce Performance Management Clear assessment criteria. Assessing employrestructuring that necessitates significantees using behavioral definitions of job perforreductions in workforce cost). Even then, itmance, metrics tied to specific, measurable jobshould be used very carefully.goals, and/or well defined attributes describing Reviewing and discussing categorization ratkey skills and capabilities. During calibration sesings. One of the hallmarks of effective calibrationsions managers are challenged to describe andsessions is the use of discussion and dialoguecategorize employees relative to these criteria.to clarify what employee behaviors, skills and Expected rating distributions. Some calibraaccomplishments are critical to the success oftion sessions encourage managers to distributethe organization. Research has shown thatemployee ratings to fit a pre-defined distribuwhen raters know they will be asked to justifytion (i.e., 15% “fails to meet expectations”, 35%assessment ratings, they put more care into“meets expectations”, 35% “exceeds expectaconsideration of performance behaviors whentions”, and 15% “greatly exceeds expectamaking their assessments22. The result is moretions”). The use of recommended distributionsthorough, consistent, unbiased assessment ofcan be an effective means to encourage managemployees. The most common type of ratingers to critically evaluate employees’ relativereview involves an employee being rated by hiscontributions. But strictly requiring managersor her manager, and then that manager’s manto “force rank” employees into different categoager reviewing the rating to ensure it aligns withries can be highly problematic. Force rankingthe company’s performance definitions and/ormay initially be effective at increasing workforcerating distribution guidelines. This method is aproductivity if a company has a high percentform of calibration, but it is prone to allowingage of under-performing employees that it hasbias to affect ratings since both managers maynot addressed. But its value wears off quickly asshare the same perspectives23, 24. Calibrationthe company begins to manage out poor persessions reduce this risk of bias by bringing to19,20formersand over time it can significantlygether a larger group of individuals composeddamage workforce quality and employee moof managers, senior leaders, and/or talent man19,20,21rale. Consequently, we do not recommendagement specialists from across multipleusing strict forced ranking unless there is a veryteams, departments, or organizations to disspecific situation that requires it (e.g., financialcuss and debate assessment ratings.10 / 43 2017 SAP SE or an SAP affiliate company. All rights reserved.
Total Workforce Performance ManagementCalibration sessions can leverage all three ofthese methods to increase assessment accuracy.For example, performance data and ratings fromindividual managers based on clear performancecriteria may be collected prior to a calibration review session. During the session managers arerequired to explain and discuss their ratings decisions to their peers. In some cases, the discussion may lead managers to change their ratings.Managers may also be challenged if their ratingsdo not conform to an expected rating distribution(e.g., if a manager rated all their employees asabove average they would need to explain to theirpeers why their team is so strong). This ensuresthat the final assessments are both accurate andconsistent with the views of and observations ofother members of the organization.removed many of the operational constraints thathistorically limited companies from making muchgreater use of calibration. As a result, the use ofcalibration is becoming more common across amuch wider range of jobs than in the past.One of the historical challenges to conducting calibration was the work required to assemble thedata on individual employees needed to hold effective calibration sessions. Because of the effortinvolved, calibration was often only done for a verysmall number of jobs in a company such as topleadership roles. Or it was done in an overly simplistic manner that emphasized compliance overquality (e.g., forcing managers to comply withstrict forced ranking guidelines). Advances inhuman capital management technology have“ We use a ‘good cop bad cop’ type strategy topush supervisors to really flush out their thoughtprocess and justify their ratings further.”Given its multiple benefits, we believe that calibration should be used by virtually every organization in some manner (an exception might bemade for very small companies). This does notmean calibration should be used in the same wayor for the same purpose in every organization.Nor should it necessarily be used for every typeof job. But given the value calibration has for ensuring fair, accurate assessments of employeecontributions, it is hard to imagine a companythat could not benefit from its use in some form.“ Disagreements are generally where leaders mustreally justify their view of a person’s performance. If I’ve interacted with the person myself, Imight say to the manager, ‘well this was my interaction with so and so, and this is what occurred.is this an anomaly? Is this typical behavior for soand so? Things like that.”11 / 43 2017 SAP SE or an SAP affiliate company. All rights reserved.
Total Workforce Performance ManagementPart III. Calibration DesignThere is no “one best way” to conduct calibrationsessions. Methods that work in one companymight not make sense for another organization.The following are critical design questions thatshould be considered when designing a calibration process or conducting calibration sessionsfor your company:1. What are you calibrating?2. What jobs and employees are being calibrated?3. H ow much time will be spent on calibration?4. How will calibration data be used?5. Which employees will be assessed in differentcalibration sessions?6. What methods will be used to structure calibration conversations and decisions?7. Who will participate in calibration sessions andwhat are their roles?8. How will the results of calibration sessions becommunicated?Each of these questions will be discussed inmore detail.1. WHAT ARE YOU CALIBRATING?It is important to ask “what purpose will calibration serve in my organization?”, and to makesure that managers clearly understand thispurpose. Calibration may be used for several purposes, including: Managing differences in job performance. Thesuccess of a company ultimately comes downto the performance of its employees. And thebest predictor of future performance is pastperformance. While job performance is criticalto business success, it is also notoriously difficult to define and measure. Performance canbe defined in terms of past accomplishments,quantitative results, knowledge and skill levels,or any number of behaviors. Managers vary inhow they evaluate employee performance,have difficulty differentiating between high andlow performers, and struggle to effectively dealwith employee performance differences. Left totheir own devices, many managers choose totreat all employees as though they perform atthe same level, rather than addressing low performers or supporting truly high performers.“ We saw ratings continue to become higher andhigher. It’s easier to say ‘your performance hasimproved’ than the opposite. So, we usedround tables to get calibrated ratings back.”“ Looking at how managers were rating employeeperformance, we knew we had to do something.Previously, everyone was getting a ‘5’. But weknew from corrective actions throughout theyear, that these ratings couldn’t possibly be accurate. Calibration seems to be working. Rarelydo we find employees in the lower two steps formore than a year. They either exit the organization, or more likely, have improved their skillsand are meeting expectations.”Calibration addresses these problems by usingcommon definitions and processes for assessing performance, and by fostering conversations that provide guidance and expectationsabout how performance differences are to bemanaged. By improving the accuracy of performance ratings, calibration provides the company with valuable data for making talentdecisions related to staffing and compensation,tracking metrics such as “turnover of high vs.12 / 43 2017 SAP SE or an SAP affiliate company. All rights reserved.
Total Workforce Performance Managementlow performers”, or using performance criteriato evaluate and refine hiring and trainingmethods.long-term incentive, or spot awards, has beenshown to significantly increase workforce performance over time27. But these ‘pay for performance’ techniques only work if the organizationhas methods to appropriately determine the level of rewards to be given to different employees.Calibration can be a key part of these methods. Assessing differences in employee potential.To maintain organizational performance overtime, companies must effectively support and invest in the potential of their employees. Accurately measuring employee potential is critical toguiding development decisions related to training, job assignments, or retention incentives.Like performance, the concept of potential canbe difficult to define. It depends on a range offactors including past performance, motivation,aptitude, qualifications, and others. Calibrationhelps ensure that assessments of potential arebased on a common, organizational definitionand not simply the untested judgment or “intuition” of individual managers. Calibration sessions can also increase visibility of high potentialemployees and identify opportunities to providethem with job assignments or resources that willhelp them realize their potential.“ The problem was that managers thought of‘high potential’ in different ways across our organization. There are so many definitions of potential, so the definition would change whetherspeaking about it in the investment business orthe distribution channel, for example. In anycase, we weren’t quantifying potential consistently across the organization, which is whatcalibration helped us to do.” Allocating compensation and rewards. It isappropriate to invest resources into employeesbased in part on the relative value they provideto the company26. Rewarding performance,either through merit increases, bonuses,It is important to stress that using calibrationfor compensation is different than calibratingperformance or potential. While performanceand potential are important factors that influence compensation, they are not the onlythings that affect compensation decisions. Forexample, current pay levels compared to themarket, comparing pay levels between employees in similar roles, previous pay increases, andperceived retention risk are all reasonable factors to consider during a calibration session focused on compensation. None of these factorswould make sense to include in a calibrationsession focused on performance.“ There are a lot of good employees. But there’sonly so much money to give out.”It is common and sensible for companies to havemultiple calibration sessions with different purposes. For example, a company might holdcalibration sessions focused on performance inthe first quarter of the year and then hold sessions focused on compensation the secondquarter. The compensation calibration sessionsmight leverage data from the performancecalibration sessions, but would also use otherdata to guide pay decisions. In the third quarterthe company might hold another series of calibration sessions focused on assessing employeepotential. These sessions might be limited to13 / 43 2017 SAP SE or an SAP affiliate company. All rights reserved.
Total Workforce Performance Managementemployees in certain roles or employee who haveachieved some minimum level of performance.Figure 4: This screen illustrates one of the morecommon methods used to calibrate employees forthe purpose of succession management. Frequently called the “9 box”, it assesses employeesbased on performance in their current role andtheir future potential to move into roles with greater levels of responsibility. This method is called a 9box because most companies assess employeeson 3 levels of performance and 3 levels of potential. Variations of the 9 box include 25 boxes, 6boxes, 4 boxes or any other combination onemight imagine based on the number of performance and potential categories used. As we discuss later, the 9 box method while widely used andbeneficial in many respects, also has some significant limitations.In addition to using calibration for performance,potential, and compensation, companies are increasingly using calibration to guide operationalstaffing decisions. For example, following announcement of an acquisition a company mighthold a calibration session to determine who toassign to the post-merger integration team.During this session leaders meet to considerwhich employees to assign to the integrationteam based on their past experiences, skills andfuture potential. Technology has made these operational, “spur of the moment” operational staffing calibration sessions much easier to conductby allowing companies to quickly identify andcreate pools of employees and to readily reviewemployee performance and potential data.There are several reasons why it is better to conduct separate calibration sessions rather than trying to calibrate performance, compensation andpotential all in a single session. First, holding multiple sessions simplifies calibration by focusing onone type of decision at a time. This tends to improve the results of calibration sessions. Researchhas shown that the broader the number of topicsand individuals included in group decisions, thegreater difficulty they may encounter in terms ofcoordination and decision making28. In otherwords, as more people become part of a conversation and more topics are covered, the greater therisk of confusion and misunderstanding.Having multiple calibration sessions also allows acompany to design each session tailored to itsspecific purpose. This may include changing thecomposition of the calibration group, the criteriaused for calibration, and the timing and steps usedin the overall calibration process. For example, itmay make sense to have one group of individualscalibrate past employee performance, but anothergroup to discuss future employee potential. Thedesign and agenda of the session will also change14 / 43 2017 SAP SE or an SAP affiliate company. All rights reserved.
Total Workforce Performance Managementdepending on whether the discussion is aboutperformance, potential, or compensation. And itmay make sense to calibrate performance acrossa broad range of jobs and employees, but only calibrate potential or compensation for a smallernumber of people. In sum, it is better to have several types of calibration sessions each focused ona si
Total Workforce Performance Management 3 / 43 This paper discusses research studying methods for total workforce performance management. The term "total workforce" is used to emphasize performance management processes designed to guide decisions regarding the management of groups of employees. This can be contrasted to methods
