WIXLIB

Journal of Industrial Technologywould begin the important process ofdeveloping a universally accepted common core of knowledge for six sigmaprofessionals and elevate the recognition of the profession.In response to this clear need, a studywas conducted that could validateASQ’s SSBB BOK topics (DeRuntz,2005). This study surveyed a broadsample of the BB trainers that were notdirectly associated with ASQ with theintent of assessing their perceptions ofthe relative importance of certificationtopics to be used as BB candidate training criteria.The purpose of the study was to explorethe perceptions of Black Belt trainerswith respect to the criteria used to trainand certify BB candidates. More specifically, it sought to understand trainers’perceptions of the relative importanceof the ten certification topics identifiedby ASQ.METHODOLOGYThe study sought to obtain a samplepopulation that was a cross sectionalrepresentation of the quality professionand independent of any professionalquality organization. The populationfor this study was identified through aSix Sigma consultant directory on thewebsite isixsigma.com. This widelyused website contains many resourcesfor Six Sigma professionals and offersconsultants the opportunity to list theirbusiness. As such, one limitation of thisstudy is that only Six Sigma professionals registered on the isixsigma.comwebsite were included in the studiessample.The sample population was comprisedof 120 Master Black Belts who havetaught Black Belt training courses.The 120 consultants represented aninternational cross-section of the SixSigma trainers who are leading thisglobal transformation. Inferences madefrom this study were limited by thisselective population that offers trainingcommercially because there are manymore Six Sigma trainers who work forcorporations, educational institutions,etc. This cross-section of Six Sigma Volume 25, Number 3 July 2009 through September 2009professionals was chosen because;they are not employed exclusively byone company, they are not associatedwith any particular professional qualityorganization, and they had the greatestvariety of training experiences.The study utilized a mixed-methodology of both quantitative and qualitativeresearch methods. Using a mixed-methodology can create cross validation,which combines two or more theoriesor sources of data that study the samephenomenon in order to gain a morecomplete understanding of it (Denzin,1970). Additionally, a mixed-methodology is used to achieve complementaryresults by using the strengths of onemethod to enhance the other (Morgan,1998). This mixed-methodology wascarried out in sequential phases (i.e.quantitative-qualitative) for complementary purposes.In the quantitative phase, a surveyinstrument was developed from an extensive literature review and interviewswith Six Sigma professionals. Theinstrument was pilot-tested to determine face validity, content validity andease of use by a group of three MasterBlack Belts not included in the surveypopulation.Instrumentation.The final version of the instrument consisted of 63 questions contained in fourparts. In Part I of the survey, participants were asked to choose a descriptoron a Likert-type scale to indicate theirperceived importance of the 10 majortopics and respective subsections ofthe ASQ BOK. Each Likert descriptorcorresponded to a value ranging from1 to 5; a higher value would be used toindicate an essential topic (i.e., 5) followed in descending order to the leastimportant topic (i.e., 1).Part 2 asked their opinions regarding the need for a project evaluationinstrument, and other data unrelatedto this article. Part 3 collected basicdemographic data regarding the trainers’ experiences, and Part 4 solicitedvoluntary contact information if therespondent wished to participate in the4 www.nait.orgqualitative portion of this study.The survey achieved a response rateof 30.0%, and while a response rate of30% may seem low, it is not unusual.Alreck and Settle (1995) noted that surveys with response rates of more than30% are rare. In a study by Colombo(2000), the author stated that “typicalresponse rates from surveys are about20%” (p. 2). While research shows that30% is an acceptable response rate, theauthor felt the study would be furthervalidated through the use of complementary research in the form of followup interviews with a randomly selectedcross-section of respondents.Using a random number table, onesubject was chosen from each of therepresentative organizations: ASQ, Education, Industry, and Consultant. Thesample size of four was chosen basedupon a projected return rate of 30%,or 40 responses. From the 36 respondents, a random sample of 10% (i.e., 4)respondents was chosen to participatein a follow-up interview.Demographic data was collected in Part3 of the survey to assess the qualifications and experience of each participant. Qualitative participants who hadcompleted fewer than ten Black Beltprojects were excluded from the dataanalysis for lack of experience. In the(2003) article, “What does it take tobecome a Master Black Belt?” Watsonrecommended that a Master Black Beltcandidate should have completed atleast ten BB projects with topics havingboth commercial and technical applications.Table 1 presents the certification affiliation of SSBB trainers who respondedto the survey. These data show wherethe respondents received their certification. Note that the response fromtrainers who received their certificationfrom Industry comprised 41.7% of therespondent population.Rigor and ValidityEstablishing survey validity usuallyrequires evidence from several sources.The internal validity or rigor with

Journal of Industrial Technologywhich this study was conducted (e.g.,the study’s design, the attention takento conduct measurements, and decisions with reference to what was andwasn’t measured) is offered through theextensive review of literature that wasconducted. Four bodies of knowledgewere reviewed in depth. These fourbodies of knowledge included: (1) Motorola, (2) The George Group, (3) Rath& Strong, and (4) American Society forQuality. Upon careful review of thesebodies of knowledge the authors choseto utilize the American Society forQuality Six Sigma Black Belt Body ofKnowledge for this study. The rationalefor selecting the ASQ Body of Knowledge as the focal point for this studywas based upon the fact that ASQ isthe only organization that uses all theother Six Sigma Bodies of Knowledgeas references and for the development of their certification exam (ASQ,2009). Additional secondary information sources were reviewed including areview of classic quality texts, qualityjournal articles, master black belt proceedings and the ASQ website.To achieve survey validity from an external perspective it is important to havea high response rate. In order to havesurvey results that genuinely reflect thepopulation, it is necessary to have a statistically valid sampling from the SSBBconstituency. The higher the responserate the more valid the results. According to Bennekom (2003) a 30% (36/120)response rate yields a statistical accuracyof 95% 15%. Ninety-five percent waschosen by convention. If the accuracy is 15% and the survey instrument utilizesquestions with a measurement scalethat ranges from 1 to 5, then there are 4intervals on the scale. Plus or minus 15%on the scale is slightly more than one fullinterval point (25% of 4). Therefore, theauthors are 95% certain that the average(population mean) would lie within aband of one point on the scale with theaverage score from a survey question(the sample mean) in the middle. Puta different way, if a particular surveyquestion had a mean score of 3.5, and theauthors conducted a census, 95% of thescores would lie in a band from 2.975 to4.025. Volume 25, Number 3 July 2009 through September 2009 www.nait.orgTable 1. Number of Survey Respondents by the Organization Granting onsultants925.0Educational Figure 1. SSBB BOK Factors Average Rating of Importance and Percentageof Times Rated EssentialThe alpha level selected for this studywas .05. Alpha is the likelihood of being wrong that the authors are willingto accept. Five percent or (.05) beingwrong is the same as 95% certainty thatthe authors findings are correct. In thisexample, if the mean for a particularsurvey question was 3.5 on a 1 to 5scale and the confidence was 0.15, thenthe authors are therefore 95% certainthe true mean or population mean liesin a band defined by 3.5 0.15. Ouraccuracy is 0.15 as a percentage ofthe size of the scale, which is (5 - 1) 4. Thus, our accuracy is 0.15/4 or3.75%. Therefore, 95% of the time themean will fall in a range from 3.35 to3.65.5QUANTITATIVE FINDINGSThe statistical descriptors of meanand frequency were chosen to assesstwo different aspects of the perceivedimportance of the major topics. A meandescriptor was used to illustrate thetopics with the highest average level ofimportance. Essential is defined in thesurvey as having the highest importance when considering initial projectevaluation criteria. Frequency wasused to illustrate the number of timesa topic received the highest ranking ofEssential. The intent of this design wasto understand both the average level ofimportance and how often each topicwas considered to be Essential. Figure 1 illustrates the average rating ofImportance and the percentage of timea factor was rated Essential.

Journal of Industrial TechnologyBased on mean scores, the hierarchyof importance of the ASQ SSBB BOKtopics is as follows: 1) Measure Stage,2)Define Stage, 3) Analyze Stage, 4)Control Stage, 5) Business ProcessManagement, 6) Project Management,7) Improve Stage, 8) Enterprise-wideDeployment, 9) Lean Enterprise, and10) Design for Six Sigma.Confidence Intervals CalculationsThe data was further analyzed by examining the confidence intervals for eachof the 10 SSBB BOK topics or factors.The width of the confidence intervalgives an idea about how uncertain theresearchers are about the differences inthe means. A very wide interval mayindicate that more data needs to becollected before anything definite canbe said. The confidence intervals wereset at 95% by convention and thus canbe interpreted as; the researchers are95% confident that the interval containsthe true difference between the twopopulation means. In other words 95%of all confidence intervals formed inthis manner (from different samplesof the population) will include the truedifference.The formula for the confidence intervalof a mean is: CI (t * SE), Where“t” is a factor that depends on the confidence desired (95%) and the degreesof freedom generated for the estimateof error. It represents the number ofstandard deviations from the estimatedmean ( ). The 95% confidence intervalrepresents approximately two standarddeviations from the mean. The formulafor the mean of each survey response(Y) is shown below, where “n” equalsthe sample size (36 for this study) and“i” is each respondent’s individualranking for the specific survey item.nY Σ i 1 YinThe mean, or was calculated by adding up each of the individual respondents rankings and then dividing by thenumber of respondents. Variance (s2)equals the sum of the squared deviations from the mean, divided by oneless than the number of survey respondents. Volume 25, Number 3 July 2009 through September 2009 www.nait.orgTable 2. Perceived Importance of Major Topics within ASQ’s BOKMajor TopicAvg. Rating ofImportance out ofa possible 5Percentage ofTimes RatedEssentialS.D.Measure stage4.67670.48Define stage4.64690.59Analyze stage4.61670.60Control stage4.61720.80Business process mgt.4.58670.65Project management4.50610.74Improve stage4.31470.82Enterprise-wide deployment4.19390.89Lean enterprise4.03390.91Design for six sigma3.92220.77n 36, Note. S.D. standard deviation.1 not at all important; 2 Somewhat not important; 3 Neither important nor notimportant; 4 Somewhat important; 5 Essential.S2 nΣ i 1[(Yi Y ])2n-1Variance is the primary statistic usedto measure variability, or dispersion, ofthe distribution. However, to get unitsback to their original (not squared) metric, it is common to report the “standarddeviation(s).” This is just the squareroot of the variance.For this study the following formulawas used to approximate the standarderror. This formula is based on thecentral limit theorem.SE S2nRemember the standard error getssmaller as the number sampled (n)gets larger. In other words the moreone samples the more precisely onecan estimate the “true” outcome. Table3 depicts the calculations required tograph the confidence intervals for theten major SSBB BOK topic areas.6Figure 2 illustrates graphically theconfidence intervals for each of theten topics or factors included in theSSBB BOK. The graphical depiction ofthe data allows for easy interpretationof the findings. For example, as oneexamines the confidence interval for themeasure stage, it is easily seen that itoverlaps with the interval for the designstage, analyze stage, control stage,business process management, project management, and improve stages.Therefore, there are no significant differences between the measure stage andany of the aforementioned factors.Continuing with examining the confidence interval for the measure stage, itis easily seen that there is no overlap ofthe measure stage confidence interval with the enterprise-wide deployment, lean enterprise, and design forsix sigma factor confidence intervals.Thus, there are significant differencesbetween the measure stage and theenterprise-wide deployment, lean enterprise, and design for six sigma factors.To summarize the confidence intervalanalysis there are no significant differences between the measure stage,define stage, analyze stage, controlstage, business process management,

CI www.nait.org (t * SE)SE S2SE Std Dev n-(t*SE) (t*SE)Measure stageDefine stageAnalyze stageControl stageBusiness process Mgt.Project managementImprove stageEnterprise-wide deploymentLean enterpriseDesign for six 50.13n 367Design for 6 6SixLean EnterpriseImprove StageProject ManagementBusiness ProcessControl StageFigure 2. ASQ CSSBB Major Topic Confidence IntervalsEnterprise-wideInterviewees were asked how datashowing the relative importance ofASQ topics can be used in their SSBBtraining program. They indicated thatthey would place greater emphasis onthe topics ranked 1-7 in their trainingand would only use these topics whenevaluating a candidate’s project(s).Because Black Belts can obtain certification that may or may not include anexam as part of its process, the interviewees were asked their perceptionof this inconsistent requirement. Allof the interviewees said they put less Table 3. Confidence Interval Calculations for the 10 SSBB Major Topic AreasAnalyze StageInformation compiled from the survey was used to develop an interviewquestionnaire that would serve ascomplimentary research. Four MasterBlack Belts were interviewed and askedabout the applicability of this study in abusiness setting. Their responses wererecorded and studied using a contentanalysis procedure. Interviewees wereasked if they thought the rank orderof the major topics seemed logical tothem. All the respondents reaffirmedthe rank order of importance of the major topics. The majority of intervieweesfelt the relative importance of certification topics ranked 1-7 are absolutelyfundamental and essential to the BOKand have the most value. They also feltthat topics 1-7 could be classified asproblem solvers and topics 8-10 couldbe classified as problem preventers. July 2009 through September 2009Define StageQUALITATIVE FINDINGSVolume 25, Number 3Measure Stageproject management, and improve stagefactors. There are also no significantdifferences between the define stage,analyze stage, control stage, businessprocess management, project management, improve stage, and enterprisewide deployment factors since theirconfidence intervals all overlap. Additionally, there are no significant differences between the project management,improve stage, enterprise-wide deployment, and lean enterprise factors.Significant differences do exist betweenthe confidence intervals of the measure stage, define stage, analyze stage,control stage, and business processmanagement when compared with thelean enterprise and design for six sigmafactors. Average Rating of ImportanceJournal of Industrial Technology

Journal of Industrial Technologyemphasis on the exam and far greateremphasis on evaluating the BB performance in conducting their BB project.The interviewees’ perceptions pointout a significant problem that certification granting agencies need to address.That being that on the job performanceis perceived to be of more value thanpassing a certification exam.DISCUSSIONThe topics that comprise the DMAICmodel (Define-Measure-AnalyzeImprove-Control) ranked in 5 of the top7 positions. This disjointed ranking issurprising considering that the 5 stepDMAIC model is the backbone of theSix Sigma methodology. It is thought tobe highly unlikely that the importanceof the sequential DMAIC model couldbe fractured by other equally or moreimportant topics. The topics of Business Process Management (BPM) andProject Management (PM) achievedhigher rankings for both Average Ratingof Importance and number of times ratedEssential. The seventh place ranking ofthe Improve stage and its discontinuityin not maintaining sequential associationwith the DMAIC model infers that it isperceived as the least important stagewithin the DMAIC model and that BPMand PM should receive greater emphasisin training and evaluation.The major topics of BPM and PM represent the importance of a BB having anenterprise-wide perspective and interpersonal skills over technical knowledge.BPM and PM’s elevated importancereinforce Hoerl’s (2001) claim thatBlack Belts are valued for what they cando; not for what they know.Two other recent studies reportedon the future continuing education/training needs of manufacturingprofessionals. Callahan, Jones &Smith (2008) reported that the mostcommonly identified continuingeducation/training needs over thenext ten years would be Lean 77.8%,Six Sigma 56.3%, Quality Management 46.7% and Statistical Analysis46.0%. Their data was collectedfrom 261 engineers and technology Volume 25, Number 3 July 2009 through September 2009professionals who had a technicalcomponent associated with theirjobs. DeRuntz and Meier (2004)in a survey of NAIT Industry Division members reported that surveyrespondents indicated the followingprograms would be most beneficial forindustry members and the inclusionof these topics in the annual conference would impact their decision onconference attendance: Lean Management (40 of 102), Quality & Six Sigma(38 of 102), and Project Management(17 of 102). Based upon the increasedneed for people with Six Sigma relatedknowledge and skills it is importantthat educators and trainers understandthe importance of each of the ten majorcomponents of the Six Sigma Body ofKnowledge.CONCLUSIONThis study identified the hierarchy ofimportance for ASQ’s Six Sigma Bodyof Knowledge and thereby the amountof training and the level of project evaluation needed for each major topic. Theresults of this research indicated that ina training context, the instructor mustensure that adequate

The American Society for Quality (ASQ) is regarded as the preeminent professional quality organization in the world and grants professional certifica-tion in 14 different areas, based almost exclusively on the respective Body of Knowledge (BOK). ASQ’s certifica-tion is highly regar