WIXLIB

EXECUTIVE SUMMARYThis document examines the use of competency-based initiatives across postsecondary educationin the United States and presents principles that underlie successful implementation drawn fromselected case studies. Conducted under the auspices of the National Postsecondary EducationCooperative, this project was informed by a Working Group of individuals selected for theirexpertise in utilizing competencies in a variety of settings. This project began in September 1998and concluded in October 2000.The world is changing and so is postsecondary education. Access to learning opportunities isgreater now than at any previous time (Rosenberg, 2000; Voorhees, 2001). The learning pathscreated by advances in information technology no longer lead solely to postsecondaryinstitutions. Organizations outside of postsecondary education have made significant inroads byproviding performance-based learning opportunities built on competencies. It is now possible formore sophisticated education consumers, (i.e., students and potential students) to acquire skillsand competencies through various means and at times they prefer. The Wingspread Group onHigher Education (1993, p. 14) noted that, “putting learning at the heart of the academicenterprise will mean overhauling the conceptual, procedural, curricular, and other architecture ofpostsecondary education on most campuses.” Some colleges and universities have undergonedramatic transformations in response to this movement toward performance-based learning, whileothers have maintained a traditional curriculum packaged in standard delivery formats andprovided in regular academic terms.Competencies are the bridge between traditional credit hour measures of student achievement andthe learning revolution. For the purposes of this project, a competency is defined as “acombination of skills, abilities, and knowledge needed to perform a specific task.” “Competencybased initiatives,” then, are those purposeful actions undertaken by postsecondary institutionsdirected at defining, teaching, and assessing competencies across their system.Why is it important to implement competency-based initiatives in colleges and universities? Onemain reason is that specific articulations of competencies inform and guide the basis ofsubsequent assessments at the course, program, and institutional levels. Secondly, specificcompetencies help faculty and students across campus, as well as other stakeholders such asemployers and policymakers, to have a common understanding about the specific skills andknowledge that undergraduates should master as a result of their learning experiences. Assumingthat faculty use a formal process to get feedback about what the competencies should be, thenstakeholders are more likely to accept and value them. Third, specific competencies providedirections for designing learning experiences and assignments that will help students gain practicein using and applying these competencies in different contexts.A major challenge for faculty, staff, and administrators is to ascertain the “data ramifications”that ensure that competencies are both valid and reliable with the ultimate aim, in practice, beingthat they are fully transportable between and outside of postsecondary entities. Dataramifications also refer to efforts to describe competencies in a uniform manner so that they canhave the same meaning in a variety of contexts and for a variety of audiences. If competenciesare to be utilitarian, strategies must be built to ensure that they are uniform. It is thisstandardization of terminology and semantics that can lead to the transportability of competenciesamong sectors of postsecondary education and across organizations that are not immediatelyaligned with colleges and universities.vii

Disappointingly, few sites in this study were actively dealing with issues of reliability andvalidity. Instead, as might be expected of relatively new endeavors, issues of process and internalacceptance for competencies were the initial focus taking time, energy, and resources that mightotherwise (with greater experience) be used to tackle transportability. In the future, asorganizations and postsecondary institutions gain more experience with competency-basedinitiatives, a focus on data ramifications will be important. By attending to concerns aboutvalidity and reliability, institutions can glean meaningful information to improve their initiativesand to satisfy external demands for accountability.Competencies are crucial for students before, during, and after their attendance at postsecondaryinstitutions. Consequently, these points in time were chosen as the organizing framework for thiswork* as shown in exhibit E-1. This project examined how competencies were used in varioussettings (including eight case studies of postsecondary institutions and a corporation) at keypoints in a student’s education.The ultimate goal of this project was to produce a final report that could serve as a hands-onresource for practitioners who seek to develop, implement, or refine their competency-basedinitiative. Accordingly, the case studies and the expertise of the Working Group were drawnupon to synthesize and recommend the following principles for practice in postsecondaryeducation. While each case study yielded fruitful insights, there are common practices gleanedfrom this study that can enhance the likelihood of success for those wishing to embark oncompetency-based initiatives. A senior administrator is the public advocate, leader, and facilitator for creating aninstitutional culture that is open to change, willing to take risks, and fostersinnovations by providing real incentives for participants. The appropriate stakeholders fully participate in identifying, defining, and reaching aconsensus about important competencies. Competencies are clearly defined, understood, and accepted by relevant stakeholders. Competencies are defined at a sufficient level of specificity that they can be assessed. Multiple assessments of competencies provide useful and meaningful informationthat is relevant to decision-making or policy-development contexts. Faculty and staff fully participate in making decisions about the strongest assessmentinstruments that will measure their specific competencies. The precision, reliability, validity, credibility, and costs are all considered andexamined in making selections about the best commercially developed assessmentsand/or locally developed approaches.*This temporal sequence complements the “across provider” framework offered by the National PostsecondaryEducation Cooperative Working Group on Student Transitions. That framework examines student movement acrossmultiple providers and assumes that competencies are adding value as a transportable unit.viii

Exhibit E-1.—Framework for organizing competency-based initiativesKey transitionixEntry into postsecondaryeducation (PSE)!Competency-basedsecondary schoolgraduation!Competency-basedadmissionsWithin PSE!Competency-basedcurricula in specificdisciplines!General educationcompetencies!Competencies at transfer(within providers)!Competencies at transfer(across providers)!Competencies certifiedthrough standardizedtestsKing’s CollegeNorthwestMissouri s and ollegeColleges(National ScienceIncumbentFoundationWorker iencybased AdmissionStandards System(PASS) inOregonXXXXXXXXXXXXFord MotorCompany

Exhibit E-1.—Framework for organizing competency-based initiatives (continued)Key transitionxExit from PSE!End of programcompetencies!PSE to employmentcompetenciesOverall institutionaleffectiveness!Internal (improvementsin student learning andacademic programs)!External (accountability,performance budgeting,accreditation)King’s CollegeNorthwestMissouri utions and ollegeColleges(National ScienceIncumbentFoundationWorker ciencybased AdmissionStandards System(PASS) inOregonFord MotorCompany

The competency-based educational initiative is embedded within a larger institutionalplanning process. The assessments of competencies are directly linked with the goals of the learningexperience. The assessment results are used in making critical decisions about strategies toimprove student learning. The assessment results are clear and reported in a meaningful way so that all relevantstakeholders fully understand the findings. The institution experiments with new ways to document students’ mastery ofcompetencies that supplement the traditional transcript.Each principle is more fully delineated in section 4 and accompanied by illustrations fromspecific case studies. Research into competencies has not been reported extensively in the highereducation literature. Nevertheless, there are useful and thoughtful resources about how toimplement competencies. Examples of recent literature are annotated and presented in thebibliography in section 6.xi

xii

Table of ContentsSectionPageFOREWORD .iiiACKNOWLEDGMENTS .ivNATIONAL POSTSECONDARY EDUCATION COOPERATIVECOMPETENCY-BASED INITIATIVES WORKING GROUP.vEXECUTIVE SUMMARY .vii1INTRODUCTION .12DEFINING AND CLASSIFYING COMPETENCY-BASED INITIATIVES:THE GUIDING FRAMEWORK.53METHODOLOGY FOR THE CASE STUDIES .134PRINCIPLES OF STRONG PRACTICE.195CONCLUDING OBSERVATIONS.31REFERENCES .336ANNOTATED BIBLIOGRAPHY .357CASE STUDY ABSTRACTS.498APPENDICES A – I .57A:B:C:D:E:F:G:H:I:King’s College .Northwest Missouri State University.Sinclair Community College .Hagerstown Community College.Community Colleges of Colorado Incumbent Worker Project.Western Governors University .Proficiency-Based Admission Standards System (Oregon) .Ford Motor Company .Case Study Interview Protocol .xiii597397109121135155165175

Table of Contents (continued)List of ExhibitsExhibitPage1Framework for organizing competency-based initiatives .142Abbreviated example of a competency statement.24List of FiguresFigure1PageA hierarchy of postsecondary outcomes .xiv8

1. INTRODUCTIONA competency is a combination of skills, abilities, andknowledge needed to perform a specific task in agiven context.In 1997, a National Postsecondary Education Cooperative (NPEC) Working Group examiningdata needs related to workforce development policies noted an increased interest nationally inskill standards, competency-based credentials, and assessment of work-based learningexperiences. NPEC’s mission is “to promote quality, comparability and utility of postsecondarydata and information that support policy development” (Public Law 103-382, Section 410,Improving America’s Schools Act of 1994). This Working Group recommended theestablishment of a NPEC project that would look at these emerging trends from the context ofdata and data ramifications. Advances in educational technology making access to competencybased learning more prevalent combined with competition from groups other than traditionalpostsecondary providers accelerated interest in this topic. In this context, performance-basedlearning has gained increasing attention.There are numerous challenges associated with developing and assessing competency-basedinitiatives based upon performance standards. Faculty and staff must decide upon thecompetencies to examine and levels of performance. They deliberate about the bestmethodologies to assess student performance, including whether to use locally developedinstruments or commercially available standardized tests. They decide who will be responsiblefor assessments and how the results will be used. It is not clear how problems related toportability of competency assessment and credentials across states and institutions will beresolved. These issues have ramifications for new data priorities as states, educationalinstitutions, and training providers encounter uncharted territory in developing performancestandards and assessing competencies.A number of states and institutions are currently supporting competency-based initiatives, andefforts to define critical skills have been underway for some time at the federal level, as well asunder the auspices of the national testing services. Even so, there is no comprehensive source fordetermining the extent to which postsecondary entities are engaged in competency-basedinitiatives and the extent that issues of transportability across a wide spread of providers areplaying into that work. At its meeting in January 1998, the NPEC Steering Committee approveda new Working Group and asked it “to analyze current practices in defining and assessing learnercompetencies and to determine their utility across a range of contexts.” The key tasks assigned tothe Working Group were to: Address the current state of the use of competencies across postsecondary education; Investigate and report the data ramifications of competency-based initiatives inpostsecondary institutions and their connections to student learning; Demonstrate the importance of competency-based initiatives in broader contexts suchas economic development, technology-based instruction, and industry interests incompetency certifications; Address terminology and definitional issues: What is a competency? How does itdiffer from a student outcome, skill, or ability? What is the relationship betweenperformance, assessment, and standards?1

Reach out to important target audiences (e.g., presidents, board members, legislators)that are not likely to avail themselves of other work in the area of competencydefinition and assessment; and Establish a set of “strong principles” or key considerations in the decision-makingprocess for defining and measuring competencies.Members of the Working Group brought wide and focused expertise to bear on these tasks,including: Developing competency-based admissions processes in higher education; Linking skills with job requirements; Developing career transcripts; Assessing critical thinking and writing skills; Defining workforce development performance measures; and Carrying out assessments at different points in the learning process.It should come as no surprise that these areas are ubiquitous in the wide arena of competencybased initiatives. It is very challenging to create, plan, and implement competency-basedsystems. In many instances, major political processes both within postsecondary education and inthe external environment dictate the will to implement them and the success that they mightachieve. The Working Group discussed purposes, principles, pitfalls, and processes for definingand assessing competencies within these areas, and concluded that future directions shouldinvolve an examination of the state of current practices through a case study approach. Theresulting case studies form the foundation for gleaning the principles of effective practice incompetency-based initiatives presented in section 8 of this document.The Working Group also struggled with the semantically sensitive issues that are attendant toefforts in this area. For example, what is considered a “competency” in one setting may beregarded as an “objective” or “skill” in another setting. There is no single definition of acompetency that cuts across all contexts, necessitating development of an operational definitionto guide this project.The Working Group also was cognizant of its relationship to other Working Groupscommissioned by NPEC. By casting its definition of competency, in examining state-of-the-artpractice, and in making recommendations for future practice, it is hoped that this work can informnot only NPEC but the wider postsecondary universe as well. For example, the conceptualframework, with its emphasis on times in a postsecondary education career that competencies areimportant, parallels the efforts of another NPEC Working Group (Student Transitions WorkingGroup) entitled “Examination of Policy Issues and Data Needs at Points of Student Transition”(Student Transition Data Systems: Phase I Report of the NPEC Working Group, November2000).2

A Quick Roadmap to This ReportFollowing this introduction (section 1), the report is divided into seven sections. The following“roadmap” may aid the reader to use this document efficiently. Section 2: Provides a conceptual overview of competencies as implemented in avariety of settings, including key concepts and definitions, which can be useful as astarting point for institutions that previously have not considered competencies andtheir connection to the learning process. Section 3: Establishes the methodology used by the Working Group to select sites forthe case study investigation. Gives practitioners a quick overview of the criteria usedto determine “cutting-edge” case study locations, as well as a glimpse of key issuesprobed during interviews. Section 4: Provides a quick read on the overarching principles and strong practicesuncovered within the case studies.Helps institutions, state agencies, andpolicymakers learn about how to improve or create competency-based initiatives withexamples from other settings. Section 5: Outlines an agenda for future work and a call for action at the institutionallevel. Section 6: Offers a current, annotated bibliography of resources in the literatureabout competencies. Serves as a guide to the best resources for both practitioners andpolicymakers. Section 7: Contains abstracts of the eight case studies. Section 8: Appendices A-H present the eight case studies for those practitioners whowant more details and analyses of current practices in assessing and utilizingcompetencies within a specific institutional or corporate context. Appendix I is thecase study interview protocol.3

4

2. DEFINING AND CLASSIFYING COMPETENCY-BASED INITIATIVES:THE GUIDING FRAMEWORKThe challenge is to determine which competencies can be bundledtogether to provide which types of learners the optimalcombination of skills and knowledge needed to perform aspecific task.What is a competency? Why is it important to assess competencies? How are assessments ofcompetencies used in making decisions and developing policy? The Working Group’sdeliberations on these topics are presented in this section. The Working Group relied upon thewide range of experiences reflected in its membership to develop a first-cut definition of thescope of competency-based initiatives and used that pragmatically derived definition to guide thecompilation of an annotated bibliography of competency-based initiatives.The world grows more complex on a daily basis as knowledge increases dramatically each year,and access to learning opportunities is greater now than at any previous time. Informationtechnology has expanded learning paths that do not lead solely to postsecondary institutions. Theinterest in learning “anywhere, anyplace, anytime” means that traditional educational deliverysystems will be challenged to address the needs of diverse learners. Along with expanded access,potential students and undergraduates will likely become more sophisticated consumers and willgrasp the concept of acquiring skills and competencies through diverse means. The WingspreadGroup on Higher Education (1993, p. 14) noted that, “putting learning at the heart of theacademic enterprise will mean overhauling the conceptual, procedural, curricular, and otherarchitecture of postsecondary education on most campuses.” This report is not intended, byitself, to change the culture of most postsecondary institutions. However, it can offer guidepostsand examples of good practice for those institutions and entities that want to become more fullyengaged in perfor

Prepared for the Council of the National Postsecondary Education Cooperative (NPEC) and its orking GrW oup on Competency-Based Initiatives by Elizabeth A. Jones and Richard A.Voorhees, with Karen Paulson, under the sponsorship of the National Center for Education Statistics (NCES), U.S. Department of Education. Report of the National Postsecondary