Organizational Cultural Competence Review Of Assessment Protocols

Transcription

Making Children’sMental HealthServices SuccessfulOrganizational Cultural Competence:A Review of Assessment ProtocolsA Monograph byMaya Harper, M.A.,Mario Hernandez, Ph.D.,Teresa Nesman, Ph.D.,Debra Mowery, Ph.D.,Janice Worthington, M.S.,Mareasa Isaacs, Ph.D.Research & Training Centerfor Children’s Mental HealthDepartment of Child & Family StudiesLouis de la ParteFlorida Mental Health InstituteUniversity of South Florida

This publication was produced by Study 5: Accessibility of Mental Health Services: Identifyingand Measuring Organizational Factors Associated with Reducing Mental Health Disparitiesof the Research and Training Center for Children’s Mental Health. This study investigates howthe concept of cultural competence can be operationalized and implemented within systems ofcare to improve access to services and supports and reduce mental health disparities. December 2006Research and Training Center for Children’s Mental HealthDept. of Child & Family Studies, Louis de la Parte Florida Mental HealthInstitute, University of South FloridaRecommended citation:Harper, M., Hernandez, M., Nesman, T., Mowery, D., Worthington, J., & Isaacs, M. (2006).Organizational cultural competence: A review of assessment protocols (Making children’smental health services successful series, FMHI pub. no. 240-2). Tampa, FL: University ofSouth Florida, Louis de la Parte Florida Mental Health Institute, Research & Training Center forChildren’s Mental Health.For more information:See the Web site http://rtckids.fmhi.usf.edu or call the Center at 813-974-4661.This document may be reproduced in whole or part without restriction provided the Researchand Training Center for Children’s Mental Health, Louis de la Parte Florida Mental HealthInstitute, University of South Florida is credited for the work.The Center is jointly funded by the National Institute on Disability and Rehabilitation Research,U.S Department of Education and the Center for Mental Health Services, Substance Abuseand Mental Health Services Administration under grant number H133B040024. The opinionscontained in this document are those of the authors and do not necessarily reflect those of theU.S. Department of Education or Substance Abuse and Mental Health Services Administration.Support for this activity is providedby the Child, Adolescent and FamilyBranch, Center for Mentl Health Services,Substance Abuse and Mental HealthServices Administration (SAMHSA)AcknowledgementsWe would like to express our gratitude to the Annie E. Casey Foundation for supporting thisendeavor through funding the document’s printing.ii Making Children’s Mental Health Services Successful

Making Children’s Mental Health Services Successful:Organizational Cultural Competence:A Review of Assessment ProtocolsContents1 Introduction.12 Literature Review.3Focus of Monograph.53 Method.7Protocol Selection.7Analysis and Compilation of Domains.104 Findings.12Definitions of Cultural Competence .12Categories for Assessing Organizational Cultural Competence .13Identification of Common Domains.14Domain 1: Organizational Values.15Domain 2: Policies/Procedures/Governance.16Domain 3: Planning/Monitoring/Evaluation.17Domain 4: Communication.18Domain 5: Human Resource Development.19Domain 6: Community and Consumer Participation.20Domain 7: Facilitation of a Broad Service Array.21Domain 8: Organizational Resources.22How Results are Measured or Given Meaning .235 Conclusions.26References.29Appendices.32Appendix A Cultural Competence Assessment Analysis: Overview of Categories.33Appendix B Definitions of Cultural Competence.37Appendix C Identified Domains and Sub-domains.39Organizational Cultural Competence: A Review of Assessment Protocols iii

FiguresFigure 1 Methods for Developing Advanced Cultural Competence.4Figure 2 Conceptual Model of Organizational Cultural Competence:Compatibility Between the Community and Organizational Domains.6Figure 3 Selection Process of the Organizational Level Cultural Competence Instruments.8Figure 4 Conceptual Model for Organizational Cultural Competence.26Figure 5 Relationship Between Organizational Infrastructure and Direct Service Functions.27TablesTable 1Organizational Level Cultural Competence Assessment Instruments.9Table 2Domain Frequencies Across Organizational Level Cultural CompetenceAssessment Instruments.14Table 3Instrument Development.23Table 4Guide for Interpreting Results Obtained from Instruments.25iv Making Children’s Mental Health Services Successful

Organizational Cultural Competence:A Review of Assessment ProtocolsIntroductionCultural competence is understood at the most basic level to be the practice ofconsidering culture in order to effectively serve people of diverse backgrounds. Afoundational definition for the concept, and one of the most widely cited, comesfrom Cross, Bazron, Dennis, and Isaacs (1989): “Cultural competence is a setof congruent behaviors, attitudes, and policies that come together in a system,agency, or among professionals that enable effective interactions in cross-culturalsituations.” (p. 13). Cross et al. (1989) further elaborate that “a culturally competent system of care acknowledges and incorporates—at all levels—the importanceof culture, the assessment of cross-cultural relations, vigilance towards the dynamics that result from cultural differences, the expansion of cultural knowledge,and the adaptation of services to meet culturally-unique needs” (p. 13). Despitemuch that has been written about cultural competence since the publication ofthe Cross et al. definition, the concept has remained largely ideological and lacksoperationalization in a format that is measurable, understandable, and usable byorganizations (Geron, 2002; Vega & Lopez, 2001). However, instruments havebeen developed to assess cultural competence in a variety of settings, with theindividual items within instruments representing a starting point for operationalization of the concept. Of particular interest to this review is how assessmentinstruments operationalize the concept for application at the organizational levelof mental health services.1IntroductionA focus on the organizational levelis prompted by the recognition thatcultural competence supports areneeded at all levels of organizations.Cultural competence assessment instruments designed for use at the organizational level are the reflection of the developers’ theories concerning whichorganizational factors are most closely associated with cultural competence inthat context. These theories are commonly developed with the use of data derivedfrom literature reviews, field experience, and/or case studies. While there arecertainly commonalities among the resources utilized in developing individualinstruments and similarities in the reasoning behind the factors they include,a collective understanding has not been established. In addition, some culturalcompetence assessment instruments lack explanation of the underlying theoriesconcerning the factors associated with cultural competence in organizations. Forthese reasons, a closer examination of cultural competence theories and how theyare operationalized in assessments is needed. A focus on the organizational level isprompted by the recognition that cultural competence supports are needed at alllevels of organizations, including decision-making and funding mechanisms, andnot only at the level of the individual service provider.This monograph presents the findings from a review of cultural competenceassessment tools designed for use at the organizational level. It serves to informthe larger goals of the Florida Mental Health Institute’s (FMHI) Research andTraining Center Study 5: Accessibility of Mental Health Services: Identifyingand Measuring Organizational Factors Associated with Reducing Mental HealthOrganizational Cultural Competence: A Review of Assessment Protocols

Chapter 1: IntroductionDisparities (Research and Training Center for Children’s Mental Health, 2004).The goals of Study 5 are to identify and describe measurable organizational factors that lead to cultural competence in organizations and are therefore associatedwith increasing service accessibility for racially/ethnically diverse children withserious emotional/behavioral disorders and their families. This monograph describes the factors perceived by the developers of cultural competence assessmentinstruments to influence accessibility and utilization of services by ethnicallydiverse individuals. The findings contribute to the goal of developing culturalcompetence in systems of care, wherein participating organizations must supportthe delivery of culturally competent services and develop collaborative relationships with diverse communities. The findings will ultimately be utilized in combination with a literature review and organizational case studies to development anorganizational cultural competence assessment instrument. Making Children’s Mental Health Services Successful

2Literature ReviewLiterature ReviewThe Supplement to the Surgeon General’s Report focusing on culture, race,and ethnicity (U.S. Department of Health and Human Services [U.S. DHHS],2001a) indicated the importance of considering cultural influences in mental health, mental illness, and mental health services when considering healthdisparities. The role of culture in service delivery includes not only the cultureof the patient, but also the culture of the clinician, societal influences on mentalhealth services, and broader societal issues and historical circumstances that affecteconomic, social, and political status (U.S. DHHS, 2000; U.S. DHHS, 2001a).All of these circumstances affect access to and appropriateness of services.Applications of cultural competence in health care settings have been promoted by the U.S. DHHS Office of Minority Health (U.S. DHHS, 2001b). TheU.S. DHHS articulated 14 standards of Culturally and Linguistically AppropriateServices (CLAS) and presented them in three major thematic areas, includingCulturally Competent Care (Standards 1-3), Language Access Services (Standards4-7), and Organizational Supports for Cultural Competence (Standards 8-14)(2001b). The U.S. DHHS (2000) also categorized these CLAS standards by levels, including overall system standards, clinical standards, and provider competencies, which point out the need for application across organizational domains. Thiswork has not only made cultural competence a public priority, but has providedspecific guidelines for acceptable practices within key areas of service provision.An underlying value of systemsof care is that services mustbe appropriate to the culturalcontexts of the lives of thechildren and families they serve inorder to truly benefit children andyouth with serious mental healthissues and their families.The Center for Mental Health Services (CMHS) (1997) has issued culturalcompetence standards for managed care mental health services targeting fourmajor ethnic groups in the United States (African American, Asian/PacificIslander, Latino, and Native American). These standards addressed culturalcompetence in areas including planning, governance, benefit design, prevention, education and outreach, quality monitoring and improvement, management information systems, and human resource development (CMHS, 1997).For each standard, the CMHS also developed implementation guidelines, performance indicators, and recommended outcomes (with benchmarks). Thesecomponents provided examples of ways in which cultural competence could beapplied to mental health settings.The issue of cultural competence has also been addressed in children’s mentalhealth settings through its incorporation as a core value of systems of care (Stroul& Friedman, 1986). Systems of care are broadly defined as integrated systems ofservices that recognize the multidimensional needs of children and families. Anunderlying value of systems of care is that services must be appropriate to the cultural contexts of the lives of the children and families they serve in order to trulybenefit children and youth with serious mental health issues and their families.Organizational Cultural Competence: A Review of Assessment Protocols

Chapter 2: Literature ReviewThe application of the concept of cultural competence to systems of care waselucidated by Cross et al. (1989). Five key organizational qualities were identifiedas important to serving culturally diverse children and their families: valuing andadapting to cultural diversity; ongoing organizational self-assessment; understanding and managing the dynamics of cultural difference; institutionalizationof cultural knowledge and skills through training, experience, and literature; andinstituting service adaptations.It is suggested thatorganizations “go beyondrhetoric and find ways tomake culturally competentservices a reality.”Isaacs and Benjamin (1991) provided additional guidance for implementingcultural competence in systems of care by conducting organizational case studies.Eleven organizations were described in case studies that documented the stepsbeing taken to make their services more accessible and effective for children andfamilies of color. The programs provided unique examples of operationalizingcultural competence for specific communities and contributed to a greater understanding of the importance of appreciating the challenging process of movingtowards cultural competence within specific contexts. Although these exampleswere helpful, strategies that are generalizable across contexts and diverse populations were not identified.Support for developing cultural competence in systems of care at the organizational as well as individual level was provided in Hernandez and Isaacs (1998).The authors suggested that organizations “go beyond rhetoric and find ways tomake culturally competent services a reality” (Hernandez, Isaacs, Nesman, &Burns, 1998, p. 21). The need to identify strategies and methods for advancingand measuring cultural competence in organizations involved in systems of carewas also described (Aguirre, 1998; Jordan, 1998). For example, Jordan (1998)outlined methods for developing advanced cultural competence through organizational efforts to improve access, availability and outcomes of services, andconducting on-going assessment as summarized in Figure 1.Figure 1Methods for Developing Advanced Cultural CompetenceComponentDescriptionMeasureAccessEqual access to services regardlessof differences in language, culturalbackground, social status, or otherdemographic variablesComparison of populations, riskpool, and client base demographiccharacteristicsAvailabilityAppropriate services tailored for allgroupsTracking enrollment and retention inservices for each group servedOutcomesEquitable outcomes that are relevantto all groups served and reflectequivalent quality of services acrossgroupsComparing satisfaction and functionaloutcomes across groups served(Jordan, 1998) Making Children’s Mental Health Services Successful

Chapter 2: Literature ReviewIn addition to describing the components of cultural competence at variouslevels, there have also been efforts to compare the measures used in cultural competence assessment protocols. The Technical Assistance Center for the Evaluationof Children’s Mental Health Systems conducted an evaluation of 14 culturalcompetence assessment instruments, publishing its findings in Roizner (1996).Nine of the 14 cultural competence protocols reviewed were designed to measure cultural competence at the organizational level, while the remaining 5 wereintended for use at the level of the individual provider. In their examination ofcultural competence instruments, both Geron (2002) and Sue (2003) concludethat there is support for the incorporation of cultural competence into agencypractice, but there is a lack of standardized definitions and measures. As noted byGeron (2002), “existing efforts to measure the cultural competency of healthcareand social service providers have been developed adhoc and suffer from severalshortcomings” (p. 44). Shortcomings that were mentioned included lack ofdefinitions, minimal client/consumer input, and the need for reliability, validity,and psychometric property testing (Geron, 2002). Although these observationsprovide some direction for further development of the general concept of culturalcompetence, a focus at the organizational level is needed to better understand thisspecific aspect of cultural competence.This review aims to increasethe understanding of howorganizational culturalcompetence has been definedand measured.Focus of MonographThe purpose of the review described in this monograph is to advance theunderstanding of how organizational cultural competence has been operationalized in existing organizational level assessment tools. The review did not includea comprehensive evaluation of all of the cultural competence assessment protocols in existence. Rather, it focused attention on 17 assessment instruments withpotential application in mental health contexts that were designed for use at theorganizational level. This review aims to increase the understanding of how organizational cultural competence has been defined and measured.A conceptual model for organizational cultural competence was informed byand guided this review (Hernandez & Nesman, 2006). The conceptual modelillustrates the relationships between the community’s populations, organizationalstructures and processes, direct service structures and processes, and the overallcommunity context (Figure 2).The model shows that alignment or discordance between the communitycontext, cultural/linguistic population characteristics, and organizational components (i.e., infrastructure and direct service domains/functions) can facilitateand/or impede access, availability, and utilization of needed mental health services/supports. Ultimately, the level of compatibility between the community’spopulations and the organization is linked to the level of mental health disparities. This monograph will elaborate upon the organizational infrastructuredomain/function (4a) within the conceptual model.Organizational Cultural Competence: A Review of Assessment Protocols

Chapter 2: Literature ReviewFigure 2Conceptual Model of Organizational Cultural Competence:Compatibility Between the Community and Organizational DomainsCommunity Context (1)Alignment or discordanceDegree of compatibility defines level oforganizational/systemic cultural competencebetween the communitycontext, cultural/linguisticpopulation characteristics,and organizationalcomponents can facilitateand/or impede access,availability, and utilizationof needed mental acteristics ofa uctureDomain/Function(4a)Outcomes:Reducing mental healthdisparitiesDirect ServiceDomain/Function(4b)Definition: Within a framework of addressing mental health disparities within a community, the levelof a human service organization’s/system’s cultural competence can be described as the degreeof compatibility and adaptability between the cultural/linguistic characteristics of a community’spopulation AND the way the organization’s combined policies and structures/processes work togetherto impede and/or facilitate access, availability and utilization of needed services/supports. Making Children’s Mental Health Services Successful

3MethodMethodThe methodology of this review involved exploration, iteration, and the application of an evolving understanding of what was relevant and important toassessing organizational cultural competence. The key processes in the methodology were:1. The identification and selection of cultural competence instruments,2. The analysis of the measurable factors associated with organizational cultural competence (i.e., domains),3. The compilation of a comprehensive list and accompanying definitions ofthose domains, and4. The assessment of methods used for instrument development and applicability to mental health organizations and systems of care.Protocol SelectionThe methodology of this reviewinvolved exploration, iteration,and the application of anevolving understanding of whatwas relevant and important toassessing organizational culturalcompetence.The instruments selected for inclusion in this review were identified primarily from a search of the tools on the National Center for Cultural Competence(NCCC)’s online resource database (http://gucchd.georgetown.edu/nccc/), withother sources including Roizner’s (1996) review, and tools referenced in thebibliographies of relevant instruments. An online bibliography developed by acontributor to a listserv of CLAS (Gilbert, 2004) was also consulted and helpedin substantiating that the compilation was comprehensive.Instruments were selected for review based on their utility in assessing organizational cultural competence, with the specific criteria for inclusion being that theinstrument:1. Was designed to address cultural competence,2. Was focused at the organizational level,3. Included operationalized domains of cultural competence,4. Was focused on health or mental health, and5. Was obtainable for the review.The search for assessment tools meeting the above criteria yielded 45 instruments, 27 of which were identified in the original search of the NCCC database.In the process of protocol selection, 7 instruments were not included becausethey did not meet the selection criteria. As an example, an instrument developedby Eng & Parker (1994) was not included in the list because it was designed toassess community competence, which is a construct similar to but not synonymous with cultural competence. Instruments assessing linguistic competencewere determined to measure a critical element of cultural competence but not anequivalent construct. In addition, those instruments assessing cross- or multi-cul-Organizational Cultural Competence: A Review of Assessment Protocols

Chapter 3: Methodtural counseling and cultural sensitivity were considered to have a much narrowerfocus than cultural competence and therefore were not included in this review.Individual level assessmentsare designed to address the skills,knowledge, and attitudes ofpractitioners, while organizationallevel assessments addressinstitutional manifestations ofcultural competence.Ten instruments intended for use in assessing the cultural competence of individual practitioners (e.g., counselors and other direct service personnel) were notincluded in the analysis, as they were determined to lack a significant focus at theorganizational level. The importance of assessing cultural competence at the level ofthe individual is widely acknowledged, given the role of individuals in daily operations and direct service. Organizations, however, are guided by structural characteristics such as policies, procedures, mission statements, and service availability, whichmust be considered in assessing cultural competence at the organizational level. AsIsaacs and Benjamin (1991) argued, “It is not enough to merely assess the culturallevel of clients, and then continue to have them treated by culturally insensitivestaff or come to agencies with culturally insensitive policies and practices” (p. 31).Individual level assessments are designed to address the skills, knowledge, and attitudes of practitioners, while organizational level assessments address institutionalmanifestations of cultural competence, in areas such as policies or staffing.Further refinement of the list of assessment instruments to be reviewed wascompleted by selecting only those instruments that assessed more than one component of cultural competence and those designed for use within a health/mentalhealth care context. Five planning and screening checklists were not included dueto their lack of operationalization of cultural competence through domains orsubcategories. Checklists were not included as a rule, given that they tended totest only for the presence or absence of elements rather than evaluating the degreeof attainment. In addition, most checklists were designed for use in the preliminary screenings of organizations or as one of the first steps in planning a culturalcompetence strengthening strategy. For example, Goode’s (2003) checklist wasdescribed as a guide in planning for cultural competence improvements andtherefore was not included in the review.Finally, four instruments that were available only in a consumer versionwere not included in this review. While consumer input is necessary and useful(Scholle, 2002) and certainly informs organizational cultural competence, thestructure of these instruments was substantially different from organizational assessments and consequently lacked sufficient relevance to organizational culturalcompetence to justify an independent analysis.Figure 3 depicts the process of selecting cultural competence assessmentinstruments. Ten (10) instruments out of the initial 45 were not included becausethey measured only individual level cultural competence. Seventeen (17) instruments remained for analysis after additional instruments were removed forthe following reasons: 1) instrument measured a construct other than culturalcompetence in health/mental health; 2) instrument was a planning/screeningchecklist; 3) instrument was only available in a consumer version; and 4) instrument was not obtainable.Figure 3Selection Process of the Organizational Level Cultural Competence uments Making Children’s Mental Health Services Successful 35Organizational/System level instruments-Checklists &consumerinstruments 17Organizational levelinstruments

Chapter 3: MethodGiven the particular interest in supporting cultural competence within organizations serving the most diverse populations, further classification of the instrumentsinvolved identifying those with a documented application for use in systems of carefor children’s mental health. Of the final 17 instruments, six met the criteria. Thisapplicability was determined by explicit statements made to this effect within theinstrument, the inclusion of system of care principles within the instrument, ora documented association with a system of care in related articles or backgroundmaterial. For example, authors of one instrument published research in an articlediscussing the measurement of culturally competent services in systems of care, yetmade no explicit statement about relevance to systems of care within the instrumentitself (Siegel, Haugland, & Chambers, 2004).Table 1 indicates the author and title of each of the 17 organizational level cultural competence instruments that were analyzed for this monograph. Included inthe table is information on how to obtain each instrument.Table 1Organizational Level Cultural Competence Assessment InstrumentsAUTHORTITLE OF INSTRUMENTAmherst H. Wilder Foundation (2002)Cultural Competence Program Self-Assessment (order info) http://www.wilder.org/contact us.0.htmlAndrulis, Delbanco, Avakian, & Shaw-Taylor (n.d.)Conducting a Cultural Competence e 9.1g.htm&module provider&language EnglishAssociation of University Centers on Disabilities(AUCD) (2001)Assessment of Organizational Cultural Competence l Competence Survey.htmChild Welfare League of America (CWLA) (2002)*Cultural Competence Agency Self-Assessment Instrument (Revised) (order info) http://www.cwl

elucidated by Cross et al. (1989). Five key organizational qualities were identified as important to serving culturally diverse children and their families: valuing and adapting to cultural diversity; ongoing organizational self-assessment; under-standing and managing the dynamics of cultural difference; institutionalization