Transcription
CULTURAL COMPETENCE SELF-ASSESSMENTQUESTIONNAIREA Manual for UsersJames L. Mason, B.S., Project ManagerAssistance with Manuscript Preparation:Tracy Williams-Murphy, B.A., Research AssistantMulticultural Initiative ProjectResearch and Training Center on Family Supportand Children's Mental HealthRegional Research Institute for Human ServicesGraduate School of Social WorkPortland State UniversityP.O. Box 751Portland, Oregon 97207-0751(503) 725-4040August 19952
Second Printing April 2002.The recommended citation for this publication is:Mason, J. L. (1995). Cultural competence self-assessment questionnaire: A manual for users. Portland, OR:Portland State University, Research and Training Center on Family Support and Children's Mental Health.This publication was developed with funding from the National Institute on Disability and RehabilitationResearch, United States Department of Education, and the Center for Mental Health Services, Substance Abuseand Mental Health Services Administration (NIDRR grant number H133B40021-94). The content of thispublication does not necessarily reflect the views or policies of the funding agencies.
TABLE OF CONTENTSPageAcknowledgments 1Introduction .3Historical .4Application . .4Administration .5Outcomes . 6Reliability 7Content Validity . 7Scoring Guide . 8Prototypical Tables, Charts, Graphs, and Narratives . 9Implications for Training . .9Practical Considerations . 10References . .13Appendix A:17Cultural Competence Self-Assessment Questionnaire/Service Provider VersionAppendix B:Cultural Competence Self-Assessment 29Questionnaire/Administration VersionAppendix C:Cultural Competence Self-Assessment . .41Scale/Demographic InformationAppendix D:Subscale Analysis 45
AcknowledgmentsThe development and refinement of any assessment measure requires the support ofmany individuals. The Cultural Competence Self-Assessment Questionnaire (CCSAQ) isno exception. Before “cultural competence” was conceptualized in the professionalliterature, many people worked to lay the foundation for the model and subsequently theCultural Competence Self-Assessment Questionnaire. The concept, however, has comeinto greater prominence in recent years as social and human service delivery systemshave been required to more effectively serve diverse children, families and communities(Cross, Bazron, Dennis & Issacs, 1989; Pinderhughes, 1989; Sue & Sue, 1990; Lum,1992).The Child Mental Health Services Branch of the National Institute of Mental Health(NIMH) and the National Institute of Disability Rehabilitation Research (NIDRR)through the Child and Adolescent Service System Program (CASSP) have mandated thatcultural competence be manifested in systems of care for children, youth, and theirfamilies. Hence, the CCSAQ was developed in recognition of the efforts to makeservices more culturally competent. It is important that people from many organizationsbe acknowledged for their commitment to effective and relevant service delivery.Recognition must also be extended to other organizations and their staff who participatedin various surveys, focus groups, and product reviews and who contributed considerabletime in the refinement of the items and subscales, which reflect various aspects of theCASSP Cultural Competence Model.Among the organizations that helped define specific behaviors consistent with theCASSP model and relevant empirical and theoretical literature include: Portland Research and Training Center on Family Support and Children’sMental Health National Advisory Committee, Portland State University, Portland, Oregon; National Research Consulting Group, Portland, Oregon; CASSP Technical Assistance Center Minority Resource Committee,Georgetown University, Washington, DC; Northwest Child Welfare Leadership Institute, University of Washington,Seattle, Washington; State Mental Health Representatives for Children and Youth, Washington,DC; National Association of State Alcohol and Drug Administration Directors(NASADAD), Washington, DC; National Indian Child Welfare Association, Portland, Oregon; and Multicultural Educational Research and Training Institute (MERTI), NewYork.1
Pilot sites which helped to identify the basic psychometric properties of the scale include:The Management Team of the Washington State Division of Mental Health; the Childrenand Youth Services Administration in the District of Columbia; the South CarolinaDivision of Mental Health; the New York State Office of Mental Health; the MonroeCounty (New York) Department of Mental Health; the Multnomah County (Oregon)Juvenile Justice Division; the Boys and Girls Aid Society of Oregon; and the CaliforniaDepartment of Health-Maternal and Child Health Branch. Many unnamed but dedicatedagency personnel who were extremely helpful in completing this effort must also beacknowledged.It is also important to recognize all of the graduate students, research assistants, projectassociates, academicians, researchers, professionals, consumers, families, advocates andothers who continue to work to assure that health and human services are culturallycompetent and relevant to culturally-, ethnically-, and racially-diverse populations.James L. MasonApril 19952
CULTURAL COMPETENCE SELF-ASSESSMENT QUESTIONNAIREA manual for UsersIntroductionIn response to the growing body of literature promoting culturally competent systems ofcare, the Portland Research and Training Center developed the Cultural CompetenceSelf-Assessment Questionnaire (CCSAQ). The CCSAQ is based on the Child andAdolescent Service System Program (CASSP) Cultural Competence Model (Cross, et al.,1989). This model describes competency in terms of four dimensions: attitude, practice,policy, and structure. The instrument helps child- and family-serving agencies assesstheir cross-cultural strengths and weaknesses in order to design specific training activitiesor interventions that promote greater competence across cultures. As cultural trainingactivities are developed, it is important that they be relevant to the audience, respectwithin- and between-group differences and acknowledge the present level of competenceof professionals, agencies and systems.The United States is rapidly becoming more racially and culturally diverse (Ponterotto &Casas, 1990). One impact of this diversification is that mental health and otherprofessionals must consider how culture influences the perception of mental illness andmental health (Sue & Sue, 1990; Ho, 1987, 1992; Cross, et al., 1989); help-seekingpractices and resources (Neighbors & Taylor, 1985; Gary, 1987; Green, 1982;Pinderhughes, 1989; Lum, 1992); and credible services and providers (Owan, 1982;Zane, Sue, Castro, & George, 1982; Solomon, 1987; Slaughter, 1988).When culture is ignored, barriers to effective services may emerge. For example,culturally diverse populations often face barriers such as out of home placements andmore restrictive settings (Cross, et al., 1989; Sue & Sue, 1990; Lum, 1992; Green, 1992;Isaacs, 1986; Katz-Levy, Lourie & Kaufman, 1987; Knitzer, 1982); culturally-biaseddiagnostic and assessment procedures (Ho, 1987, 1992; Sue & Sue, 1990; Flaskerud,1986; Chin, 1983; Ponterotto & Casas, 1990; Solomon, 1987); and professionals whomay be well-intentioned but who have not been trained to work with culturally diversepopulations (Woody, 1991; Atkinson & Hackett, 1988; Lum, 1992; D’Andrea, Daniels &Heck, 1991; LaFromboise & Foster, 1989; Gallegos, 1982; Chau, 1991). The negativeimpact of each of these issues is compounded because professionals of color andprofessionals trained to work specifically with diverse populations are not increasingcommensurate with growth rates of the diverse groups (Ponterotto & Casas, 1990; Isaacs& Benjamin, 1991; Persico, 1990). Moreover, while many programs seek to deliverservices in a culturally competent way, there are few systematic approaches to evaluatinga worker’s preparation for working cross-culturally (Mason, 1988; Cross, et al., 1989;Woody, 1991; D’Andrea, Daniels & Heck, 1991).3
Historical OverviewWhile designed for use in child and adolescent mental health systems, the CCSAQ hasbeen applied in other human service disciplines such as maternal and child health,alcohol and drug abuse treatment, child welfare, juvenile justice, public health, andeducation. In each instance, the measure was utilized to identify the organization’scultural competence training needs in such areas as: (a) improving service delivery toculturally diverse populations; (b) identifying cross-cultural strengths that currently existwithin an organization, system, or network of professionals; and (c) focusing onbeneficial training topics for providers of services. While this tool is intended to assistservice providers who work with groups of color, it will also have applicability to otherdiverse groups as well.The CCSAQ has proven useful in a variety of organizations around the country. Inseveral instances, the sample population consisted of attendees at local, regional, ornational conferences. In some of these sites, the measure was used prior to theconference to help identify the general training needs of the group. In some cases, theassessment was completed early enough so that relevant curricula could be prepared andincluded in the conference. At other sites, the measure was used to establish a baseline,which could later be compared to subsequent assessments of the same group.The CCSAQ was also demonstrated at the local agency- or system-level in children’smental health and other child and family service settings (e.g., child welfare or juvenilejustice systems) in Oregon, New York, Arizona, North and South Carolina, Pennsylvania,and Illinois. At these sites, analysis of the CCSAQ was based on the entire network ofprovider agencies and reported as an aggregate. The analysis was then used to developcultural competence plans for the aggregate group. For a few of these sites, however, anadditional analysis was conducted on smaller units (sub-samples) such as departmentswithin an organization. These smaller units of analyses are usually avoided to preventunfair and inappropriate comparisons across individuals or departments. When utilizedat the state or regional level, the goal of analysis was to help service deliverybureaucracies identify their cultural competence training needs as well as identify waysin which cultural competence could be manifested.ApplicationThere are two versions of the Cultural Competence Self-Assessment Questionnaire. Oneversion is for use with direct service providers and the other is for administrative staff.These different versions of the CCSAQ are useful when designing specific traininginterventions for either administrative or service-level personnel. Comparing resultsfrom direct service providers with those gathered from administrators can help clarifyperspectives between them regarding staff and board composition, agency policy,practice procedures, cultural group characteristics and client demographics, community4
based resources and linkages and community involvement.Administration of the CCSAQ in a variety of organizational settings have led the authorsto several conclusions: (1) many areas of cross-cultural training could be addressed bypeople with expertise who are already within a system; (2) one intervention does not fitall situations, and; (3) hidden resources such as diverse linguistic and religious insights,information about culturally-specific belief systems and protocols or experience in ethnicenvironments can be identified as a part of the analysis process. An equally importantaspect of the process is that completion of the self-assessment questionnaire is just thebeginning for an organization intent upon becoming more culturally competent. It is, infact, a means to an end. In most cases, the CCSAQ should be coupled with an activity(i.e., cross-cultural training intervention) so that subsequent measures of the same groupreveal some variance or positive change. Therefore, results of the CCSAQ should not beviewed as a static measure but rather should serve as an indication of areas where growthneeds to occur.In planning a training intervention, culturally-informed staff might be used in break-outsessions, panel presentations, brown bag sessions, pre- or in-service activities, or asrespondees to presentations made by outside consultants or presenters. This alsoacknowledges talent that may have been previously overlooked and generates a sense ofvalue for that particular worker. Also, using agency- and community-based resourcesconserves limited training dollars that can then be used for general sessions or sessionsfor which outside experts are needed. In any event, in-house personnel may hold hiddentalents and should be strongly considered as training resources. Also, the assessmentprocess is optimized when it spans several agencies, enhancing the potential foruncovering cultural training resources or the value of interactive learning.AdministrationThe process of self-assessment of cultural competence can be fairly intimidating.Responding to questions about racial and cultural differences can create problems forindividuals or an entire staff. These feelings can create tension when agencies andorganizations attempt to diversify staff, prepare for cultural competence, or begin toreach out to culturally diverse communities. Because cultural competence is adevelopmental process, the goal of assessment is not to tell people where they are on ascale, but to inform them where they need to enhance their skills cross-culturally.Moreover, since the CCSAQ is a self-reported measure, the veracity of responses iscritical. Therefore, administration of the measure must be preceded by an in-housediscussion about the reasons for participation and how each respondent’s confidentialitywill be assured.5
Several key issues should be considered prior to and during the administration of themeasure:1. Stress that there is no way to perform poorly since cultural competence is adevelopmental process.2. Assure respondents that the results of the CCSAQ will not be used forcomparisons between individuals, programs, or systems; rather, the goal of thiseffort is to identify the cultural training needs of a particular work force.3. Stress completeness, asking subjects to respond to every question on the scale tothe best of their ability.4. Utilize the demographic face sheet to identify agency-based cultural strengths,which may be overlooked.5. Keep track of how many questionnaires were distributed and how many werereturned; this will allow a return rate to be computed.6. Be certain that subject responses remain confidential and cannot be tracked tospecific individuals.Typically, it takes twenty to thirty minutes to complete the CCSAQ; however, somerespondents may require an hour. We recommend identifying lead personnel who will beresponsible for distribution and collection of the questionnaires. The logistics will varyfrom site to site; however, careful consideration should be taken (e.g., by a task group) toensure a smooth process.OutcomesThe CCSAQ provides a general score based on five subscales. For both serviceproviders and administrative personnel, the subscales include knowledge of community,personal involvement, resources and linkages, staffing, organizational policies andprocedures, and reaching out to communities. When analyzed and ranked, the subscalesprovide information to an organization for establishing training and/or policy andprocedural priorities. When examined individually, a subscale can suggest behaviorscongruent with the cultural competence model or assess the degree to which specificbehaviors routinely occur.Because there are two versions with many parallel items, comparisons can be made andinferences drawn based on the similarities and differences of perspectives between directservice workers and administrative staff. This facilitates the development of trainingactivities that reflect the experiences and needs of the two levels of personnel.6
ReliabilityMost of the psychometric analysis for this measure concerned the issue of internalconsistency reliability. This type of reliability deals with the extent to which items (i.e.,questions) in a scale correlate with one another to measure a specific construct or thedegree to which they consistently measure the same phenomenon (Vogt, 1993). Data toaddress this issue were collected at the New York, South Carolina, California,Washington DC, and Washington state sites.Calculating alpha coefficients, which can range from .00 to 1.00, typically assessesinternal consistency. Measures of .70 on new measures are deemed respectable(Nurally, 1978). For the CCSAQ, the majority of subscales have yielded alphacoefficients of .80 or higher. However, the coefficients for one subscale (PersonalInvolvement) average around .60. While the suggested behaviors in this subscale arequite important, the subscale may be revised or some items eliminated based onsubsequent experience by users of the CCSAQ. Since the CCSAQ is a relatively newinstrument, analysis of internal consistency is advisable. In settings where moreexpertise is available, additional psychometric analyses should also be considered.Content ValidityVogt (1993) states that content validity is addressed when the items in a scale or measureaccurately represent the phenomenon being measured, suggesting that it is not astatistical property as much as it is a qualitative judgment. With respect to the CCSAQ,the author conducted extensive reviews and consulted with acknowledged experts todefine subscales, identify item content, and refine item wording.The development of the CCSAQ began with an extensive review of historic andcontemporary literature relevant to the delivery of health and human services. Thisliterature is both research- and theory-based. The general goal of the literature reviewwas to establish convergent theories about what constitutes culturally competentbehaviors. After the literature review was conducted, focus groups were convened todiscuss the development of subscale items. Focus group members were professionalsfrom the service disciplines of mental health, child welfare, special education, maternaland child health, and alcohol and drug treatment. Academic disciplines whichcontributed to the development of this measure included social work, antibias/discrimination, intercultural communication, race relations, sociology, psychology,cultural anthropology, and public health administration. Based upon comments fromthese recognized experts, the authors constructed items for placement in each of theseven subscales.The goal of the instrument is to evaluate the degree to which respondents were exhibitingappropriate behaviors. On one hand, the items describe areas in which new behaviors7
can occur; however, in other instances, they suggest ways in which professionals candemonstrate cultural competence.Steps were taken to assure that the CCSAQ was developed in accordance with the higheststandards of instrument construction. As cultural competence becomes more refined,subsequent measures will emerge. These future measures must be developed with evenmore scientific rigor and normed on an expanded and diverse sample.The CCSAQ represents a process tool. In the hands of an experienced facilitator, it is abeneficial instrument. With the scaling removed, the items will generate usefuldiscussions of what can be done to stimulate culturally competent practices andbehaviors. It is our expectation that the CCSAQ will evolve into a better tool over timewith increased utilization.Scoring GuideThe method of analyzing CCSAQ scores has typically involved descriptive statistics.Because Likert scaling was used instead of nominal scaling, behavioral changes can bemeasured. Thus, it is necessary to compute item averages and subscale means. Thesemeans can then be ranked to determine which items deserve consideration. This methodallows for a ranking of subscales to prioritize areas in which interventions will beapplied. However, simple ranking may not be totally useful. A discussion aboutpriorities for change by an agency- or system-based work group may be particularlyvaluable. The group can review the rankings and--based on their knowledge of the areaand program in question--place the findings into a site-specific perspective.The CCSAQ can lead to interventions based on the pre-test analysis or agency-specificgoals. The impact of these interventions can then be evaluated with a post-testadministration. So, while the tool can be used in a one-shot case study, it seems to haveoptimal utility when used in pre- and post-test designs.To analyze the results of the aggregated CCSAQ scores, means are computed for eachitem and each subscale. This allows an in-house discussion of the analysis that can betranslated into specific training approaches. Because there are two versions (DirectService and Administrator), comparisons can be made so that distinct training tracks canbe configured to meet the distinct needs of these two levels of service. In cases whereoverlapping needs are identified, general sessions can be designed; when the needs aredivergent, job-specific interventions can be considered.When used as a process tool, the items can engender discussions centering around whoon a given staff currently engages in certain behaviors. This may stimulate anopportunity to8
share cross-cultural successes with colleagues, reflecting a method of identifying inhouse training resources that might otherwise be overlooked.It is certainly possible, however, to look at more intensive forms of analysis that involvebivariate or even multivariate analysis. Tests of significance can be computed on itemand subscale means, and comparisons can be made between administrator and directservice provider versions of the CCSAQ. Moreover, tests of internal consistencyreliability can be computed by an agency or an organization; given a large sample size,other psychometric tests may be performed. For example, if a pre- and post-test design isused, test-retest reliability can be analyzed. In this instance, correlations with valid orassociated measures can be considered or additional evaluations of validity can beexamined.Prototypical Tables, Charts, Graphs, and NarrativesAfter administration of the CCSAQ, the resulting data should be presented in ways thatare comprehensible to a given audience. For some, narratives will be appropriate. Forothers, charts, tables, and other graphics may be helpful. Tables that have been used withpositive results by organizations administering the measure are shown in Appendix ?.Bar graphs and pie charts can also be effective in presenting analyses of the CCSAQ.When using the tables, items that score beneath the subscale mean can be identified withan asterisk or shading to stimulate discussion.In most cases, subscale means can be ranked from low to high and considered by theresponding group as topical themes for training. Training interventions could then bedeveloped emphasizing activities for items below the subscale mean. One goal of atraining intervention could be to generate positive movement on the cultural competencecontinuum with respect to a given item or subscale.Implications for TrainingThe CCSAQ was designed to aid organizations that wish to consider training and otherinterventions to improve services to children and families of color. A tenet of the culturalcompetence model is that professionals can become more culturally competent withtraining and experience. When an organization identifies the convergent and divergentneeds of staff, all agency personnel within an organization can become moreaccomplished cross-culturally.The CCSAQ can assist with internal improvements in organizations. For example, theself-assessment process will reveal if certain behaviors, as captured by a given item, areactually occurring. For instance, respondents who reveal that they are reaching out tonatural helpers may wish to be identified as in-house resources who can share this9
information with colleagues, precluding the need (at least initially) for outsideconsultation. In this case, two things can be accomplished: the use of in-house resourcesas training personnel who can disseminate site-specific best practices information, andthe acknowledgment of in-house personnel for talents and attributes deemed important bythe agency. Thus, the measure can generate greater sharing around cultural issues withrespect to the organization’s service delivery approaches, while bestowing distinction(and possibly other benefits or compensation) to staff who present relevant information.The goal of a training intervention after use of the measure is to increase the culturalcompetence of staff. Moreover, post-test administration of the CCSAQ can be used as amethod of evaluating the impact of a given intervention. Trainers can review aggregatedresults and design presentations and conferences to have the greatest impact for thepersonnel within the network of provider agencies. One format is interactive, where thefacilitator or trainer asks a specific item, allows others in the audience to share theirexperiences, and develops a menu of strategies that lead to the desired behavior. Trainerscan then suggest to administrators and staff the use of literature, program examples, ortheorists and consultants that can address specific items.Practical ConsiderationsUse of the measure can generate anxiety. Staff may feel intimidated by the questions orwonder what impact their responses will have. It will be important to share with staffthat there is no way to perform poorly on the CCSAQ. The instrument is not an aptitudetest or a measure of goodness. The administration of the CCSAQ is simply a part of aprocess attempting to identify cultural blind spots in order to address them. The processshould not be experienced as ominous but rather as an endeavor that, at worst, will havebenign consequences; at best, consequences of the process can be positive for staffdevelopment and the delivery of services to all populations.It will be important for staff to know why the agency is conducting the assessment, whycultural competence is being pursued, and the agency’s commitment to this value.Agencies should circulate this information prior to distribution of the questionnaires. Itwill be important to explain to prospective respondents the importance of candor becausea desire for political correctness or other social acceptability may impact their responses.Therefore, confidentiality must be assured whenever the questionnaire is used.A central tenet of the CASSP model is that individuals are on different points on thecultural competence continuum and everyone can improve. Therefore, comparisonsbetween individuals and programs should be avoided. The goal for an organization is notto rate individual staff members but rather to help staff determine where they are, wherethey need to grow in cultural competence and help them gain that competence.10
Logistics should also be considered, including issues of distribution and collection ofsurvey forms because this is an area where confidentiality can be breached. Care shouldbe taken to protect the respondents. Assuring workers that they will have the necessarytime to complete the survey is also important. Typically, the completion time hasaveraged around twenty-five (25) minutes. It is important to record the number of formsdistributed and collected in order to compute an accurate return rate.Sampling may also prove to be an issue. In a small organization, the entire staff may bepolled. In larger organizations, a preliminary review of completed questionnaires mightbe helpful to assure that a representative sample can be presumed. Since there are twoversions of the CCSAQ, time will need to be taken to consider who gets which version.Staff who do not routinely interact with clients in clinical situations get the administrativeversion. Administrative support personnel (e.g., clerical and reception staff) as well asauxiliary personnel (e.g., drivers, food preparers, volunteers) get the direct serviceversion.Part of the survey analysis process includes answering questions related to organizationaldemographics. The demographic face sheet is designed to help agencies better describetheir work force and to identify personnel with cultural expertise who are under-utilizedas a training resource.11
12
REFERENCESAtkinson, D.R. & Hackett, G. (Eds.) (1988). Counseling non-ethnic Americanminorities. Springfield, IL: Charles C. Thomas.Chau, K.L. (1991). Social work with ethnic minorities: Practice issues and potentials.Journal of Multicultural Social Work, 1(1), 23-39.Chin, J.L. (1983). Diagnostic considerations in working with Asian Americans.American Journal of Orthopsychiatry, 53(1), 100-109.Cross, T.L., Bazron, B.J., Dennis, K.W. & Isaacs, M.R. (1989). Towards a culturallycompetent system of care. Washington, DC: Georgetown University ChildDevelopment Center, CASSP Technical Assistance Center.D’Andrea, M., Daniels, J. & Heck, R. (1991). Evaluating the impact of multiculturalcounseling training. Journal of Multicultural Counseling and Development, 70,143-150.Flaskerud, J.H. (1986). Diagnostic and treatment differences among five ethnic groups.Psychological Reports, 58, 219-235.Gallegos, J.S. (1982). Planning and administering services for minority grou
Graduate School of Social Work Portland State University . P.O. Box 751 Portland, Oregon 97207-0751 (503) 725-4040 August 1995 . 2 . Second Printing April 2002. The recommended citation for this publication is: Mason, J. L. (1995). Cultural competence self-assessment questionnaire: A manual for users.
