Transcription
VALIDATION OF THE ADOLESCENTSELF-ESTEEM QUESTIONNAIRETechnical report
2Validation of the Adolescent Self-esteem QuestionnaireSuggested citation: Hafekost K., Boterhoven de Haan K., Lawrence, D., Sawyer M.G., and Zubrick S.R.,(2017) Validation of the Adolescent Self-Esteem Questionnaire: Technical Report: Telethon KidsInstitute and the Graduate School of Education, The University of Western Australia, Perth, Australia.Acknowledgments: Young Minds Matter: The Second Child and Adolescent Survey of Mental healthand Wellbeing was funded by the Australian Government Department of Health.The authors would like to acknowledge the contribution of the families involved in YMM, andschools, students and teachers who contributed their time to this study.
Validation of the Adolescent Self-esteem QuestionnaireContentsValidation of the Adolescent . 1Self-Esteem Questionnaire . 1Technical report . 1Project Summary . 5About Young Minds Matter . 6Background . 7Project Aims. 9Methods . 10Study 1 . 10Participants . 10Procedure. 10Diagnostic Interview Schedule for Children (Youth Informant) Version IV (DISC-IV) .11Statistical Analysis . 11Study 2 . 12Participants . 12Procedure.13The Adolescent Self-Esteem Questionnaire (ASQ).13The Rosenberg Self-Esteem Scale (RSES; 1965) .13The Beck Depression Inventory – Youth (BDI-Y).13Statistical analysis . 14Results .15Study 1 .15Demographic characteristics .15Factor analysis.15Scale characteristics . 18Study 2 . 23Demographic characteristics . 23Scale characteristics . 23Missing data . 23Internal consistency and reliability . 24Construct validity . 24Findings . 25Limitations . 26Conclusion . 2733
4Validation of the Adolescent Self-esteem QuestionnaireData Access . 28References . 29Supplementary material . Error! Bookmark not defined.
Validation of the Adolescent Self-esteem QuestionnaireSUMMARYThis document describes the validation of the adolescent Self-Esteem Questionnaire (ASQ) whichwas developed for the purpose of measuring self-esteem in Young Minds Matter: the secondAustralian child and adolescent survey of mental health and wellbeing (YMM).The existing gold standard measure of global self-esteem, the Rosenberg Self-esteem Scale (RSES)was identified to be somewhat outdated, in language and concepts, for use in a contemporaryAustralian youth population. Therefore, we sought to develop a short scale which captured globalself-esteem which could be included in the youth self-report component of YMM. In order tovalidate the scale we used data from two study populations. The first was the nationallyrepresentative sample of 2,964 11 to 17 year-olds who participated in YMM. These data were usedto undertake a factor analysis of the items, and also to establish population norms for the scale. Thesecond study included 304 Western Australian school students aged between 14 and 17 years, whichwas used to compare the ASQ with the RSES and to assess the test-retest stability of the ASQ.The properties of the scale were examined, and its reliability and validity were compared to theexisting gold standard measure of self-esteem, the Rosenberg Self-Esteem Scale (RSES). Factoranalysis allowed for investigation of the underlying concepts captured by the ASQ, and populationnorms and a clinically meaningful threshold which was used to define low levels of self-esteem, werecalculated.The ASQ performed as well as the RSES. In addition, the ASQ was highly predictive of depression,suggesting good construct validity. The inclusion of the ASQ in YMM provided us with nationallyrepresentative data from which sex-specific population norms, and a clinically meaningful definitionof self-esteem, could be derived. Population norms are provided in this report, and a threshold forlow self-esteem has been identified to assist researchers using the ASQ measure with the YMMConfidentialised Unit Record File (CURF). This information may also allow the ASQ to be useful as atool in future research and practice.55
6Validation of the Adolescent Self-esteem QuestionnaireAbout Young Minds MatterMental health problems are among the most common and burdensome health conditions in childrenand young people. Without appropriate treatment, they can be persistent and disabling, and canadversely affect children’s learning and development. Young Minds Matter, the second AustralianChild and Adolescent Survey of Mental Health and Wellbeing was conducted as part of theAustralian Government Department of Health’s National Survey of Mental Health and Wellbeingprogramme. The survey was conducted to provide information to support improving mental healthservices for children and adolescents experiencing mental health problems and their families. Over6,300 families with children and adolescents aged 4-17 years from around Australia participated inthe survey. Information was collected from primary carers via a face-to-face diagnostic interview,and directly from adolescents aged 11-17 years via a self-complete questionnaire administered usingtablet computers.The main aims of the survey were to determine how many children and adolescents had which mental health problems and disorders the nature and impact of these problems and disorders how many children and adolescents had used services for mental health problems; and the role of the education sector in providing these services.Further information about YMM is available at www.youngmindsmatter.org.au
Validation of the Adolescent Self-esteem QuestionnaireBACKGROUNDYoung Minds Matter sought to collect information about mental health and wellbeing of Australianchildren and young people including identifying the prevalence of mental health problems anddisorders, the environments in which children live and the relationship of mental health andwellbeing to family and school characteristics, the impact of mental disorders, risk factors associatedwith mental health and wellbeing, and use of services and unmet need for services.Self-esteem has been identified as being associated with mental health and wellbeing, and has beenassociated with health risk behaviours that are also associated with poor mental health. Recognisingthat self-esteem is an important component of adolescent wellbeing, a measure of self-esteem wassought for inclusion in YMM. As there were a substantial number of content areas to be covered inthe YMM questionnaire, and practical constraints on the length of the questionnaire, a briefmeasure of global self-esteem was desirable.While there is no single agreed definition and theory of self-esteem, it is generally considered torepresent an individual’s overall appraisal of themselves and their value as a person 1. It isconsidered to be the evaluative and affective components of the sense of self 2, 3. Self-esteem isthought to be an assessment of competency and worth as a person in relation to others 4. Selfesteem reflects the components of self-concept that are considered to be important by theindividual. It spans across domains and roles, and consists of both positive and negative selfjudgements 5.A number of different constructs of self-esteem have been discussed in the literature. For example,global or trait self-esteem refers to an individual’s assessment of themselves overall, or generalfeelings about themselves relative to others. In contrast, domain-specific self-esteem refers toevaluations in a particular area. For example, an individual could have low academic self-esteem andhigh self-esteem regarding their athletic ability. State self-esteem is typically used to capture selfesteem in a specific moment or setting 6. While self-esteem is a topic of much research, thedifficulties in measuring self-esteem, and using self-esteem scores to accurately predict lifeoutcomes and behaviours, have been documented 6.Self-esteem is not static but rather it fluctuates in response to situational factors, and over the lifecourse. Robins and Trzeniewski note that self-esteem is typically high in childhood and adulthoodbut is lower in adolescence and old age 7. The authors attributed the decline in self-esteem inadolescence to improvements in abstract thinking, greater comprehension of abilities relative to77
8Validation of the Adolescent Self-esteem Questionnaireothers, growing awareness of body image, and academic and social challenges associated withschool transitions. This decline in self-esteem has been linked to antisocial behaviour, academicunderachievement and risk-taking behaviours such as alcohol and substance use, and early sexualactivity 8 9.Self-esteem has also been demonstrated to be associated with mental health and has beenimplicated in the diagnostic criteria for several mental health disorders including mood and anxietydisorders 3. Low self-esteem in adolescence has been associated with problem eating behaviours,difficulty with peer interactions, and suicidal ideation and it has been found to be an indicator ofpoor long-term outcomes and impairments in functioning 10 11. It is well accepted that earlyidentification and intervention for mental health problems is critical. Given that self-esteem can beseen as an indicator for how an individual approaches the challenges of adolescence, and suggestiveof mental health status, measures of self-esteem which are specifically developed for an adolescentpopulation are an important tool for identifying and assessing at-risk youth 12.Due to the link between self-esteem and outcomes such as mental health status, there are a largenumber of self-esteem measures. These measures vary depending on the construct (i.e. globalversus domain or state specific self-esteem) they aim to capture, the population of interest and thesetting in which they are used. In studies such as YMM where self-esteem is one of several conceptsthat need to be measured, short measures of global self-esteem, which have low respondentburden, are favoured. One of the most widely used measures in this setting, which is considered tobe the current gold standard measure of global self-esteem, is the Rosenberg Self Esteem Scale(RSES) 2 13. The RSES was designed in 1965 for adolescent populations. It is a short, self-reportquestionnaire which contains both positively and negatively worded items relating to the feelings aperson has about themselves 4. The scale was designed for use in population studies rather than inclinical settings 2. The RSES has been demonstrated to have sound psychometric properties and hasbeen used extensively in research 14 15. However, since the development of the RSES in the 1960s,there have been many societal changes which may impact on the relevance of the items within theRSES and their ability to capture self-esteem in surveys of contemporary adolescent population.One factor, which has substantially changed the adolescent experience over recent years, is themarked increase in access to the internet and social media. This vastly different environment fortoday’s young people, compared to previous generations, may impact on the issues relevant to selfesteem and, as a result, its measurement. For example, greater social media use and image sharingmay have altered the relationship between self-esteem and body image in young people. Recentstudies have identified body image and perception of physical appearance as one of the strongest
Validation of the Adolescent Self-esteem Questionnairesources of self-esteem in adolescents 12 16. The RSES does not include an item relating to body imageor perceptions and, as a result, may not evaluate a key factor which contributes to youth selfesteem. A further consideration is the language of the RSES. Hinkin suggests that the language ofscales needs to be familiar to the target audience, and revision of scales is needed to maintain theireffectiveness 17. The language included in the RSES could be considered somewhat outdated. Forexample, the RSES asks participants to respond to the statement ‘I feel that I’m a person of worth, atleast on an equal plane with others’. The outdated language may impact on a young person’s abilityto understand and respond appropriately to the scale and, as a result, the ability of the scale toaccurately capture self-esteem.Therefore, we hypothesised that a revised measure of self-esteem, the Adolescent Self-EsteemQuestionnaire (ASQ) would provide an improved measure for assessing self-esteem in an adolescentpopulation setting. It was anticipated that this updated measure of self-esteem, which was designedto be more relevant for young people in their current context, would allow for more accurateassessment of self-esteem and would enable better detection of youth with low self-esteem whomay be at risk of mental health problems.Project AimsThe aims of this project were to: Determine whether the ASQ was a valid and reliable measure of self-esteem in a youthpopulation Ascertain whether the ASQ performed better than current gold standard measure, the RSES,in relation to scale consistency, reliability and construct validity Determine population norms and clinically meaningful threshold scores for the ASQ whichare of use to future research and in practice.99
10Validation of the Adolescent Self-esteem QuestionnaireMETHODSValidation of the ASQ involved two studies. The first study used data collected in YMM. Factoranalysis of the ASQ items was undertaken using these data, as well as the establishment ofpopulation norms. In order to measure additional aspects of the performance of the ASQ, a secondstudy was also conducted. In this study, both the ASQ and the RSES were administered to the samegroup of students, allowing a comparison of the performance of both instruments. A subset ofstudents also completed the instrument on two occasions to assess the test-retest reliability of theinstrument.Study 1Young Minds Matter: The Second Australian Child and Adolescent Survey of Mental Health andWellbeing (YMM)18 was funded by the Australian Government Department of Health . The aims ofYMM were to determine the prevalence of mental health problems and disorders among childrenand adolescents in Australia, examine the nature of these disorders, assess the degree of functionalimpairment associated with these disorders, determine the mental health services used by youngpeople with disorders, and examine the role of the education sector in providing services to youngpeople with mental health problems and disorders.Of relevance to this analysis, YMM provided a population representative sample of Australian youthaged between 11 and 17 years. This allowed for the underlying constructs and population norms ofthe ASQ to be assessed. In addition, this sample allowed for derivation of clinically meaningfulthresholds.ParticipantsThe YMM survey had a final sample of over 6,000 families with children aged 4 – 17 years fromacross Australia. In total, 2,964 young people between the ages of 11 and 17 years completed theASQ self-esteem questionnaire within the self-report youth interview. This sample, which included1,529 boys and 1,435 girls, represents 89% of eligible youth.ProcedureThe YMM survey approached Australian households over an 11 month period in 2013/2014. YoungMinds Matter was a population representative sample of over 6,000 families. Data were collectedfrom parents using household-based interviewing. For children aged 11 years and older, where
Validation of the Adolescent Self-esteem Questionnaire 11parent and child consent was provided, additional information was collected from the young personvia a self-complete interview on a tablet computer. The parent questionnaire collected informationabout the child’s mental health, service use, and demographic and family information. The youthself-complete survey collected information on mental health status, service use, wellbeing, risktaking behaviours, use of technology, bullying, health, and self-esteem.The current analysis primarily made use of the youth self-report data. Variables of interest included:demographic information; self-esteem as measured by the ASQ; risk taking behaviours includingalcohol use, drug use and sexual activity; being a victim of bullying; self-harming behaviours; andmental health status. The primary measure of mental health status was the diagnosis of majordepressive disorder which had impacted on functioning in the previous 12 months. This diagnosiswas obtained with use of the Diagnostic Interview Schedule for Children (Youth Informant) VersionIV (DISC-IV) which is described below.Diagnostic Interview Schedule for Children (Youth Informant) Version IV (DISC-IV)The Diagnostic Interview Schedule for Children (Youth Informant) Version IV applies Diagnostic andStatistical Manual of Mental Disorders criteria to allow for the existence, or absence, of specificmental disorders to be identified in young people. The DISC-IV is a structured interview whichincludes six modules covering 22 specific psychiatric diagnoses. Only the major depressive disordermodule was included in the YMM youth self-complete survey.Statistical AnalysisStatistical analyses were carried out using SAS 9.3 (SAS Institute, Inc., Cary, NC). Confirmatory factoranalysis was performed using Mplus Version 7.4 19.Survey data was weighted using calibration on marginal totals to population figures obtained in the2011 Census of Population and Housing. All analyses took into account population weights.For the purpose of factor analysis, the data set was randomly split into two equal samples to providean analysis and test dataset. This was completed to allow for cross validation, and improving thegeneralisability of results. In order to examine the structure of the ASQ, an exploratory factoranalysis using maximum likelihood estimation and a promax rotation was performed. The number offactors was determined based on associated eigenvalues of greater than one and the shape of thescree plot. Fit statistics included measures of absolute fit (chi-square goodness of fit), parsimonycorrected fit (root mean square error of approximation (RSMEA), and comparative fit (comparativefit index (CFI)). Cut-off values against which fit was assessed were a significant RMSEA and its11
12Validation of the Adolescent Self-esteem Questionnaireconfidence interval 0.08 and CFI 0.90. Confirmatory factor analysis, using the weighted leastsquares method, was completed with the fit of both a one and a two factor model examined. Itemresponse theory analysis was used to examine the ability of items to distinguish between individualswith low and high levels of self-esteem. In order to help ascertain whether the negative and positiveitems captured two meaningful factors, the ability of these sub-scores to predict depression, whichhad impacted on functioning in the previous 12 months, was examined. The Area under the ReceiverOperating Characteristic Curve was compared to determine the relative predictive ability of theindividual subscales and the total ASQ score.Population norms were determined for ASQ scores for both males and females. In addition, ASQscores were compared to the predicted probability of the young person being identified as havingmajor depressive disorder which had impacted on functioning via the DISC using logistic regression.Generalised additive models were used rather than restricting the nature of the relationshipbetween ASQ scores and major depressive disorder to be linear or any other fixed functional form.This was undertaken with the aim of identifying meaningful a cut-point or threshold in the scalewhich could distinguish those with low self-esteem. The use of this threshold was subsequentlytested using logistic regression models. Demographic and mental health factors which weretheoretically associated with low self-esteem were also included in the model. Non-significantvariables were excluded from the model (alpha 0.05) and the most parsimonious model wasreported.Study 2The second study included 304 school students aged between 14 and 17 years. The data obtainedfrom this study allowed the ASQ to be compared to the RSES. In addition, this sample allowed thereliability, validity and test-retest stability of the ASQ to be determined.ParticipantsYoung people between the ages of 14 and 17 years were recruited from four schools in metropolitanWestern Australia. The sample comprised 160 girls and 144 boys. The young people were agedbetween 14 and 17 years old with the majority of children aged 14 (n 117) and 15 (n 133) years.Students from three schools were asked to complete the questionnaire on two occasionsapproximately two weeks apart. As the data did not contain an identifying number or name, resultsfrom the two time points were matched on school, gender and date of birth. Records which werenot unique, based on these identifying characteristics, were excluded from the analysis. The final
Validation of the Adolescent Self-esteem Questionnaire 13sample of young people who completed two questionnaires, and whose records could besuccessfully matched, was 81.ProcedureSchools were approached for permission for researchers to speak to students about theirparticipation in the study. Participant consent was sought from all students. No students refused toparticipate in the study. In addition, parental consent was required for students in one school. Oncethe required consents were obtained, participants were provided with a link to the onlinequestionnaire. Participants completed the questionnaire in the classroom, on a personal computer,and were asked to answer the questions individually, without the input of others. The questionnairecomprised three scales which were combined to form the single, online questionnaire. The threescales were the ASQ, the RSES and the Beck Depression Inventory – Youth scale. These scales aredescribed below.The Adolescent Self-Esteem Questionnaire (ASQ)A 13-item measure of global self-esteem. The ASQ includes both negatively and positively wordeditems which are on a 5-point Likert scale. Item scores were summed, with positively worded itemsreverse coded, to determine an individual’s self-esteem score. A higher score reflects higher levels ofself-esteem. The ASQ, and sample scoring code (SAS), is included in Appendix 1.The Rosenberg Self-Esteem Scale (RSES; 1965)A 10-item scale which provides a measure of global self-esteem. Items are answered on a four pointLikert scale that range from strongly agree to strongly disagree. The RSES uses both positive andnegatively worded items. Negatively worded items are reverse coded and item scores are summedto calculate a total score. Higher scores are associated with higher global self-esteem.The Beck Depression Inventory – Youth (BDI-Y)A 20-items self-report assessment that is used to identify symptoms of depression (Beck, 2008). TheBDI-Y measures children’s and adolescents’ negative view of self, their world and their future and isdesigned for young people aged between seven and 18 years old. It captures a range of thoughts,feelings, and behaviours which are characteristic of depression such as hopelessness, suicidalideation, and sleep disturbance. The BDI-Y scale is scored by summing all the items to obtain a totalscore. This raw total score is then converted into a t-score equivalent which is specific to age-groupand gender.13
14Validation of the Adolescent Self-esteem QuestionnaireWith the aim of reducing potential order effects in the data, two versions of the online questionnairewere developed. The first version included the ASQ first and RSES last (i.e. ASQ, BDI-Y, RSES). Thesecond version of the questionnaire included the RSES first and the ASQ last (i.e. RSES, BDI-Y, ASQ).As no identifying information was collected, the survey version was randomised at the classroom,rather than individual level. In addition age, sex and date of birth information was obtained.Participants from three schools were asked to complete the questionnaire on a second occasion. Thefourth school did not complete a second testing occasion due to school holidays. The time betweenthe two testing occasions was approximately two weeks. As we did not collect names on thequestionnaire, in an effort to preserve anonymity and reduce social desirability biases, anindividual’s questionnaires were matched based on school, classroom, sex, and date of birth. In thecase of duplicate date of birth and sex within a single classroom, questionnaires were excluded fromthe test-retest analysis. Questionnaires excluded from the test-retest analysis due to duplicatematching criteria remained in all other analyses.Statistical analysisThe internal reliability of the scale was determined using Cronbach’s alpha. Test-retest stability wasdetermined via correlations between time one and time two scores. In addition, paired t-testsallowed for comparisons of scores at the two time points for the same individual. Theoretically, lowself-esteem is linked with higher levels of depression and depressive symptoms. Therefore, in orderto examine construct validity, the relationship between the ASQ and BDI-Y was examined.Correlations between scores obtained on the self-esteem scales and scores obtained on the BDI-Ywere determined.
Validation of the Adolescent Self-esteem Questionnaire 15RESULTSStudy 1Demographic characteristicsDemographic characteristics of the sample, which includes weighted population estimates fromYMM, and population comparisons from the 2011 Census of Population and Housing, are displayedin Table 1. These data demonstrate that the YMM data were comparable to the Australia populationin relation to age and sex. The YMM data were weighted based on Estimated Resident Populationcounts for Australian children and adolescents. These figures adjust for under-enumeration in thecensus, and explain why the YMM weighted est
high self-esteem regarding their athletic ability. State self-esteem is typically used to capture self-esteem in a specific moment or setting 6. While self-esteem is a topic of much research, the difficulties in measuring self-esteem, and using self-esteem scores to accurately predict life outcomes and behaviours, have been documented 6.
