Reliability Of An On-line System To Assess . - UNT Digital Library

Transcription

RELIABILITY OF AN ON-LINE SYSTEM TO ASSESS PHYSICAL ACTIVITY BEHAVIORS IN AN ACTIVEGROUP OF KINESIOLOGY UNDERGRADUATE STUDENTSGregory KnellThesis Prepared for the Degree ofMASTER OF SCIENCEUNIVERSITY OF NORTH TEXASAugust 2013APPROVED:James R. Morrow, Jr., Major ProfessorJakob L. Vingren, Committee MemberAllen W. Jackson, Committee Member andChair of the Department of Kinesiology,Health Promotion, and RecreationJerry R. Thomas, Dean of the College ofEducationMark Wardell, Dean of the Toulouse GraduateSchool

Knell, Gregory. Reliability of an on-line system to assess physical activity behaviors in anactive group of kinesiology undergraduate students. Master of Science (Kinesiology), August2013, 42 pp., 7 tables, 1 figure, references, 32 titles.Engaging in muscle strengthening activities (MSA) as part of a physical activity programoffers health benefits. Although the merits of physical activity are well documented, manyadults fail to meet appropriate levels as recommended in the 2008 Physical Activity Guidelinesfor Americans (PAGA). To get a more complete understanding on an individual’s physicalactivity behaviors, the Tracking Resistance Exercise and Strength Training (TREST) internetbased survey was developed. The purpose of the current study was to determine the test-retestreliability of TREST items. Additionally, the prevalence of participants meeting the 2008 PAGAwas reported by gender. The survey was completed approximately two weeks apart by 224(52% male) undergraduate kinesiology students. Analysis of the survey items presented TRESTas a reliable instrument in assessing an individual’s physical activity behavior with a focus onMSA. Among the convenience sample of 445 participants (56% male) that completed the surveyin assessment #1, 73% met the 2008 PAGA minimum recommendations for MSA ( 2days/week) and aerobic activity ( 150 min MVPA). A more complete MSA and MVPA criteriawas established (requiring MSA of all seven major muscle groups) and only 32% of participantsmet this guideline. In general, men engaged in aerobic exercise and MSA more than women.These results cannot be generalized due the age, activity level, and education of the study’sparticipants. Future studies should investigate the validity of TREST items among a sample ofvarying fitness levels, races/ethnicities, ages, and educational levels.

Copyright 2013byGregory Knellii

ACKNOWLEDGEMENTSThis project would not have been possible without the support of many people. Deepestgratitude goes to my mentor, Dr. James R. Morrow, Jr., who has been tremendously helpful andoffered knowledge, guidance, and support throughout the entirety of this project. In addition, Iwould like to offer appreciation to the members of my supervisory committee, Drs. AllenJackson and Jakob Vingren. Their assistance with this project has been invaluable andinstrumental to its success.Finally, I wish to thank my family, especially my wife, Megan and my parents, for theirlove and support throughout my studies. I would not have been able to complete this projectwithout their understanding and patience.iii

TABLE OF CONTENTSPageACKNOWLEDGMENTS . iiiLIST OF TABLES AND FIGURES . vINTRODUCTION AND LITERATURE REVIEW . 1METHODOLOGY . 5Study Design. 5Survey. 5Statistical Analysis . 6RESULTS. 9Prevalence. 9Reliability. 11DISCUSSION. 13AppendicesA.2001 BRFSS PHYSICAL ACTIVITY MODULE QUESTION BANK . 26B.COMPLETE COPY OF TREST SURVEY . 29REFERENCES . 38iv

LIST OF TABLES AND FIGURESPageTables1.Frequency of MSA Participation . 102.Frequency of MSA Participation by Gender . 103.List of Exercises Demonstrated in TREST and Corresponding Muscle Group . 194.TREST Question/Answer and Appropriate Statistical Analysis . 205.Summary of Continuous Reliability of TREST. 226.Summary of Physical Activity Prevalence and Categorical Reliability of TREST . 237.Criteria for Categorization of Dichotomous TREST Data . 24Figures1.Flow chart of the questions offered in TREST. 25v

INTRODUCTION AND LITERATURE REVIEWEngaging in physical activity is one of the most effective health promoting activities anindividual can undertake to lengthen and improve quality of life (US Department of Health andHuman Services [USDHHS], 2010). Physical activity is associated with decreased risk ofpremature death from various chronic diseases such as coronary heart disease, stroke, somecancers, and type 2 diabetes, while also improving sleep quality, balance, flexibility, cognitivefunctioning, and aerobic capacity (Haskell et al., 2007). In particular, engaging in musclestrengthening activities (MSA) as part of a physical activity program offers additional survivalbenefits for adults (Brito, Ricardo,& Araujo, (in press); Schoenborn & Stommel, 2011; Williamset al., 2007). Increased muscle mass is inversely related to excess body fat and abdominal fat,further decreasing the risk of chronic diseases (Lee, Blair, & Jackson, 1999). Muscular strength isparticularly important for adults as they age in preventing falls, decreasing functionallimitations, and controlling bone density loss (Chodzko-Zajko et al., 2009; Nelson et al., 2007).Although the merits of physical activity are well documented, many Americans fail toengage in appropriate levels (Carlson, Fulton, Schoenborn, & Loustalot, 2010; Dinger &Waigandt, 1997; Tucker, Welk, & Beyler, 2011; Wallace & Buckworth, 2002). Therefore, majorhealth organizations (e.g., American College of Sports Medicine [ACSM], American HeartAssociation [AHA], Centers for Disease Control and Prevention [CDC], USDHHS, US SurgeonGeneral’s Office [Surgeon General],World Health Organization [WHO]) have deemed itnecessary to identify physical activity as a key component of a healthy lifestyle (CDC, 2010;Garber et al., 2011; Surgeon General, 2001; Thompson et al., 2003; USDHHS, 2008; USDHHS2010; WHO, 2010). In 2008, the USDHHS released the first national guidelines for physical1

activity, the physical activity guidelines for Americans (Physical Activity Guidelines for Americans[PAGA], 2008). It is recommended that adults achieve 75 minutes of vigorous to 150 minutes ofmoderate aerobic physical activity (MVPA) per week, as well as engage in MSA that aremoderate to high intensity and involve all major muscle groups (i.e., legs, hips, back, chest,abdomen, shoulders, and arms) on two or more days a week (PAGA, 2008). No specific durationor intensity for MSA are specified, other than the exercises should be completed to nearexhaustion.The CDC’s Office of Surveillance, Epidemiology, and Laboratory Services monitors thepopulation’s progress toward meeting the 2008 PAGA through the Behavioral Risk FactorSurveillance System (BRFSS [CDC, 2012]). The BRFSS is a yearly statewide survey that hascontinuously collected data on residents of all 50 states, as well as the District of Columbia,Puerto Rico, the U.S. Virgin Islands, and Guam since 1984. The BRFSS collects information onparticipants’ health risk behaviors, preventative health practices, and health care access.Several studies have analyzed the reliability and validity of the BRFSS questions to assessphysical activity. Everson and colleagues (2003) examined test-retest reliability of the BRFSSamong a large, diverse group of women, and found an ICC 0.69. In 2005, Everson and McGinnexamined the ability of the BRFSS to repeatedly categorize adults as meeting the minimumrecommendations, insufficiently active, or inactive and found an intraclass correlationcoefficient (ICC) of 0.44. Unfortunately, the 2001 version of the BRFSS used in both of Everson’sstudies did not include any MSA specific questions. A copy of the 2001 BRFSS physical activitymodule question bank can be found in Appendix A.2

In a later study, Yore et al. (2007) examined the reliability of the MSA query that reads:In a usual week, do you do any activities designed to increase muscle strength or tone,such as lifting weights, pull-ups, push-ups, or sit-ups?The strengthening question produced excellent test-retest reliability (κ 0.85 – 0.92) andmoderate validity (κ 0.40 – 0.52) among a group of Columbia, South Carolina adults. Thesereliability results were similar to Brown et al.’s (2004) study of Australian adults, using the sameBRFSS question on MSA (ICC 0.69 – 0.84). The positive results from reliability and validitystudies such as these offer support to conclusions that 18% of adults, and as much as 42% ofwomen meet the 2008 physical activity guidelines for Americans MSA recommendations(Carlson et al., 2010; Trudelle-Jackson, Jackson, & Morrow, 2011). There are, however, severallimitations to these studies which may call into question their results. Brown et al. (2004) andYore et al. (2007) defined the MSA criteria as performing MSA two or more times per week, butfailed to include the portion of the criteria that states that each of the seven major musclegroups should be exercised in this bi-weekly activity. If the respondents reported participationin any muscle strengthening activity at least two times per week, then they were noted asmeeting the criteria, not considering if each of the seven muscle groups was activated in thatweek. Although several studies have produced excellent test-retest reliability for physicalactivity behavior queries, some of the questions required participants to rely uponretrospective recall. Retrospective recall has been found to lack reliability and validity in healthrelated surveys where the answers may be altered to achieve a more socially desirable behavior(Shepard, 2003). Current physical activity behavior assessments fall short in determining ifindividuals achieve at least 150 minutes per week of MVPA and two days per week of MSA thatengage all seven major muscle groups. To address these shortcomings and to get a more3

complete understanding of an individual’s physical activity behaviors, the Tracking ResistanceExercise and Strength Training (TREST) survey was developed. TREST is an internet basedsurvey, developed at the Cooper Institute and the University of North Texas with support fromthe Discovery Foundation, utilizing visual modeling of specific muscle strengthening activitiesfor all seven major muscle groups. The purpose of this study is to determine test-retestreliability of TREST responses in undergraduate individuals when reporting physical activity witha focus on MSA.4

METHODOLOGYStudy DesignA convenience sample of approximately 800 participants was recruited in January 2013from undergraduate kinesiology courses at the University of North Texas (UNT) in Denton,Texas. Some participants were offered extra credit points in their respective kinesiology course(dependent upon the course instructor) for participation in the study. Before completing thesurvey, each person read and agreed to the informed consent notice, which had been approvedby the UNT institutional review board for protection of human rights.Participants were sent a link to the tracking resistance exercise and strength training(TREST) survey via their online course management platform (Blackboard). The link directed theparticipant to an informed consent notice, and after agreeing to the details outlined, theparticipant completed the survey. After a two-week period, participants were sent another linkvia Blackboard to the same TREST survey. To assure they were responding in the most truthfulmanner possible, participants were not made aware at any point during recruitment nor duringthe study that the intent of the study was to determine reliability or that they would be wouldtaking the same survey two times.SurveyThe TREST survey consists of three sections. Section I includes demographic questions (n 5), and questions on the participant’s general physical activity behaviors (n 12). Thesegeneral physical activity questions are similar to previously used Behavioral Risk FactorSurveillance System (BRFSS) physical activity questions regarding moderate and vigorousexercise, and also include items on muscle strengthening activities (MSA), yoga, Pilates,5

resistance bands, flexibility, calisthenics, leisure time physical activity, and time spent sitting.The answer to the MSA query in Section I determines if Section II is offered to the respondent.Section II aims to identify specific resistance exercise strength training (REST) behavior. Itcontains 15 yes/no questions that determine if the participant performs a muscle strengtheningexercise. Each of the seven major muscle groups is addressed through various exercises. Foreach exercise, accompanying looping video files (graphics interchange file) demonstrate theexercise. This ensures that if the respondents are not familiar with the terminology used, theyare still able to see what exercise is being asked of and can more accurately respond to thequestion. For each of the exercises, if the respondents answer “yes” to performing that type ofexercise, then a follow up question is asked regarding the frequency and intensity of theexercise (i.e., how many repetitions, sets, and days per week they perform each exercise, and atwhat intensity they perform the exercise [scale 1-10]). The last portion of the survey, Section III,asks questions on the respondent’s history of MSA (n 1), type of environment the participantengages in MSA (n 2), if the participant’s occupation requires MSA (n 1), and finally an openended question limited to 200 characters that allows the respondents to detail any MSA notincluded in the TREST survey. The complete TREST survey is available in Appendix B. See Figure1 for a flow chart of the questions offered in the survey.Statistical AnalysisDescriptive statistics for the distribution of participants meeting each of the 2008physical activity guidelines for Americans (PAGA) were reported by gender. Musclestrengthening activity was examined on two levels based on the 2008 guidelines: minimummuscle strengthening activities (MSA) guideline reported (i.e., engaging in MSA two or more6

times per week); expanded MSA guideline (i.e., engaging in MSA two or more times per weekthat includes all seven major muscle groups [legs, hips, back, chest, shoulders, arms, andabdominals]). Prevalence of participating in all other physical activities (i.e., Section I: GeneralPhysical Activity; Section II: Resistance Exercise and Strength Training [REST]) was reported onthe basis of if the individual participated in that activity in a typical week, with no considerationof meeting the 2008 PAGA. For a list of exercises and corresponding muscle group used todetermine if the participant met the qualification for all seven major muscle groups, refer toTable 3.Participant’s aerobic physical activity behavior was categorized as meeting themoderate-vigorous aerobic activity (MVPA) recommendation by engaging in at least 150minutes per week of moderate-intensity aerobic activity, at least 75 minutes per week ofvigorous-intensity aerobic, or an equivalent combination of 150 minutes of MVPA where twominutes of moderate physical activity equals one minute of vigorous physical activity. Only theresults from the first administration were used to determine the prevalence of participantsmeeting the 2008 PAGA. Repeated measures (test-retest) reliability was calculated for 22 of thetracking resistance exercise and strength training (TREST) survey questions, including sevengeneral physical activity questions and 15 REST questions. Each of the general physical activityquestions (Section I) analyzed included a yes/no portion with follow-up questions to a “yes”response (i.e., how many days per week, how many minutes per day). The REST questions(Section II) also included a yes/no question and the follow-up portion included how many daysper week, number of sets, number of repetitions, and relative intensity. For the general physicalactivity questions continuous data, a total number of minutes per week was calculated by7

multiplying the number of days per week by the number of minutes per day. Data wereanalyzed for reliability using appropriate measures (Morrow, Jackson, Disch, & Mood, 2011) fordichotomous data (i.e., yes/no) and continuous data (i.e., total number of minutes per week).Agreement between the first and second administrations of dichotomous data were analyzedusing percent agreement, chi square (x²), phi coefficient (φ), and kappa coefficient (κ). For thecontinuous data, alpha coefficient (α) was used to compare the administrations. The questions,with appropriate statistical analysis administered for test-retest reliability, are listed in Table 4.Analyses were performed using SPSS 19 (SPSS Inc, Chicago, IL).8

RESULTSPrevalenceThe 445 participants of Assessment 1 had a mean SD age of 21.4 4.1, a mean SDbody mass index (BMI) of 25.2 5.1, and 56% (n 248) were male. Using the 2008 PAGAcriteria, 85% of the respondents self-reported meeting the moderate-vigorous physical activity(MVPA) recommendation, and 73% met the standard for minimum muscle strengtheningactivities (MSA). With the expanded MSA criteria, 32% of the participants (38% male, 24%female) met the minimum recommendations to fully conform to the 2008 PAGA. For each ofthe categories (MVPA and MSA), prevalence was higher among men than women. Among thegeneral physical activity questions in Section I, including yoga, Pilates, exercise bands andflexibility exercises, females had higher prevalence of self-report participation than males.Among the resistance exercise and strength training (REST) questions, differences in prevalenceby gender were found to depend on the area of the body. Females were more likely to engagein exercises of the lower body and trunk, including abdominals, inner and outer thigh, and legs(i.e., leg extensions [quadriceps] and leg curls [hamstrings]). Males were more likely to engagein strength training of the upper body including, chest, back, shoulders, and arms. Men did,however, show a higher participation in exercises of the calves and legs and hips (i.e., squatsand/or leg press). Most of the differences in prevalence by gender were small except inexercises of the inner and outer thigh, where 25% and 23% of females engaged in this activityversus 11% and 10% of males, respectively, and exercises of the chest (75% males, 49%females). Analysis of frequency evidenced that most males and females participated in MSAthree days per week (29% males, 31% females, 30% overall). Tables 1 and 2 show the complete9

frequency of MSA participation by gender and overall, respectively.Table 1Frequency of MSA ParticipationDays 611.716.929.621.919.46.44.2100.0Table 2Frequency of MSA Participation by GenderDays 023.67.76.7100.02.029.430.719.013.74.60.7100.010

ReliabilityThe data were initially analyzed to determine the participants that had completed bothsurveys completely. First, responses with missing data from the first page of the survey (noentries for height, weight, age, gender) were deleted from the data set (Assessment 1: n 24;Assessment 2: n 17). Responses were also eliminated that did not complete the final page ofthe assessment to remove all the individuals that may have started the survey but failed tocomplete the entire survey (Assessment 1: n 9; Assessment 2: n 7). A total of 231 individualscompleted Assessments 1 and 2. After constructing scatterplots of demographic data (i.e.,height, weight, age, gender) between Assessments 1 and 2, seven additional outlier responseswere deleted. It is possible that these individuals were arbitrarily responding in order tocomplete the survey in the most timely manner possible. After completing this portion of thedata analysis, 224 individuals had completed both surveys, 52% male (n 117), 48% female (n 107). The first and second administrations were taken two weeks apart (mean SD 14.6, 3.7days).The TREST survey is reliable in categorizing the participants into meeting the 2008physical activity guidelines for Americans (PAGA) for moderate-vigorous physical activity(MVPA), for meeting the minimum muscle strengthening activities (MSA) guideline, for meetingthe minimum MSA guideline and MVPA, for meeting the expanded MSA guideline, and formeeting the expanded MSA guideline and MVPA. Among these classifications, the percentagreement was greater than 79%, all had significant chi square values indicating a relationshipbetween the responses (p 0.001), all had phi coefficients and kappa statistics greater than0.56. Agreements between the two assessments when categorizing the individuals not meeting11

the 2008 PAGA (i.e., did not meet minimum MSA and did not meet the MVPA guideline) had apercent agreement of 91%, but not a significant chi square value (p 0.19), and only κ 0.06.Among the other physical activity questions in Section I, the TREST survey was most reliable indetermining participation in yoga and Pilates. Both questions had a percent agreement greaterthan 92%, κ 0.68, φ 0.68, and significant chi square value (p 0.001). The moderate physicalactivity question had a percent agreement of 81% and significant chi square value (p 0.001),but only φ 0.29 and κ 0.28, marking the only instance of kappa being less than 0.41. Each ofthe 15 resistance exercise and strength training questions from Section II of the survey, hadpercent agreement between 75 and 94%, significant chi square values (p 0.001), and phicoefficients and kappa statistics greater than 0.47.The reliability of total minutes per week of muscle strengthening activities reported wasα 0.80. Agreement of the total minutes of MVPA reported was α 0.71, while flexibilityexercises and moderate physical activity only had alpha values of 0.52, and 0.67, respectively.12

DISCUSSIONParticipation in moderate-vigorous physical activity (MVPA [85%]) and musclestrengthening activities (MSA [73%]) among this sample is higher than other what Carlson et al.(2010) found among participants of similar age (18-24 years), education level (some college),geographic location (southern United States) and body mass index (BMI) category (25 - 30).That study found that 13-16% met the MVPA guideline, and 20-30% met the MSA guideline.Additionally, participation in in yoga and Pilates among the current study’s sample is moreprevalent (13% and 8%) than has been found in previous studies of larger samples of US adults(5% and 2% [Birdee et al., 2008; Chang, 2000]). This is most likely because the participants inthe current study were recruited from undergraduate kinesiology courses and therefore arepotentially more interested in physical activity, while also aware of the benefits of physicalactivity through their coursework and career interests. The current study established a criterionfor determining if individuals met the MSA portion of the 2008 physical activity guidelines forAmericans (PAGA) that had not been used in previous research. The MSA portion of the 2008PAGA states that individuals should engage in at least two days per week of MSA. Additionally,it is recommended that MSA incorporates all seven major muscle groups. Previous studies haveclassified individuals as meeting the guideline if they met the threshold for MVPA and if theydid any MSA at least two days per week, regardless of the seven major muscle groups (Carlsonet al., 2010). If this criteria is applied to the current study’s participants, 73% of the respondentsmeet the minimum guideline. When the more complete criteria, termed expanded MSA andMVPA ( 2 days/wk of MSA including all 7 muscle groups and 150 minutes MVPA), is applied,only 32% of the individuals meet the criteria. The more complete criteria of the 2008 PAGA13

guideline resulted in far fewer participants meeting the guideline in this study’s highly activepopulation. This is an important finding considering Carlson et al. (2010) found 18% of a morediverse and less active population sample met the 2008 PAGA for MSA and MVPA.The reliability of the tracking resistance exercise and strength training (TREST) survey’sdichotomous data was categorized (see Table 7) as poor, fair, moderate, strong, and almostperfect depending on criteria for percent agreement, chi square (x²), phi coefficient (φ), andkappa coefficient (κ; Viera & Garrett, 2005). The reliability of categorizing the participants intomeeting the 2008 PAGA was moderate for MVPA, strong for the minimum MSA guideline, andmoderate for expanded MSA and MVPA guideline. Although the reliability of the survey indetermining those individuals that did not meet the MSA and MVPA criteria was poor tomoderate, the percent agreement for those categories was over 89%. Among the generalphysical activity questions, only the moderate physical activity question was less than moderate(fair), while yoga and Pilates questions had strong reliability. The most unique portion of theTREST survey, the resistance exercise and strength training (REST) questions, had moderate tostrong reliabilities. The reliability of continuous data (i.e., total minutes per week) between thetwo assessments was considered acceptable if Cronbach’s alpha was greater than 0.70(Nunnaly, 1978). Total minutes per week of flexibility exercises (α .52) and moderate physicalactivity (α .67) did not reach the acceptable threshold. Each of the other general physicalactivity questions had acceptable reliabilities. In general, the TREST survey’s dichotomousquestions have moderate to strong evidence for test-retest reliability, and is reliable fordetermining the total minutes per week of physical activity.14

In 2007, Yore and colleagues tested the reliability of the behavioral risk factorsurveillance system (BRFSS) items used to assess physical activity behaviors, including the MSAcriteria. They found the BRFSS to have reliabilities of κ 0.85 and 0.92 with percent agreementsof 0.93 and 0.97 in assessing individuals meeting the MSA criteria, and κ 0.67 and 0.84 withpercent agreements of 0.87 and 0.93 for the MVPA criteria. The multiple kappa coefficients andpercent agreements are due to the difference in days between the assessments. The BRFSS wasadministered three times; Assessment 1, five days after Assessment 1, and again 10-19 daysafter Assessment 1. The reliabilities were consistently higher for the Assessment 1 to five dayAssessment than for the Assessment 1 to 10-19 day Assessment. The current study found thesame MSA criteria ( 2 days/week of MSA) to have κ 0.70 with percent agreement of 0.92,and the MVPA criteria ( 150 min MVPA) to have κ 0.57 with percent agreement of 0.82.When comparing the current study’s data to the Assessment 1 to 10-19 day Assessment, wesee that Yore et al.’s reliabilities are higher. The TREST’s MSA criteria items’ percent agreementwas slightly less (92% vs. 93%), but the difference in kappa statistic was greater (0.70 vs. 0.85).Yore et al. also showed the BRFSS had better reliability results for the MVPA criteria thanTREST. It should be noted that in Yore et al.’s study, the sample size was small (N 60), butmore diverse in terms of age and fitness level than the current study’s sample.There are several limitations to the current study that could account for the lowerreliability than Yore et al. (2007) found using the BRFSS to assess MSA and MVPA. The TRESTsurvey takes approximately 15-20 minutes to complete, so it is relatively time consuming andcould lead to some user fatigue. Since the survey has several follow-up questions to a “yes”response, there is the possibility that the participants learned that a “no” response moved15

more quickly through the survey, and therefore began answering “no” so that they couldcomplete the survey in a more timely fashion. It is theorized that this learning effect may havedeveloped during Assessment 2. This notion of answering “no” to all the questions would becompounded by the fact that most of the participants were offered extra credit in theirkines

such as lifting weights, pull-ups, push-ups, or sit-ups? The strengthening question produced excellent test-retest reliability (κ 0.85 - 0.92) and moderate validity (κ 0.40 - 0.52) among a group of Columbia, South Carolina adults. These reliability results were similar to Brown et al.'s (2004) study of Australian adults, using the same