WIXLIB

For the 2012‐2013 testing year, the certification examination was administered in five 14‐day test windows:March/April, May/June, July/August, November, and February. The BOC certification examination forms consist ofscored and experimental items, with scored items in common with an anchor form. Candidates who fail are notrestricted in their retakes during the testing year.Number of Test FormsTwo sets of scored items were developed for 2012‐2013. Each scored set was assigned different experimental sets forthe year, creating six different test forms. Forms 362(7), 362(8), and 362(11) comprised scored set A, and Forms 362(9),362(10), and 362(12) comprised scored set B. Scored set A was first administered in April 2012 and equated to Form362(3), which was administered in June 2011 to 1,177 candidates. Scored set A and Form 362(3) had 70 items incommon. Scored set B was first administered in June 2012 and also equated to Form 362(3), sharing 65 items incommon.Standard Setting and Equating of Test FormsIn February 2011, a panel of 10 currently certified athletic trainers was convened to establish the performance standardto be implemented for the revised test blueprint (RD/PA6). The panel reviewed the scored questions for Forms 362(1)and 362(2) introduced in April 2011. The panel participated in three rounds of data collection and used a modifiedAngoff model, the Yes/No technique (Impara & Plake, 1997).All later forms of the examination are equated following the protocols for common‐item non‐equivalent groups designusing the Levine True Score Method Applied to Observed Scores with internal anchors (Kolen & Brennan, 2004). Thisdesign compares the performance of one group of test takers on one examination form to another group of test takerson an earlier examination form with a known cut score.The protocol for equating is to equate the current test forms to a form used within the last two years in order to avoiditem overexposure through repeated selection of the standard setting examination versions, the removal of outdated orinappropriate items, and a potential shift over time of candidate demographics and experiences that impact theperformance.Use of Scaled ScoresSince examination forms are possibly of different difficulty, providing raw scores can be misleading. As a result, manyprograms, including the ACT and SAT examinations, use scaled scores. Scaled scores are particularly useful atproviding the basis for long‐term, meaningful comparisons of results across different administrations of an examination.Scaled scores are used because, over the life of every testing program, there are situations when changes in test lengthoccur: a decision is made to assess more or fewer areas, the numbers of items that are scored versus unscored(experimental) changes, or different examination forms of different difficulty are being compared.For scaled scores, the passing standard (number of items answered correctly) on any examination form is alwaysreported as the same scaled score.The equated scores for the BOC certification examination are converted via linear transformation so that the passingstandard for all test forms are reported to candidates as 500 on a scale of 200 to 800.BOC 2012‐13 Testing Year ReportConfidential materials5

Score ReportingThe BOC provides scaled scores and pass/fail decisions to candidates approximately two weeks after closure of a testwindow. Candidates pass or fail based on their scaled score performance compared to a criterion‐referencedperformance standard.Certification Examination DevelopmentDuring 2010‐2011, new test specifications and the associated passing standard were introduced. All later forms of theBOC certification examination are equated back to this standard.Since 2006, the BOC has provided a computerized certification examination. Prior to 2007‐2008, the certificationexamination consisted of three separate components. Since that period, the certification examination has consisted ofone assessment experience for candidates. During the 2008‐2009 testing year, focused testlets were introduced to thetesting model.Two‐day meetings for item review and test form development were held in 2012 in February, July, and November. Themeetings focused on the development and review of focused testlets and the review of stand‐alone items.BOC 2012‐13 Testing Year ReportConfidential materials6

ANALYSIS OF THE CERTIFICATION EXAMINATIONCandidate PerformanceStatistics reported refer to the performance of analyzed candidates for the BOC certification examination. Statisticalreports are generated for a particular time (e.g., a test window). Some candidates are excluded from the pool ofanalyzed data, specifically those candidates who completed less than 25% of their examinations. It is likely that thesecandidates experienced problems, such as being late to the site or other issues, and therefore, their data is problematic.As of 2007, the three cohorts of candidates reported for the BOC certification examinations are:1. First‐time candidates – candidates from athletic training education programs accredited by the CAATEreported as first‐time test takers of the certification examination.2. Retakes – candidates who re‐sat for the certification examination one or more times.3. All – candidates who tested.Candidates Excluded from this ReportThe report does not include, except where noted, those candidates who were administered the BOC certificationexamination via paper and pencil or those candidates with incomplete data. As a result, the number of candidatesanalyzed for this report may not match the number of candidates who sat for the BOC certification examination. Datafrom previous years may only include two of the three cohorts.Data for individual tables also may differ due to exclusion of some candidates from the analysis for that table. Data priorto April 2007 is excluded from the remainder of this report, except where noted, because the program used to assesscandidates was not equivalent to the current BOC certification examination protocol.There were 4,950 reported administrations of the BOC certification examination during the 2012‐2013 testing year, anincrease of 1% from 2011‐2012 (4,886). Continuing an upward trend since 2008‐2009, of the 4,950 administrations,3,631 (73%) examinations were administered to first‐time candidates, compared with 66% in 2011‐12, 52% in 2010‐2011and 46% in 2008‐2009 and 2009‐2010.Pass RatesTable 1 provides annual pass rates for the BOC certification examination since 2005‐2006. Data for 2005‐2006 and2006‐2007 are for the multiple‐choice component of the three‐part assessment used by the BOC at the time. Formsprior to 2011‐2012 were administered under a different blueprint and standard, and information is provided forhistorical purposes only.BOC 2012‐13 Testing Year ReportConfidential materials7

Table 1: Number of Candidates in Three Cohorts and Pass Rates for BOC Certification Examination, 2005‐2006 to2011‐2012.1YearFirst‐timePass% PassRetakePass% PassAllPass% e 2 details the pass rates for each form by test window for the administrative year.Table 2: Passing Rates for Each Test Form for All Candidates for BOC Certification Examination, 2012‐2013.FrequencyPercentTest 2005‐2006 and 2006‐2007 data are for the multiple‐choice component only.BOC 2012‐13 Testing Year ReportConfidential materials8

Distribution of Candidate ScoresTable 3 details the overall scaled score performance for the BOC certification examination for 2012‐2013 with acomparison of the performance of candidates since 2008‐2009.Table 3: Number of Candidates in Three Cohorts, Minimum, Maximum and Average Scaled Score, Median and ModeScaled Score, and Standard Deviation (Scaled Score) for BOC Certification Examination.CohortNAvg.MedianStd. Dev.MinMaxAll 2012‐13First‐timeRetakeAll 2011‐12First‐timeRetakeAll l 2009‐106,17147648258200638All 2008‐096,13547347679200686Figure 1 presents a cumulative frequency distribution for 2012‐2013 retake and first‐time candidates. The figurerepresents the proportion of candidates who scored at a scale score or lower.BOC 2012‐13 Testing Year ReportConfidential materials9

Figure 1: Cumulative Percentage of First‐time and Retake Candidates by Scaled Score, BOC 2012‐2013.Ideally there should be a sharp increase in the cumulative proportion of candidates around the passing standard; that is,the slope of the curve would be more vertical around the passing standard. Test forms in which the slope is before orafter the passing standard would not be functioning optimally. If the candidates were generally well‐prepared for thecertification examination, a relatively constrained set of scores with no long tails to the upper or lower end of the scalewould be expected. The data in Figure 1 shows that the majority of candidates are performing consistently with thisideal. The figure also shows that first‐time candidates are more successful in their performance than retake candidates.Figure 2 provides information on the distribution of scale scores for the two cohorts of candidates.BOC 2012‐13 Testing Year ReportConfidential materials10

Figure 2: Scale Score Distribution of First‐time and Retake Candidates, BOC 2012‐2013.BOC 2012‐13 Testing Year ReportConfidential materials11

Test Form Summary StatisticsTable 4 provides test form descriptive statistics for each test window by form and retake status (see Appendix A forinformation on the statistics reported).Table 4: Summary Test Form Statistics in Scaled Scores for Candidates for BOC Certification Examination, 2011‐2012.Test WindowAprilExam lForm Totals362(7)362(8)362(9)362(10)BOC 2012‐13 Testing Year ReportConfidential materialsRetake StatusFirst TimeRetakeTotalFirst TimeRetakeTotalFirst TimeRetakeTotalFirst TimeRetakeTotalFirst TimeRetakeTotalFirst TimeRetakeTotalFirst TimeRetakeTotalFirst TimeRetakeTotalFirst TimeRetakeTotalFirst TimeRetakeTotalFirst TimeRetakeTotalFirst TimeRetakeTotalFirst TimeRetakeTotalFirst TimeRetakeTotalFirst TimeRetakeTotalFirst 5528484Std. 49543952543952534257534257474450484150524151523912

Test WindowExam Form362(11)362(12)TotalRetake StatusTotalFirst TimeRetakeTotalFirst TimeRetakeTotalFirst ,950Mean514511482494539482516539484524Std. Dev.53554149544056514154As shown in Table 4, consistent with previous test administration years, there were differences in the scaled scores foreach test window and by retake status. A Univariate General Linear Model (GLM) was conducted to assess theinteraction between test form, test window, and retake status. The results indicated that there was no statisticallysignificant performance difference by candidates on each test form, that there was no statistical interaction betweenretake status and test form (i.e., retake candidates performed the same across all test forms, as did first‐timecandidates), and that there were statistical differences in the performance of candidates for different test windows(month) and by retake status. Table 5 provides the Between‐Subjects results for the Univariate GLM.BOC 2012‐13 Testing Year ReportConfidential materials13

Table 5: Univariate Between Subject Effects Assessing Interaction Between Exam Form, Test Window, and Retake Statusfor BOC Certification Examination, 2012‐2013.SourceCorrectedModelInterceptTest WindowRetakeExam FormTest Window *RetakeRetake * ExamFormErrorTotalCorrected TotalType III Sumof SquaresdfMeanSquareSig.Partial 9542278.805FThe candidates for the June 2012 test window had the highest scaled score, although their performance was notstatistically different from April and February.Difficulty and DiscriminationDuring the test administration year, item and test form performance are reviewed at every administration. For theannual summary, the item difficulty and discrimination statistics are reported for the first administration of the twoscored sets administered. Data on the range of difficulty and discrimination statistics obtained for the firstadministration of the scored items is contained in Table 6 below.BOC 2012‐13 Testing Year ReportConfidential materials14

Table 6: Summary of Item Discrimination and Difficulty for First Forms Tested, BOC 2012‐2013.FormDifficultyDiscriminationScored SetAScored SetBAverageMedianMinimumMaximumRange 00 to 0.10.1 to 0.20.2 to 0.30.3 to 0.40.4 to 0.50.5 to 0.60.6 to 0.70.7 to 0.80.8 to 0.9 0.9TotalAverageMedianMinimumMaximumRange 00 to 0.10.1 to 0.20.2 to 0.30.3 to 0.40.4 to 0.50.5 to 0.60.6 to 0.70.7 to 0.80.8 to 0.9 , discrimination statistics for the items were acceptable, and average difficulty for the BOC certificationexamination forms was appropriate.Domain PerformanceTest validity is a concept that refers to how well an examination measures what it is designed to measure. Test forms forthe BOC certification examination were constructed according to test specifications that were based on the results ofthe role delineation/practice analysis study (RD/PA6) introduced in April 2011. This study was undertaken to define thejob‐related activities, knowledge, and skills required of entry‐level athletic trainers. To ensure that test items account forthe content areas presented in the test specifications, each item has been classified by content experts according to itsapplication to the practice domains and tasks of RD/PA6.BOC 2012‐13 Testing Year ReportConfidential materials15

Each test item has been linked to a specific content area of the test specifications, and items meet minimum standardsof criticality for work as an entry‐level athletic trainer. Thus, the procedures used to construct the BOC certificationexamination support the inference that the examination has been built to achieve its stated purpose. Consistent withthe objectives of the BOC certification examination program, the examination is designed to separate candidates intotwo distinct groups: candidates whose knowledge and skill levels are deemed acceptable for entry‐level certification as apractitioner and candidates whose level of knowledge falls below the minimum requirements for certification. Testforms for the BOC certification examination are not intended as predictors of future success within the profession.There are five performance domains in the content framework for the BOC examination, consistent with RD/PA6 uponwhich the certification examination is based. Table 7 reports descriptive statistics at the domain level using raw scores.Table 7: Domain Level Statistics for Each Test Form for All Candidates for BOC Certification Examination, 2012‐2013 (RawScores).FormNMinimumMaximumMeanStd. ionEvaluationImmediate iate iate iate iate iate CareTreatmentOrganizationBOC 2012‐13 Testing Year Repor

The major objective of the BOC's athletic trainer certification program is to establish that individuals have the knowledge and skills necessary to create and provide safe and effective athletic training services. . (e.g., LSAT and GMAT ). . Items are constructed using guidelines established by the BOC for the development and review of .