Transcription
CPA ExaminationHow is theUniform CPAExaminationscored?Effective April 1, 2017
2
Contents4A passing score is 754Uniform CPA Examination structure6Score scale and passing score6Score frequently asked questions and answers10 Is there more that I can readabout the Examination?3
A passing score is 75Anyone who has taken the Uniform CPA Examination, prepared for the Examination, or been involved inthe CPA licensure process knows that the passing score is 75. But very few understand what that 75 means.The AICPA regularly hears from candidates, state board representatives, educators, and others who wonderhow the Examination is scored. This non-technical overview of the scoring process will help answer the mostfrequently asked questions.Uniform CPAExamination structureThe Examination comprises four sections: Auditingand Attestation (AUD), Business Environment andConcepts (BEC), Financial Accounting and Reporting(FAR) and Regulation (REG). All four sections containmultiple-choice questions (MCQ) and task-basedsimulations (TBS). BEC has also has a portion forwritten communication tasks.Testing within each Examination section isadministered in blocks called testlets. Testletscontain operational and pretest questions.Operational questions are scored, while pretestquestions are not scored. Pretest questions areindistinguishable from operational questions.Pretest questions that meet certain statisticalcriteria are used as operational questions on futureexaminations. This strategy for pretesting questionsis common practice in high-stakes testing.MCQ testlets vary in difficulty — there are two levelslabeled “medium” and “difficult.” Within the testlets,items often vary in their difficulty levels, but acrosstestlets, those labeled “difficult” contain harderquestions on average than testlets labeled “medium.”Every candidate receives a medium testlet first.The succeeding testlet can be either medium ordifficult, depending on a candidate’s performance(for more information about multi-stage testing, seeFAQ No. 3). The scoring procedures take the difficultyof all questions into account so that candidates arescored fairly regardless of the difficulty of thetestlets they take.TBSs are case studies that allow candidates todemonstrate their knowledge and skills by generatingresponses to questions rather than simply selectingthe correct answer. TBSs typically require candidatesto use spreadsheets and/or research authoritativeliterature provided in the Examination.In addition, the BEC section contains three writtencommunication tasks. Each requires the candidate towrite a short passage, letter, or memo.4
Examination structure by sectionSectionItem types*TestletAuditing and Attestation (AUD)72 MCQsTime to complete: 4 hours8 TBSsNo. 1: 36 MCQsNo. 2: 36 MCQsNo. 3: 2 TBSsNo. 4: 3 TBSsNo. 5: 3 TBSsBusiness Environment and Concepts (BEC)62 MCQsTime to complete: 4 hours4 TBSs3 Written communicationNo. 1: 31 MCQsNo. 2: 31 MCQsNo. 3: 2 TBSsNo. 4: 2 TBSsNo. 5: 3 Written communicationFinancial Accounting and Reporting (FAR)66 MCQsTime to complete: 4 hours8 TBSsNo. 1: 33 MCQsNo. 2: 33 MCQsNo. 3: 2 TBSsNo. 4: 3 TBSsNo. 5: 3 TBSsRegulation (REG)76 MCQsTime to complete: 4 hours8 TBSsNo. 1: 38 MCQsNo. 2: 38 MCQsNo. 3: 2 TBSsNo. 4: 3 TBSsNo. 5: 3 TBSs* Each section includes 12 pretest MCQs and 1 pretest TBS. BEC also includes 1 pretest written communication task.Before appearing on the Examination, all operationaland pretest questions have passed through severalextensive and rigorous subject matter reviews toensure they are technically correct, have a single,best answer, are current, and measure relevantcontent knowledge and skill as specified in theExamination Blueprints. The blueprints specify thepercentage of questions that should be devoted toeach content area, and to each skill level. Operationalquestions have been statistically evaluated to ensurethey meet the psychometric requirements of theExamination.5
Score scale and passing scoreSection scores are reported on a scale that rangesfrom 0 to 99. A total reported score of 75 is requiredto pass each section. This is not a percentage correctscore nor can it be interpreted as a percentage. Thetotal score in the AUD, FAR and REG sections is aweighted combination of scaled scores from theMCQs and TBSs. For the BEC section, the total scoreis a weighted combination of the scaled scores fromthe MCQs, TBSs, and written communication tasks.Scaled scores on the MCQ and TBS portions of theExamination are calculated using formulas that takeinto account factors such as whether the questionwas answered correctly and the relative difficulty ofeach question.Percent of score contributed by item typeItem typeAUDBECFARREGMultiple choice questions (MCQs)50%50%50%50%Task-based simulations (TBSs)50%35%50%50%Written communication tasksn/a15%n/an/aScore frequently askedquestions and answers1. Can you compare scores of candidates who areadministered Examinations with variabledifficulty and different questions?Yes. Scores from the different administrations ofthe Examination are placed on a standard scale sothey can be compared to each other. This processaccounts for any differences in difficulty amongthe versions. All total scores are reported on theExamination 0 to 99 scale. Given the significantchanges made to the Examination (effective April2017), a new passing score will be established viathe standard setting process (see FAQ No. 14). Itshould be noted that scores for tests administeredafter April 1, 2017, are not comparable with scoresfrom prior administrations except that 75 is thepassing score. The use of a standard reporting scaleis common practice in the testing industry. Similarexamples are the SAT scale or the ACT scale.62. Are some administrations of the Examinationmore difficult than others?There may be minor differences among differentadministrations; however these differences areaccounted for during scoring. The AICPA enhancestest security by creating multiple forms of theExamination with different questions for differentadministrations. Each form is comparable but notidentical. Great care is taken to match the formsin terms of content and item difficulty. Remember,a candidate may be given questions of varyingdifficulty depending on his/her performance (seepage 2, Uniform CPA Examination structure). Thedifficulties of the questions are accounted forduring scoring. Therefore, it does not mean thatit is easier to get a higher score simply because acandidate receives easier questions.
CPA examination testlet selectionStronger performanceTestlet 1mediumWeakerperformanceTestlet 2bmediumTestlet 2amedium3. When are easier or more difficultquestions given?Candidates take two multiple-choice testlets.The first testlet is always a medium testlet. If acandidate performs well on the first testlet, he/she will get a more difficult second testlet whilethose who do not perform well on the first testletwill receive a second medium difficulty testlet. Thistesting process is called multi-stage testing (MST).Please note that the task-based simulations arenot chosen based on candidate performance onthe multiple-choice testlets. They are pre-assigned.The diagram on page 6 shows how it works. (seeFAQ No. 9 for further explanation of multi-stagetesting).4. Can I compute my score from the number ofquestions I answered correctly?No. The total reported score is a scaled valuethat takes into account both the response toand the statistical characteristics of eachquestion administered. (see FAQs No. 12and No. 16 for further information about thestatistical characteristics and how scores areproduced, mmunication5. In college some of my professors gave teststhat had questions that were worth one pointand others that were worth two points. If onestudent got five of the one-point questions right,he got five points. If another student answeredfive two-point questions right, he got 10 points.Is that what you’re doing?Conceptually, yes. But the professor assigned theweights based on judgment. In the Examination,the weights are determined from candidateresponse data using item response theory (IRT).(see FAQ No. 12 for further information about IRT)6. How do you score the writtencommunication tasks?Most responses are scored by a computer gradingprogram, which is calibrated using human scorers.In some cases, responses are scored by a networkof human graders (all CPAs). If a candidate scoreis close to the passing score, his/her writtencommunication tasks will be automaticallyre-graded by human graders. When there ismore than one grader for a response, theaverage of the scores is used as the final grade.Note: The remainder of the FAQs offer more detail about theExamination. Read on to gain a better understanding of thescoring process.7
7. Can I pass by only doing well on themultiple-choice questions?No. The portion contribution from task-basedsimulations and written communication tasks inBEC is too large. A candidate would need to getsome of the TBS questions correct in order to pass.8. Is multi-stage testing fair? Why are youusing it?Yes, it is fair. Since the characteristics of the testquestions are taken into account in the scoring,there is no advantage or disadvantage to beingassigned testlets of different difficulty. TheAICPA uses multi-stage testing because the testquestions are matched to proficiency levels, andtherefore fewer questions are needed to obtainaccurate estimates of proficiency.9. How do you decide which questions are difficultand which are medium?The difficulty levels of the test questions (andother statistics that are used to describe eachtest question) are determined through statisticalanalysis of candidate responses. At the questionlevel, difficulty is not quantified as a category(e.g., moderate or difficult), but as a numeric valuealong a scale. Testlets are classified as eithermedium or difficult based on the average difficultyof the questions within that testlet.10. Does that mean difficult testlets can have easierquestions and medium testlets can have difficultquestions?Yes. All testlets have questions ranging indifficulty. Questions in difficult testlets just havea higher average level of difficulty than those inmedium testlets.811. What if I’m well-prepared but do poorly onthe first testlet? Can I still pass?Yes, a candidate can still pass the Examinationbut will need to do better on the second testlet;and that is just the MCQ portion. Fifty percent ofa candidate’s score comes from the TBSs in theAUD, FAR and REG sections while 35% comesfrom TBS and 15% from written communicationtasks in BEC.12. What do you mean when you say “statisticalcharacteristics” of test questions?There are three statistics used to describe thequestions: Difficulty — whether the question isgenerally easier or more difficult for candidates,discrimination — how well the questiondifferentiates between more able and less ablecandidates, and guessing — the chances ofcandidates answering the question correctlyjust by guessing. The statistics are generatedwhen the questions are administered as pretestquestions and used in the scoring when thequestions are operational. The formulas forgenerating the statistics and scoring theExamination come from a scoring approachcommonly referred to as — Item Response Theory(IRT). IRT is currently being used or has recentlybeen adopted by nearly all of the large licensingexamination programs in this country.
13. Can two candidates answer the same numberof questions correctly, but get different scores?Yes, because scores depend on the characteristicsof the questions, not just how many are answeredcorrectly. For example, all else being equal, acandidate who correctly answers 10 difficultquestions would score more than a candidate whocorrectly answers 10 easy questions.14. How is the passing score set for each section?Volunteers who are licensed CPAs with recentexperience supervising entry-level CPAs participatein a passing score study. They review testquestions and how candidates performed on thosequestions in order to make judgments of what testperformance is required to ensure the protection ofthe public interest. The results of the study are usedby the Board of Examiners (BOE) as a guide whenit establishes the passing scores for each section.These passing scores are then mapped to a scoreof 75 on the scale used to report scores. Thisprocess, known as standard-setting, is commonpractice within high-stakes testing.15. How do I find out my scores for each contentarea of the Examination? The AICPA does notrelease subscores by content area, but does reportcategories of performance. Candidates should usecaution in interpreting content area performance.The subscores are calculated on fewer items andtherefore not as reliable as the final score. Theperformance comparisons of weaker, comparable,and stronger are provided to candidates as a generalindicator of performance. The subscores are notdisclosed because the AICPA does not want toconvey a sense of precision in those scores thatis unwarranted. That’s why, in the event a candidatedecides to retake any section of the Examination,he/she should study all content areas. If a candidatestudies only the areas where there is weakness, he/she might do better on those areas, but worse onothers when taking the Examination again.9
16. In general terms, what are the stepstaken to produce the reported score?Initially, for purposes of score reporting, eachcomponent (multiple-choice questions, task-basedsimulations and written communication tasks) istreated separately. For the multiple-choice andtask-based simulation components, item responsetheory (IRT) is used to obtain the scaled score foreach type of question. The multiple-choice estimateis then mapped to a score on a multiple-choicescale. Similarly, the task-based simulationAUD, FAR and REG sectionsestimate is mapped to a score on a taskbased simulation scale. The t
the CPA licensure process knows that the passing score is 75. But very few understand what that 75 means. The AICPA regularly hears from candidates, state board representatives, educators, and others who wonder how the Examination is scored. This non-technical overview of the scoring process will help answer the most frequently asked questions. MCQ testlets vary in difficulty — there are two .
