Transcription
2021A Brief Review ofParticipation inMOCA-Peds in 2021With comparisons to datafrom 2020 where possibleIn 2021, the American Board of Pediatrics (ABP) continued the Maintenance of Certification Assessment for Pediatrics(MOCA-Peds) program. MOCA-Peds is an online, non-proctored longitudinal assessment of pediatric medical knowledgedeveloped as an alternative to the proctored exam given in a secure testing center. Achieving a passing score in MOCA-Pedsduring one's Maintenance of Certification (MOC) cycle meets the Exam requirement (Part 3) and provides Self-Assessment(Part 2) credit.2021 MOCA-Peds Cognitive Exam (Part 3) Participation by Exam cinePediatric ic Critical CareMedicinePediatric alPediatrics201Pediatric Nephrology122Child Abuse Pediatrics181Pediatric Endocrinology117Pediatric Rheumatology53Pediatric HospitalMedicine43* Includes MOCA-Peds Exam (Part 3) participants who answered at least 80% of all available questions. Source: ABP Certification Management System.Almost 39,000 pediatricians were eligible to participate in MOCA-Peds for Part 3 credit in 2021. 25,281 pediatricians (73.4% of total eligible) completed 80% or more of available questions in MOCA-Peds for GeneralPediatrics. This was an increase of over 18,000 from 2020. This increase was anticipated as individuals rotate intoMOCA-Peds. 3,347 pediatric subspecialists (77.9% of total eligible) completed 80% or more of available questions in MOCA-Peds inone of the eleven subspecialty examinations offered in 2021. This was an increase of 2,278 participants from 2020.See table below for more details. The number of subspecialists participating in subspecialty exams is expected toincrease as MOC cycles continue to renew each year and additional subspecialty exams become available.These statistics do not account for duplications of individuals taking MOCA-Peds in more than one area.1
2021 MOCA-Peds Exam (Part 3) Question Completion Rates by Exam AreaMOCA-Peds was offered across numerous exam areas including those certified in General Pediatrics and in the 11 pediatricsubspecialties offered in 2021 (Child Abuse Pediatrics, Developmental-Behavioral Pediatrics, Neonatal-Perinatal Medicine,Pediatric Critical Care Medicine, Pediatric Endocrinology, Pediatric Gastroenterology, Pediatric Hospital Medicine, PediatricInfectious Diseases, Pediatric Nephrology, Pediatric Pulmonology and Pediatric Rheumatology). Eligibility to participate wasdetermined based on each pediatrician's MOC cycle.The majority of participants in MOCA-Peds, regardless of exams areas offered, completed 80% or more questionsadministered in 2021 (see below).Total numbereligible toparticipateAnsweredone or l PediatricsChild Abuse Pediatrics154Developmental-Behavioral Pediatrics255Neonatal-Perinatal Medicine1,683Pediatric Critical Care Medicine386Pediatric Endocrinology151Pediatric Gastroenterology454Pediatric Hospital Medicine50Pediatric Infectious Diseases488Pediatric Nephrology240Pediatric Pulmonology370Pediatric Rheumatology68Answered atleast 80% 7%35478%4386%35673%18175%29279%5378%Answered 77%35378%4284%35172%17774%29179%5378%Source: ABP Certification Management System.Participation Demographics: Exam (Part 3)*GenderMedical School LocationAgeFemaleAmerican Medical School Graduate 3940–4950–5960–6970 70%Male78%International Medical School Graduate33%22%19%34%30%15%2%* Includes MOCA-Peds Exam (Part 3) participants in General Pediatrics and the 11 available subspecialties who answered at least 80% of all availablequestions. Source: ABP Certification Management System.2
Percent Currently Meeting Passing Standard by Exam Area and QuestionsAnsweredThe following figure provides a breakdown of the percent of pediatricians currently meeting the passing standard inMOCA-Peds in 2021 (i.e., achieved a scaled score of 180 or above across all questions offered that year). The datareflected below are not the cumulative passing rates for all participants. Final exam scores are calculated over aperiod of four years, with the four lowest-scoring or skipped quarters being dropped automatically for each individual.Two additional quarters were dropped in 2020 due to the impact of COVID-19 on health care.Answered at least 80%of questionsAnsweredall questionsGeneral Pediatrics95%96%Child Abuse Pediatrics98%98%Developmental-Behavioral Pediatrics96%95%Neonatal-Perinatal Medicine100%100%Pediatric Critical Care Medicine93%94%Pediatric Endocrinology91%91%Pediatric Gastroenterology91%91%Pediatric Hospital Medicine100%100%Pediatric Infectious Diseases96%96%Pediatric Nephrology87%88%Pediatric Pulmonology97%97%Pediatric Rheumatology96%96%Exam AreaSource: ABP Certification Management System.Includes all MOCA-Peds participants answering at least 80% of their MOCA-Peds questions in General Pediatrics and the 11 available subspecialties.More information about scoring can be found at: lecting on your experience with MOCA-Peds in 2021, how much time did youspend preparing to answer questions?E.g., studying, reviewing/reading featured readings or other literature, past questions.The majority of participants( 73%) studied less thanfive hours before startingMOCA-Peds questions.Note that those whoselected "I did not preparein advance" were notoffered the study resourcequestion on the followingpage. 20 hours10 to 20 hours5 to 10 hours2 to 5 hours8%8%11%16%35%I did not prepare in advance202112%10% 1 hour1 to 2 hoursSource: Includes data from survey respondents who were participating for Part 3 credit in any practice area. Data from General Pediatrics andsubspecialty exam participants are presented in aggregate.3
Please indicate the resources you used to prepare to answer MOCA-Peds in 2021?Note: Participants could select all that apply, so the percentages will add up to more than 100%.MOCA-Peds: Featured readings available in Exam Content64.8%UpToDate54.7%MOCA-Peds: Question history/answer feedback (rationale and references)54.6%MOCA-Peds: Learning objectives available in Exam Content54.0%AAP’s PREP products (e.g., PREP The Curriculum)33.7%General search engines (e.g., Google, Yahoo, Bing)31.2%Free information website (e.g., WebMD, Mayo Clinic, Wikipedia)30.1%Peer-reviewed publications (e.g., Pediatrics, JAMA, NEJM)29.7%Government website (e.g., CDC, NIH, NICHD)19.1%Textbooks (hardcopy or digital)18.2%Professional association websites (e.g., AAP, American College of Cardiology)16.4%Personal notes16.1%14.7%Red Book Online12.4%Board review guide or textbookOther resource not listed hereBoard review course (online or in-person)Source: Includes data fromsurvey respondents who wereparticipating for Part 3 creditin any practice area. Datafrom General Pediatrics andsubspecialty exam participantsare presented in aggregate.7.7%4.8%What reference material(s) did you use while answering MOCA-Peds questions in 2021?Note: Participants could select all that apply, so the percentages will add up to more than 100%.I did NOT use any resources during 202110.9%UpToDate54.0%General search engines (e.g., Google, Yahoo, Bing)44.4%42.0%MOCA-Peds: Featured readings available in Exam Content32.1%Free information website (e.g., WebMD, Mayo Clinic, Wikipedia)25.4%Peer-reviewed publications (e.g., Pediatrics, JAMA, NEJM)22.0%Government website (e.g., CDC, NIH, NICHD)Professional association websites (e.g., AAP, American College of Cardiology)15.6%Personal notes14.6%Textbooks (hardcopy or digital)12.8%Red Book Online12.7%AAP’s PREP products (e.g., PREP The Curriculum)Board review guide or textbookOther resource not listed here44410.3%8.7%4.9%Source: Includes data fromsurvey respondents who wereparticipating for Part 3 creditin any practice area. Datafrom General Pediatrics andsubspecialty exam participantsare presented in aggregate.
Reflecting on your participation in MOCA-Peds in 2021, please estimate thetotal amount of time you spent doing MOCA-Peds?E.g., including study preparation time, looking up references, answering questions, reviewing feedback.Time spent participating inMOCA-Peds — whether studyingmaterial prior to participationor in taking questions — variedgreatly among participants.Those certifying in two ormore areas reported moretime spent participating thanthose only certifying in onearea, which was not surprising.Overall, 66% of pediatricianstook less than 10 hours, while34% took 10 or more hours tocomplete MOCA-Peds in 2021. 40 hours20 to 40 hours2 hours5.5%11%11%2 to 5 hours10 to 20 hours18%202128%27%5 to 10 hoursSource: Includes data from survey respondents who were participating for Part 3 credit in any practice area. Data from General Pediatrics andsubspecialty exam participants are presented in aggregate.Reflecting on your participation in MOCA-Peds over time, please complete thefollowing statement(s) with the most appropriate ending.DecreasedNeither increasednor decreasedIncreasedThe ease of using the MOCA-Pedssystem has 5.1%37.2%57.7%My comfort level answeringquestions in MOCA-Peds has 4.8%30.9%64.3%The amount of time I spend onMOCA-Peds has 19.2%67.8%13.0%My ability to fit MOCA-Peds into myregular schedule has 12.4%59.1%28.6%2.8%37.3%59.9%The amount I have learned byparticipating has.Source: The survey response scale has been collapsed to a 3-point scale with Decreased comprised of the responses of Significantly decreasedand Decreased, and Increased comprised of Increased and Significantly increased. The figure includes data from 2021 survey respondentswho completed all quarters in MOCA-Peds and were participating for Part 3 credit, regardless of exam area. Data from General Pediatrics andsubspecialty exam participants are presented in aggregate.55
Feedback on MOCA-Peds QuestionsSurvey respondents were asked to provide their level of agreement with a series of items designed to gather feedback onpediatricians’ experience with MOCA-Peds, including question difficulty and relevance. Responses are shown for 2020 and2021 for each item.Neither AgreeDisagree nor DisagreeQuestion difficultywas appropriate.20202020 N 9,1472021 N 18,38320219%7%Agree14%77%11%82%Neither AgreeDisagree nor DisagreeQuestions assessed myclinical judgment, goingbeyond factual recall.20202020 N 9,1472021 N 18,38320219%7%Agree14%78%11%89%Neither AgreeDisagree nor DisagreeI had enough time toanswer each question.2020 N 9,1472021 N 18,383Featured readings werea valuable part of thelearning experience inMOCA-Peds in 2021.2020 N 9,1472021 N %22%57%10%11%22%59%8%20202021Neither AgreeQuestions in the Generalnor DisagreeDisagreePediatrics version ofMOCA-Peds were relevant 2020to General Pediatrics.7%9%2020 N 9,1472021 N 18,383N/A, I have notcompleted questionsin this areaNeither Agreenor DisagreeAgreeN/A, I have notcompleted questionsin this area83%1%20217%9%77%668%
Questions in thesubspecialty version(s) ofMOCA-Peds were relevantto my subspecialty.2020 N 1,1142021 N 18,383Neither Agreenor DisagreeDisagreeN/A, I have notcompleted questionsin this areaAgree20205%8%77%10%78%8%20217%Questions in MOCA-Pedswere relevant to mypractice.20202020 N 11,4012021 N 18,38320219%DisagreeNeither Agreenor DisagreeAgree16%17%63%13%16%N/A, I am notcurrently practicing4%69%2%NOTE: The survey response scale has been collapsed to a 3-point scale with Disagree comprised of the responses of Strongly Disagree and Disagree,and Agree comprised of Agree and Strongly Agree. The figure includes data from survey respondents who completed all quarters in MOCA-Pedsand were participating for Part 3 credit, regardless of their exam area. Data from General Pediatrics and subspecialty exam participants arepresented in aggregate.Overall Quality of the Questions in MOCA-PedsSurvey respondents were asked to rate theoverall quality of MOCA-Peds questions basedon their experience participating in 2021.*The majority of respondents (56.7%)indicated the quality of questions was “VeryGood” or “Excellent,” with more than 88.2%indicating MOCA-Peds questions were “Good”or better. This was up from 51% and 86%respectively in 2020.* Includes data from survey respondents who wereparticipating for Part 3 credit in any practice area.Data from General Pediatrics and subspecialtyexam participants are presented in lent
Quality of the Questions in MOCA-Peds by Exam AreaSurvey respondents were asked to rate the overall quality of MOCA-Peds questions based on their experience participatingin 2021. Below, we have categorized responses by each participant's exam area.*89%General Pediatrics92%Child Abuse Pediatrics82%85%Developmental-Behavioral PediatricsNeonatal-Perinatal MedicinePediatric Critical Care Medicine82%Pediatric Endocrinology82%82%Pediatric Gastroenterology70%77%Pediatric Hospital MedicinePediatric Infectious DiseasesPediatric Nephrology86%Pediatric Pulmonology82%86%78%Pediatric RheumatologyMore than one subspecialty050100* Includes data from survey practice area respondents who were participating for Part 3 credit by exam area. Data presented show the percentage ofparticipants by each exam area who indicated the quality of the exam was “Good”, “Very Good”, or “Excellent.” The percentage reported for GeneralPediatrics includes those who were only participating in MOCA-Peds General Pediatrics and not also participating in a pediatric subspecialty.Learning and Applying KnowledgeSurvey respondents were asked if they learned, refreshed, or enhanced their knowledge from participation in MOCA-Pedsin 2021. A follow-up question was then asked to gauge if and how pediatricians were able to apply what they learned totheir clinical practice.Did you learn, refresh, or enhance your medical knowledge based on participatingin MOCA-Peds?The majority of respondents (91%) indicated they had learned, refreshed, or enhanced their medical knowledge as aresult of participating in MOCA-Peds.* This is roughly equivalent to what pediatricians said in 2020 (90%).Unsure 5%No 5%Unsure 5%No 4%20202021Yes 90%Yes 91%* Includes data from survey respondents who completed all quarters in MOCA-Peds and were participating for Part 3 credit, regardless of exam area.Data from General Pediatrics and subspecialty exam participants are presented in aggregate due to small sample sizes in some subspecialties.8
Were you able to apply any of what you learned from MOCA-Peds in yourclinical practice?The majority of survey respondents indicated they had either already applied what they had learned from MOCA-Peds totheir clinical practice (62.7%), or they planned to do so (20%) moving forward. These numbers were slightly higher thanin 2020.*2020202159%Yes, I have alreadyNo, but I plan to moving forward62%21%No, because my practice area is different thanthe focus of the questions in MOCA-Peds20%14%No, I am not clinically active4%No, for another reason2%11%4%2%* Includes data from survey respondents who were participating for Part 3, regardless of exam areas and indicated they had learned from MOCA-Pedsin a previous question (n 18,383). Data from General Pediatrics exam and subspecialty participants are presented in aggregate due to small samplesizes in some subspecialties.How likely are you to recommend MOCA-Peds to a fellow pediatrician?Survey respondents were asked to rate, on a scale from 1 to 10, where 1 “Extremely unlikely” and 10 “Extremelylikely,” how likely they would be to recommend MOCA-Peds to a fellow pediatrician. The majority indicated that theywould be “Extremely likely” to recommend MOCA-Peds to a fellow pediatrician.*56.5%2020202112.4%1.3% 1.1% 0.5% 0.6% 0.7% 1.1% 0.6% 1.1%1Extremelyunlikely23453.3%15.6%14.0% 15.4%5.9% 6.1%3.7% 4.4% 3.2% 4.0%5678910Extremelylikely* Includes data from survey respondents who were participating for Part 3 credit in any exam area. Data from General Pediatrics and Subspecialty examparticipants are presented in aggregate.99
What are these data used for? The original purpose in collecting these data was to evaluate the success of MOCA-Peds (e.g.,acceptability, effectiveness, feasibility for participants) These deidentified data are shared with pediatricians to demonstrate the variation in participantusage and opinion. These deidentified data are shared with internal teams at the ABP and question writingcommittees (e.g., subboards) to improve the content and quality of the assessment in future years.How are these data collected? The data are drawn from (1) administrative information in the ABP Certification ManagementSystem on participation, and (2) an end-of-year survey distributed after completion of quarter 4.— The end-of-year survey survey was delivered to those who completed the 2021 Quarter 4questions. Email reminders were sent to those who had yet to complete evaluation survey.The delivery method was the same in 2020.— The survey response rates were 68.6% in 2020 and 63.7% in 2021 for survey respondentswho were participating for Part 3 credit.— The primary limitation is that those who did not complete MOCA-Peds Quarter 4 questionsare not represented here. Their opinions of MOCA-Peds may vary from the aggregatedresponses shown here.Data presented are from the ABP Certification Management Systemand the 2021 and 2020 MOCA-Peds End-of-Year Surveys.For questions about MOCA-Peds, please contact mocapeds@abpeds.org.For questions about the survey research, please contact research@abpeds.org.Developed in collaborationwith RTI International1010
Pediatric Gastroenterology 454 403 354 353 Pediatric Hospital Medicine 50 48 43 42 Pediatric Infectious Diseases 488 410 356 351 Pediatric Nephrology 240 212 181 177 Pediatric Pulmonology 370 330 292 291 Pediatric Rheumatology 68 62 53 53 Source: ABP Certification Management System. 2021 MOCA-Peds Exam (Part 3) Question Completion Rates by Exam .
