WIXLIB

Security of ExaminationsTo ensure the integrity of the NBME Subject Examination program and the validity of the scores reported, it isessential to maintain the security and confidentiality of test materials at all times. All items used in NBME SubjectExams are owned and copyrighted by NBME. In addition, any unauthorized reproduction or disseminationof test materials or any part of them through any means, including but not limited to copying, printing,photographing or downloading of electronic files, or reconstruction through memorization and/or dictation, isstrictly prohibited. NBME reserves the right to pursue its rights and remedies to the fullest extent permitted bylaw without further notice, including taking legal action. NBME also reserves the right to discontinue SubjectExamination services in the event of a breach or suspected breach in the security of test materials.NOTE: If your institution wouldlike to administer SubjectExams at other campuses,the ECP must designatean AECP to oversee theadministration at thenew campus location.AECPs must follow thesame procedures asECPs to maintainthe security oftest materials.Test Delivery OptionsWeb-Based Examinations at Your InstitutionWeb-based exams must be administered in a secure, proctored setting.The Chief Proctor (CP) will have access to a centralized Chief ProctorResource Site, which houses all the system applications in support oftest administration. The CP must conduct the exam in strict compliancewith the procedures outlined in the Chief Proctor’s Manual.NBME provides a web-based utility that checks both hardware andsoftware configurations and downloads a plug-in that secures thebrowser. The computer is locked down, preventing the examinee fromaccessing applications such as e-mail or notepad.It also includes a function designed to detect any recording applicationsand programs that may be running on workstations or examinees’personal laptops prior to the start of the exam.Web-Based Examinations at Prometric Test CentersThe Clinical Science and Comprehensive Subject Examinations are availablefor administration at Prometric’s worldwide network of secure test centers.Prometric testing may be preferred for students serving on clinicalrotations in a distributed hospital network or for those located at a sitewithout web-based test delivery capability. It may also be useful formakeup tests.This option requires advance planning to allow students sufficient timeto schedule testing appointments. See page 10 for detailed informationabout this process.7

SUBJECT EXAMINATION CONTENTDescription of Subject ExaminationsSystems Science curricula and educational initiatives.Together with clinical and basic science instruction,Health Systems Science prepares medical studentsfor effective clinical practice.The Basic Science and the Clinical ScienceExaminations are designed to assess performanceat the end of a course, clerkship, or other unit ofinstruction.Advanced Clinical Examinations: The EmergencyMedicine and Internal Medicine Examinations aredesigned to measure the effectiveness of the fourthyear emergency medicine and internal medicinesub-internship experiences. NBME staff memberscollaborated with a task force of clinical faculty frommedical schools and teaching hospitals to developthese exams.The Introduction to Clinical Diagnosis Examination isdesigned to measure student performance at the endof history-taking and physical diagnosis courses.The Ambulatory Care Examination was developedin collaboration with clerkship directors in internalmedicine and a task force of clinical faculty fromseveral US medical schools. It is intended forassessment at the end of an ambulatory clerkship inthe third or fourth year.Comprehensive Examinations: The ComprehensiveBasic Science Examination and the ComprehensiveClinical Science Examination are achievement testscovering material typically learned during basicscience education and core clinical clerkships. Theyreflect content similar to that of the USMLE Step1 and Step 2 CK exams and are typically used toidentify areas of strength and weakness prior totaking the USMLE.NBME worked with the American MedicalAssociation to develop and pilot The Health SystemsScience Examination in collaboration with medicalfaculty experts in Health Systems Science. Facultycan use this high quality, standardized assessmenttool to support the implementation of Health8

NBME Subject ExaminationsBasic ScienceMicrobiology onlyMicrobiology ImmunologyBehavioral SciencesBiochemistryGross Anatomy onlyGross Anatomy EmbryologyClinical ntroduction toClinical DiagnosisPhysiologyAmbulatory CareMedicine*Clinical Neurology*Obstetrics and Gynecology*Family Medicine Modular*CoreCore Chronic CareCore Chronic Care MusculoskeletalPediatrics*Health SystemsScience ExaminationHealth Systems ScienceAdvanced ClinicalExaminationsEmergency Medicine*ComprehensiveExaminationsComprehensive Basic Science*Psychiatry*Surgery*Internal MedicineComprehensive Clinical Science** Available for Administration at Prometric Test CentersContent Outlines and Sample ItemsSubject Exam outlines developed by NBME committees of content experts are used to construct each test.Items are selected based on content and performance statistics. The final form of each exam is reviewedand approved by the appropriate committee chair prior to release. Multiple forms of each exam are typicallyavailable at any time so that alternate forms may be used for successive courses and clerkships or for makeupexams. Content outlines and sample items are available for all exams on the NBME website under Schools andResidency Programs.Exam PreviewExam Preview is a free service which allows medical school faculty to preview representative content for aspecific subject exam so they can determine whether that exam is appropriate for a particular class/curriculum.Exam Preview is open to medical school faculty and to authorized staff of other approved medical educationinstitutions.9

ORDERING AND TEST ADMINISTRATIONMyNBME Services PortalOnline Ordering SystemServices and reports related to Subject Exams areavailable through the MyNBME Services Portal.This is a secure website and a primary source forauthorized users to obtain information about theirstudents for a variety of activities related to USMLE,NBME Subject Exams, and other services. It providesaccess to the Online Ordering System, CustomizedAssessment Services, roster management for webbased exams, and Subject Exam scores and year-endreports.All Subject Examination program services arerequested through the Online Ordering System.The Online Ordering System allows the user toenter order information, select one of four differentpayment methods, review the status of orders, andreceive e-mail notifications at specific points duringthe ordering cycle.Online Ordering of ExaminationsA major feature of the portal is the capability for aninstitution to create and manage its own users andto assign privileges to access specific areas of theportal.Detailed information, test-location options andinstructions for ordering web exams are availablein the Online Ordering System. All exams must beordered in a time frame that is sufficient to allowfor processing and scheduling administrative ortechnical support. Ordering timelines for each examdelivery option are shown below.Each institution has at least one User AccountAdministrator who assigns users the privileges theyneed to access areas of the portal appropriate fortheir responsibilities.Web-Based AdministrationsWeb-BasedAdministrationsThrough Your InstitutionOrders may be placed up to and including the 2nd calendar day prior tothe test date.Web-Based PrometricAdministrationsA testing window of 7-14 calendar days is required.Orders lock 2 calendar days before the test date.Orders must be placed no later than 28 days prior to the start of thetesting window.Orders lock 21 calendar days prior to the start of the testing window.All exam orders can be revised or cancelled any time prior to the lock date.After the lock date, NBME must be contacted to make order changes.10

Changes to Subject Examination Form AssignmentConsistent with best practices and national standards for high qualitytesting programs, NBME periodically reviews its examination programs,policies and procedures. In order to maintain the examination securityand protect the integrity of NBME examinations, NBME will no longerprocess requests for specific examination forms to be assigned forupcoming administrations effective January 2, 2018.For security purposes, NBME routinely employs multiple forms forSubject Examinations available for test delivery during an academicyear. The scores on each examination form have been equated acrossforms and administrations so that your examinees’ performance isdirectly comparable, regardless of which form has been administered.Equating is a process that statistically adjusts for variations in test formdifficulty so that scores on forms of different difficulty can be usedinterchangeably. Equated scores can then be used to compare and trackschool and examinee performance over time.When you order Subject Exams, an examination form is assigned atthe order or examinee level depending on whether the exam isadministered locally or at Prometric test centers. Form assignmentsenhance security. While examinees may see the same or different forms,all scores are comparable regardless of the form administered.11

The Roster System For Web-based ExaminationsThe NBME roster system supports the administration and scoring of itsweb-based examinations, including those administered at Prometric testcenters. This system also facilitates administration of different SubjectExams in the same testing room. The roster system, which is availablethrough the MyNBME Services Portal, is divided into two parts: themaster roster and the test administration roster.Master RosterThe master roster is a list of all examinees at your institution (covering allclass levels) who will be taking NBME web-based exams. In addition toname, date of birth, and gender, each record includes a unique local IDnumber. Each examinee must enter this ID number into the test deliveryinterface in order to start an exam. Other optional fields are available topopulate (such as program type and graduation month/year), in orderto expedite records searches. For examinees who will take an exam ata Prometric test center, the master roster must also include their e-mailaddresses.Test Administration RosterThe test administration roster is a list of examinees scheduled to takean exam on a specific test date or within a specific testing window. Themaster roster must be populated before a test administration roster canbe created.For local administrations at schools or other institutions, the testadministration roster can be populated any time prior to the test dateand before a test session is launched. However, a test administrationdelay may occur if a roster is not completed prior to the start of a testsession.For examinations administered at Prometric test centers, the roster mustbe completed no later than 21 calendar days prior to the start of thetesting window.Detailed instructions are available in the roster system.12

Scheduling Web-based Exams atPrometric Test CentersMyNBME Services Portal features a “My BillingAccounts” portlet through which those who orderor arrange for payment of exam services can viewup-to-date information about the status of invoices,access monthly summaries, and pay for servicesdirectly.After the test administration roster has beencreated for a web-based exam at a Prometric testcenter, NBME generates a scheduling permit foreach examinee in approximately 2-3 business days.There are exceptions if the examinee requires testaccommodations (See page 18). Each examinee willreceive an e-mail from NBME with a URL to accessand print/download his/her scheduling permit.Information printed on the scheduling permit isrequired to schedule a testing appointment throughPrometric’s website. The permit also has informationabout admission to the test center, cancelling/rescheduling appointments, and other policies andprocedures.Payment OptionsPayment for exam services may be made by: Credit card (Visa, MasterCard or AmericanExpress) Check or money order Wire transfer Institution account with a credit balancePayment Policies and ProceduresPayment TermsExam FeesNBME policies for payment are as follows:There is a separate fee structure for the Prometrictest center option. Current fees for Subject Examsare listed on the NBME website under Schools andResidency Programs: Subject Examination Fees. Payment of invoices generated for NBME examservices must be received by NBME prior to thescheduled test date. Payment must be in US currency only and in thepayment forms specified in the Payment Optionssection above.Billing Accounts Payments, regardless of form, from examineesare not accepted.NBME requires that accounts be set up for thepurpose of applying fees received and for computingany credit or payment due after scores have beenreported. No refunds are provided. Score reports will not be released by NBME unlesspayment for an order has been received.There are two types of accounts: an institutional orECP Account, which is required, and departmentaccounts, which are optional. Typically, theseaccounts reflect funding sources at an institution.Accounts carry credits only and no refunds areprovided.If an account accrues more than 1,000 in invoicesthat remain unpaid for more than 90 days followingthe test date, all score reports associated with thataccount will be held by NBME until the account isbrought current. This applies even if other orders forthat account were prepaid.13

Administering Subject ExaminationsA variety of test-delivery environments aresupported: both wired and wireless networks, PCand Mac operating systems, and laptops or fixedworkstations. Also, it is possible to schedule multipleseparate testing sessions on the same day or to runtesting sessions in different buildings or at separatecampuses at the same time.The ECP has overall responsibility for managementand planning of test administration events. TheECP is also required to recruit chief proctors andother support staff, as needed, to perform the tasksnecessary to ensure a successful and secure testadministration.Web Administrations at Your InstitutionTest Administration SupportNBME provides the tools necessary to administer aweb-based exam in a secure, proctored setting.Web Examinations at Your InstitutionThese features include:NBME staff will be available for test administrationsupport on your test date to respond to technical orprocedural issues that may arise. A centralized Chief Proctor Resource Site thatprovides resources to support all tasksassociated with the test administration cycle.This includes access to the Chief Proctor’s Manual,tutorials, a practice exam, and features formanaging the test session online.NBME staff are available 24 hours a day, 7 days aweek by calling 215-590-9298 (exclusive of NBMEholidays). Software that “locks down” the browser duringtesting, preventing examinees from accessing anyother applications (such as e-mail or notepad),during the test session.Web Administrations at Prometric Test CentersPrometric test center staff will administer NBMEexaminations in a secure setting and respond toany technical or procedural issues. Informationabout Prometric test centers is available atwww.Prometric.com. An automated compatibility check that helps toensure every computer used for testing purposesis properly configured to deliver the exam.Web-Based Testing Outage Guidelines for Customized AssessmentServices and Subject ExaminationsIn order to improve communication and better aid your decision making in the event of a webbased testing outage, guidelines are now available in the following locations for your reference: Web-Based Testing Status Page: https://www.nbme.org/nbme-testing-status Executive Chief Proctor (ECP) Manual: Available on the MyNBME Services Portalunder ECP Resources tabThe guidelines apply when the source of the outage is NBME or its vendors, andthey apply to Customized Assessment Services/Subject Exam administrators atboth medical schools and Prometric. Also included are communication venuesthrough which NBME will send notifications and status updates when outagesoccur. Please note that the guidelines are subject to change, and NBME willprovide notification when/if they should be updated.14

WEB-BASED TESTING OUTAGE GUIDELINESCommunication during SubjectExamination and CustomizedAssessment Web-Based TestingSystem InterruptionsTherefore, if such an interruption occurs causing yourexaminees to wait up to 30 minutes or more to startor complete an examination, you have the followingoptions:When NBME determines that a universal outageis affecting Subject Examination and CustomizedAssessment Services testing, communications will besent notifying all customers testing on the affectedday of the situation. NBME will provide frequentstatus updates from the time NBME determines thatthere is an outage. All notifications will be providedvia the following venues:Examinations Administered at your Medical SchoolPrior to Students Starting Exam Test students later in the day—you may launchyour examination anytime on your scheduled testdate regardless of the session start and endtimes. There is no need to make adjustmentsto your order. Chief Proctor Resource Site Executive Chief Proctor (ECP) and proctoremail addresses on file with NBME Cancel your Order—Contactsubjectexams@nbme.org to cancel your order.You will receive a credit to your account withinone to two business days. You may use thecredit as payment when you order the rescheduledexamination. If the examination needs to berescheduled sooner than two business days fromthe cancellation date, please contact us forassistance.Policy for Subject Examination andCustomized Assessment ServicesWeb-Based OutagesNBME is committed to providing excellent customerservice. However, technical problems may occurprior to or during an examination which may preventexaminee testing at institutions (e.g., medicalschools) or Prometric test centers, from accessingor completing examinations. While we are workingto lessen the frequency of such situations, it isimpossible to eliminate their occurrence altogether.15

Examinations Administered at Prometric TestCentersAfter Students have Started Exam Reschedule your students — if you choose toreschedule your students, the chief proctor mustexclude students from the score roster prior toconcluding the administration (See Confirming theRoster – pgs. 31-34, Web-Based ExaminationsChief Proctor’s Manual). This will ensure thatyou do not receive scores for partially completedexaminations, you are not charged for thoseexaminations, and that partial data is not includedin your year-end reports and/or schoolperformance profiles. If you are unable to removestudents from the score roster, please contactsubjectexams@nbme.org immediately and we willexpunge the partial scores and credit the originalorder for the corresponding examinations.Prior to Students Starting Exam Reschedule within eligibility period — if a technicalproblem occurs that prevents a student fromstarting an examination at a Prometric test center,Prometric will ask the student to wait for 30minutes while they attempt to resolve the issue.If the issue cannot be resolved within 30 minutes,the student may continue to wait for the issueto be resolved or leave the test center. If there istime remaining in the student’s testing window,the student may work with Prometric toreschedule his/her test appointment to anotherdate within the testing window. Resume exam administration when service isrestored —If you choose to wait for service to berestored, you have the following options:ww Submit new order — If a student cannot testwithin the time remaining in the test window orif there is no time remaining in the test window,the medical school should contactsubjectexams@nbme.org so that the studentcan be placed on a new exam roster. We willopen a new test window so that you can submita new order. We will credit your account for themissed examination. The student wil

in collaboration with clerkship directors in internal medicine and a task force of clinical faculty from several US medical schools. It is intended for assessment at the end of an ambulatory clerkship in the third or fourth year. NBME worked with the American Medical Association to develop and pilot The Health Systems