Transcription
Quality AssurancePrinciples, Elements and CriteriaDecember 1998Publication code: A0798Published by the Scottish Qualifications AuthorityHanover House, 24 Douglas Street, Glasgow G2 7NQ, and Ironmills Road, Dalkeith,Midlothian EH22 1LE Scottish Qualifications Authority 1998The information in this publication may be reproduced in support of SQAqualifications. If it is reproduced, the Scottish Qualifications Authority should beclearly acknowledged as the source. If it is to be used for any other purpose, thenwritten permission must be obtained from the Publications Officer at the ScottishQualifications Authority. It must not be reproduced for trade or commercialpurposes.
ContentsForeword1Overview of Assessment and Quality Assurance2Quality Assurance Principles8Quality Assurance Elements9General Criteria10Approval as an SQA centre11Approval to offer specific SQA qualifications12Validation of SQA qualifications13Internal moderation (verification) of internal assessment16External moderation (verification) of internal assessment17Quality control of external assessment19Monitoring of SQA’s quality assurance elements20
ForewordDeveloping its policy on assessment and quality assurance was one of the first tasksundertaken by the Scottish Qualifications Authority (SQA) on its formation. SQAinherited policies and procedures from its predecessor bodies, and these had to beintegrated. It also had to take account of decisions already made in relation to the newHigher Still provision. Assessment and quality assurance of Higher National Certificatesand Diplomas were also under review, as were the criteria applied to ScottishVocational Qualifications. Convergence, where possible, with the emerging policies ofthe Quality Assurance Agency for Higher Education (QAA) has been another importantconsideration.SQA works in partnership with centres to ensure that all of its qualifications are subjectto rigorous quality assurance and has now drawn up a number of quality assuranceprinciples to maximise the effectiveness of its partnership. Specific elements of qualityassurance are based on these principles, and each element consists of a number ofcriteria. The elements and criteria are designed to ensure that all SQA qualifications areassessed to national standards.By reading this publication, staff in centres should develop an understanding of thecriteria and of the ways we can work together to ensure that all SQA qualificationscontinue to meet the requirements of the people of Scotland.1
Overview of Assessment and QualityAssurance1 IntroductionThis document sets out SQA’s quality assurance principles, elements and criteria.Responsibility for meeting some of the criteria falls mainly on SQA itself, while for othercriteria it falls more on the centre.How SQA monitors the way a centre carries out its responsibilities depends on the typeof centre and the type of qualification being offered. For a qualification which isassessed wholly by an externally-marked question paper, such as some Standard Gradesubjects, most of the quality assurance burden falls directly on SQA. For an internallyassessed qualification, responsibility is shared with the centre, so a different range ofquality assurance procedures is required. Another consideration was that the assessmentprocesses of education authority schools and further education colleges are subject tothe quality assurance procedures of a number of external bodies, whereas for someother organisations it may be SQA alone which externally monitors assessmentdecisions.The most important objective of SQA’s assessment and quality assurance principles andprocedures is to ensure that assessment of SQA qualifications is valid and reliable; iethat candidates who have demonstrated the competences required by a qualificationreceive certification and that those who have not demonstrated competence do not. Thishas to be achieved in a practical and cost-effective manner, and we have taken accountof concerns about the quality assurance burden expressed by some centres. Every efforthas been made to keep the quality assurance demands on centres to the minimum.The rest of this section gives an overview of how quality assurance will be carried out inthe various contexts in which SQA qualifications are assessed.2 Primary SchoolsSQA’s involvement in primary schools relates to 5 – 14 National Testing. Policydirection for National Tests lies with The Scottish Office and, of course, National Testsdo not result in a qualification or the issue of a certificate. Some of the centre criteria arenonetheless relevant but are only of an advisory nature for primary schools. Forexample, internal moderation of the marking of National Tests is strongly encouraged.2
3 Secondary SchoolsApproval as an SQA centreEducation authority schools will receive centre approval automatically, as the educationauthority is obliged to ensure that minimum standards of resourcing are attained and allteachers have to be registered with the General Teaching Council. Independent schoolsfully registered with The Scottish Office Education and Industry Department (SOEID)will be approved following a development visit by a member of SQA’s staff.Approval to offer specific qualificationsA school wishing to introduce a new National Qualification in an area in which it hasexperience simply needs to let SQA know. This will help SQA to plan assessmentmoderation.Where a school wishes to offer a qualification in an area in which it has no experience,there may be specialist skills or equipment required, or a specific learning environment.In such instances approval to offer the qualification will be required. The completion of aform will normally be sufficient, though an approval visit may take place if SQA needsmore informationValidationSchools, for the most part, will be delivering National Qualifications which aredeveloped nationally. Schools’ involvement in validation will, therefore, focus on theirwillingness to release staff on request to serve on national development groups, and torespond to consultation on proposals for introducing new qualifications or revisingexisting ones.Internal moderation/verificationIt is important that schools ensure that internal assessments and estimates are made in aconsistent manner. SQA will assume that all candidates in the school entered for thesame qualification are assessed to a common standard, so methods of ensuring thatassessments are consistent should be applied. It is also important that any new memberof staff has access to all the relevant SQA documentation.Where external moderation reveals a significant problem in assessment the externalmoderator may look into the way internal moderation is carried out. Should a centrewide problem be identified through external moderation, then a visit by a systems verifiermay be arranged.External moderationExternal moderation will follow the principles and procedures which SQA consulted onearly in 1998. The system will involve both visiting and central (including postal)moderation. External moderation will focus on the validity and reliability of the school’sassessment. There will be a greater role for holistic assessment at outcome level.Development visits will be offered. A separate charge will be made for these but nocharge will be made for moderation visits instigated by SQA. The National AssessmentBank items should enhance the validity and reliability of internal assessment of theNational Units covered by the Bank. This resource, coupled with the use of externalassessment, will allow less obtrusive moderation of these units. Further details on3
moderation will be found in the Summary of the Outcomes section of SQA’sAssessment Moderation/External Verification Project: Outcomes of theConsultation.External assessmentThe conduct of national examinations requires a high degree of organisation andpreparation. Most of this falls to SQA, but centres have an important role to play insubmitting estimates and entering candidates at the appropriate course level.Accommodation for large numbers of students sitting examinations is required, as is theschool’s co-operation in special assessment arrangements, and procedures forabsentees and assessment appeals. The production by candidates of folios, projects etcoutwith the examination room demands different quality assurance measures for ensuringthat the work is the candidate’s own. Schools will already be familiar with theserequirements but the emphasis on supervised ‘write-ups’ will increase.4 CollegesApproval as an SQA centreAs independent institutions, colleges new to SQA will require individual approval inorder to become an SQA centre. Full registration with SOEID and/or the ScottishQuality Management System (SQMS) or with another awarding body should result inan abbreviated approval process.Approval to offer specific qualificationsFor most colleges, responsibility for this function is devolved after a successful audit.The responsibility may be withdrawn should difficulties in assessment arise frominadequate resourcing and staffing.ValidationResponsibility for validating National Qualifications will not be devolved as NationalQualifications will be developed nationally in consultation with centres. Colleges’involvement will focus on their willingness to release staff on request to serve on nationaldevelopment groups, and to respond to consultation on proposals for introducing newqualifications or revising existing ones.For Higher National Units, where a college has obtained devolved responsibility forvalidation of units and/or group awards on SQA’s behalf, the extent of this devolution isdefined in the centre’s quality contract with SQA. In the centre’s exercise of itsdevolved responsibility for validation, it must: have a system and procedures which ensure that all validation proposals areappraised against the validation criteria have documentation describing all validations which have been carried out, includingthe result of the appraisal against each of the criteria ensure that the outcomes of validations are timeously reported to SQA for recordingin the appropriate catalogueAccreditation of Scottish Vocational Qualifications is not devolvable.4
Internal moderation/verificationIt is important that colleges ensure that internal assessments and estimates are made in aconsistent manner. SQA will assume that all candidates in the college entered for thesame qualification are assessed to a common standard, so methods of ensuring thatassessments are consistent should therefore be applied. It is also important that any newmember of staff has access to all the relevant SQA documentation.Where external moderation reveals a significant problem in internal assessment, theexternal moderator may look into the way internal moderation is carried out. This maylead to a visit by a system verifier, or to monitoring of the centre by an auditor if theproblem appears to extend beyond a single department.External moderationColleges are familiar with visiting moderation, but have also had some experience ofcentral moderation. In the new system central moderation will play a larger role thanpreviously. For National Units developed as part of Higher Still, it will be the prevalentform. The advantages of this method are that moderation decisions are closelymonitored by the Senior Moderator, and it is cost-effective. There will be a role forvisiting within central moderation, however; where major problems in a college’sassessment of a unit are identified through central moderation, a visit to the centre will bemade without charge.Most National Units not part of Higher Still, Higher National Units and SVQs willcontinue to be subject to visiting moderation, although over time units will be scrutinisedto see if they could more effectively be centrally moderated.Where the candidate evidence is ephemeral, for example a performance in Drama or aprocess in Catering, or results in a product which cannot be readily transported, visitingmoderation will always be used.External moderation will focus on the validity and reliability of the college’s assessmentof the candidates’ evidence. There will be a greater role for holistic assessment atoutcome level, although moderation of all SVQs will continue to be based on a pc by pcapproach.Development visits will continue to be part of the system. A separate charge will bemade for these visits, but no charge will be made for moderation visits instigated bySQA. The National Assessment Bank items should enhance the validity and reliability ofinternal assessment of the National Units covered by the Bank. This resource, coupledwith the use of external assessment, will allow less obtrusive moderation of these units.Further details are given in the Summary of the Outcomes section of SQA’sAssessment Moderation/External Verification Project: Outcomes of theConsultation.External assessmentThe conduct of national examinations requires a high degree of organisation andpreparation. Most of this falls to SQA, but centres have an important role to play insubmitting estimates and entering candidates at the appropriate course level.Accommodation for large numbers of students sitting examinations is required, as is thecollege’s co-operation in special assessment arrangements, and procedures forabsentees and assessment appeals. Most colleges are familiar with these procedures,but the scale will increase with the implementation of Higher Still. The production by5
candidates of folios, projects etc outwith the examination room demands differentquality assurance measures for ensuring that the work is the candidate’s own. Becauseof the form of attendance this probably represents a greater challenge for colleges thanfor schools, and supervised write-up sessions have a particular importance inmaintaining credibility.Quality auditingQuality auditing ensures that colleges have the systems in place to enable them to takeon responsibility for aspects of quality assurance normally held by SQA. Concerns havebeen expressed about over-auditing and SQA is sensitive to them. The devolution ofquality elements to colleges will depend on a successful audit against the criteria set outin this document. After auditing, feedback from external moderation and, whererelevant, knowledge of the performance of candidates in external assessment will allowSQA to monitor the quality of assessment in the college. If all is well, further intensiveaudits should not be necessary, although there will be on-going monitoring. Wheredifficulties arise which are not restricted to one department, further action may be takenby systems verifiers or auditors.5 UniversitiesIt is important that SQA procedures and those of the Quality Assurance Agency forHigher Education (QAAHE) are aligned where possible. Discussions with QAAHE andthe universities which are SQA centres are continuing, and the outcomes will be madeknown in due course.6 Private training providers and employersApproval as an SQA centreAll private training providers and employers will require individual approval to becomean SQA centre. Some will be visited to ensure the centre criteria are met, and to discusshow their systems and SQA’s can dovetail together. Others which have been approvedby the Scottish Quality Management System (SQMS) or another awarding body will,providing they can make suitable arrangements for assessment moderation (verification),be approved without a visit.Approval to offer specific qualificationsThe approval process will depend on the centre’s track record in the specialist areafrom which the qualification is drawn. If the centre is new or has not offeredqualifications in the specialist area before, a visit by an external moderator (verifier) willbe necessary. If the centre has experience in the specialist area and a good trackrecord, and has made an approval submission which provides satisfactory information,then a visit may not be necessary.ValidationThough responsibility for accrediting and re-accrediting SVQs can not be devolved,staff of private training providers and employers will have the opportunity to makerepresentations to National Training Organisations (NTOs) to develop SVQs. Staff mayalso be invited by SQA to assist in developing qualifications comprising National orHigher National Units.6
Internal moderation/verificationIt is important that private training providers and employers ensure that internalassessments and estimates are made in a consistent manner. SQA will assume that allcandidates in a centre entered for the same qualification are assessed to a commonstandard, so methods of ensuring that assessments are consistent should be applied. It isalso important that any new member of staff has access to all the relevant SQAdocumentation. Where external moderation/verification reveals a significant problem ininternal assessment the external moderator will look into the way internalmoderation/verification is carried out. This may lead to a visit by a systems verifier if theproblem appears to extend beyond a single specialist area.External moderation/verificationPrivate training providers and employers will be familiar with visitingmoderation/verification and that will continue to be the main approach. Centralmoderation, where paper-based evidence is posted to SQA in an envelope supplied byus, and returned to the centre as soon as possible after scrutiny, may be used to a smallextent. External moderation/verification will focus on the quality of the centre’sassessment of the candidates’ evidence.Development visits will continue to be part of the system. A separate charge will bemade for these visits, but there will be no charge for moderation visits instigated bySQA. Further details are given in the Summary of the Outcomes section of SQA’sAssessment Moderation/External Verification Project: Outcomes of theConsultation.External assessmentPrivate training providers and employers are unlikely to be involved in examination roomtypes of external assessment, though other forms such as reports, folios, investigationsand so on may be introduced if independent assessment becomes part of the designrules for SVQs under the revised criteria. If so, the administrative procedures would besimilar to those for central moderation.7
Quality Assurance PrinciplesSQA is committed to quality assuring all its qualifications to ensure public recognitionand credibility through the maintenance of standards. SQA quality assurance is basedupon the following principles: the SQA assessment and quality assurance system should be understandable tostakeholders, effectively administered, publicly accountable and cost effectiveto operate qualifications should be accessible to all candidates who have the potential toachieve them the criteria which define the performance required of candidates for them toachieve qualifications should be appropriate to purpose, explicit and in thepublic domain each unit, course and group award should be unique and necessary, and shouldcomply with the relevant qualification specification assessments should be valid, reliable and practicable, and assessment resultsshould be in line with qualification criteria qualifications should be offered in centres which have the resources andexpertise to assess candidates against the qualification’s criteria staff in centres should be provided with effective support in assessingcandidates for certification responsibility for quality assurance should be devolved to centres where this isconsistent with the maintenance of national standards8
Quality Assurance ElementsIn order to ensure that the qualifications SQA offers are designed, delivered andassessed to acceptable national standards, we have identified key quality assuranceelements, based on the quality assurance principles. These elements underpin all SQAqualifications, and are the mechanisms through which national standards are establishedand maintained.We have divided each element into requirements or criteria. SQA and centres havecorresponding responsibilities for these. Quality provision requires an effectivepartnership, based on the quality assurance criteria, between SQA and staff in centres.Some criteria relate to all of the elements and they have been extracted and describedseparately.The elements are: approval as an SQA centrethe criteria relate to the management procedures which underpin the implementationand assessment of SQA qualifications in centres approval to offer specific SQA qualificationsthe criteria relate to resources required of centres for the implementation andassessment of specific SQA qualifications validation of SQA qualificationsthe criteria relate to ensuring that SQA qualifications are fit for purpose internal moderation of internal assessmentthe criteria relate to the processes by which centres ensure that all internalassessment is valid, reliable, practicable and cost-effective external moderation of internal assessmentthe criteria relate to external processes by which SQA ensures that internalassessment is in line with the national standards set out in the qualifications.'Verification' has the same meaning and will continue to be found in publicationsreferring to SVQs. quality control of external assessmentthe criteria relate to the processes by which SQA and centres ensure that externalassessment is in line with the national standards set out in the qualifications monitoring of SQA’s quality assurance elementsthe criteria relate to the processes which are used to measure the success of theother elements in supporting the consistent application of national standards9
General criteriaThe following criteria are implicit in all of the quality assurance elements and have therefore been setout separately.CriterionSQAThe centreA1 Processes whichsupport all SQAqualifications arereviewed andcontinuously improved reviews and updates internal qualityassurance processes as required identifies staff roles andresponsibilities facilitates staff participation inconsultation exercises and takesaction on feedback from staffcontinuously monitors qualityassurance processes andupdates them in line withqualification and centrerequirements consults with stakeholders andtakes action on feedbackA2 All main processesare in the public domainand information andguidance are provided gives targeted information andguidance to centres and otherstakeholders as required disseminates information to staff,candidates and other stakeholders asrequiredA3 Special assessmentarrangements are madefor candidates whorequire them provides guidance on possiblespecial assessment arrangementsfor candidates makes fair and consistentdecisions on individual casessubmitted by the centreseeks SQA approval for specialassessment arrangements andimplements these arrangements asagreedA4 Data processes areaccurate and timeous maintains a comprehensiverecord of candidate achievement issues results data andcertificates accurately andtimeouslyallocates the Scottish CandidateNumber (SCN) to new candidatesonly; ensures existing candidatesuse correct SCN submits entry data timeously10
Element: Criteria for approval as an SQA centreFor centres not yet established, the potential to fill the criteria should be demonstrated.CriterionSQAThe centreB1 Quality assuranceis managed effectively provides information,guidance and support onquality assurancerequirements to centres manages effectively the internalprocesses which underpin SQAprovisionB2 Information ismanaged effectivelywithin the centre andbetween the centre andthe SQA provides information on SQArequirements eg qualificationinformation includingarrangements documents andunit specifications stores information on SQA qualifications,procedural requirements and candidates disseminates SQA related information tostaff and candidates as appropriate records information about candidates,qualifications and proceduralrequirementsB3 Resources aremanaged effectively processes informationreceived from centres submits information to the SQAaccurately and timeously approves assessment andquality assurance processeswithin centres and monitorsthem for continuingcompliance ensures that staff are competent in thesubject /cognate/occupational areaswithin which they operate and that theyare competent to assess provides accommodation, referencematerial, equipment, learning andassessment material which supports theassessment of SQA qualifications maintains financial viability ensures that staff understand the processand SQA requirements allows access for SQA representatives makes available relevant materials in theappropriate locations makes curricular, vocational andqualification-related guidance andsupport available to candidates demonstrates a commitment to equalopportunity in the selection, admissionand assessment of candidates makes material relating to SQAqualifications available operates a satisfactory internalassessment appeals procedure forcandidates disseminates information on the appealsprocedures to staff and candidates maintains records of appealsB4 There arearrangements in placewhich facilitate externaland internal qualitycontrol of SQAqualifications B5 Candidates areprovided with induction,guidance and support monitors processes withincentres provides guidance to centresand candidatesB6 There is asatisfactory appealsprocedure within thecentre for candidates makes available informationon external and internalquality control requirementsapproves the centre appealsprocess for internalassessment decisions andmonitors it for continuingcompliance11
Element: Criteria for approval to offer specific SQA qualificationsThis element can be devolved for all qualifications.CriterionSQAThe centreC1 An approvalprocess for specific SQAqualifications is defined defines procedures for obtainingapproval for specificqualifications defines internal processes whichunderpin specific qualificationsapproval operates an approval system tomeet the requirements of centresand qualifications records and stores information onobtaining approval for specificqualifications monitors the effectiveness of thecentre's approval system records and stores informationrelating to specific qualificationapproval gives guidance on the resourcesrequired for specific SQAqualifications approves resources for specificqualifications and monitors themfor continuing complianceprovides accommodation andequipment which is suitable for thequalification and takes account oftechnical/specialist needs provides sufficient numbers ofqualified/experienced staff to assessand internally moderate internalassessments and estimates provides all candidates with access toreference and learning materials whichsupport the qualification uses assessment material which isappropriate to the requirements of thequalificationC2 Resources meet therequirements of: the qualifications the method ofassessmentthenumber and needs of candidates the mode(s) ofdeliveryCentres with devolved responsibility for approval must: have a system and procedures which ensure that all approval proposals are appraised against theapproval criteria have documentation describing all approvals which have been carried out, including the result ofthe appraisal against each of the criteria ensure that the outcomes of approvals are timeously reported to SQA for recording in theappropriate catalogue12
Element: Criteria for the validation of SQA QualificationsThis element can be devolved only for HN Qualifications.CriterionSQAThe centre publishes the system, proceduresand timescales for the validation ofnew and revised National and HigherNational Qualifications and makesthem known to centres ensures that submissions to SQAof locally-devised proposals forvalidation comply with the SQA’ssystem, procedures and timescales publishes design rules and othertechnical requirements and guidanceon their use develops National and HigherNational Qualifications and validatesthem to ensure that all of thesequalifications comply with thedesign rules and the technicalrequirements prior to inclusion in theappropriate SQA qualificationscatalogue*uses the SQA’s design rules andother technical requirements as thebasis for development of proposedqualifications to be submitted toSQA for validation validates qualifications to ensurethat they comply with the designrules and other technicalrequirements prior to inclusion in theappropriate SQA qualificationscatalogue*D3 There is ademonstrable need forthe National or HigherNational Qualificationwhich fills a gap in thequalificationsframework and makes avaluable and uniquecontribution to theSQA’s portfolio ofqualifications works with stakeholders to identifythe qualifications required to meetthe needs of Scottish society* consults with appropriate bodies toidentify the need for the National orHigher National qualification*provides feedback to SQA on theextent to which the qualificationsin the National and Higher NationalQualifications catalogues meet thiscriterion has a feedback system throughwhich it collects stakeholders viewsin order to regularly review theNational and Higher NationalQualifications with a view toengaging in a programme ofupdating where appropriate*D4 The qualification isaccessible to allcandidates who couldpotentially achieve it ensures that all National and HigherNational Qualifications which itvalidates are free from unnecessarybarriers to achievement strives to serve the best interestsof prospective candidates anddoes not create artificial barriers toachievement provides feedback to SQA on theextent to which qualifications inthe National and Higher NationalQualifications catalogues meet thiscriterionN and HND1 The validationprocess forqualifications which areto be included in theSQA National or HigherNational Qualificationscatalogue is defined andin the public domainD2 The qualificationconforms to the SQAdesign rules and othertechnical requirementswhich are current at thetime of i
the Quality Assurance Agency for Higher Education (QAA) has been another important consideration. SQA works in partnership with centres to ensure that all of its qualifications are subject to rigorous quality assurance and has now drawn up a number of quality assurance principles to maximise the effectiveness of its partnership.
