Transcription
Independent Evaluation of the Rural HealthMultidisciplinary Training ProgramFinal Report to theCommonwealth Department of HealthMay 2020Australia
AcknowledgementsKBC Australia would like to acknowledge and thank the many people who participated in andcontributed to this evaluation. We are grateful to the RCS and UDRH Directors and staff fortheir assistance in facilitating site visits and providing additional information as required, aswell as for their insights and willingness to provide the evaluation team with open and honestfeedback.We acknowledge and thank all the people who participated in interviews and surveys includinguniversity staff, clinical supervisors, students, health service providers, health professionals,jurisdictional officers and community representatives. We appreciate the contribution oforganisations and peak bodies through written submissions and participation in round tables.We thank the Expert Reference Group for their advice provided in shaping the methodologyand responding to the evaluation findings. We also acknowledge and appreciate the input ofthe officers of the Department of Health throughout all phases of the evaluation.Project TeamDr Kristine Battye, KBC AustraliaDr Cath Sefton, KBC AustraliaDr Joseph (Mark) Thomas, KBC AustraliaProfessor Janie Smith, Bond University/Dr Janie Smith ConsultingDr Shannon Springer, Bond University/Springer Health ConsultantsDr Isabelle Skinner, Decision Support AnalyticsDr Emily Callander, Griffith UniversityMs Sally Butler, KBC AustraliaMs Rachel Wilkins, KBC AustraliaMs Jane Gordon, KBC AustraliaMs Kate Kelly KBC AustraliaKBC AustraliaABN: 44 101 153 913PO Box 2428, Orange NSW 2800Suite 7, Level 1, 24 Sale Street, Orange NSW 2800Phone: 02 6361 4000www.kbconsult.com.auKBC Australia2
AcronymsACCHOAboriginal Community Controlled Health OrganisationACRRMAustralian College of Rural and Remote MedicineAhpraAustralian Health Practitioner Regulation AgencyAIHWAustralian Institute of Health and WelfareAMCAustralian Medical CouncilANMACAustralian Nursing and Midwifery Accreditation CouncilAPHCRIAustralian Primary Health Care Research InstituteARCAustralian Research CouncilARHENAustralian Rural Health Education NetworkASGSAustralian Statistical Geography StandardRA1Australian Statistical Geography Standard Remoteness Area 1—Major CityRA2Australian Statistical Geography Standard Remoteness Area 2—Inner RegionalRA3Australian Statistical Geography Standard Remoteness Area 3—Outer RegionalRA4Australian Statistical Geography Standard Remoteness Area 4—RemoteRA5Australian Statistical Geography Standard Remoteness Area 5—Very RemoteATARAustralian Tertiary Admissions RankCRRHCentre for Rural and Remote Health (Mt Isa)CSPCommonwealth Supported PlaceCAGRCompound Average Growth RateCATSINaM Congress of Aboriginal and Torres Strait Islander Nurses and MidwivesDDSDemand Driven SystemDTERPDental Training Expanding Rural PlacementsDMODistrict Medical Officer (Northern Territory)EIFEducation Investment FundERPEstimated Resident PopulationERGExpert Reference GroupKBC Australia3
ECPPExtended Clinical Placement ProgramFRAMEFederation of Australian Medical EducatorsFTEFull-time EquivalentGDHGove District HospitalGPGeneral PractitionerGIRSGeographic Index of Relative SupplyHECSHigher Education Contribution SchemeIAHAIndigenous Allied Health AustraliaIRTPIntegrated Rural Training PipelineIMGInternationally Trained Medical GraduateKBCKBC AustraliaKPIKey Performance IndicatorLHDLocal Health DistrictLHNLocal Health NetworkMDMedical DoctorateMDANZMedical Deans Australia and New ZealandMSODMedical Schools Outcomes DatabaseMABELMedicine in Australia: Balancing Employment and LifeMMModified Monash (Model)NHMRCNational Health and Medical Research CouncilNRHANational Rural Health AllianceNGONon-government OrganisationNCMECNorth Coast Medical Education CollaborationNTBMSNorthern Territory Bonded Medical SchemeNTMPNorthern Territory Medical ProgramOSCEObjective Structured Clinical ExaminationPGYPost Graduate YearPIPPractice Incentive PaymentKBC Australia4
PHCREDPrimary Health Care Research Evaluation and DevelopmentPHNPrimary Health NetworkPSMPropensity Score MatchingRACGPRoyal Australian College for General PractitionersRANZCOG Royal Australian and New Zealand College of Obstetrics and GynaecologyRTHRegional Training HubRHPPRemote Health Practice ProgramRTORegional Training OrganisationRVTSRemote Vocational Training SchemeRFDSRoyal Flying Doctors ServiceRCSRural Clinical SchoolRCSWARural Clinical School of Western AustraliaRCTSRural Clinical Training and SupportRGRural GeneralistRHCRural Health ClubRHMTRural Health Multidisciplinary TrainingRJDTIFRural Junior Doctor Training Innovation FundRWARural Workforce AgencySARRAHServices for Australian Rural and Remote Allied HealthSTPSpecialist Training ProgramTEHSTop End Health ServiceUMATUndergraduate Medicine and Health Science Admission TestUCRHUniversity Centre for Rural Health (Lismore)UDRHUniversity Department of Rural HealthVETVocational Education and TrainingWACRHWestern Australia Centre for Rural HealthKBC Australia5
ContentsAcknowledgements2Acronyms3Executive Summary141.352.3.4.Background and Introduction1.1RHMT Program. 351.2RHMT Program context. 441.3Policy Environment. 491.4Higher Education. 531.5Intersection of higher education and health policy. 57Evaluation Overview2.1Purpose . 592.2Focus and Scope . 592.3Objectives of the Evaluation. 592.4Intended Audience. 592.5Key Evaluation Questions. 602.6Program Logic. 602.7Roadmap to the Report. 602.8International students. 63Methodology653.1Project Scoping. 653.2Ethics. 663.3Surveys. 663.4Analysis of Workforce Outcomes . 683.5Program Data Analysis. 683.6Cost analysis. 693.7Consultations. 703.8Data synthesis and analysis. 733.9Reporting. 74Student Placements and Supervision4.1KBC Australia5980Introduction . 806
5.6.7.8.4.2Student Enrolment. 804.3Student placements. 834.4How well have program objectives been met?. 954.5Elements of placement quality . 994.6Enablers of quality placements. 1024.7Challenges in delivering quality placements. 106Research, Academic and Professional Networks5.1Introduction. 1155.2Context. 1155.3Description of Academic and Professional Networks. 1165.4Meeting Program Objectives—Academic and Professional Networks. 1175.5Enablers to building and maintenance of academic and professional networks. 1185.6Challenges to building and maintenance of academic and professional networks. 1195.7Building Research Capability and Capacity. 1205.8Meeting Program Objectives. 1225.9Future directions. 128Local Capacity Building and Community Engagement1336.1Introduction. 1336.2Student Activities. 1336.3Community development and engagement activities. 1336.4Benefits of RHMT program to local communities. 1376.5Rural Commitment. 1396.6Challenges. 140Aboriginal and Torres Strait Islander Health1457.1Introduction. 1457.2Enrolment and graduation of Aboriginal and Torres Strait Islander students. 1457.3Mentoring. 1467.4Education and Training. 1477.5Cultural Safety and/or Awareness . 1497.6Employment of Aboriginal and Torres Strait Islander staff. 1527.7Parameter 6. 1537.8Challenges. 1547.9Future directions. 155Regional Training Hubs8.1KBC Australia115159Introduction. 1597
9.8.2Focus of the evaluation. 1598.3Overview of approaches and activities. 1608.4Challenges to implementation of the RTH initiative. 1668.5Enablers to implementation of the RTH initiative. 1678.6Future directions. 1688.7Allied health and Nursing. 169Rural Health Clubs10. Northern Territory Medical Program17317810.1 Overview of the Northern Territory Medical Program. 17810.2 Pathways into the NTMP. 17910.3 Supporting course progression and completion. 18110.4 Promoting Aboriginal and Torres Strait Islander health. 18210.5 Student Placements. 18510.6 Workforce Outcomes. 19210.7 Consolidation. 19310.8 Recommendations: NTMP. 19411. Dental Training Expanded Rural Placements19911.1 Introduction. 19911.2 Rural-origin dental students. 19911.3 Dental placements. 19911.4 Challenges and barriers. 20211.5 Benefits of rural placement. 20212. Impact of Consolidation and Performance Management20612.1 Opportunities for interdisciplinary training. 20612.2 Flexibility and innovation in delivery models. 20712.3 Resource Management. 20712.4 Program requirements. 20812.5 Reporting and monitoring. 20812.6 Summary. 20913. Workforce Outcomes21613.1 Introduction. 21613.2 Medicine. 21613.3 Nursing, midwifery and allied health. 22113.4 Dental. 222KBC Australia8
13.5 Multidisciplinary Health Workforce Survey. 22313.6 RHMT Program Workforce Conversion. 22913.7 Tracking rural health workforce outcomes. 23013.8 Future Directions. 23214. Value for Money23914.1 Program Income and Expenditure. 23914.2 Value of the Program - Achievements and Benefits. 24114.3 Cost analysis – Student placements. 24614.4 Cost-effectiveness of investing in rural multidisciplinary clinical placements. 25014.5 Limitation to the cost analysis. 25114.6 Limitations on determining Value for Money. 25114.7 Monitoring program efficiency. 25215. Conclusion and Recommendations25415.1 Introduction. 25415.2 Program Outcomes, Objectives and Principles (Recommendations 1, 2). 25615.3 Student selection (Recommendation 3). 25815.4 Curricula (Recommendations 4,5). 25915.5 Placement Quality (Recommendations 6-14). 26015.6 Responding to community and workforce need (Recommendation 14). 26715.7 Strengthening research networks (Recommendations 15,16). 26815.8 Transitioning medical students to rural work (Recommendations 17, 18). 26915.9 Social Accountability (Recommendation 19, 20). 27115.10 Measuring program impact (Recommendations 21,22). 27215.11 Program performance (Recommendations 23, 24). 27315.12 Funding and Innovation (Recommendations 25-29). 27415.13 Future Direction. 278ReferencesKBC Australia2839
List of TablesTable 1-1 University Departments of Rural Health.36Table 1-2 Rural Clinical Schools.39Table 1-3 Estimated resident population (ERP) by remoteness area (2013, 2017).44Table 1-4 Australian health workforce, total full-time equivalents, FTE rate and % change (2013, 2017).46Table 3-1 Summary of RHMT program site consultation locations.71Table 3-2 Consultation informants by stakeholder group.72Table 3-3 Written submissions by stakeholder group.72Table 4-1 Rural-origin medical student enrolments (2016-2018).81Table 4-2 Rural-origin nursing students (2016-2018).81Table 4-3 Rural-origin allied health students (2016-2018).82Table 4-4 Placements, placement weeks and average duration, medicine (2018).84Table 4-5 Number, duration and average duration of long RCS medical placements (2018).84Table 4-6 UDRH placements and placement weeks completed by Australian students by discipline (2018).89Table 4-7 UDRH placements and placement weeks completed by Australian students by Remoteness Area (2018).90Table 4-8 Placement target achievement (2016-2018).96Table 4-9 Assessed placement quality by discipline.97Table 4-10 Assessed supervision capacity building by programmatic unit.100Table 5-1 Assessed research capability by programmatic unit.123Table 5-2 Scoping review of UDRH publications.125Table 7-1 Mentoring access by Aboriginal and Torres Strait Islander students (2016-2018).146Table 7-2 UDRH provision of Indigenous cultural training to domestic students (2015).150Table 8-1 RTH strategies to promote integrated rural medical training.161Table 10-1 NTMP Aboriginal and Torres Strait Islander student commencement and graduation (2016 -2018).180Table 10-2 Flinders University Northern Territory medical placements, by clinical setting and region (2018).186Table 10-3 Domestic graduates completing six-months of rural placement in the Northern Territory (2016-2018) .186Table 10-4 Residential rural placements of at least 6 weeks duration (2016-2018).187Table 11-1 Rural-origin dental students (2016-2018).199Table 11-2 Number of DTERP placements and placement weeks completed by Australian students, 2018.200Table 11-3Number of DTERP placements and placement weeks completed by Australian students, by Remoteness Area(RA), 2018.200Table 13-1 Predictors of rural medical practice.220Table 13-2 System level contribution to developing the rural medical workforce.221Table 13-3 Multidisciplinary Health Workforce Survey response by profession.224Table 13-4 Average clinical placement weeks undertaken in ASGS RA2-5 by quintile and discipline.225Table 13-5 Matching estimates—Hours worked in ASGS-RA 2-5 by exposure status, Allied Health.226Table 13-6 Matching estimates—Hours worked in ASGS-RA 2-5 by exposure status, Nursing and Midwifery.226Table 13-7 Main reasons for choice of primary work location—Allied Health.227KBC Australia10
Table 13-8 Main reasons for choice of primary work location—Nursing and Midwifery. .228Table 14-1 Consolidated RHMT program income and expenditure (2018). .240Table 14-2 Overview of programmatic outcomes and benefits. .241Table 14-3 Cost-effectiveness ratio estimation. .250KBC Australia11
List of FiguresFigure 1-1Population growth by remoteness area and age group (2013-2017)†.44Figure 1-2Employed health professionals—full-time equivalent rate, by remoteness area (2017).47Figure 1-3GIRS index scores, Specialist General Practitioners by SA2 (2014).48Figure 1-4Full-time equivalent Registered Nurses, Midwives and Enrolled Nurses by remoteness (2013, 2017).49Figure 1-5Australian Government funded Rural Medical Training Pipeline.50Figure 1-6Commencing enrolments for a General Nursing course by student status (2001–2018).54Figure 1-7Commencing enrolments for an Allied Health† course by student status (2001–2018).54Figure 3-1Key elements of the evaluation methodology.65Figure 4-1Crude RCS placement weeks by year and clinical setting.87Figure 14-1Cost per rural placement week (RA2-5). 247Figure 14-2RCS - Average cost per-placement week by university and proportion in RA3-5. 248Figure 14-3UDRH - Average cost per-placement week by university and proportion in RA3-5. 249Figure 15-1 A university’s sphere of influence. 256KBC Australia12
Executive Summary
Executive SummaryIntroductionThe Rural Health Multidisciplinary Training (RHMT) program is one of several Commonwealthrural health workforce programs aiming to increase the number of health professionals workingin rural, remote and regional Australia.Through the RHMT program and its precursors, the Commonwealth has demonstrated atwenty year commitment to support the training of health students in rural, remote and regionalAustralia as an initial step towards a rural health career. Twenty-one universities are currentlyfunded under the RHMT program, establishing a national network of 19 Rural Clinical Schools(RCSs) and 16 University Departments of Rural Health (UDRHs) to provide the infrastructureand academic networks for teaching and training.The RHMT program operates in a complex and somewhat fragile environment wheregeographic health workforce maldistribution exists. It is a health workforce program deliveredthrough the higher education sector where education policies can take precedence over ruralworkforce policies and as such is impacted by changes in higher education policy and funding.While Commonwealth, state and territory health workforce training and employment policiesintersect with the RHMT program, alignment of these programs is not consistently visible ormaximised.In April 2019 KBC Australia was commissioned to undertake an evaluation of the RHMTprogram to assess the extent to which the current design and delivery of the program isachieving the program’s aim of improving the recruitment and retention of medical, nursing,dental and allied health professionals in rural and remote Australia. In addition, the evaluationhas considered the benefits to local health delivery from engagement in teaching and trainingthrough the RHMT program.The evaluation used a mixed-methods approach. This included interviewing over 980stakeholders in semi-structured interviews, focus groups and roundtables, 30 writtensubmissions from peak and professional bodies, two national electronic surveys(Multidisciplinary Health Workforce Survey 4,000 responses; RHMT program staff survey,411 responses), review of longitudinal workforce data, and review of program reports andexpenditure data. An Expert Reference Group provided advice to the Department of Health(the Department) and the evaluators in the development of the methodology and execution ofthe evaluation.It should be noted that the evaluation was at a program level, not of individual universitiesor organisational units and inherent differences between universities based on the length oftime they had been running and historical funding received was recognised. However, theevaluation found great variability between universities in terms of delivering on targets, qualityof placements, financial support from the individual university to their RCS and/or UDRH and,alignment of the university goals with the RHMT program goals and intent. This may reflectdiffering commitment to the RHMT program at the universities’ central campuses which canrequire strong advocacy from the RCS/UDRH Directors to promote rural training and to bringKBC Australia14
about changes towards program objectives.This evaluation found that overall, the RHMT program has been an appropriate response andimportant contributor to addressing rural workforce shortage. After two decades it is a strongfoundation for rural health workforce training and research in rural, remote and regional areaswhich is now conside
Suite 7, Level 1, 24 Sale Street, Orange NSW 2800 Phone: 02 6361 4000 www.kbconsult.com.au. KBC Australia 3 Acronyms ACCHO Aboriginal Community Controlled Health Organisation ACRRM Australian College of Rural and Remote Medicine Ahpra Australian Health . ANMAC Australian Nursing and Midwifery Accreditation Council APHCRI Australian Primary .
