WIXLIB

Culminating Experiences: A National Survey of Accredited InstitutionsElizabeth BjerkeUniversity of North DakotaABSTRACTDuring the summer of 2007 the Aviation Accreditation Board International (AABI) approved a newset of criteria to be used in accrediting aviation degree programs. One addition to the new criteria is theinclusion of creating a culminating experience for all aviation program options. The purpose of this studywas to discover the importance placed on incorporating a culminating experience in the aviation programcurriculum, and to determine which of the general outcomes were being assessed. The results of thesurvey (N 55) indicate that all programs felt the culminating experience was an important part of theirdegree program. However, there was a variance in the number of outcomes assessed and theappropriateness to assess each outcome.project thatbuilds on prior course work.Evidence may include student portfolios andother records of student achievement. (p. 19)Prior to the new criteria, only the AviationManagement option required a culminatingexperience.Another addition in the new criteriaincluded 10 general outcomes for whichgraduates of accredited aviation programs mustdemonstrate either ability or knowledge. TheAABI general outcomes are often referred to as‘outcomes a-j’, thus the remainder of this articlewill post the corresponding letter to theoutcome. Refer to Table 1 for a listing of theAABI approved general outcomes at the time ofthis study.The purpose of this study was to discoverwhat types of culminating experiences aviationprograms were incorporating into theircurriculum, and to determine which of thegeneral outcomes were being analyzed, and howstudent learning was being assessed.INTRODUCTIONIn July of 2007, the Aviation AccreditationBoard International’s (AABI) Board of Trusteesapproved the new criteria based accreditationmanual (AABI, n.d.). The new outcomes-basedcriteria replaced an older standards basedapproach to accrediting institutions of highereducation. Outcomes based criteria places agreater emphasis and accountability on studentlearning, rather than focusing on curricularinputs.AABI is a specialized accreditor thatfocuses on collegiate aviation education for bothtwo-year and four-year, non-engineering relatedaviation programs. Specialized accreditation isnot mandatory for institutions offering aviationeducation; in fact only 26% of UniversityAviation Administration’s member institutionshave AABI accredited programs (Prather, 2008).When seeking aviation accreditation there aresix program options available: AviationManagement, Aviation Maintenance, AviationElectronics, Aviation Studies, Flight Education,and Safety Science.One significant change in the new AABICriteria Manual is the inclusion of a culminatingexperience requirement for all program optionsfalling under the baccalaureate criteria. TheAABI (2009) culminating experience criterion isas follows:Each program MUST provide evidence of asignificantculminatingupperdivisionexperience in (AABI Program Option).Examples of a culminating experience include acapstone course, an internship, or a specialREVIEW OF THE LITERATUREA culminating experience, or capstonecourse, is an ideal part of the curriculum thatallows students to demonstrate mastery of theknowledge and skills acquired during theireducational journey. It can also serve as a toolfor assessing program learning outcomes, aswell as aiding faculty in conducting overallprogram evaluations of existing curriculum.Although culminating experience outcomes tendto be similar across fields of study, the approachtaken may differ.9

Table 1. General OutcomesAABI General Outcomesa. An ability to apply knowledge of mathematics, science, and applied sciencesb. An ability to analyze and interpret datac. An ability to function on multi-disciplinary teamsd. An understanding of professional and ethical responsibilitye. An ability to communicate effectively, including both written and verbal communication skillsf. A recognition of the need for, and an ability to engage in, life-long learningg. A knowledge of contemporary issuesh. An ability to use the techniques, skills, and modern technology necessary fori. An understanding of the national and international aviation environmentj. An ability to apply pertinent knowledge in identifying and solving problems.Note. From “Accreditation Criteria Manual”, by Aviation Accreditation Board International, 2008, Form201, p. 7.Pocock, Zuraski, Meade, Mitchell & Farrington,2002; Padmanabhan & Katti, 2002). In asynthesis of research conducted after the initialAccreditation Board for Engineering andTechnology (ABET) criteria change in requiringa ‘senior-level design course’, it was found thatalthough a proliferation of new courses werecreated to meet the accreditation requirement,the format tended to vary greatly betweenprograms (Dutson, Todd, Magleby & Sorensen,1997). In a follow up study, Howe andWilbarger (2005) surveyed ABET accreditedinstitutions and found similar differences informatting nearly a decade later. The surveyalso revealed a stronger trend towards industryinvolvement.McKenzie, Trevisan, Davis and Beyerlein(2004) conducted a national survey of ABETaccredited institutions regarding their use of the‘senior-level design course’ in assessing theABET general outcomes (a-k). It was found that92% of the respondents felt that the capstonecourse played an important role in the overalleducational experience of the students. Thestudy also broke down the role in which theculminating experience played in the evaluationof each general outcome. Due to the likenessand similarities between the ABET generaloutcomes and the newly approved AABI generaloutcomes, permission to revise and use thissurvey tool for aviation education was obtainedfrom the lead author. The survey tool was usedto assist in answering the following researchquestions:Strategic Management or Business Policy isa typical capstone course for a businessmanagement degree program (Parente, Brown,& Warner, 2005). Due to the large number ofaccredited business programs there is anextensive amount of literature focusing on theimplementation and forms of assessment ofbusiness capstone courses. Payne, Whitfield andFlynn (2002) propose a four phase approach toassessing business capstone courses combiningthe scholarship of teaching and learning withstakeholder theory. The four phases articulatedwere as follows: 1.) explore perspectives andpractices elsewhere, 2.) examine institutionalfaculty perceptions and curricular concerns, 3.)discover student perceptions, and 4.) explorebusiness community stakeholders. A moretraditional approach to assessing studentlearning in a business capstone course is throughcompetency based testing (Parente et al., 2005).Yet in another study (Payne, Flynn, & Whitfield,2008) student’s were interviewed when enteringa capstone course in order to assess their degreeof motivation. This approach aided faculty indeveloping and changing the course to fulfill theperceived needs of the students.While the field of business managementeducation tends to favor a capstone courseapproach to the culminating experience, thediscipline of engineering employs a seniordesign project as its culminating experience.Many of these senior design courses workclosely with industry to solve real worldproblems (Todd & Magleby, 2005; Jenkins,10

tasked with assisting AABI members during thetransition to an outcomes-based accreditation.A sample of the survey tool used is foundin the appendix of this article.1.) What percentage of aviation programs havea defined culminating experience in theircurriculum?2.) How important is the culminatingexperience in the aviation program?3.) Which AABI general outcomes areconsidered appropriate to assess in aculminating experience, and which AABIgeneral outcomes are being assessed in theculminating experience?ProceduresA survey packet was sent out via first classmail to 23 accredited institutions which arelocated in the United States. An identical packetwas sent electronically to the two internationallyaccredited programs. The packet consisted ofthe following items: cover letter from researcher,support letter from AABI, separate surveyquestionnaire for each aviation accreditedprogram at the institution and a self addressedstamp envelope for return purposes.Forexample, if an institution had three differentaviation accredited programs such as AviationManagement, Flight Education and AviationStudies, it would receive three separate surveytools to fill out. The rationale behind thismethod is that these are three different programswhich may all have different culminatingexperiences defined.The completed surveys were imported intoboth Microsoft Excel and SPSS version 16.0 fordata analysis. In order to answer the statedresearch questions, both descriptive andinferential statistics were used. The followingsection states the results.METHODOLOGYParticipantsDuring the fall of 2008 a survey wasadministered to all four-year AABI accreditedinstitutions. A current listing of accreditedprograms was obtained from AABI. Onlycurrently accredited programs were chosen forthis study in order to better assess theimplementation of the new outcomes-basedcriteria. At the time of this study there were 25baccalaureate institutions consisting of 79accredited programs. However due to thetimeframe, no schools were yet accredited underthe new criteria.MaterialsThe survey tool used was adapted, withpermission, from a similar study conducted forengineering education (McKenzie et al., 2004).Due to the similarity between the ABET and theAABI criteria, the survey tool was easilyadapted for aviation education.By replicating a previously conductednational study (McKenzie et al., 2004), manyconcerns of validity and reliability wereaddressed. However, the survey tool was alsopiloted by the Outcomes Resource and Trainingcommittee of AABI. This committee was madeup of educators and industry representativesFINDINGSOf the 25 institutions holding AABIaccredited status, 16 (64%) responded to thissurvey. More importantly, since the data wasanalyzed at the program level, of the 79 differentaccredited programs surveyed, 55 (70%)responded. Table 2 lists the program optionswhich responded to the survey.Table 2. Program OptionsProgram OptionAviation ManagementFlight EducationAviation StudiesAviation MaintenanceAviation ElectronicsNumber Responded1815114311Percentage of Overall Responses35.3%29.4%21.6%7.8%5.9%

The mean of the response on a five point scalewas 4.75, with 5 indicating the highest level ofimportance.In order to fully understand the role of theAABI general outcomes in the culminatingexperiences defined by each program, a series ofquestions were asked. First the respondentswere asked which general outcomes theyconsidered were appropriate to assess in thecourse.The mean numbers of outcomesconsidered appropriate to assess for theprograms was 6.88 with a standard deviation of2.18. Next the respondents were asked toidentify which of the general outcomes they willassess in their culminating experience. Themean of the number of outcomes in which theywill assess was 6.45 with a standard deviation of2.27. Table 3 lists the results in order of greatestpercentage to least percentage of assessments,and Figure 1 gives a graphical representation.NumerousAnalysisofVariances(ANOVAs) were run between the programoptions and different variables on the surveywith no significant differences being found.Thus the rest of the data analysis uses the entiredata set together, and does not distinguishbetween program options.Of the programs surveyed, 86.3% alreadyhad a culminating experience defined in theircurriculum. The majority of the programs(60.8%) are utilizing a capstone course, whileanother 19.6% are using a combination ofmethods to fulfill the requirement such asvarious courses or an internship experience. Thevast majority (82.4%) of the programs areconducting the culminating experience withinone semester.When asked of the importance that theculminating experience has on the institutionsoverall program all programs answered witheither an important or very important response.Table 3: General Outcomes and Assessment (N 55)AABI General OutcomeAppropriate to Assess inCulminating ExperienceWill Assess inCulminating Experiencej. An ability to apply pertinent knowledge inidentifying and solving problems.96%96%b. An ability to analyze and interpret data94%94%e. An ability to communicate effectively, includingboth written and verbal communication skills84%82%c. An ability to function on multi-disciplinary teams80%76%g. A knowledge of contemporary issues71%69%d. An understanding of professional and ethicalresponsibility71%55%h. An ability to use the techniques, skills, and moderntechnology necessary for professional practice69%65%i. An understanding of the national and internationalaviation environment45%45%a. An ability to apply knowledge of mathematics,science, and applied sciences41%33%f. A recognition of the need for, and an ability toengage in, life-long learning37%29%12

Figure 1. Outcomes in Graphical Form.Further analysis was conducted todetermine if there were any significantdifferences between general outcomes that therespondents felt were appropriate to assess,however were not planned on being assessed. Aseries of paired sample t-tests were performedbetween each paired variable.Only twovariables were found to have a significantdifference in the paired testing: d.) anunderstanding of professional and ethicalresponsibility and f.) recognition of the need for,and an ability to engage in, life-long learning.The most significant difference occurredfor the general outcome concerning professionaland ethical considerations. While 71% ofresponders felt the outcome was appropriate toassess, only 55% stated that they plan to assess.This resulted in a t(55) -3.050, p .004. Theother outcome regarding life-long learning had37% of the responders stating they felt itappropriate to assess, however only 29% statedthat they plan to assess that outcome in theirculminating experience. This resulted in a t(55) -2.063, p .044.DISCUSSIONS ANDRECOMMENDATIONSAlthough the criteria of requiring aculminating experience are rather new toaccredited aviation programs, the vast majorityof accredited programs have already identifiedand/or created the experience in theircurriculum.The programs surveyed allindicated the importance of this experience intheir overall aviation education curricula.Aviation programs also appear to be tying theassessment activities of the culminatingexperience to specific AABI general outcomes.Of the 10 listed AABI general outcomes,schools plan to assess over six outcomes onaverage in their culminating ess to assess and planning to assessneeds to be further evaluated. For instance,when evaluating the two variables mentioned inthe previous section (professional/ethicalconsiderations and life-long learning) twoquestions occur; why do significantly moreprograms feel that some outcomes areappropriate to assess yet have no plan to assessthese outcomes in their culminating experience?If these goals are not assessed in the culminatingexperience, when will they be assessed? Lastly,13

students for a successful career in the aviationindustry.aviation educators need to collectively determineand discuss what are some appropriate andeffectives means of which to assess these lesstangible general outcomes.Prather (2008) found that the aviationindustry as a whole is not fully aware of AABIor the role that it plays in accrediting aviationprograms. Perhaps by following the lead takenby both business and engineering education,who closely tie their culminating experienceswith industry support, aviation programs couldbenefit as well. It could prove to be the catalystneeded to achieve better industry recognitionand awareness of collegiate aviation programs.Further research needs to be conductedconcerning culminating experiences in aviationeducation. A qualitative study consisting ofinterviews and document analysis of variouscapstone course syllabi would help facilitate aneven more in-depth discussion on currentpractices and assessment techniques. Anotherarea for further research would include thenumerous non-accredited aviation programs todistinguish if any of them currently have, or planto incorporate a defined culminating experiencein their programs. Lastly, a similar follow-upsurvey needs to be conducted after all currentlyaccredited programs complete their initial reaffirmation of accreditation under the newcriteria.Aviation programs may use thatprocess as a time of reflection to change theirexisting culminating experience to better assessthe AABI general outcomes.CONCLUSIONAs aviation educational programs embarkon this new path toward outcomes-basedaccreditation, careful thought and planningneeds to be exercised in meeting the newcriteria. Aviation programs should use thisopportunity to create effective and worthwhileculminating experiences to not only meet thenew requirements of AABI accreditation, but toimprove the quality of the educationalexperience for its students. By participating indialogues with other aviation educators, anddetermining best practices in the field, allaviation programs can add a new element totheir curriculum in order to best prepare its14

REFERENCESAviation Accreditation Board International. (2008). Accreditation Criteria Manual, Form 201. Availablefrom the AABI website, http://www.aabi.aero/forms.htmlDutson, A., Todd, R., Magleby, S. & Sorenson, C. (1997). A review of literature on teaching engineeringdesign through project-orientated capstone courses. Journal of Engineering Education, 86(1), 17-28.Howe, S. & Wilbarger, J. (2006). 2005 national survey of engineering capstone design courses.Proceedings of the American Society of Engineering Educators Annual Conference, Chicago, IL.Jenkins, S. R., Pocock, J., Zuraski, P., Meade, R., Mitcherll, Z. & Farrington, J. (2002). Capstone coursein an integrated engineering curriculum. Journal of Professional Issues in Engineering Educationand Practice, 128(2), 75-82.McKenzie, L., Trvisan, M., Davis, D. & Beyerlein, S. (2004). Capstone design courses and assessment: Anational study. Proceedings of the American Society of Engineering Educators Annual Conference,Salt Lake City, UT.Padmanabhan. G. & Katti, D. (2002). Using community-based projects in civil engineering capstonecourses. Journal of Professional Issues in Engineering Education and Practice, 128(1), 12-18.Parente, D., Brown, R. & Warner, A. (2005). Review sessions and results: Competency testing in thecapstone business course. Journal of Instructional Psychology, 32(3), 233-243.Payne, S., Whitfield, J. M. & Flynn, J. (2002). Assessing business capstone course through a methodbased on the SOTL and the stakeholder process. Journal of Education for Business, 78(6), 69-74.Payne, S., Flynn, J. & Whitfield, J. M. (2008). Capstone business course assessment: Exploring studentreadiness perspectives. Journal of Education for Business, 83(1), 141-146.Prather, C. D. (2008). Stakeholder perceptions of specialized accreditation by the Aviation AccreditationBoard International: Part two – aviation students and industry employers. Colleigiate AviationReview, 26(2), 69-77.Todd, R., & Magleby, S. (2005). Elements of a successful capstone course considering the needs ofstakeholders. European Journal of Engineering Education, 30(2), 203-214.15

APPENDIXCulminating Experience SurveyInstitution Demographics1. Name of Institution:2. AABI Program Option:3. Degree Title:4. In what year did or will you have your first accreditation visit under the new AABI outcomes basedcriteria?5. How many students are currently in this aviation program option?Culminating Experience Questions6. Does your program currently have a culminating experience defined asrequired per AABI Criteria 4.0? (Please circle) Yes NoIf not, are you currently in the process of developing such an experience?7. What best describes your program’s culminating experience? Capstone Course Internship Special Project Other (specify)8. What is the duration of your culminating experience? 1 semester 2 semesters other (please specify)9. How important do you feel this course is for your program? (circle a number)Not Important 12345 Extremely Important10. Which of the following AABI defined general outcomes do you consider appropriate to assess inthe culminating experience?(Check all that apply) a. An ability to apply knowl

aviation programs. Specialized accreditation is not mandatory for institutions offering aviation education; in fact only 26% of University Aviation Administration's member institutions have AABI accredited programs (Prather, 2008). When seeking aviation accreditation there are six program options available: Aviation