Transcription
DOCUMENT RESUMEED 116 029AUTHORTITLEINSTITUTIONSPONS AGENCYBUREAU NOPUB DATEGRANTNOTE'EDRS PRICEDESCRIPTORS95gCE 005 944Tuttle, Francis T.The Oklahoma State Department of Vocational andTechnical Education Curriculum Laboratory Grant.Research Project in Vocational Education. FinalReport.Oklahoma State Dept. of Vocational and TechnicalEducation, Stillwater.Office of Education (DREW), Washington; D.C.V25702430 Jun 75OEG-0-72-.4682(361)70p.MF- 0.76 HC- 3.32 PlUs PostageCareer Education; *Curriculum Development;*Curriculum Evaluation; Educational Coordination;Evaluation Methods; Information Dissemination;.*Instructional Materials Centers; Program Budgeting;*Program Evaluation; State ProgramsABSTRACTThe goals of the project were to strengthen thecurriculum management capabilities of the Oklahoma Curriculum andInstructional Materials Center'(CIMC); to expand its efforts towardcurriculum management in career education t 'ough coordination ofefforts in development, dissemination, 4W1 fusion; and to maximizeresources for curriculum management through,c ordination of efforts,in development, disemination; and diffusion. The CIMC conductedseveral activities to meet these objectives: together with theResearch Coordinating Unit, 34 instructional units of curriculummaterials were field-tested to ,determine if the objective-basedinstructional unit-was an effective learning technique; itcoordinated efforts with industry in curriculum development to carryout its career 'edhcation activities; and the Ten-State CurriculumNetwork was formed as part bf the National Network for CurriculumCoordination which focused on new developments in curriculumtechnology. Over.twto-thirds of the document contains favorablethird-party evaluation of the project and related appendixes. Theproject's considered to be strong points were field-testing andrevision of curriculum materials and workshop planning. Theevaluation was conducted through team meetings; meetings with keyCIMC and State personnel, instructional staff involved in pilotingcareer education materials, and the project's advisory committee; anda project audit. (Author/EC)Documents acquired by ERIC include many informal unpublished materials not available from other sources. ERIC makes everyeffort to obtain the best copy available. Nevertheless, items of marginal reproducibility are often encountered and this affects thequality of the microfiche and hardcopy reproductions ERIC makes available via the ERIC Document Reproduction Service (EDRS).EDRS is not responsible for the quality of the original document. Reproductions supplied by EDRS are the best that can be made fromthe original.
.v.S DEPARTMENT OF NE ALTH.4EDUCATION & WELFARENATIONAL INSTITUTE,OFEDUCATION-THIS DOCUMENT HAS BEEN. REPRO.DUCED EXACTLY AS RECEIVED FROMFINAL:REPORTProject No.:Grant No.:THE PERSON OR ORGANIZATION ORIGINATING IT POINTS OF VIEW OR OPINIONSSTATED DO NOT NECESSARILY REPRE.SENT OFFICIAL NATIONAL INSTITI4TE OFEDUCATION POSITION OR POLICYV257024OEG-0-72-4682 (361)The Oklahoma State Department of VocationalAnd Technical Education. Curriculum Laboratory GrantResearch Project in Vocational EducationConducted UnderPart I of Public Law 90-576.The project report herein was performed pusuant to a- gr t from the Office ofEducation, U.S. Department of Health, Education, and Welf re. Contractors orgrantees undertaking such projcts under Government spons rship are encouragedto express freely their professional judgment, in the conduct of the project.Points of view or opinions stated do not, therefore, necessarily representofficial Office of Education position or policy.4Francis T. TuttleOklahoma State Department]Departmentrlocational and Technical Education1995AvenueStillwater; Oklahoma 74074June 30, 1975kt2.o
i!rTable of ContentsPageSummary of Report1Body of Repbrt3Table I7o.4-, ".:',.t.I.Oc:4aa
SUMMARY OF THE REPORT.1.\a.Time Period Covered by the Report: July 1, 1972 to June 30, 1915b.Goals and Objectives of the Project: The goals of the project were:(1) To strengthen the curriculum management capabilities of the CurriculumndInstructional Materials Center; (2) to exparKI curriculum management efforts ofthe Curriculum and Instructional Materials Center in Career Education throucoordination of activity in development, dissemination and diffusion; (3)tR,maximize resources for curriculum management through coordination of effortsin development, dissemination, and diffusion.c.Procedures followed to achieve the objectives: The Center'S staff concentrated .their.efforts on: (1) implementing the Management by Objectives concept, for administeringthe responsibilities and functions of the center; (2) collecting and evaluatingcurriculum material presently on the shelf; (3) developing curriculum for skirl trainingand for Career Education; (4) validating and disseminating materials through.appropriate channels; (5) in-service and Rreservice training of personnel tomaterialto promote diffusion; (6) coordinating the development 'of curriculum to preventduplication; and (7) dissemination of materials and diffusion of the philosophy anduse of educatiohal technology in curriculum materials for the 10 State Curriculum.Network of the National Network for Curriculum Coordination in vocational andtechnical education.\Additional personnel was employed to increase the capabilities of the Oklahoma CIMCto develop curriculum for Career Awareness, Career Exploration, Guidance andCounseling and Vocational Education and to provide in-service and preservice trainingfor teachers and state staff in the use of curriculum materials.d.,Results and Accomplishments: The major results and accomplishments were:(1)Adoption of the "Management by Objectives" system to increase efficiency ofCIMC management.k.(2)Employment of additional staff. Increases in staff were:Assistant Coordinator--1Librarianfrom 1/2 time to full timeto 21/2 to 3Editors--fromArtists- -from11Secretarial - -from' 2 ,to 6, four of which were not paid with grant money.(3)Increased development of curriculum materials for vocational education;(4)Validation' of career education materials in Oklahoma. schools.(5)Development of curriculum materials for career education.(6)Dissemination of career education materials.(7)Diffusion of the career education concept through three state-wide workshops.(8) irnplemention of the regional network concept for curriculum and instructionalmaterials coordination and management.s.1'4.
2(9)Development of a curriculum evaluation model.Career education curriculum guitles were gleveloped in the following areas: CareerAwareness, Came; Exploration, and Guidance and Counseling. The materials were fieldtested by seven school systems in Oklahoma and revised by the curriculum specialists.DiffusiOn of the career education Concept was accomplished through career educationarticles; speeChes and workshops in-state and out-of-state, and preservice presentations inuniversity classes.Manuals for sixteen vocational education programrevised.were developed; three manuals were.e.Evaluation: A third-party evaluation was conducted by a team of four members fromthe areas of industrial education, computer science education, vocational education,and Vocational business and office education. The team members collected dataprimarily through s'tructured interviews with various teaching and counseling personnelthroughout the state who .lead assistbd in the field-testing of career education materialsand had utilized the services of the center, for determining the validity of the project.f.Conclusions and Recommendations: The total project received support from classroom.teachees, teacher educators, administrators, and state department personnel in assessingJteeds, development of instructional materials, and the field testing of career educationmaterials. All states in the 10 State Curriculum Network reported increased activity.in Curriculum. Examples of such activities are: (1) establishment of curriculumlaboratories where there had been none; (2) exchange of instructional materials amongthe states; and (3) representation of all states ,ori numerous curriculum committees.Additional positive comments' and recommendations are located in the third-party'evaluation report.,.
NBODY OF THE REPORTa.Statement of the ProblemThe major purpose of education is to prepare young people to live satisfying andproductive lives. However, many young people are leaving school without.entry levelsjob skills, the behavioral characteristics, and the basic general knowledge necessaryto live and participate successfully in society.,In typical schools throughout the country young people have charged that curriculumsare dull and irrelevant and that their education is not equipping them for a rewardingand productive adulthood. Some parents and teachers also question the value ofeducational experiences in preparing students to live useful -and satisfying lives.Employers are finding that young people are poorly trained in the occupational skillsand are lacking the behaiiioral characteristics necessary to perform competently inthe world of work.4These problems underscore the need for reorganizing the total curriculum and findingmeans for developing and testing promising, innovative instructional materials toprepare the stadent to enter the job market with a salable skill and/or to continuehis/her education.To meet this need the Oklahoma State Department of VoCational and TechnicalEducation proposed to: 1) strengthen the present curriculum management capabilitiesof the Oklahoma tariculum and Instructional MaterialsSenter to further *researchthe social and theoretical foundations for development and implementation ofcurricula, and 2) blend the present curricula in vocational educational education,general education, and college preparatory education into one curriculum.6
%b.Goals and ObjectivesThe project goals were:1.To strengthen the curriculum management capabilities of the OklahomaCurriculum and Instructional Materials Center.2.To expand the efforts of the Oklahoma Curriculum and InstructionalMaterials Center toward curriculum management in career education throughcoordination of efforts in development, dissemination and diffusion.3.To maximize resources for curriculum management through coordinationof efforts in development, dissemination and diffusion.Objectives used to .accomplish the- project 'goals were:.,e1.To improve the curriculum management capabilities of the OklahomaCurriculum and Instructional Materials Center.2.To coordinate career education efforts within the state- including collecting,developing, evaluating, validating, disseminating and diffusion of curriculummaterials for curriculum management in career education and establishingchannels of communication for transmitting results of the aforementioned. activities.3.To coordinate more effectively the development o-f curriculum activitiesin order to prevent duplication, provide information dnd materials throughdissemination to each participating state of the Ten State CurriculumNetwork and diffuse the philosophy of educational technology in the useof curriculum materials.? 1.
5Ic.Description of the general project design and proceduresProcedures for accomplishing objective #1:The "Management by Objectives" approach for administering the responsibilities andfunctions of the Center was instituted to increase the efficiency of operation.Operating under the Management by ObjeCtives system involved:.1.Planning the desired accomplishments and setting terminal dates for theirachievement.2.Executing the plan within the time lines:3.Reviewing and evaluating results.Increased efficiency of operation allowed a small amount of time for superviiing theuse of curriculum materials.PersonnelAn assistant coordinator was employed to expand the capabilities of the Oklahoma CIMC.His major responsibilities were to coordinate and increase ,development of curriculummaterials for skill preparation and to coordinate in-service and preservice training in theuse of curriculum materials. This training provided for more individuali-zed instruction forteachers and increased the opportunity for evaluating curriculuM forjevision. Intensivein-service training through workshops for district supervisorS in,thp vocational and technicalareas of training afforded more and better supervision for teachers in the use of material.The Curriculum and Instructional .Materials Center cooperated with other state agenciessuch as the Research Coordinating Unit. Two projects resulted from this joint effort: 1)validating curriculum materials, and 2) developing a system for task analysis of incumbentworkers , in business and industry. A description of these projects follows.Validation of Curriculum Material Through Field TestingThe Curriculum, and Instructional Materials Center in cooperation with the ResearchCoordin4ting Unit validated selected units of curriculum materials through field testing.A model was developed by research assistants employed by the RCU and field testingwas supervised by CIMC staff.The pu'rpose of the validation project was to develop a model curriculum evaluation systemand to test its efficiency on instructional materials developed at,the Oklahoma Curriculumand Instructional Materials Center. The major question the evaluation attempted to answerwas: is the objective-based instructional unit an effective learning technique?Thirty-four instructional units from twelve instructional manuals were evaluated. Pretestand post-test data were collected from 397 high school students. For purposes of theevaluation, three criteria were arbitrarily adopted to define an effective unit: (1) 80%of the students would attain 80% mastery on a criterion. referenced posttest, ige thelowest posttest score is greater than 59%, or (3) the post-ten mean is at least 4R, andthe standard deviation is less to 10.0. Research is needed to determine criteria levels whichwould discriminate between masters and non-masters, however; the selected criteria seemedreasonable' for purposes of testing thc.model:8
6ResultsResults of the testing related to the criteria are reported in Table I. Fifteen of thethirty-four units (44%) tested satisfied at least one criterion for an effective unit. Additionalanalyses with regard to the model and its weaknesses are reported more fully in the finalreport of the project.The fact that only 44% of the units satisfied the criteria for effectiveness, does not implythat objective-based instruction is ineffsctiye. This finding only JaiseetAhe question as towhy some units were effective while Others were not. Three alternative hypotheses wereconsidered: (1) objective-based instructional design is an unreliable education method, (2)individual differences in aptitudes of classes account for differences in effectiveness, or(3) individual differences in teacher ability accounts for the differences.To test hypothesis 2, i.e., differences between classes, a test for differences between meangain scores was computed between classes taught the same unit by the same teacher. Ofthe seven units where two classes were taught by the same teacher, three tests weresignificant. This finding implies that, at least in some cases, large differences did existbetween classes.Post-test means by classes were inspected for each unit. It was noted that mean differencesbetWeen classes on the same unit were often larger than 10% with some differences over20%. Because the effects of teacher differences and class differences are combined, suchdifferences seem to support both hypotheses 2 and 3.Hypothesis 1, i.e., that objective based instruction in unreliable, was also supported byobservation. Correlations between pretest and post -tess in any unit do not seem to bestable or consistent; further, the standard deviations of post-test scores were extremelyhigh.ConclusionsFifteen of the thirty-four units satisfied at least one of, the criteria for an effective unit.An additional six units were -at least marginally effective. In addition, almost all of theunits produced sizable learning gains. Thus, it is recommended that none of the instructionalunits be abandoned.A formative evaluation of the post-test indicated that some units had a number of difficultquestions in the sense that, students answered them incorrectly. These difficult questionsfrequently required the student to solve a problem. A major step in revising the ineffectiveunits Should be tb provide additional instruction in the difficult areas.It is the investigatorikcontention that much of the problem with the instructional unitslies not in poor design', but rather, appears to stem from improper use of the instructionalunits by the teachers. The teachers seem to be using the instructional manuals just asthey would a textbook and norm-referenced test. If the teachers could be pursuaded touse the mastery model with the instructional manuals, the problems would hopefullydisappear. This means that a student who does not achieve the mastery level the firsttime he takes the unit test should be given remedial work in his area of weakness. Onlywhen he reaches mastery should he proceed with the next unit. The mastery model meansmore work for the teacher, however, the work load could be reduced by designing remediallessons into each unit. 01?jective based units with criterion tests are ideally suited forthis type of program.
7TABLEI"UNMAIRY STA 1ICS FOR ALL UNITSManual & UnitNAgriculture I** Livestock Industry38Livestock Feeding38** Plant &.Soil Sciende15Agriculture III* Market Grades26Marketing Livestock26Agriculture IVSOurces of Credit38Legal Land Description 38** Farm Utility Building 38Auto Mechanics** Brake Systems28Ignition Systems28** Fuel Systems28Business and Office** Telephone Communica.51** Letter Writing51* Reference Materials51Child Development** Care of Mother37* Parent Responsibility 37* Infancy35Distributive Education III** Marketing.16Research Procedures'16** Capital Resources16** Financial Statements16Home Economics 1Personal Income86Managing Income86Machine ShopSafety10** Engine Lathe10Nursing** Vital Signs17** Pati'ent's Environment17,** Patient's Cleanliness 17Residential Carpentry* Roofing Materials31Exterior Wall Covering 31Roof Framing31Welding* OxyaCetylene Cutting19Braze Welding19Metal Arc Welding19.Students atlowMeanS.D.O% %79%3284.8t9.768%78.292.217.2 86.873.298.787.69.7747.100'' 4645010010010083%533960477. 775.414.216.112.690%s-** This unit met at least one of the criteria for an effective unit.* This unit approached the criteria anti may be considered marginallyeffective.97100100100.
8Several pieces of information infer that teachers are not using the mastery model. (1)there were extremely lOw scores in twenty* of the thirty-four' units which indicat thatsome students were not achieving mastery, (2) twelve,of the thirty-four units hay largestandard deviations indicating student grades are strung out-rover a 'very wide ran , and(3) personal communications with several of the teachers indicated their la& of fami lathywith concepts of the mastery model,Recommendations1.Testing of the evaluation model should be repeated with much more stringentcontrols over differences, such as teacher differences and class differences, whichmight relate to the effectiveness criteria. The criteria should also be researchedto discover the point which discriminates between mastery and non-mastery.2.There are many indications in the data that the criterion referenc unit testsmight not be reliable instruments. It is recommended that in:service trainingfor the Center staff with regard to statistical properties:of test constructionshould be accomplished. The key problem in conducting,any evaluation is todevelop or obtain a reliable and valid measuring instrument.3.It is strongly recommended that the evaluation model become a standard partof the curriculum design process. Although there are bugs in the model, thereis no reason why it cannot be. used Wyi.ts present form. The data from thepresent evaluation could te used toevise all of the units which were tested.4.It needs to be emphasized that the instructional material in each of the unitsevaluated appeared to be pedagogically sound. Improvement in the units couldbe made by adding remedial sections to each unit, providing a diagnosticinterpretation for the unit tests, and providing more detailed information sheets.The second project, developing a system for task analysis of workers in businessand industry is in progress at this time.IndustryThe Oklahome Curriculum and Instructional Materials Center had direct involvement withindustry in curriculum development. The Associated General Contractors of America andthe Oklahoma Curriculum and Instructional Materials Cente.r have developed instructionalmanuals for commercial carpentry and cement masonry. Don Diehl, Construction leSuperintendenp with Ringland, Johnson-Crowley Company, Des Moines, Iowa, representing 4"the Associated General Contractors, served as a member of the State Department of:Vocational and Technical EducatiOn staff to write commercial carpentry curriculum.Commercial carpentry is a course of study designed to train carpenters who,will maketheir careers in industrial and commercial construction.Laborn "Bud" Hendrix, Concrete Superintendent with the George Bahre ConstructionCompany in Indianapolis, Indiana, served as a member of the State Department ofVocational and Technical Education staff for eight months. He wrote the cement masonrycurriculum. which is a course of study for use in secondary and post secondary levelprograms and is designed to train apprentices and to upgrade skills in cement masonry.1i0
f.or.Procedpres for Accomplishing *Objective #2.An assistant coordinator, three curriculum specialists, and a media resource specialist (1/2time) were employed to carN out the activities of the career education component for the center.'The training of the staff consisted of, "Behatiioral 'objectives," and "Writing A Unit of:"Instruction," (Units of instruction developed by the center staff for in-service training),the Vimcet Series of filmstrip nd tapes on curriculum by Popham and Baker, and"Designing Effective Instructi ," a 30-hour programmed workshop published by GeneralProgi-ammed Teaching, a di 'mon of Commerce Clearing HOuse, Inc. The 30-hour workshopwas directed and instrud by the center's coordinator and two assistant coordinators.StaffThe assistant coordinator was qualified both by experience and training In curriculumdevelopment and admihistration. Her major responsibilities included:, (1)0 assisting theCoordinator in coordinating the activities of the Curriculum .and Instructional MaterialsCenter, and representing the coordinator when necessary; (2) coordinatihg the in- serviceand preservice training as needed by the center and training of new personnel; and (3)coordinating the development of Career Education Curriculum for the Oklahoma Center,Curriculum SpecialistsThe three curriculion specialists for career awareness, career exploration, and counselingand guidance were qualified both by formal training and experience. The career awarenessspecialist held a master's degree in elementary education and had-six years of teachingexperience. The career exploration specialist held a bachelor's degree in industrial artsand had three years of teaching experience. The guidance specialist held a master's, degreein guidance and counseling and had five years of classroom teaching and counselingexperience. The specialists' major respontibilities included: collecting, developing, diffusing,disseminating and revising career education curriculum material. Diffusing the careereducation concept in Oklahoma 'schools was an added responsibility. The specialists alsoserved as career education consultants as needed.A Career Education Advisory Committee was organized to provide input for dire-ctionsin curriculum development. The committee established priorities for curriculumdevelopment and helped establish channels of communication for dissemination of materialsand career information. Quarterly meetings of the committee and career education staffwere held to explore avenues of cooperation with business, industry, labor, the EmploymentSecurity Commission, the Manpower Division (located in the Vo-Tech complex), and otheredion agencies, such as Oklahoma State Regents of Higher Education, State Boardof Vo tional and technical Education, Oklahoma Curriculum Improvement Commission,State Department of Education, Guidance and Counseling Service, Adult Basic Education,and The Oklahoma Education Association (see section e. Evaluation of the Project foradvisory committee members). In .addition to the Career Education Advisory Committee,each curriculum specialist organized an advisory group for her/his particular area.Elementary teachers, principals, and teacher educators served on the career awarenesscommittee; industrial arts teachers (junior high or middle schooTand teacher educatorsserved on the career exploration committee, and counselors and counselor educatownrvedon the counselor's committee.12'4
.Diffusion of the career education curriculum was accomplished through in-serviceworkshops and preservice training for teachers at the university (level. In-service andpreservice training was devoted to appropriate teaching techniques of the material.Dissemination was accomplished through workshops and regionalization.Validation was accomplished through %the advisory committee for the career educationstaff, advisory committee for each curriculum specialist, and the field-testing of the materialby nine elementary schools, six junior high schools, six secondary counselors, and fourcounselor education 'workshops (see Section e. Evaluation of the Project).Procedures, for Accomplishing Objective #3Ten State Curriculum NetworkThe Ten State Curriculum Network was formed as a part of, the National Network forCurriculum Coordination. The participating states were: Arkansas, Colorado, Kansas,Missouri, Nebraska, New Mexico; North Dakota, Oklahoma,sSduth Dakota and Texas. Thefollowing procedures and policies were adopted:1-Each' participating state will have one contact person to work with the OklahomaCurriculum and Instructional Materials Center.2.The Oklahoma 'Curriculum and Instructional Materials Center will publish aquarterly newsletter (the third month in each quarter) which will include thefollowing items:a.Listing of curriculum projects under development, in each participating stateincluciing information such as title of project and description of materialas to grade ;level -and content. Each state will report on its curriculumprojects, and other news to the Oklahoma Center by the end of the secondmonth of each quarter.b.Listing of activities and accomplishments of the other regional laboratoriesalong with information selected by Vocational Instructional Materialssections of American Vocational Association.All curriculum and instructional material developed by the Oklahoma Curriculumand Instructional Materials Center as.well as from each of the participating stateswill be developed in accordance with the standardized format. The format shallconsist of -the. following components:a.Behavioral objectives.,Suggested activities for bbth ihstructor'and student.c.Iriformation (coihent) necessary for reaching each objective.d.Assignment isheas 'for application of problem solving approaches.e.f.vwJob;, Sheets for rdotor- perfcrrhance skills.Evaluation (tts) fdr measuring the objectives.,10
114.The Oklahoma Curriculum anenstructional. Material's Center will send one copyof each publication produced by the Center to each representative of theparticipating states.5.The Oklahoma- Curriculum and Instructional Materials Center will providematerials developed by the Center at special rates for quantity purchases toparticipating states.6.a.Teachers manual - Fifty or more in any combination. Original order eachyear will receive forty percent discount.'b.For guaranteed delivery student material orders must be made one yearprior to our August 1 printing date. Approximate cost will be 3 per set.Other orders will be filled subject to the availability of materials.Additional services to be provided by the Oklahoma Curriculum and InstructionalMaterials Center are to:N.a.Collect and disseminate priorities as submitted to participating states.b.Provide consulting services to participating states at each state's expbnse.c.Notify contact personnel' in each participating state prior to curriculumcommittee meetings inviting them to send representatives for the evaluationof materials- under development--0Disseminate the evaluation of curriculum materials to each participatingstate by using an evaluation sheet to be developed.and validated .by theOklahoma-Center.e.Establish a method ,of negotiating contacts for printing and disseminationof instructional materials developed by representatives of participatingstates.The participating states were encouraged to assess their vocational-technical curriculum'development needs and report to the Ten State Curriculum Network.New developments in curriculum technology (design, validation, testing, infution, methods)were a prime focus of the Ten State Curriculum Network. As new developments wereperfected they were disseminated through the Network to curriculum specialists who inturn applied them in their developmental wdrk program. Each CeRter has an advisorygroup which is
AUTHOR Tuttle, Francis T. TITLE. The Oklahoma State Department of Vocational and. Technical Education Curriculum Laboratory Grant. Research Project in Vocational Education. Final. Report. INSTITUTION. Oklahoma State Dept. of Vocational and Technical. Education, Stillwater. SPONS AGENCY. Office of Education (DREW), Washington; D.C. BUREAU NO .
