Transcription
Volume 3, 2000ISSN: 1523-4320Approved October 2000www.ala.org/aasl/slrThe Effects of Concept Mapping on theSearching Behavior of Tenth-Grade StudentsCarol A. Gordon, Head Librarian, Educational Resources Library, Boston UniversityA qualitative study addressed the effect of concept mapping on the searching behavior of tenthgrade students engaged in research projects based on their instruction in a classroom-basedgenetics unit. The setting was an automated library of a private American school in Europe. Tenbiology students were chosen by purposive sampling. Selection criteria, monitored by userprofiles, included student age, computer experience, native language, grades, and test scores.One group used concept mapping, while the other received the same classroom instructionwithout mapping. Data on searching behavior were collected using think-alouds, interviews,debriefing, and journals. Calculations based on Bayesian statistics and the Fano measure frominformation theory were triangulated with qualitative analysis of data. While the entire researchprocess, as defined by Kuhlthau’s model, was examined to include stages from pre-focusformulation to writing the research paper, this article centers on the information search. Fano’sinformation measures showed there was a greater probability that concept mappers will useprint rather than electronic means, that they will search in SIRS rather than the OPAC, and thatin electronic searching they will use subject heading rather than keyword. In print, as opposed toelectronic searching, measures showed mappers applied a larger number of search terms,employed opening moves, re-formulations, search operations, and relevancy judgments moreoften and executed more depth than breadth searching. In all cases probability measured at leasthalf a nat (one nat equals 2.718), indicating chances were approaching twice as likely thatsearchers exhibiting these characteristics in print indexes will be mappers. Larger differencesbetween the groups emerged in electronic searching, where mappers spent less time.Quantitative data verified mappers were more thorough and efficient, reformulating by shiftingsynonyms and moving from general to specific search terms and terminating searches to readrather than when they depleted their search terms. Stronger focus formulation emerged as themost important determinant of searching behavior. Further research is recommended toreplicate the study with a larger sample, using information theory as an alternative to classicalstatistics in hybrid qualitative-quantitative studies.The ProblemThe paradox confronting students-as-information searchers is that they must ask for what they donot know. Without a conceptual framework buttressed by enough knowledge to generate searchterms, students have difficulty locating and recognizing relevant information. To build aknowledge base, however, new knowledge must be integrated with prior knowledge (Ausubel
Volume 3 ISSN: 1523-43201963). A lack of knowledge results in inability to ask for what is needed and an inability todetermine the relevance of what is found. Will concept mapping, used in the classroom as aheuristic learning device, affect the information searching of ten students? Will it help searchersto overcome an anomalous state of knowledge (Belkin, Oddy, and Brooks 1982)? This studyfollows ten students from a classroom-based unit on genetics to a library-based researchassignment and is based on the belief that inadequacies in searching vocabulary are symptomaticof poor concept formation. Is it sufficient to examine searching behaviors as isolated phenomenawithout benefit of the context of the research process?The question of whether one instructional method is more effective than another is, in itself, aproblem in school library research. A content analysis of the research literature by Eisenberg andBrown (1992) revealed that many of the assumptions upon which we have based libraryprograms have gone untested. The authors identified four areas for future research: (1) the valueof instruction in library and information skills; (2) the skills curriculum should emphasizegeneral information problem solving research and research processes; (3) these skills should notbe taught in isolation; (4) innovative instructional methods and technologies can enhance theteaching of information skills. Eisenberg and Brown also concluded that models that define theresearch process had not been verified.The Process Model for Research (Kuhlthau 1986), verified in longitudinal studies, has filled thisgap over the past decade and is used as the framework for structuring the design of this study.Table 1 outlines the model in terms of the six stages: task initiation, topic selection, pre-focusexploration, focus formulation, information collection, and search closure, which is followed bythe writing.Table 1. Kuhlthau Model of the Search smconfusion,frustration, doubtclaritysense ------------------- specificity-------------------------- increased interestActionsseeking relevant information---------------------------------- seeking pertinent informationKuhlthau’s model documented student progress and how it impacts their confidence and finalproducts. Both thoughts and feelings were considered as searchers advanced from seekingrelevant information to seeking pertinent information. Increased confidence correlates with morefocused papers and higher grades (Kuhlthau 1989). When applied to high-, middle-, and lowachieving high school seniors, the stages seemed to indicate that information seeking was acomplex learning process that involved finding meaning. Aggregate scores from the three groupsverified that thoughts move from general to specific to focused during the search process.Thought processes progressed from ambiguity to specificity, and interest increased after the2School Library Media Research www.ala.org/aasl/slr
Volume 3 ISSN: 1523-4320focus had been formed. It is in the context of this model that student searching behaviors wereobserved and analyzed.While the information search was used as the touchstone for determining how well students wereprepared for the research process in this study, observation and other data spanned the entirescope of Kuhlthau’s model. A qualitative mode was chosen in order to look rigorously at thisexperience in its natural context and to describe it in detail. For this reason the study wasconducted on a limited sample. The intent was to work with two groups of students andgeneralize to existing or new theories of searching behavior, rather than generalizing fromsample to population. The researcher’s intent was to treat each of the ten students as a case studyto track his/her progress through the research process. Consequently, conclusions are limited tothe sample and the study should be replicated to verify findings and for generalizability. Thisarticle will focus on the data culled from studying the searching behavior of concept andnonconcept mappers.Theoretical FrameworkAusubel (1963) noted a distinction between two types of learning: receptive and discovery.Classroom learning has been traditionally dominated by receptive, or rote, learning wherebywhat is to be learned is presented to, rather than discovered by, the learner. While he concededthat both types of learning can be meaningful—i.e., related to the learner’s existing structure ofknowledge—Ausubel noted the dominance of receptive learning in the classroom. “As a result,not only have advances in the efficient programming of verbal classroom learning been impeded,but teachers have also been encouraged to perceive meaningful verbal materials as rote incharacter” (1). While progressive educational practice has mitigated “teach and test” and “teachto the test” syndromes, rote learning is surfacing again as a viable teaching model. Competencytesting by state departments of education tends to test content, rather than skills and theapplication of knowledge. On the other hand, the school library setting is a discovery-learningsituation where students interact with information as independent learners. In both classroom andlibrary, students struggle to relate new information to what they already know to make itmeaningful. Assimilation theory (Ausubel 1963) has served as the conceptual framework for thedevelopment of heuristic devices, such as concept mapping and advance organizers.Concept Mapping: A Review of the LiteratureThe heuristic device of concept mapping has emerged as a well-tested and viable tool forpromoting concept development. In modifications of Piagetian clinical interviews (Pines et al.1978), "concept mapping” (Cardemone 1975; Moreira 1979; Stewart, VanKirk, and Rowell1979; Rowell 1978; Novak 1981) was developed to assess the framework of concepts orpropositions that an individual possesses (Novak, Gowin, and Johansen 1983). Novak, Gowin,and Johansen (1983) constructed maps from evidence provided by learners, adding labels to thelines, or linkages, to indicate different "linkage meanings” between the same two concepts. Thehierarchical nature of the maps revealed the multi-dimensionality of cognitive structures andscoring keys were developed to evaluate the maps themselves.The first pedagogical use of concept maps was to help students learn subject matter meaningfullyin the areas of mathematics and science at the college level. The first comprehensive study3School Library Media Research www.ala.org/aasl/slr
Volume 3 ISSN: 1523-4320utilizing concept maps was conducted with junior high school students, who outperformed by awide margin members of the control group on a test of novel problem solving (Novak, Gowin,and Johansen 1983). In another study mean scores on a problem-solving test in high schoolphysics were two to three standard deviations higher for students preparing concept maps whencompared with students who followed a traditional physics program (Novak 1990). Willermanand MacHarg (1991) found that a concept map used as an advance organizer resulted in apositive significant difference in the science achievement of eighth-grade students. Esiobu andSoyibo (1995) verified the efficacy of concept mapping under cooperative, cooperativecompetitive, and individualistic whole-class learning conditions in improving studentachievement in ecology and genetics.Using quantitative measures to evaluate concept maps, Austin and Shore (1995) found conceptmaps were useful in assessing the understanding of relationships between the concepts requiredfor multiple-step problem solving in physics. Trygestad (1997) used qualitative inspection, aswell as quantitative measures, of students’ maps to generate data of misconceptions and growthin student understanding. Enger (1998) qualitatively analyzed a set of 22 seventh-grade maps andfound changes in knowledge representations (reorganization in pre-maps and post-maps, changesin vocabulary usage, the nature of new knowledge representations, and the presence ofmisperceptions. Enger’s study also noted statistically significant differences in pooled preinstruction and post-instruction concept maps, but not in a class set of pre- and post-instructionmaps. Rice (1998) reported that a year-long study implemented in seventh grade life scienceclasses indicated that concept mapping is useful in assessing declarative and proceduralknowledge. Markow (1998) found no significant differences between control (essay-writing) andtreatment (concept-map-constructing) groups on post-lab multiple-choice achievement tests forfirst-year, nonmajor college chemistry students (n 32), while interviews with students indicatedpositive attitudes toward concept mapping. Gold (1998) tested concept mapping in teachingcomplex notions in urban geography and found that concept mapping techniques increasedstudents’ understanding.The Research DesignThis study was conducted in a naturalistic setting that included instruction in the classroom and aresearch project in the library to investigate the effect of an instructional approach. The studyschool was an American private school located in a major European city. The student population,composed of 49 nationalities, numbered 1,208. Seventy-four percent of students were American.The faculty was 80% American and almost 20% British. The college preparatory curriculum wastypical of American schools. The study took place in a well-equipped middle/high schoollibrary/media center with two full-time librarians.A stratified random sample of ten college-bound students was drawn from two tenth-gradebiology classes. Both groups were given a checklist and asked to indicate which biologycurriculum concepts were familiar. The list was presented at the beginning of the year toestablish that the groups started with similar knowledge, and at the end of the year to ensure thatthe instruction of both groups covered the curriculum. The researcher also compiled user profilesfor each participant in the study. The profiles included gender, ethnicity, facility with English,reading levels as recorded on student records from the standardized tests, biology grades fromthe last grade reports, and the students’ ratings of their familiarity with online searching on a4School Library Media Research www.ala.org/aasl/slr
Volume 3 ISSN: 1523-4320scale of 1 to 5. Teacher input was used to interpret student records and to compile this data. Theresults, shown in table 2, guided the selection of participants.Table 2. Characteristics of the Key adeIowascoresSex Age BirthPl 3B98F15USADutch3.5C9089B99F14Philip.English4D60NAC 90M15USAEnglish2E8082B-81F15 EnglandEnglish2.5CM 4 B91.6153NCM153.8The use of concept mapping may have had the Hawthorne effect—that is, the experimentalgroup was more able or motivated or was doing more. Sample selection measures, in addition toteacher-made tests administered throughout the unit, minimized this effect.An important factor in selection was the willingness of students to participate as key informantsin the study. The sample was thus drawn from a pool of volunteers. The sample was not intendedto be representative of the school in which the study took place. Five students were chosen askey informants from the class that was taught concept mapping and they served as theexperimental group. Five students were chosen from a class that received the same instruction,research assignment, and library instruction without concept mapping. This small sample madean in-depth analysis of qualitative data possible. The study encompassed the entire researchprocess from topic selection through writing the paper, with the focus on the information search.5School Library Media Research www.ala.org/aasl/slr
Volume 3 ISSN: 1523-4320The two study groups were taught by the same teacher to control for differences in teachereffectiveness and method. The biology teacher taught one class in the conventional manner,covering all curriculum topics, including an introduction to genetics, as illustrated in figure 1.Figure 1. Teacher's Concept Map of the Genetics UnitTeaching methods used by the biology teacher included class discussion, demonstration, handson activities, small group work, science labs, and limited lecture. Instructional aids included atextbook, (Biological Science: A Molecular Approach, BSCS Blue version, 6th ed., D.C. Heathand Co.), audiovisual materials, and computer-assisted instruction using interactive software.The software, Biology Explorer: Genetics, was used to reinforce the concept of monohybridinheritance (i.e., pure lines; dominant and recessive alleles; phenotype and genotype;homozygote and heterozygote; Mendel’s First Law) and the monohybrid test cross. The teacheralso used the program to elaborate on, and give more examples of, the elementary laws ofprobability using large sample sizes. "Basically,” she noted, "rather than drilling them withrepeated genetics problems, they ran models and used questions from me on paper to guide theirlearning.”The other class received the same instruction. In addition, this group was taught to use conceptmapping continuously and consistently during the year as detailed in Novak’s book, Learn toLearn, which was used as a guide by the teacher. Since the experimental group was asked to dosomething extra, it may be conjectured that the mappers spent more time learning concepts orthinking about searching. To minimize this perception, search strategy was taught by thelibrarian to both groups as part of the research unit and included the mechanics of searching. Thegroups also received the same instruction regarding vocabulary and concepts within the biologyunit of study. The same amount of time was spent on the topics and concepts in each unit. The6School Library Media Research www.ala.org/aasl/slr
Volume 3 ISSN: 1523-4320benefit of concept mapping accrues to the person who designs the map and building skills inmapping takes place over several months time (Novak, Gowin, and Johansen 1983). For thesereasons, contact between members of the two groups would not have compromised the effects ofconcept mapping on searching behaviors.The librarian taught a library unit that included an introductory lesson to the reference tools andelectronic searching. Students were required to write a proposal subsequent to three days ofsustained silent reading in the library that precluded note taking. This method was intended topromote focus formulation. The information search in the library immediately followed, lastingabout three weeks. The biology teacher and librarian collaborated with the researcher to design aconventional research assignment culminating in a written paper. Students chose topics from ateacher-made list, which included topics such as using monoclonal antibodies to immunizehumans, the use of DNA testing in police work, the benefits and disadvantages of transgenicanimals, and cloning. Basic concepts underlying these topics were taught to both groups, butstudents were expected to expand their knowledge of the topic through research.The librarian instructed classes in the use of library sources of information, including printindexes to New Scientist and Critical Issues, the Readers’ Guide to Periodical Literature, and theuse of automated search systems, including a computerized OPAC and the Social IssuesResource Series (SIRS) in electronic format. These library reference tools were used becausethey are common to most school libraries and were accessible to all students. The period of datacollection on students’ searching behavior was two weeks, but data collection extended fromtopic initiation through writing the paper. Table 3 provides a summary of data collectionmethods. Data on searching were collected from three to five think-aloud sessions for each keyinformant. These sessions were audio recorded by the researcher as students searched forinformation. Informants wrote transaction logs from memory immediately following each search.These sessions were followed by structured interviews. Stimulated recall was used as aninterview method; key informants were provided with written transcriptions of their think-aloudsessions, which were discussed in interviews. These methods were replicated with the biologyteacher and librarian serving as “experts” to compare their searching with student searching.Students, the teacher, and librarian maintained journals from the beginning of the genetics unit tothe end of the research project to document their thoughts, feelings, and observations. Debriefingtook place immediately following each session of data collection.Table 3. Summary of Data CollectionI. The Classroom UnitA. Control Group1.2.3.4.Teacher-made pre-test on vocabulary and concepts administered.Traditional lessons taught.Two teacher-made tests of student knowledge administered.Student key informants selected using data from user profiles.B. Experimental Group1. Teacher-made pre-test on vocabulary and concepts administered.7School Library Media Research www.ala.org/aasl/slr
Volume 3 ISSN: 1523-43202.3.4.5.6.Studies on concept mapping given to teacher.Teacher trained in concept mapping using Learning How to Learn.Lesson plans that incorporate concept mapping methods generated by teacher.Students received hand-outs, introduction to concept mapping.Students taught characteristics of maps using good and bad examples. Students receivedirections on concept mapping; made individual maps.7. Students given feedback from the teacher for at least 3 sub-topics of the unit includingscoring using Novak and Gowin's (1984) comprehensive scoring scheme.8. Two teacher-made test of student knowledge administered.9. Expert concept maps made by the teacher for at least 3 unit sub- topics.10. Student key informants selected using data from user profiles.II. The Library UnitA. Control and Experimental Groups1. Teacher and librarian collaborated to design the research unit and packet of supportmaterials for students.2. Students chose topics from teacher-made list.3. Librarian gave instruction in use of packet (Glossary, Note-taking Sheets, BibliographyCharts), sources of information, and using automated search systems.4. Teacher/Librarian graded research paper and research process (packet) respectively.5. Librarian conducted three expert searches.III. Methods of Data Collection from Key InformantsA. Think-alouds1. Think-alouds conducted for each key informant (minimum of three per student, two perteacher, two per librarian).2. Each session audio-taped and transcribed.B. Interviews1. Individual interviews conducted for students, teacher and librarian for a minimum ofthree times at staggered intervals during the search process for a minimum of ten minuteseach.2. Stimulated recall using transcriptions of think-alouds during sessions incorporated withinterviews.3. Each session is audio-taped and transcribed.C. Journals1. Students kept journals detailing their thoughts and feelings throughout the researchprocess as well as their searching strategies.2. Students made transaction logs from memory immediately after each search.3. Teacher and librarian kept journals of their observations of students.8School Library Media Research www.ala.org/aasl/slr
Volume 3 ISSN: 1523-43204. Teacher and librarian made transaction logs from memory after each the observed expert""searches they per formed, including time lines, time logs and flow charts.D. Debriefing1. Debriefing occurred following each data collection session for all key informants.Data AnalysisThe data were analyzed both qualitatively and quantitatively. The former analysis used theconstant comparative method and grounded theorizing to find trends and patterns in verbal data(Glaser and Strauss 1967). While searching behavior is the focus of this article, the studyincluded data drawn from each stage of the research process and is presented as supporting orcontradictory evidence, as the case may be, for the quantitative data established by a formulabased on probablility and Bayesian statistics. Qualitative data was triangulated with quantifiabledata generated by information searches using a mathematical formula based on informationtheory proposed by Fano (1961) and developed in the field of Bayesian statistics.Information theory, which has been used in the development of artificial intelligence systems andthe construction of knowledge bases, is a viable alternative to classical statistics. Used indecision theory, it seems especially appropriate for researching the information search in whichdichotomous situations constantly confront the searcher. Information theory has developed indirections that are of interest as statistical measures and as quantifiable concepts in libraryscience (Parsaye and Chignell 1988). Generally, information theory is not concerned withconfidence levels but with the amount of information in favor of an event happening. Theamount of information is a kind of degree of confidence. Rather than rejecting a hypothesis, or inthe case of qualitative research, a theory or proposition leading to theory formulation, a certainthreshold related to the value of the information can be used to make a decision.The kind of information that most closely relates to perceived meaning or new knowledge can berepresented by the statementInformation now learned about A what you now know about A - what you knew orexpected about A.Related to this statement is the Fano measure, a measure of mutual information that can bedescribed as measuring the information relating A to X. Fano’s measure embodies these conceptsof expected information, expressed in logarithms:I(A;X) log [(number of times A and X are observed together(number of times A is observed, i.e., with or without X)]Interpreted in terms of one of the measures of this study, this formula reads:9School Library Media Research www.ala.org/aasl/slr
Volume 3 ISSN: 1523-4320The information that X (concept mapping) provides about searching behavior (A) is equalto the log of the number of times A and X are observed together divided by the number oftimes A is observed, with or without X (concept mapping).In broader terms, the concept can be stated as:Information learned about A What you now know about A - What you knew orexpected about A.Bayesian statistics refined the formula as follows:I(use print : electronic ; concept mappers) log[ P(concept mappers, print) / P(concept mappers, electronic) ] - log[ P(print) /P(electronic) ]which is read as:the amount of information (I) provided by (indicated by a semi-colon) concept mappersthat the student will use print at any time as opposed to (indicated by a colon) electronicmeans is equal to the log of concept mappers in print divided by concept mappers inelectronic minus everyone searching in print over everyone in electronic (whichrepresents the expected probability that the searching behavior would occur anyway, orby chance).Mathematical probability is based on a model that assesses the frequencies of sequences ofevents. The formula based on Bayesian statistics addresses expected probablility: the probabilityof A occurring by chance, which is the same principle used in standard statistics. However, thechance situation is not always the most useful or meaningful basis. For example, there is morehope of finding Oliver Twist in a library than in an average room in an average house (Parsayeand Chignell 1988). It is possible to introduce a kind of probability that is pragmatic so that onewill make a decision that is more profitable or less risky. For example, the decision to look for abook in a library rather than in an average room might be reassessed if entering a library cost alot of money. A condition can be introduced into the formula so that we can state there is morehope of finding Oliver Twist in a library than in an average room in an average house conditionalupon free access to a library.The Bayesian approach to probability relies on the concept that one should incorporate the priorpossibility of an event into the interpretation of a situation (Parsaye and Chignell 1988). Bayes’sequation is a special application of conditional probability.The information measure resulting from applying this formula can be positive (A has morechance than you would expect), zero (the occurrence of A is just what you would expect andthere is no information), or negative (A has less chance than you would expect.) The units ofmeasure depend on the logarithms used:Base 2 - binary units or BITSBase 10 - decimal units or HARTLEYSNatural logs (base e) - natural units or NATS10School Library Media Research www.ala.org/aasl/slr
Volume 3 ISSN: 1523-4320NATS is recommended for statistics: the natural logarithm is usually the automatic setting on acalculator.The following points of grammar translate the formula from its mathematical language tocolloquial speech:I informationSemi-colon provided byColon as opposed to conditional uponUnlike classical statistics, information measures speak of results that are “meaningful” ratherthan “significant.” Significance levels, traditionally set at .05 and .01 to determine the probabilityof the outcome occurring by chance, are replaced by a measure of one nat, which is considered“meaningful.”The researcher identified ten searching behaviors as points of analysis to determine if there was ameaningful difference between the searching behaviors of concept and nonconcept mappers: (1)preference for print vs. electronic formats; (2) preference for SIRS vs. OPAC; (3) preference forsubject heading vs. keyword searching; (4) total number of search words in searching repertoire;(5) number of unique search words; (6) incidence of opening moves; (7) the number ofreformulations; (8) the number of search operations; (9) the incidence of breadth searching; and(10) the incidence of depth searching.FindingsPrint versus Electronic SearchingWill concept mappers be inclined to use print or electronic resources? Figure 2 illustrates thatalthough the total searching time of both groups was virtually the same, concept mappers spent96 minutes in electronic indexes while nonconcept mappers spent 156 minutes.Figure 2. Total Time Spent Searching on Print and Electronic Search Tools11School Library Media Research www.ala.org/aasl/slr
Volume 3 ISSN: 1523-4320Another way of highlighting the differences in apportionment of time is shown as percentages infigures 3 and 4.Figure 3. Search Time Allotment—Concept MappersFigure 4. Search Time Allotment—Non-Concept MappersA u
The Effects of Concept Mapping on the Searching Behavior of Tenth-Grade Students Carol A. Gordon, Head Librarian, Educational Resources Library, Boston University A qualitative study addressed the effect of concept mapping on the searching behavior of tenth-grade students engaged in research projects based on their instruction in a classroom-based
