Understanding The Basic Reading Skills Of U.S. Adults .

Transcription

Understanding the BasicReading Skills of U.S. Adults:Reading Components in thePIAAC Literacy SurveyJohn Sabatini Educational Testing Service Princeton, N.J. December 2015

TABLE OF CONTENTSThis report was written by:Introduction . 1Theoretical Background: Understanding the ReadingProcesses of Adults Using Reading ComponentSkills Tasks . 4Results . 13Review of Key Findings . 31Conclusions . 39Endnotes . 40John SabatiniEducational Testing ServiceThe views expressed in this report arethose of the author and do not necessarilyreflect the views of the officers andtrustees of Educational Testing Service.View and download the report at:ets.org/research/report/reading-skillsCopyright 2015Educational Testing Service.All rights reserved.ETS and ETS logo are registered Trademarksof Educational Testing Service (ETS).MEASURING THE POWER OF LEARNING isa trademark of ETS. All other trademarksare the property of their respectiveowners.December 2015ETS Center for Research on Human Capitaland EducationResearch and DevelopmentEducational Testing ServiceRosedale RoadPrinceton, NJ 08541-0001

INTRODUCTIONThe results of the Programme for the International Assessment of Adult Competencies(PIAAC) survey paint a troubling portrait of the literacy skills of adults in the UnitedStates.1 The survey included a direct assessment of skills and was conducted in 23countries with nationally representative samples of adults ages 16 through 65.Assessed were cognitive and workplace skills needed for success in the 21st-centuryglobal economy. In a report entitled Time for the U.S. to Reskill? What the Survey ofAdult Skills Says,2 prepared by the OECD at the request of the U.S. Department ofEducation, it was found that the skills of adults in the United States have remainedrelatively unchanged in the decade since the previous report,3 while other countrieshave been showing improvements, especially among adults with low basic skills.The ability to read fluently and for understanding—to be able to learn from text—isperhaps the most important foundational skill for U.S. adult citizens' health, well-being,and social and economic advancement. It is a gateway to lifelong learning, education,and training. With the emergence of the Internet and social networking (which operateprimarily through the written word), reading literacy provides control over animmeasurable, readily accessible library of the world's knowledge, as well as the abilityto communicate with friends, family, and employers. While the digital revolution hasincreased the prevalence of and, access to, visual/aural media, written text—whetheron paper or screen—continues to be an omnipresent currency of communication andcommerce, except for adults who continue to struggle to read.4Adults who have trouble reading, using mathematics, solving problems, and usingtechnology are at a disadvantage when competing for jobs in the 21st-centuryworkforce.5 The situation is perhaps most dire for those at the lowest level of readingliteracy skills, because limited literacy skill reduces their access to print-based trainingand educational opportunities that could be used to enhance their social andworkforce skills. Low literacy adults are not necessarily isolated, thanks to theever-present visual media and communications available. However, their potential islimited because they cannot use printed media to learn, grow their knowledge, andseek opportunities. Interpersonally, it is often painfully obvious to adults when theycannot read well, as it also is to the casual observer. When confronted with text and atask, they can be observed puzzling and lingering for longer than proficient readers dowhen performing the same literacy activity.International surveys have consistently documented percentages of adults who scoreat or below Level 1 on the reading literacy proficiency scale,6 with internationalUNDERSTANDING THE BASIC READING SKILLS OF U.S. ADULTS: Reading Components in the PIAAC Literacy Survey1

averages at 3.3% and 12.2% for Below Level 1 and Level 1, respectively, in the mostrecent survey.7 Before the PIAAC 2011 survey, however, essentially all that one couldinfer about the literacy skills of adults below Level 1 was that they could notconsistently perform accurately on the easiest literacy tasks on the survey. One couldnot estimate what literacy tasks they could do successfully, if any.One primary reason for introducing a battery of reading component tasks to the PIAACliteracy assessment was the desire to have richer information from which to drawimplications for policy, as well as for learning and instruction, for adults who score at orbelow Level 1 in literacy proficiency. What do we know about the reading literacyprofiles of adults with low literacy scores in the United States in comparison to othercountries? What are the underlying reading skills of adults below Level 1 proficiency?Do they truly have no literacy skills at all? For adults at Level 1, is there evidence ofmastery of foundational component skills?Policy makers and educators can benefit from understanding what kinds of skills thatadults bring to learning programs, because the learning needs of those with very lowskill levels may differ from those with more intermediate levels of skills,8 as perhapsbest explained in the seminal work of Jean Chall.9 Chall distinguished learning toread—that is, the mastery of decoding, word recognition, and reading fluency—fromreading to learn or to do—that is, using text to build one's knowledge or accomplishspecific goals. Adults at or below Level 1 have needs at both levels.10 To build fluent,efficient foundational reading skills may require direct knowledge and skill instruction,as well as practice with applying skills to build up fluency of application in literacycontexts at home or in the workplace.11The most elementary applied literacy tasks of the general, cognitive survey (forexample, locating a single piece of information in a paragraph of text), while easyrelative to the other tasks, are not the most basic, foundational tasks that indicatereading literacy skill. Also, they are not aligned with evidence-based instructionalapproaches typically used when teaching beginning readers.12 Component readingliteracy tasks, on the other hand, assess the foundational skills that enable proseliteracy comprehension. Such tasks can probe knowledge of the alphabet, decoding,word recognition, word meaning knowledge, sentence comprehension, and basicpassage reading.The introduction of reading component tasks in the 2011 PIAAC survey provided a richopportunity to better understand adults with low literacy proficiency scores in theUnited States in comparison to similar populations in other countries. Readingcomponents results help us to understand what adults with scores at or below Level 1can and cannot do. Can they identify the meaning of high-frequency vocabulary wordsUNDERSTANDING THE BASIC READING SKILLS OF U.S. ADULTS: Reading Components in the PIAAC Literacy Survey2

when they appear in print? Can they evaluate the meaning of single sentences? Canthey read for local meaning in simple passages? That is, what is the range and variationin foundational skills among the lowest scoring adults in a country? These are thequestions addressed in this report.In sum, the reading components tasks in PIAAC were designed to complement theapplied literacy tasks in order to provide a richer sense of what adults scoring at orbelow Level 1 can and cannot do when engaging and processing basic written words,sentences, and passages. In the remainder of the report, we describe in more detail a)the reading component measures, including the theoretical and empirical rationale foradopting this framework; b) the results in a select set of countries that participated inthe PIAAC survey; and c) implications of those findings for policy and practice.UNDERSTANDING THE BASIC READING SKILLS OF U.S. ADULTS: Reading Components in the PIAAC Literacy Survey3

THEORETICAL BACKGROUND: UNDERSTANDING THEREADING PROCESSES OF ADULTS USING READINGCOMPONENT SKILLS TASKSThe PIAAC survey assessed the proficiency of adults in three information-processingskills: literacy, numeracy, and problem solving in technology-rich environments. ForPIAAC, literacy is defined as "understanding, evaluating, using and engaging withwritten texts to participate in society, to achieve one's goals, and to develop one'sknowledge and potential."13 In the main literacy assessment, texts could be digital(such as Web pages or emails) or print-based (such as newspapers, books, orpamphlets). Regardless of the format, texts could be continuous (such as passages),noncontinuous (such as tables or forms), mixed, or multiple (such as a blog post thatcontains an initial text and related responses or comments). These texts may naturallyappear in work-related, personal, social, community, or educational and trainingcontexts. Survey respondents were required to complete tasks that require applicationof cognitive strategies such as accessing, identifying, integrating, interpreting,evaluating, or reflecting on information in texts.14Reading components, introduced for the first time in the 2011 PIAAC survey, are part ofthe literacy assessment framework, but targeted toward adults near the bottom of theproficiency distribution.15 A detailed account of the PIAAC components assessmentframework can be found in Sabatini and Bruce,16 as well as in the reading literacyframework.17 The rationale for component skills rests on the premise that the"meaning construction" processes of reading are built upon a foundation of knowledgeof how one's language is represented in one's writing system, that is, component printskills.18 Empirical studies in the reading literature over the past several decades haveyielded a rich literature for understanding component reading processes in adults.19Evidence of an individual's level of print skill can be captured in tasks that can be usedto provide evidence of a reader's ability and efficiency in processing the elements ofthe written language—letters/characters, words, sentences, and larger, continuous textsegments.Reading components are derived from a view of reading literacy skill as a dimension oflanguage proficiency.20 Language learning is typically thought of in four dimensions—reading, writing, speaking, and listening. For native speakers of languages,foundational speaking and listening skills are acquired developmentally. Childrenacquire a productive/receptive vocabulary of words that we can refer to as theirlistening/speaking lexicon. Syntax or grammar is also acquired implicitly, allowingindividuals to interpret or produce meaningful phrases, sentences, and longerUNDERSTANDING THE BASIC READING SKILLS OF U.S. ADULTS: Reading Components in the PIAAC Literacy Survey4

discourse in their native language. As native speakers, we can understand what otherspeakers are saying in "real time," that is, we process meaning at the speed that thespeaker is speaking, which is generally in the range of 150 to 175 words per minute forthe average adult English speaker.21The challenge, in part, in becoming a fluent, skilled reader, is processing printed text aslanguage written down. That is, if a reader can recognize the words rapidly,automatically, and with ease, then the meaning of words can be processed seamlessly,using functional areas of the brain devoted to understanding language, that is,morphology, syntax, semantics, pragmatics, discourse analysis, and semiotics.22Proficient reading component competencies in any language are characterized by theease, speed, and minimal attentional resources required when cognitively processingwritten text, sometimes referred to as automaticity.23 Skilled readers are not typicallyaware of the spelling, punctuation, or typography of a text; rather their awareness andcognitive effort is applied toward constructing meaning. Thus, an association betweenreading rate and proficiency is consistently found in studies of adult readers.24Individuals who struggle with reading, on the other hand, expend considerablecognitive effort and attention in translating print into language, which expendscognitive resources that could better be utilized in constructing, interpreting, andevaluating meaning.25 They struggle with learning to map the writing system (i.e., theprinted visual symbols individually and in combination) to the spoken form of thelanguage (i.e., the phonetics, phonology, and morphology).26 Their reading behavior ischaracterized by slow, effortful processing of text. This can impact not only theirrecognition of individual words, but also building meaning from sentences andparagraphs of text.27The irregular mapping of sight-to-sound patterns (and vice versa) in English spellingpresents an additional challenge to learning to read fluently.28 When the spellingsystem of a language is highly regular—which means that with a pronunciation guide,the same spelling pattern routinely maps to the same sounds when spoken—then it isreferred to as a transparent orthography.29 German, Finnish, Italian, Spanish, andSwedish are examples of languages with relatively transparent spelling systems. Whensight-to-sound (and vice versa) mappings are less consistent, more learning andpractice is demanded before fluency is achieved.30The above account of learning to read focused on native speakers who face thechallenge of learning to apply their native language knowledge and skills whenprocessing written texts. For nonnative speakers of a language in which they arereading, the challenge is even greater. All four dimensions of language—speaking,listening, reading, and writing—may need to be learned. In the United States,UNDERSTANDING THE BASIC READING SKILLS OF U.S. ADULTS: Reading Components in the PIAAC Literacy Survey5

nonnative speakers of English constitute a sizable and significant proportion of thesubpopulation scoring at or below Level 1 in literacy. The PIAAC component taskscannot be used to distinguish whether the source of challenge for nonnative speakersis their ability to read in English versus more general lack of English language ability;thus, the specific instructional implications may differ. Nonetheless, the componentsprovide an indicator of what nonnative speakers scoring at or below Level 1 can andcannot do when reading in English.The PIAAC reading components framework covers decoding and word recognitioncomponents.31 However, the variations in how a writing system maps to a languagediffer widely across languages, ranging from alphabetic systems like English andSpanish, to syllabic like the Korean Hangul system, to logographic like the JapaneseKanji system. This cross-language variability led to the decision not to buildcomponent tasks for word recognition or decoding in the main PIAAC survey.Instead, the PIAAC reading components targeted three other levels of reading literacythat correspond to structural levels of language—word meaning, sentence processing,and basic passage comprehension. The word meaning, or print vocabulary,32 taskstargeted the mental lexicon of frequent, concrete words. The sentence processing taskstargeted basic syntactic and semantic knowledge and processing. Finally, the basicpassage comprehension tasks targeted discourse processes. Each of these task setsand their corresponding interpretation is discussed more fully in the following sections.The PIAAC component tasks were administered in a paper-based booklet format to allindividuals who took the paper-based pathway of the survey.Print VocabularyRecognizing the printed symbols on the page as representing meaningful words isfoundational to reading literacy.33 In the print vocabulary task set, the respondentidentified everyday words that the average adult speakers of the language wouldunderstand if they heard the words spoken aloud. Target words were concrete,imageable nouns of common objects. The items did not include specialized technicalor academic words that would only be known by more educated individuals in thepopulation. The words were commonly known across country contexts (e.g., sun,triangle, foot).Each item in this task set presented an image and four word choices. The respondenthad to circle the correct word choice that matched the picture. Figure 1 provides asample print vocabulary item. Distracters were designed to tap similar semanticand/or orthographic features of the target word. This way, it was less likely thatindividuals could use only partial knowledge of spelling or visual symbols to guess theUNDERSTANDING THE BASIC READING SKILLS OF U.S. ADULTS: Reading Components in the PIAAC Literacy Survey6

correct answer. For example, in the sample item in Figure 1, a reader might guess basedon the first sound of the word "ear" that the spelling starts with "e." However, there aretwo choices that start with "e," making it more challenging to guess.Figure 1. Sample print vocabulary item.Sentence ProcessingThe sentence is a natural "chunk" when reading continuous text.34 To build meaningfrom a sentence includes understanding all the words, parsing the syntactic structure,and encoding the propositions in memory. Depending on the specifics of a sentence,other operations might include making anaphoric (e.g., relating pronouns to theirreferent), causal, or knowledge-based inferences. Thus, each sentence requires somesyntactic and semantic processing.The sentence processing measure presented sentences of increasing difficulty (asindexed by length and density of information) and asked the respondent to make asensibility judgment about the sentence with respect to general knowledge about theworld or about the internal logic of the sentence. For these items, the respondent readthe sentence and circled YES if the sentence made sense or NO if the sentence did notmake sense. This task demand is consistent with the "evaluation" goal of reading in thePIAAC reading literacy framework.35 Even at the most basic reading level,comprehension or understanding may require evaluating text meaning against one'sknowledge of the world, to judge its veracity.36 Figure 2 shows a set of samplesentence processing items.UNDERSTANDING THE BASIC READING SKILLS OF U.S. ADULTS: Reading Components in the PIAAC Literacy Survey7

Figure 2. Sample sentence processing items.Passage ComprehensionSkilled reading (whether silent or aloud) is rapid, efficient, and fluent. The PIAACpassage comprehension task set targeted silent reading for basic comprehension inmultiparagraph prose texts.37 The integration of decoding, word recognition,vocabulary, and sentence processing was required to construct the basic meaning of ashort passage. Fluent, efficient performance on a basic, integrated reading task is abuilding block for handling longer, more complex literacy texts and tasks.The passage comprehension measure presented four passages, each with embeddeditems. Passages were constructed based on the kinds of text types that adults typicallyencounter: narrative, persuasive, and expository. The design used a forced-choice clozeparadigm—that is, a choice was given between a word that correctly completes asentence in a passage and an option that was incorrect. As the adults read silentlythrough a passage, they would see a word-choice item in selected sentences. Therespondent was asked to read the passage and circle the word among the alternativesthat made the sentence make sense (in the context of the passage). The incorrectchoice was meant to be obviously wrong to a reader with some basic comprehensionskills. The incorrect choice could be grammatically or semantically wrong. A samplepassage is shown in Figure 3 with the options for selection underlined within thesentences.UNDERSTANDING THE BASIC READING SKILLS OF U.S. ADULTS: Reading Components in the PIAAC Literacy Survey8

Figure 3. Sample passage comprehension items.IMPLEMENTATION OF READING COMPONENTS IN THE PIAACSURVEYThe PIAAC survey was administered via computer for most adults, but a subsample ofadults was routed to a paper-based pathway. The design and procedures for assigningthe computer- versus paper-based instrument to adults were somewhat complex.38Adults were branched to the paper-based pathway if they lacked any computerexperience, failed a core block of basic computer literacy or numeracy skills, or simplyopted not to take the survey via computer. Reading components were administered toall adults who took the paper-based pathway, regardless of their literacy proficiencylevel. Across the entire 23-country PIAAC sample, 91% of the adults who took thepaper-based pathway passed the literacy/numeracy core and therefore completed aliteracy or numeracy block as well as the reading component tasks. Thirty-one percentof the adults on the paper-based pathway scored at or below Level 1 (versus 15.5%total across the full 23-country sample). Thus, the paper-based subsample had a higherproportion of at or below Level 1 adults than the full, combined sample.The literacy proficiency scores of adults who took the paper-based pathway are on thesame scale as adults who took the computer-based pathway.39 However, the readingcomponents scores could not be put on the general literacy proficiency scale, becausethe paper-based subsamples were not random subsamples of the full country samples.UNDERSTANDING THE BASIC READING SKILLS OF U.S. ADULTS: Reading Components in the PIAAC Literacy Survey9

Consequently, we do not discuss reading components scores in terms of cross-countrypopulation estimates, but rather relative patterns of reading component mean scoreswithin and across the common international literacy proficiency score scale levels.The reading components tasks always were administered after the adults completedthe literacy/numeracy core and assessment blocks. The reading components bookletbegan with simple instructions read to the adult by the survey administrator. The threetask sets always appeared in the same order—print vocabulary, sentence processing,and passage comprehension. For the print vocabulary task, the administrator read thedirections: "Circle the word that matches the picture you see." Two items appeared oneach page. The administrator started the timer when the individual turned the page forthe first item, then stopped it when the adult completed the final item.For the sentence processing task, the administrator read the directions: "Please readeach sentence, then circle YES if the sentence makes sense, or circle NO if the sentencedoes not make sense." Then, the adult was asked to complete three practice items. The22 items appeared across two printed pages, with 13 on the first page and nine on thesecond. The administrator started the timer when the adult turned the page for the firstitem, then stopped it when the individual completed the final item.For the passage comprehension task, four passages were administered. Theadministrator read the directions: "Read the following articles. When you come to twowords that have been underlined, circle the one word that makes the sentence makesense." The administrator started the timer when the adult turned the page for the firstitem, then stopped it when the individual completed the final item. The public PIAACdatasets report three separate time estimates, one for each passage (with the last twopassages combined into one time, presumably because they were shorter in length).Table 1 shows the total items per reading components task set, the number of choicesfor each item in the specific reading components subtest, and the approximate totalscore for an adult scoring at chance levels. That is, an adult who could not read at alland guessed at every item was likely to receive by chance a total score of about 8 to 9for print vocabulary, 11 for sentence processing, or 22 for passage comprehension. Thisshould be taken into account when interpreting mean total scores or percentagecorrect on components for different proficiency levels. Analyses were conducted usingIEA Analyzer 3.1.140 and SPSS.41 The software is designed to apply iterations ofanalyses with plausible values. Weights were applied as appropriate.UNDERSTANDING THE BASIC READING SKILLS OF U.S. ADULTS: Reading Components in the PIAAC Literacy Survey10

TABLE 1. Total items, number of choices, and chance level performance for readingcomponent task setsTotalItemsper SetNo. ofChoicesper ItemChanceLevelTotalScoreChance LevelPercentageCorrectPrint ssageComprehension4422250%Total (Sum) AllItems1002 or 441-4241-42%ReadingComponentRATIONALE FOR COUNTRY SAMPLE USED IN THIS REPORTUnlike when translating the tasks on the main reading literacy survey, translatingreading component items across languages may result in different item level difficultyestimates. The relationship between the oral form of the language and its written formmay be a determining factor in how easy or difficult it is to learn to read in thatlanguage. Some languages may be easier to learn than others, perhaps because theprint-to-sound correspondences are highly regular or transparent, making it easier tolearn to sound out words once one knows the basic pronunciation key for decoding inthat language. This may in turn be reflected in a higher prevalence of adults who showsome basic reading skills in one language versus another, because even very basicinstruction will yield productive literacy skills. Similarly, differences in syntactic ormorphological structures in a language may influence ease or difficulty in interpretingphrases or sentences.As our main interest in this report is understanding the U.S. adult population, we choseto include English-speaking countries (United States, Canada, United Kingdom,42 andIreland) as the international comparison reference group. By including Englishspeaking countries, we can be assured that differences in performance levels of groupsare reflective of differences in the population and its educational opportunities, notdifferences in the relative difficulty of learning to read in the particular language.43 ForCanada, which administered the survey in French and English, we included only thosewho were administered the English-language test.UNDERSTANDING THE BASIC READING SKILLS OF U.S. ADULTS: Reading Components in the PIAAC Literacy Survey11

We also included Italy and Spain in this international comparison, because Italian andSpanish are known to have highly regular sight-to-sound correspondences, which, allthings being equal, should make it easier to learn to read in these languages. On theother hand, the populations of these two countries had among the lowest overallperformance levels of all 23 countries. Thus, they provide an opportunity to investigatethe hypothesis that there is a relative advantage in acquiring foundational readingcomponent skills in these languages even though overall these countries fare less wellacross the higher proficiency levels in the literacy skills assessed in the main survey.UNDERSTANDING THE BASIC READING SKILLS OF U.S. ADULTS: Reading Components in the PIAAC Literacy Survey12

RESULTSAs a first step, we estimated the prevalence of adults with low reading literacy skilllevels on the PIAAC general literacy scale for each country. Table 2 shows the estimatedpercentage of adults in a country by proficiency level for the six countries analyzed inthis report (Sample), as well as for the 23 countries (International) in the full (computerplus paper-based samples) PIAAC sample. In the subsample of six countries, theestimated size of the population below Level 1 or at Level 1 are 4.7 and 15.8,respectively, with Spain and Italy showing larger percentages of their respectivepopulations at or below Level 1. Note that the six-country subsample (Sample) has ahigher average percentage of adults in comparison to the 23-country full PIAAC sample(International) both at Below Level 1 (4.7 versus 3.3) and at Level 1 (15.8 versus 12.2).Table 3 shows the same information but for the paper-based subsamples only. Note thehigher percentage of adults at lower proficiency levels in the paper-based pathwayonly in comparison to the full PIAAC sample. This shows that the procedures for routingadults to the paper-based pathway were somewhat effective in directing more adultswith lower literacy proficiencies toward the paper-based instruments. Nonetheless,about 60% of the Sample group has proficiency levels of Level 2 or higher.In the remainder of the report, we will only report analyses using the six-country,paper-based pathway subsample (Sample) to facilitate comparison among the Englishspeaking countries, along with Italy and Spain. In subsequent tables and figures, wecontinue to position the Sample mean after the U.S. and Canada samples to facilitatevisual interpretation, because this is the position where the subsample mean generallyfalls. Thus, the reader can estimate the magnitude of difference between theU.S./Canada and the Sample means.UNDERSTANDING THE BASIC READING SKILLS OF U.S. ADULTS: Reading Components in the PIAAC Literacy Survey13

TABLE 2. Percentages of population by literacy proficiency level (paper and computerpathways)Below Level1Level 1Level 2Level 3Level4/5United nited taly5.522.242.026.43.3I

READING PROCESSES OF ADULTS USING READING COMPONENT SKILLS TASKS. UNDERSTANDING THE BASIC READING SKILLS OF U.S. ADULTS: Reading Components in the PIAAC Literacy Survey 4. The PIAAC survey assessed the proficiency of adults in three information-processing skills: lit