WIXLIB

ror in the survey data, b) environmentalchanges related to spending behaviors,c) a need for greater flexibility in themode of data collection, and d) theability to modernize data collection.The new design of the CE must addressthese issues.Reducing measurement error is theprimary mission of the Gemini Project,yet as certain social behaviors and technological changes become more common, there is an increased likelihood ofintroducing measurement error to thesurvey. For example, purchases madeonline or recurring bill payments madeby automatic debit may be less salientto respondents. Flexibility in CE datacollection strategies better positionsthe program to respond to such changesover the long term. Furthermore, whilethe spending behaviors that the CEseeks to measure have changed considerably over the past 30 years, thefundamental design of the survey hasnot. Although a number of improvements have recently been incorporatedinto the survey design, including thetransition to a computer-assisted personal interviewing (CAPI) instrumentin the Interview Survey, there has beenno large-scale, comprehensive changeto the survey design since 1980.Gemini Project objectivesThe primary short-term objective of theGemini Project is to develop a detailedredesign planning document. Thisdocument will describe the priorities,individual steps, timeframe, resourceneeds, and costs required to develop,pilot test, evaluate, and implement aredesigned CE. The redesign planning document will also guide thedevelopment and implementation ofCE research studies throughout theGemini Project lifecycle.As previously noted, the long-termobjectives of the redesign initiativeare to introduce design changes in theCE that reduce measurement error,improve overall data quality, enhancethe analytic value of CE data to users, and support a greater operationalflexibility to respond to changes inthe interviewing environment. Thesechanges must occur within specifiedbudget levels. Therefore, all proposedchanges will be subject to budgetaryconstraints, and implementation decisions will be considered in terms ofpriorities and trade-offs. To allow foran unpredictable budgetary environment, the redesign planning documentwill address both a complete redesignof the CE as well as more limited.Gemini Project timelineA high-level project timeline includesinformation gathering in 2009 and2010, to be followed by informationsynthesis and research planning in2011. By the end of 2012, the projectaims to have a proposed redesignplanning document in place for thedevelopment and implementation ofa redesigned CE. The development,testing, and implementation of a redesigned survey will occur in 2013 andbeyond.Redesign challengesEven at the outset of the survey redesign process, a number of challengesare evident. Defining survey requirements is challenging, particularly giventhe CE’s diverse user community.Despite identifying the needs of varied users and reconciling competinginterests, there are certain to be someusers with unmet needs. Additionally,it is a challenge to gather, respond,and act on stakeholder concerns andsuggestions while maintaining forwardproject progress. Finally, because oftime constraints and uncertainty overthe direction and nature of the forthcoming recommendations, the team iscurrently investigating several redesigntopics, and this research will be usedto support or reject some proposals.Ultimately, however, the redesign process is constrained by two overridingfactors: the final survey design mustproduce the estimates required by theCPI, and, as mentioned above, longterm operational survey costs mustkeep to specified budget levels.Project structureThe Gemini Project is composed of fiveteams, all of which report to the GeminiSteering Team. (See exhibit 1.) Eachteam addresses a specific objectivethat serves the overall Gemini mission.The Research Project Tracking SystemTeam’s primary objective was to createa research database for all completed,in-progress, and proposed CE researchprojects that could potentially informthe survey redesign. This team hasfinished its work, and the ResearchProject Tracking System is currentlyin use. The Data Quality DefinitionTeam’s objective is to produce a framework for assessing CE data quality, addressing both disparate user groups aswell as key elements for an operationaldefinition of data quality. The DataQuality Definition Team focused onsix dimensions: relevance, accuracy,coherence, timeliness, accessibility,and interpretability.2Three other teams were formed toplan, conduct, and summarize eventsthat provide input and recommendations on issues related to the redesign.One of these teams—called the DataUser Needs Forum Team— hosted anevent in June 2010 where customersdescribed their uses of the data, datarequirements, priorities, and recommendations for changes. The team willsummarize the findings from the forumand a subsequent user survey in a CEprogram office statement on CE datapriorities.The Conference Team held a Methods Workshop in December 2010. Inaddition, this team has organized aSurvey Redesign Workshop where fiveFederal survey teams gave an overviewof their redesign activities, and a DataCapture Technology Forum whereNielsen, National Opinion ResearchCenter, RTI International, and Westatpresented various technologies usedto collect survey data. The Conference Team also organized a panel atthe 2010 American Association forPublic Opinion Research (AAPOR)on respondent record use.Finally, the User Impact Team(which is not yet formed) is chargedwith assessing the impact of possible redesigns on data user products.2Brackstone, Gordan. "Managing Data Quality in aStatistical Agency." Survey Methodology. vol. 25 no.2, 1999, pp. 139-149.Consumer Expenditure Survey Anthology, 2011 3

Exhibit 1. Gemini project structureGroupGemini Steering TeamObjectiveOutputsOversee the Gemini ProjectTeam charters, Gemini Project plan, redesignplanning documentResearch Project Tracking System TeamCreate a research database for all completed, inprogress, and proposed CE research projects thatpotentially inform the survey redesignResearch Project Tracking SystemData Quality Definition TeamProduce an operational definition of data qualityfor CE and a framework for assessing CE dataquality by its disparate user groupsReport summarizing a data quality definitionand framework for CEConference TeamCoordinate methodology events to solicitexternal input and recommendations on issuesrelated to the redesign Survey Redesign Panel Data Capture Technology Forum Respondent Records Use AAPOR Panel CE Methods WorkshopData User Needs TeamCoordinate an event and survey to solicit inputfrom data users on current use, requirements, andpotential use of the dataTeamsUser Impact TeamAfter alternative designs have beenproposed, this team will explore theimpact of the redesign proposals onusers and their uses of the data.Major activitiesSeveral major products have been, orwill be, created as part of the Gemini Project. The first is the ResearchProject Tracking System (RPTS) andaccompanying report. The RPTS isa database that enables status tracking for all proposed, approved, inprogress, completed, and deferred CEmethods research projects that havethe potential to influence the CE redesign. The system stores project information in an organized structure,providing a mechanism for GeminiProject teams to better align currentand proposed projects with specificredesign needs.The second major product is theData Quality Definition Team’s report, which provides an operationaldefinition of data quality for the CEprogram. The definition includes astatement of the procedures or waysin which the program should measuredata quality, provides a frameworkfor assessing the overall quality ofCE survey data, and addresses theReport summarizing data user needs, priorities,and recommendationsIdentify the impact of possible redesign alternatives Report summarizing impact of proposed redesignon data useralternatives on data usersdata usability concerns of individualprogram stakeholders.A third major product is a set ofsummary documents created aftereach Gemini Project event. Thesereports identify the presenters, keypoints, and implications of the event.For example, a document summarizing the Data Capture Technology Forum will serve as a starting point forfuture exploration into data collectiontechnologies by identifying strengthsand weaknesses of the technologiespresented, and identifying possibleavenues of additional research.The User Needs Team will createa report summarizing the input provided by data users. This input, alongwith similar input from the CPI, willbe used to create a list of CE prioritiesthat identify the minimally requiredelements of CE data, as well as secondary data requirements. This is animportant document for the GeminiProject because it will serve as a keyreference for all subsequent eventsand discussions regarding potentialdesign alternatives.Redesign topicsTopics for current discussion. Manydifferent elements of survey methodology influence the plans of a survey re-design. Listed below are topics deemedmost important for current discussionin planning a redesigned CE. Thesetopics are addressed first because theyare fundamental to making later, moretailored decisions about issues suchas mode, technology, recall aids, andoverall respondent burden. Key issuesassociated with each topic, includingbenefits and risks of related designalternatives that need to be exploredearly in the redesign process, are alsodescribed.Global questions. The Interview Survey averages 65 minutes.3 The lengthis a result of both the breadth of expenditure categories included in the surveyand depth of detailed informationrequired for each category. The aimof the current Interview Survey is tocollect information about every expenditure category from every household.One proposed approach to accomplishthis objective (while also making theinterview shorter) is to replace someof the detailed questions with globalquestions. Global questions ask aboutan expenditure category from an aggregate standpoint (for example, “How3U.S. Department of Labor. Bureau of Labor Statistics.BLS Handbook of Methods. Washington: U.S. Government Printing Office: 2007Consumer Expenditure Survey Anthology, 2011 4

much did you spend on clothing?”),as compared with a detailed series ofquestions about individual expendituretypes within the larger category (forexample, “How much did you spendon shirts, pants, sweaters, vests, etc.?”).Replacing some of the detailed question sections with global questionswould reduce the length of the surveywhile still providing high-level expenditure information about the givencategories. However, the quality ofdata collected with global questions isdebatable, and global questions havethe potential to increase the cognitiveburden of the respondent because ofquestion difficulty. Both of these issuesneed to be further explored.Interview structure. Currently the Interview Survey is a highly structuredand standardized CAPI interview. Theinstrument is programmed to administer the questions in a set order, andthe questions are designed to be readby the interviewer exactly as worded.The questions are organized into sections by topic (for example, housing,utilities, vehicle expenses, insurance,educational expenses, and so forth),a structure which forces respondentsto report expenditures in a prescribedformat, rather than allowing them todraw on information in a sequencethat might be easier for them to recall.Nonstructured interviewing may holdpromise for collecting higher qualitydata, but will require a very differentapproach to data collection. In addition, it is more difficult to administerand may require more skilled interviewers to ensure the collection of allrequired data.Proxy reporting. Currently only onerespondent reports expenditures inboth the Interview Survey and DiarySurvey, answering questions for theentire household. The accuracy of thedata provided by the proxy dependson the extent to which the respondent has detailed knowledge aboutthe expenditures of other householdmembers. Given the complexity ofmany household situations, a proxyrespondent may often be unaware of,or unfamiliar with, purchases madeby others in the household. One approach to eliminating proxy reportingis to collect information from eachperson, either through an individualdiary or an interview. The impact ondata quality and response rates, and therisk that the same expenditure will bereported by more than one householdmember, are all issues requiring furtherinvestigation.Recall period. The Interview Surveycurrently asks respondents to reportexpenditures for the past 3 months. Thelength of this recall period, combinedwith the wide range of questions, maycontribute to incomplete or less accurate reporting, and may also presenta substantial cognitive burden for respondents. A shorter recall period mayresult in improved recall and thereforehigher quality reporting; however, thepotential increase in the number ofinterviews per household required tosupport annual estimates may negatethe benefits of increased reporting accuracy or decreased respondent burdenresulting from the change. An increasein the size of the sample could countersome of these drawbacks, but wouldincur a significant increase in surveycosts and thus would not fit within theconstraints presented for the redesign.Split questionnaire design. Currently,all Interview Survey respondents areasked about the same set of expenditure categories during each interview.The resulting collected data are usedto estimate average annual expenditures in the various categories at thehousehold level. Covering the same setof expenditure categories with everyrespondent means that the interview islong and burdensome. It may be possible, however, to administer subsets ofexpenditure categories to separate subsets of respondents and still generateaverage annual household-level expenditure estimates. Calculation of theseestimates may depend on the statisticalcapability to model the noncollecteddata (for imputation) or the willingnessof the survey stakeholders to permitother types of data manipulation, suchas matching similar households.Future topicsIn addition to the topics reviewedabove, several other areas will beexplored during the redesign process.Before a final redesigned CE can bedeveloped, some topics should beaddressed, including the use of administrative records or external data,mixed-mode designs, new technology,and recall aids.Administrative records/external data.Some of the information that the CEcollects is also compiled by othersources. Use of external data could reduce respondent burden and potentiallyimprove data quality. Whether thesedata can be used to reduce the amountof data that the CE collects depends onthe availability of procedures to link ormatch CE sample units to external data,the quality of those data, and respondent willingness to allow the linkage.Mixed-mode designs. Offering morethan one data collection mode has thepotential to increase response ratesand reduce nonresponse bias, althoughresearch evidence in this area is mixed.Because the Interview Survey was designed to be administered by personalvisit, and a substantial percentage ofcases are administered by telephone,redesigning the survey questions tobe either mode-neutral or tailoredmore specifically to the mode of actual administration has the potentialto improve data quality and reducemeasurement error.New technology. New data collectiontechnologies, such as personal digitalassistants (PDAs), smartphones, orvarious types of scanning technologies,may have the potential to improve dataquality, reduce respondent burden, andincrease completion or response rates.Such electronic data capture deviceseliminate information transfer frompaper diaries to database, instead keeping the data in electronic form through-Consumer Expenditure Survey Anthology, 2011 5

out the process. Additionally, theportability and potentially increasedconvenience of electronic data capturedevices may lend an immediacy andease of use to the recording process thatmay substantially reduce recall burden.Records and recall aids. Increasedreliance and improved guidance onrecords, receipts, and recall aids, including electronic records, also hasthe potential to improve data quality. Ifthe CE is able to identify a process toextract data from existing respondentrecords, respondent burden would alsobe reduced.Redesign-related researchTo help make informed decisions in theredesign process, a number of researchprojects have either been initiatedor are in the planning stages. Theseredesign-related research activities arepresented below.Current studies. Several researchprojects directly address the effect ofquestionnaire length on data qualityand response rates. It is often assumedthat the longer the questionnaire, thelower the quality of the data, andthe less likely respondents will be tocomplete the entire interview, eitherin the current wave or in subsequentwaves. The literature provides mixedresults on the relationship betweenquestionnaire length and data quality orresponse rates, with no clear implications for the CE. Therefore, the CE isexploring this issue directly.The Measurement Issues study hasfour goals, two of which are to assessthe impact of a shorter interview witha split questionnaire design and the useof global questions. By dividing theInterview Survey into portions and administering each portion individually,the study will be able to draw someconclusions about the impact of questionnaire length, as well as the qualityof data obtained via global questionsinstead of detailed questions. In addition to studying questionnaire lengthand the use of global questions, theMeasurement Issues study will alsoinvestigate the effect of recall periodon data quality. The Interview Surveycurrently uses a 3-month referenceperiod, which may be too long forrespondents to reliably recall all typesof expenditures; the test allows acomparison with data collected usinga 1-month reference period. However,a shorter reference period may requiremore frequent data collection to allowfor annual expenditure estimates, sounderstanding the impact of monthlydata collection on response rates is thefourth goal of the study.The Order Effects Test study alsoexamines questionnaire length. Thistest will provide data on whether theorder of the interview section (thatis, earlier or later placement) has aneffect on the quality of data .This testrandomly assigns first wave cases toone of two conditions. The first condition will have respondents proceedthrough the standard Interview Surveyinterview. The second condition movesa section from the middle of the Interview Survey to an earlier stage of theinterview. If moving a section earlierin the interview significantly improvesdata quality, the results would supportefforts to shorten the interview.Over the past several years, the CEhas explored the use of split questionnaire methods as an alternative to thecurrent data collection procedures.These methods involve dividing aquestionnaire into subsets of questionsand then administering each subset toa subsample of respondents. Currently,the feasibility of this type of split questionnaire design is being investigatedas one of the test conditions in theMeasurement Issues Study, describedabove. Serving as a counterpart to theMeasurement Issues Study, ongoinganalyses use historical CE data to runstatistical simulations investigating thevalue of an allocation method to assignsurvey items to groups of householdsbased on previous waves of data. Sincesplit questionnaire methods do notask all survey items of all households,additional research has focused onmethods to “fill-in” this missing data.Specifically, the CE has explored theuse of imputation-based procedures toestimate from all households insteadof only those households that were directly asked about a given expenditurecategory. Future research will look atrefining the allocation and imputationprocedures so that more efficient estimates can be obtained.Planned studies. There are two otherkey issues that need to be addressedthrough research before major redesigndecisions can be made. The first issueis the effect of proxy reporting. Asnoted above, the CE program relies onone respondent to provide expenditureinformation for the entire household, aprocedure suspected of being a majorsource of measurement error fromunderreporting. The Individual DiaryStudy will investigate the feasibilityand impact of collecting data from allhousehold members. This study willprovide an individual diary to each eligible household member. The collecteddata will be analyzed for improvementsin data quantity and quality over current production data, as well as theimpact of the procedural change oncollection costs and response rates. Anonline diary component is planned aspart of this test.The second issue that the CE plansto address through research is measurement error. Comparison studies withexternal data, such as the PersonalConsumption Expenditures, suggestthat CE data may have significant measurement error, especially in certaincategories. Currently, there is a limitedunderstanding as to the sources of thiserror, and no estimates of the magnitude of the errors. The Records Study isplanned to address both of these issues.In this study, participants will completea partial CE interview. They will thenbe asked to locate all available recordsthat are relevant to the expenditurescovered in the interview. In a subsequent visit, interviewers will attemptto match the records with reported expenditures and to identify differencesbetween

weights used in the Consumer Price Index (CPI). Consumer expenditure data supplied by the survey are a criti-cal component in the calculation of the CPI because they are used to estimate weights for the CPI's consumer goods and services classification structure. In the construction of the CPI, CE data serve four distinct functional uses: (1)