WIXLIB

250Writing Dissertation and Grant Proposals Blinding interviewers in a case–control study—to avoid interviewer bias Incorporating repeated administrations of questionnaires over thecourse of follow-up—to minimize nondifferential misclassification of exposure due to changes in behaviors over time Use of life events calendars—to boost the accuracy of recallthereby reducing nondifferential misclassification of exposureExample analysis techniques to minimize study limitations include: Comparing baseline characteristics of the experimental and standard care group in a clinical trial—to ensure that the randomization was successful Performing subgroup analyses among participants with and without missing data on key variables of interest—to address potentialselection bias Conducting analyses among participants with asymptomaticdisease—to address concerns regarding temporality, that is, thatwhether preclinical symptoms of disease may have influencedexposureIn Part II, I provide specific examples of design and analysis techniques to address eachof the classic study limitations in epidemiology and preventive medicine proposals.13.2.5  Conclusion to Fourfold Approachto Address LimitationsThis fourfold approach of identifying the study limitation, describing its potentialimpact on study findings, discussing alternatives considered, and ending with methodsto minimize limitations has a key strategic benefit. By ending with the steps thatyou are taking to minimize your limitations, you leave the reviewer with a positiveimpression. This leads us to the issue of where to place your study limitations in agrant proposal.13.2.6  Where to Place Your StudyLimitations in a Grant ProposalIn general, there are two schools of thought on where to place your study limitationsin a dissertation or grant proposal. The first is to place your limitations section nearor at the end of the Approach section. The second school of thought is to intermingleyour limitations within each relevant subsection of the Approach. Below, I discuss theadvantages and disadvantages of each technique. Regardless of which technique you

13 How to Present Limitations and Alternatives251choose, the Study Limitations section can be titled, Limitations and Alternatives. Thisis a key catch phrase that reviewers will search for—and will criticize proposals forfailure to include.13.2.6.1 Limitations section at the endof the approach sectionThis technique involves writing one section with a subheading titled, Limitations andAlternatives in which you discuss all the potential limitations of your proposal. Foreach limitation, you use the above fourfold approach—discussing the source of thelimitation, the potential impact on the findings, the alternatives considered, and themethods that you will use to minimize this problem.The advantage of this technique is that in one centralized section, you can carefullyand thoroughly evaluate and discuss each potential limitation.The first disadvantage of this technique is that, as the reviewer reads your proposal,they will be thinking of limitations in real time. However, the reviewer will be forced towait until the end of the Approach section to see if you have addressed their concerns.A careful reviewer will be forced to keep a list of concerns as they arise in your application and will then have to cross-check this list with your limitations summary at theend. Therefore, this approach is less kind to reviewers.The second disadvantage of this approach is that you are essentially ending thegrant on a fairly negative note. Accumulating all study limitations in one section at theend of the Approach can inadvertently lead to a diminished enthusiasm for the proposalon the part of the reviewer. This can be particularly risky as this section comes atthe end of the reviewer’s reading of the application—immediately before they need toassign their score.One way to modestly diminish this concern is to add a final section to the Approach,immediately after this Limitations section, titled Summary of Significance, where youhave a few lines rehighlighting the importance of the application. However, with strictpage limitations on grant proposals, it is often difficult to have space for this final upbeatnote. In addition, reviewers may find it repetitive of your initial Significance sectionthat already appeared earlier in the proposal.13.2.6.2 Intermingled limitations sectionsIn contrast, the technique I prefer is to intersperse limitations—as they arise—throughout the Approach section. In this manner, you can address in real time concernsthat arise for the reviewer and don’t leave them waiting and concerned until the end ofthe application. This approach is kinder to the reviewer—just as they are about to putpen to paper to note a concern, you immediately address it.For example, when you are describing the study design, you intersperse a few linesdiscussing limitations of your study design and your rationale for choosing it. Furtheron, when you discuss exposure assessment, you insert another small limitation sectiondiscussing limitations to your exposure assessment and your rationale for choosing it.In other words, each of these limitations sections is a microversion of the fourfold

252Writing Dissertation and Grant Proposals approach presented above—dismissing each limitation individually, as it occurs. Eachof these subsections can be titled Limitations and Alternatives.13.3 Part II: Methods to MinimizeClassic Limitations—Designand Analysis Techniques13.3.1  How to Present NondifferentialMisclassificationThere are a number of different techniques that can be used to minimize nondifferentialmisclassification—both via study design and via data analysis.13.3.1.1 Design techniques to minimizenondifferential misclassificationDesign techniques to minimize nondifferential misclassification of exposure can includeshorter recall periods, use of validated questionnaires, interviewer administration ofquestionnaires, use of calendars to assist participant recall, and many other techniques.Design techniques to minimize nondifferential misclassification of outcome include theuse of clear diagnostic criteria to identify disease outcomes (e.g., based on publishedconsensus guidelines).Imagine a proposal to conduct a prospective study to assess the impactof coffee on bladder cancer. In this study, coffee consumption wasmeasured via an FFQ. Identify the limitation: Women may generally underreporttheir coffee consumption. Describe the impact on your findings: The effect of suchmisclassification, however, will be to underestimate any trueassociation between coffee consumption and the outcome. Discuss alternatives: We selected an FFQ, as opposed to24 h dietary recalls, as FFQs are less prone to error due tothe day-to-day variability in diet and have demonstrated relationships between dietary patterns and cancer incidence.1 Methods to minimize: Because we collected dietary information every year, nondifferential misclassification will likelybe modest. Also, validation studies have indicated that selfreported coffee intake correlates well with true intake.2

13 How to Present Limitations and Alternatives253Of course, in your proposal, you would write this up as one complete paragraph:Women may generally underreport their coffee intake. The effect of such misclassification, however, will be to underestimate any true association betweencoffee intake and the outcome. We selected an FFQ, as opposed to 24 h dietaryrecalls, as FFQs are less prone to error due to the day-to-day variability in dietand have demonstrated relationships between dietary patterns and cancer incidence.1 Because we collected dietary information every year, nondifferential misclassification will likely be modest. Also, validation studies have indicated thatself-reported coffee intake correlates well with true intake.213.3.1.2 Analysis techniques to minimizenondifferential misclassificationOne example of an analysis technique to minimize nondifferential misclassificationwould be to propose to use findings from a validation study to correct for measurementerror. Such a validation study may be available from your preliminary studies or fromthe prior published literature. Measurement error techniques are discussed in detail inseveral excellent textbooks on the topic, and you could consult a statistician for assistance in this regard.Imagine a proposal to evaluate physical activity and risk of breastcancer.Physical activity will be based upon self-report and therefore issubject to misclassification. Due to the prospective nature of thestudy, this misclassification should not be differential according to breast cancer diagnosis. To the extent that nondifferentialmisclassification occurs, our observed odds ratios will be biasedtoward the null. As prior studies have observed strong relationships between self-reported physical activity and diseases suchas cancer and cardiovascular disease, this threat should notbe substantial. In addition, we will use data from our physicalactivity questionnaire validation study to evaluate the extent ofmeasurement error (see “Data Analysis” section).13.3.2 How to Present Selection BiasUnlike nondifferential misclassification, selection bias cannot be removed in data analysis after it has occurred. Instead, it must be prevented in the design of the study. On theother hand, data analysis techniques such as sensitivity analyses can be used to evaluatethe extent of selection bias.

254Writing Dissertation and Grant Proposals 13.3.2.1 Study design techniques to minimize selection bias ecall the proposal to conduct a case–control study of the associationRbetween multiple sexual partners and HPV presented in Chapter 12.People who have HPV (cases) and who have had multiple sexual partners(exposed) may be more motivated to participate because they are concerned that their HPV infection was caused by having multiple sexualpartners. The following design techniques could reduce this concern:Participants will be identified via random sampling of medicalrecords. In addition, we will ensure that participants are blinded tothe proposed hypothesis. Finally, questions regarding sexual partners will be embedded in a long questionnaire that includes reproductive history as well as other medical and psychosocial factors.13.3.2.2 Analysis techniques to minimize selection biasThe following sensitivity analysis could also be proposed to address the extent of selection bias:In addition, we will compare characteristics of cases and controls to see if theydiffer on sociodemographic and other medical history variables.13.3.3 How to Present Information BiasJust as with selection bias, information bias can only be prevented through studydesign approaches. Analysis techniques can, however, be used to address the extent ofinformation bias—but cannot remove information bias once it has already occurred.13.3.3.1 S tudy design techniques tominimize information biasThe best way to reduce the threat of detection or surveillance bias is to blind the assessor to the participants’ exposure (in a cohort study) or to the participants’ case/controlstatus in a case–control study. For example, in a prospective cohort study, if medicalrecord abstractors are blinded to exposure status, information on exposure cannot influence the collection of information on the outcome. Similarly, in a case–control study,if interviewers are blinded to case/control status, then information on outcome cannotinfluence the collection of information on the exposure of interest. In addition, bias isalso reduced when the hypothesized association between the exposure and the outcomeis not known to the assessor.

13 How to Present Limitations and Alternatives255Imagine again our proposal to conduct a cross-sectional study of ALAand risk of vitamin B12 deficiency. Participants in this study will beasked to self-report their ALA during a home interview.It is possible that people with vitamin B12 deficiency will be moremotivated to remember ALA use than people without vitamin B12deficiency. Such a recall bias would result in an overestimate ofthe relationship between ALA use and vitamin B12 deficiency.However, the home interviews were conducted by trained interviewers and participants were blinded to the study hypothesis.Additionally, vitamin B12 deficiency was ascertained byserum concentration and results were not shared with participants until after exposure quantification; thus, they wereunaware of their disease status at the time of the interview.Thus, it is unlikely that information bias occurred in this study,and if it occurred, we would expect its effect to be minor.Note that the bold phrases indicate the design techniques used to minimize information bias.13.3.3.2 Analysis techniques to minimize information biasSensitivity analyses can be used to examine the extent of information bias. For example,if there are concerns about detection bias or surveillance bias in a cohort study, oneapproach would be to compare the amount of surveillance applied to the exposed vs.unexposed groups—by comparing such factors as number of medical visits, recencyof medical visits, and number of screenings. Another sensitivity analysis to addresssurveillance bias would be to repeat the analysis after excluding nonsymptomatic cases(i.e., in situ cases) as these cases would only tend to be detected during regular medical surveillance. In the examples below, the techniques to minimize information biasare bold.Imagine a proposal to conduct a prospective study of postmenopausalhormone use and risk of breast cancer.One potential source of bias in our study is the difference in therates of mammographic screening between hormone users andnonusers. Those who are taking postmenopausal hormonesare more likely to have a mammogram and thus more likelyto be diagnosed with breast cancer than those women not taking postmenopausal hormones. This would lead to an overestimate of the association between postmenopausal hormones and

256 Writing Dissertation and Grant Proposals breast cancer. We will address this problem in several ways.We will compare rates of mammography screening amongpostmenopausal hormone users as compared to nonusers.We will also exclude in situ breast cancers because they aremore likely to be diagnosed through mammography.Imagine a proposal to conduct a prospective study of dietary factorsand their impact on incident diabetes.Bias could arise if those with early signs of diabetes, but not a formal diabetes diagnosis, started to change their diet in responseto these early indicators. This bias would cause an overestimateof the association between diet and diabetes. To minimize thispotential bias, we will perform a subanalysis only amongparticipants without reported symptoms of diabetes. If weobserve a comparable association between diet and diabetes inthis subgroup as in the overall sample, this would reduce concerns about the impact of information bias.13.3.4 How to Present Confounding13.3.4.1 Study design techniques to minimize confoundingThere are a number of study design techniques to minimize the threat of confounding—as summarized in Table 13.1.1. Subject restrictions: Design approaches to address confounding includerestricting subjects to particular characteristics. Specifically, you could propose to restrict subjects to particular strata of a confounding factor suchthat your exposed and unexposed groups will have this factor in common.Because most study designs already limit inclusion criteria, by definition,most proposals are already taking a first step in limiting confounding!I magine that you are proposing to conduct a study of depressionand risk of preterm birth. You are concerned about parity as apotential confounding factor. Therefore, you propose to restrictthe study sample to nulliparous women (women who have not hadprior children). In this way, parity cannot act as a potential confounder of the observed association between depression and preterm birth because all the women will have the same level of thatpotential confounding factor (i.e., no children). That is, havinghad children cannot be responsible in any part for the observedassociation between depression and preterm birth.

13 How to Present Limitations and Alternatives257Table 13.1 Study design techniquesto minimize confounding Subject restrictionsCollect information on confoundersMatched designRandomizationEven if you cannot fully restrict inclusion criteria to rule out all confounders,you can try to limit the range of a potential confounder. For example, if you wereconcerned about confounding by age, you could exclude extreme ages (e.g., children or the elderly). This will not remove the need to address potential confounding by age in your analysis, but will limit the potential extent of confounding.Imagine a proposal to evaluate vitamin D intake and cataract incidence. You will be using an existing dataset of dental hygienists.Although we controlled for many cataract risk factors in the analysis, we did not have information on exposure to sunlight. Becausesunlight is positively associated with vitamin D, and positivelyassociated with cataract incidence, the lack of control for sunlightmay lead to an overestimate of the association between vitamin Dand cataract incidence. However, because the cohort is not occupationally exposed, variation in sunlight is not likely to be as largeas in a general population sample.Note that the example ends by minimizing the threat of confounding. Thelast sentence points out that the study population does not vary significantlyaccording to the level of sunlight. Remember, as described above, that if allparticipants do not differ according to the confounding factor (e.g., all onegender, or all one age, or in this case, all one level of sunlight exposure), thenthere cannot be confounding by this factor. That is, differences between participants in levels of this confounding factor will not be responsible for theobserved association between vitamin D and cataract incidence. However,some residual confounding is likely to remain.2. Collect information on confounders: The second approach involves designingyour study to collect information on potential confounders. By having this datain hand, you will be able to adjust for these potential confounders once you getto the data analysis phase. This approach requires careful consideration of allthe potential confounding factors when writing the proposal. Reviewing theprior literature will help to identify potential confounders. The construction ofDAGs (see Chapter 10, Data Analysis Plan) is also a common approach.The flip side of this approach is the potential for heavy participant burden.That is, collecting information on each potential confounder (e.g., either

258Writing Dissertation and Grant Proposals through questionnaires or biomarkers) may take up an inordinate amountof participant time or require a high amount of biomarker assessment (e.g.,multiple blood draws, biopsies).It is also important to note that this approach does not remove the potential threat of residual confounding, but does minimize this concern. Forexample, if you are concerne

student has an understanding of each type of study limitation regardless of whether or not it is a serious threat to their approach. In contrast, in the context of a grant proposal, there is no room for this type of exercise given space limitations. Instead, you are expected to comment only on the most important/major limitations of your proposal.