WIXLIB

Difference-in-Differences in Stata 17StataCorp LLCJune 16, 2021(StataCorp LLC)Difference-in-DifferencesJune 16, 20211 / 29

Difference-in-differences (DID)One of the most popular causal effects estimators (1855)Understand the effect of a treatment on an outcome for the treatedgroupSubsidy on productivityA drug on cholesterol levelsAn after-school program on GPA(StataCorp LLC)Difference-in-DifferencesJune 16, 20212 / 29

Difference-in-differences (DID)One of the most popular causal effects estimators (1855)Understand the effect of a treatment on an outcome for the treatedgroupSubsidy on productivityA drug on cholesterol levelsAn after-school program on GPAHow is it different from other treatment effects estimators?Observational data for repeated cross-sectional and panel data(StataCorp LLC)Difference-in-DifferencesJune 16, 20212 / 29

Difference-in-differences (DID)One of the most popular causal effects estimators (1855)Understand the effect of a treatment on an outcome for the treatedgroupSubsidy on productivityA drug on cholesterol levelsAn after-school program on GPAHow is it different from other treatment effects estimators?Observational data for repeated cross-sectional and panel dataIdentification does not depend on controlling for covariatesIdentification hinges on control for group and time unobservablecharacteristics(StataCorp LLC)Difference-in-DifferencesJune 16, 20212 / 29

Difference-in-differences (DID)One of the most popular causal effects estimators (1855)Understand the effect of a treatment on an outcome for the treatedgroupSubsidy on productivityA drug on cholesterol levelsAn after-school program on GPAHow is it different from other treatment effects estimators?Observational data for repeated cross-sectional and panel dataIdentification does not depend on controlling for covariatesIdentification hinges on control for group and time unobservablecharacteristicsEstimate of causal effect of a treatment controlling for unobservables(StataCorp LLC)Difference-in-DifferencesJune 16, 20212 / 29

Stata implementationTwo-way fixed effects also known as generalized DID (default)Allows 2x2 designProvides a wide range of standard errorsProvides diagnostics and testsBinary or continuous treatmentDifference-in-difference-in-differences (DDD) with group and timeinteractions(StataCorp LLC)Difference-in-DifferencesJune 16, 20213 / 29

Stata implementationTwo-way fixed effects also known as generalized DID (default)Allows 2x2 designProvides a wide range of standard errorsProvides diagnostics and testsBinary or continuous treatmentDifference-in-difference-in-differences (DDD) with group and timeinteractionsCaveatsTreatment effects are homogeneousStandard error literature is large and growing(StataCorp LLC)Difference-in-DifferencesJune 16, 20213 / 29

OutlineBasic conceptsStata examples(StataCorp LLC)Difference-in-DifferencesJune 16, 20214 / 29

Basic Concepts(StataCorp LLC)Difference-in-DifferencesJune 16, 20215 / 29

Treated group(StataCorp LLC)Difference-in-DifferencesJune 16, 20216 / 29

What have we learnedClearly there is a change in the outcome after treatment for thetreatedIs it causal?Time specific effects. Another policy. Covid-19.Group unobservable characteristics correlated to covariates. Jargon.(StataCorp LLC)Difference-in-DifferencesJune 16, 20217 / 29

What have we learnedClearly there is a change in the outcome after treatment for thetreatedIs it causal?Time specific effects. Another policy. Covid-19.Group unobservable characteristics correlated to covariates. Jargon.What can we do?Control for time-specific effectsControl for group-specific unobservables (fixed-effects)Use a causal-inference framework(StataCorp LLC)Difference-in-DifferencesJune 16, 20217 / 29

Generalized DID or two-way fixed effectsyits γs γt Dst β εitsDst is an observation level indicator of treatment Dst {0, 1}In panel data if individuals are nested in s individual effect absorbstate effectsYou may include covariates in the specification above(StataCorp LLC)Difference-in-DifferencesJune 16, 20218 / 29

2 x 2 specification DIDyits γ1treated γ1post 1treated 1postβ εitsWorks when all units are treated at the same time (balanced)This model is nested in the generalized DID1treated is a linear combination of the group dummies1post is a linear combination of the time dummiesThis model assumes all post periods and all treatment groups areequivalent.(StataCorp LLC)Difference-in-DifferencesJune 16, 20219 / 29

Standard error computationTreatment occurs at the group level, state, county, country, etc. and timeCluster at the group level Bertrand, Dufflo, Mullainathan (2004)Few number of elements in the group:Donald and Lang (2007) aggregation and other aggregation methodsWild-cluster bootstrapBias-corrected standard errors with Bell and McCaffrey (2002) degreesof freedom adjustment(StataCorp LLC)Difference-in-DifferencesJune 16, 202110 / 29

Stata Examples(StataCorp LLC)Difference-in-DifferencesJune 16, 202111 / 29

Artificial data. webuse hospdd, clear(Artificial hospital admission procedure data). describeContains data from ervations:7,368Artificial hospital admissionprocedure dataVariables:57 Mar 2021 luelabelVariable label%9.0g%9.0g%8.0g%9.0g%9.0gsizemnthpolHospital IDHospital visit frequencyMonthAdmission procedurePatient satisfaction scoreSorted by: hospital(StataCorp LLC)Difference-in-DifferencesJune 16, 202112 / 29

Graphical representation III(StataCorp LLC)Difference-in-DifferencesJune 16, 202113 / 29

Estimation. didregress (satis) (procedure), group(hospital) time(month)Number of groups and treatment timeTime variable: monthControl:procedure 0Treatment:procedure aximumDifference-in-differences regressionNumber of obs 7,368Data type: Repeated cross-sectional(Std. err. adjusted for 46 clusters in hospital)satisCoefficientRobuststd. err.8479879.0321121ATETprocedure(NewvsOld)t26.41P t [95% conf. interval]0.000.7833108.912665Note: ATET estimate adjusted for group effects and time effects.(StataCorp LLC)Difference-in-DifferencesJune 16, 202114 / 29

Diagnostic plotsestat trendplotFirst plot: Mean of the outcome for treated and untreated unitsSecond plot: Trend of treated and control groups (group interactedwith time)(StataCorp LLC)Difference-in-DifferencesJune 16, 202115 / 29

Diagnostic plots(StataCorp LLC)Difference-in-DifferencesJune 16, 202116 / 29

Tests: estat ptrends. estat ptrendsParallel-trends test (pretreatment time period)H0: Linear trends are parallelF(1, 45) 0.55Prob F 0.4615Augmented model with trends for treated vs. control group beforeand after treatment. Test if the pretreatment trends are parallel.(StataCorp LLC)Difference-in-DifferencesJune 16, 202117 / 29

Tests: estat granger. estat grangerGranger causality testH0: No effect in anticipation of treatmentF(2, 45) 0.33Prob F 0.7239Augment the model to include dummies as if treatment had occurredin the past. Test coefficients jointly.(StataCorp LLC)Difference-in-DifferencesJune 16, 202118 / 29

A 2 2 specificationCreate dummy variables for treated group and post time periodTell didregress not to include group and time effectsAdd dummies to the outcome equation(StataCorp LLC)Difference-in-DifferencesJune 16, 202119 / 29

A 2 2 specificationCreate dummy variables for treated group and post time periodTell didregress not to include group and time effectsAdd dummies to the outcome equation. bysort hospital: egen treated mean(procedure). replace treated 1 if treated 0(3,064 real changes made). generate post 0. replace post 1 if month 3(3,684 real changes made)(StataCorp LLC)Difference-in-DifferencesJune 16, 202119 / 29

A 2 2 specification. didregress (satis treated post) (procedure),/// group(hospital) time(month) nogteffectsNumber of groups and treatment timeTime variable: monthControl:procedure 0Treatment:procedure aximumDifference-in-differences regressionNumber of obs 7,368Data type: Repeated cross-sectional(Std. err. adjusted for 46 clusters in hospital)satisCoefficientRobuststd. err.8479879.0320051ATETprocedure(NewvsOld)t26.50P t [95% conf. interval]0.000.7835263.9124494Note: ATET estimate adjusted for covariates.(StataCorp LLC)Difference-in-DifferencesJune 16, 202120 / 29

Difference-in-difference-in-differences DDDAugmented DID(StataCorp LLC)Difference-in-DifferencesJune 16, 202121 / 29

Difference-in-difference-in-differences DDDAugmented DIDSelection on unobservables provides identificationWhat if there are unobservables that vary at the group and time levelFind a new group not exposed to treatment but exposed to theproblematic time-varying confounderSubtract the effect of that group from the original DID(StataCorp LLC)Difference-in-DifferencesJune 16, 202121 / 29

Difference-in-difference-in-differences DDDAugmented DIDSelection on unobservables provides identificationWhat if there are unobservables that vary at the group and time levelFind a new group not exposed to treatment but exposed to theproblematic time-varying confounderSubtract the effect of that group from the original DIDIn our example think about individuals frequency of visit affectingsatisfaction(StataCorp LLC)Difference-in-DifferencesJune 16, 202121 / 29

DDD estimation. didregress (satis) (hightrt), group(hospital frequency) time(month)(output omitted )Number of groups and treatment timeTime variable: monthControl:hightrt 0Treatment:hightrt imeMinimumMaximumTriple-differences regressionNumber of obs 7,368Data type: Repeated cross-sectional(Std. err. adjusted for 46 clusters in hospital)satisCoefficientRobuststd. err.tP t [95% conf. ed).764154.040260318.98.8452425Note: ATET estimate adjusted for group effects, time effects, and group- andtime-effects interactions.(StataCorp LLC)Difference-in-DifferencesJune 16, 202122 / 29

Other estimation alternativesdidregress (y x1 .didregress (y .)xk) (c, continuous), .(d.), group(g1 g2)xtdidregress (y x1 .(StataCorp LLC)xk) (d), group(groupvar ) time(timevar )Difference-in-DifferencesJune 16, 202123 / 29

Standard error considerationsDefault standard errors are cluster robust standard errors at the grouplevel BDM (2004)didregress is equivalent to areg considers group fixed effects asregressors in the degrees of freedom adjustmentxtdidregress is equivalent to xtreg does not consider group fixedeffects as regressorsWhen the number of elements per groups (states, counties, countries)is small cluster robust standard errors do not work well. Alternativesare:Wild cluster bootstrapBias corrected standard errorsAggregation methods(StataCorp LLC)Difference-in-DifferencesJune 16, 202124 / 29

Wildbootstrap I. didregress (satis) (procedure), /// group(hospital) time(month) wildbootstrap(rseed(111))computing 1000 replicationsFinding p-value. 50%. 100%Confidence interval lower bound.Confidence interval upper bound.(output omitted )(StataCorp LLC)Difference-in-DifferencesJune 16, 202125 / 29

Wildbootstrap II. didregress (satis) (procedure), /// group(hospital) time(month) wildbootstrap(rseed(111))(output omitted )Number of groups and treatment timeTime variable: monthControl:procedure 0Treatment:procedure aximumDID with wild-cluster bootstrap inferenceNumber of obs 7,368No. of clusters 46Replications 1,000Data type:Repeated cross-sectionalError weight: rademachersatisATETprocedure(New vs Old)Coefficient.8479879t26.41P t [95% conf. interval]0.000.7806237.9157614Note: ATET estimate adjusted for group effects and time effects.(StataCorp LLC)Difference-in-DifferencesJune 16, 202126 / 29

Bias-corrected SEs. didregress (satis) (procedure), group(hospital) time(month) vce(hc2)Computing degrees-of-freedom:procedure .Number of groups and treatment timeTime variable: monthControl:procedure 0Treatment:procedure aximumDifference-in-differences regressionNumber of obs 7,368No. of clusters 46Data type: Repeated cross-sectionalsatisCoefficientRobust HC2std. err.8479879.0325552ATETprocedure(NewvsOld)t26.05P t [95% conf. interval]0.000.7819941.9139816Note: ATET estimate adjusted for group effects and time effects.(StataCorp LLC)Difference-in-DifferencesJune 16, 202127 / 29