Session 11: Causal Inference (2/2)

What session 11 is for

Your Python so far

Everything you need to type is on one sheet — how to start Python in VS Codium on your platform, how to make a file and run it, the DataFrame basics, and every name from sessions 1 to 11, each with a line showing it in use.

Python Cheatsheet · Session 11 (PDF) · in the repository at 11-…/PYTHON-CHEATSHEET.pdf

One sheet per package, functions explained with pictures: pandas · NumPy · matplotlib · statsmodels

The key concept

The difference-in-difference model is used to estimate the effect of a policy intervention or treatment by comparing changes over time in an outcome for a treated group to changes over time for a comparison (control) group.

1. It measures how the mean of an outcome changes before and after treatment for the treatment group.
2. It compares this change with the mean change over time for a control group that is not exposed to the treatment.

We observe average outcomes for two groups, before and after the intervention.

The key concept (cont.)

Difference in difference model

The key concept (cont.)

The DiD estimator is the difference between the average change in the treatment group and the average change in the control group:

\[ (\text{Treatment_post} - \text{Treatment_pre}) - (\text{Control_post} - \text{Control_pre}) = \widehat{\text{DiD}} \]

Using the values in :

\[ (85 - 50) - (55 - 35) = 15 \]

The key concept (cont.)

A more detailed representation is shown in the following table.

Difference in difference, an example

The red number in the bottom-right cell represents the DiD estimator. When this estimator is different from zero, the treatment is interpreted as having an effect.

The key concept (cont.)

In this simple DiD setup:

• We work with grouped data, comparing a treatment and a control group.
• Coefficients represent group means and differences between group means; there is no time trend slope as in a standard time series regression.

The statistical model

A basic DiD model can be written as:

\[ Y = \beta_0 + \beta_1 \cdot \text{Treatment} + \beta_2 \cdot \text{Post} + \beta_3 \cdot (\text{Treatment} \times \text{Post}) + e \]

Where:

• 25. is the outcome variable.
• Treatment is a dummy variable equal to 1 for units in the treatment group and 0 for the control group.
• Post is a dummy variable equal to 1 for the post-treatment period and 0 for the pre-treatment period.
• Treatment × Post is an interaction term, equal to 1 only for treated units observed after the intervention.

The statistical model (cont.)

A schematic dataset for a simple DiD design is:

Two units belong to the treatment group and two units to the control group. For each unit there is one observation before and one observation after the treatment.

The coefficients

We can estimate a simple DiD model corresponding to

Note: p<0.1; p<0.05; p<0.01

The coefficients (cont.)

Each coefficient corresponds to a group mean or a difference between group means.

The coefficients (cont.)

Diff-in-Diff, beta 3

To obtain the post-treatment mean for the treatment group, we add:

\[ \beta_0 + \beta_1 + \beta_2 + \beta_3 \]

Each coefficient corresponds to a distinct hypothesis:

The counterfactual

To interpret \(\beta_3\), it is useful to consider the counterfactual.

The counterfactual is the outcome the treatment group would have experienced in the absence of the policy intervention. In the DiD framework, the counterfactual is inferred from the time trend observed in the control group.

The counterfactual (cont.)

DID counterfactual

The counterfactual (cont.)

Using the simple four-cell example:

\[ (\text{Treatment_post} - \text{Treatment_pre}) = (\text{Control_post} - \text{Control_pre}) \\ (X - 50) = (55 - 35) \\ (X - 50) = 20 \\ X = 20 + 50 = 70 \]

So the counterfactual outcome for the treatment group in the post-treatment period is 70.

The counterfactual (cont.)

Using the regression coefficients, the counterfactual can also be written as:

\[ \text{Counterfactual} = \beta_0 + \beta_1 + \beta_2 \]

Counterfactual, betas

A Difference-in-Difference model

A Difference-in-Difference model (cont.)

A DiD design can be used to examine whether housing projects reduce nearby house prices, as in Ellen et al. (2007).

We simulate data on housing prices for:

• Houses located close to a subsidized housing project (treatment group)
• Houses located far from such projects (control group)

A Difference-in-Difference model (cont.)

Prices are observed before and after the construction of the subsidized housing site.

Research question: Do affordable housing projects reduce the prices of nearby houses?

Hypothesis: Houses near subsidized housing projects experience a larger decrease in prices over time.

Data

# In the VS Codium terminal, with .venv active:  python
import pandas as pd

data = pd.DataFrame({
    "House_Price": [213679.8, 229355.4, 190580.1, 188669.0, 253573.7,
                    320094.0, 188525.7, 282998.4, 184424.0, 290409.4],
    "Group":          [0, 0, 1, 1, 0, 0, 1, 1, 0, 0],   # 1 = treated area
    "Post_Treatment": [0, 1, 0, 1, 0, 1, 0, 1, 0, 1],   # 1 = after the policy
})

data

   House_Price Group Post_Treatment
1     213679.8     0              0
2     229355.4     0              1
3     190580.1     1              0
4     188669.0     1              1
5     253573.7     0              0
6     320094.0     0              1
7     188525.7     1              0
8     282998.4     1              1
9     184424.0     0              0
10    290409.4     0              1

Variables:

• House_Price: house price
• Group: 1 if close to a subsidized housing site, 0 otherwise
• Post_Treatment: 1 if observed after the site opens, 0 otherwise

Analysis

The DiD regression is:

\[ Y = \beta_0 + \beta_1 \cdot \text{Group} + \beta_2 \cdot \text{Post_Treatment} + \beta_3 \cdot (\text{Group} \times \text{Post_Treatment}) + e \]

import statsmodels.formula.api as smf

# The interaction IS the difference-in-differences estimate
reg_housing = smf.ols("House_Price ~ Group * Post_Treatment", data=data).fit()

print(reg_housing.summary())
print(f"\nDiD estimate = {reg_housing.params['Group:Post_Treatment']:+,.0f}")

The interaction IS the difference-in-differences estimate (cont.)

================================================
                         Dependent variable:    
                     ---------------------------
                           Housing prices       
                                                
------------------------------------------------
Intercept (B0)              217,225.80***       
                             (24,879.25)        
                                                
Treatment group (B1)         -27,672.93         
                             (39,337.54)        
                                                
Post-treatment (B2)           62,727.10         
                             (35,184.57)        
                                                
Diff in Diff (B3)            -16,446.30         
                             (55,631.68)        
                                                
================================================
================================================
Note:                *p<0.1; **p<0.05; ***p<0.01

The interaction IS the difference-in-differences estimate (cont.)

Illustrative table (as in the original document):

• Intercept (\(\beta_0\)): 216,672.60
• Treatment group (\(\beta_1\)): 16,461.22
• Post-treatment (\(\beta_2\)): 35,230.86
• Diff in Diff (\(\beta_3\)): 34,066.02

Discussion of the results

With the housing example, the main questions are:

• Is the average price in the control group at time 1 different from zero?
• Did the treatment and control groups differ in average price at time 1?
• Did the control group experience a significant change in average price between time 1 and time 2?
• Does the treatment group at time 2 differ from its counterfactual?

Discussion of the results (cont.)

Interpretation (following the original text):

• \(\beta_0\) is statistically different from zero: average control-group price at time 1 is 216,672.60.
• \(\beta_1\) is statistically different from zero: treatment-group houses cost 16,461.22 more than control-group houses at time 1. The average price in the treatment group is 233,133.80.
• \(\beta_2\) is statistically different from zero: prices in the control group increase by 35,230.86, reaching 251,903.5 at time 2.
• \(\beta_3\) is statistically different from zero: the treatment group’s prices increase by 34,066.02 more than they would have in the absence of treatment.

The counterfactual

Using the coefficients, the counterfactual for the treatment group at time 2 is:

\[ \text{Counterfactual} = \beta_0 + \beta_1 + \beta_2 = 216,672.6 + 16,461.2 + 35,230.9 = 268,364.7 \]

Because the intervention occurred, the observed average price is:

\[ 268,364.7 + 34,066.0 = 302,430.7 \]

The counterfactual (cont.)

These values are summarized graphically:

Results from the working example

Instrumental Variables: Your turn!

• https://rpubs.com/Sternonyos/913599

data = pd.DataFrame({
    "Treatment":  [1, 0, 1, 1, 1, 0, 0, 1, 0, 1],
    "age":        [75, 62, 60, 60, 57, 55, 50, 48, 60, 51],
    "sex":        [1, 2, 1, 2, 2, 2, 2, 2, 1, 2],
    "education":  [3, 2, 1, 2, 1, 1, 1, 2, 2, 2],
    "marital":    [4, 4, 4, 4, 4, 4, 4, 4, 3, 3],
    "information":[1, 1, 1, 1, 1, 1, 1, 1, 1, 1],
    "income":     [3000, 3000, 11000, 1000, 1000, 4000, 3000, 2000, 2000, 3000],
    "newvar":     [1.45, 1.00, 1.45, 1.45, 1.45, 1.00, 1.00, 1.45, 1.00, 1.45],
    "info_index": [2.2967648, -0.6487551, 1.5254940, 1.5254940, 1.3712398,
                   -0.7515912, -0.8250455, 0.9084774, -0.6781368, 1.0627315],
})

data

   Treatment age sex education marital information income newvar info_index
1          1  75   1         3       4           1   3000   1.45  2.2967648
2          0  62   2         2       4           1   3000   1.00 -0.6487551
3          1  60   1         1       4           1  11000   1.45  1.5254940
4          1  60   2         2       4           1   1000   1.45  1.5254940
5          1  57   2         1       4           1   1000   1.45  1.3712398
6          0  55   2         1       4           1   4000   1.00 -0.7515912
7          0  50   2         1       4           1   3000   1.00 -0.8250455
8          1  48   2         2       4           1   2000   1.45  0.9084774
9          0  60   1         2       3           1   2000   1.00 -0.6781368
10         1  51   2         2       3           1   3000   1.45  1.0627315

Conclusion

• The difference-in-difference model is a powerful tool for estimating causal effects in observational data.
• It relies on the parallel trends assumption, which must be carefully considered and validated.
• Proper data structuring and interpretation of coefficients are crucial for accurate analysis.
• DiD models can be extended to include multiple time periods, covariates, and more complex designs.
• Always visualize your data and results to ensure the validity of your findings.
• IV models are another tool for causal inference when endogeneity is a concern.
• Continue practicing with real-world datasets to solidify your understanding of these methods.

Conclusion (cont.)

• You have now an incredible toolbox to analyse social and business phenomena! I am really proud of you! You will change the world!

Before the next session