---
title: "Getting Started with PyFixest"
format:
  html:
    html-table-processing: none
toc: true
toc-title: "On this page"
toc-location: left
---

# OLS with Fixed Effects

## What is a fixed effect model?

A fixed effect model is a statistical model that includes fixed effects, which are parameters that are estimated to be constant across different groups.

**Example [Panel Data]:** In the context of panel data, fixed effects are parameters that are constant across different individuals or time. The typical model example is given by the following equation:

$$
Y_{it} = \beta X_{it} + \alpha_i + \psi_t + \varepsilon_{it}
$$

where $Y_{it}$ is the dependent variable for individual $i$ at time $t$, $X_{it}$ is the independent variable, $\beta$ is the coefficient of the independent variable, $\alpha_i$ is the individual fixed effect, $\psi_t$ is the time fixed effect, and $\varepsilon_{it}$ is the error term. The individual fixed effect $\alpha_i$ is a parameter that is constant across time for each individual, while the time fixed effect $\psi_t$ is a parameter that is constant across individuals for each time period.

Note however that, despite the fact that fixed effects are commonly used in panel setting, one does not need a panel data set to work with fixed effects. For example, cluster randomized trials with cluster fixed effects, or wage regressions with worker and firm fixed effects.

In this "quick start" guide, we will show you how to estimate a fixed effect model using the `PyFixest` package. We do not go into the details of the theory behind fixed effect models, but we focus on how to estimate them using `PyFixest`.

## Read Sample Data

In a first step, we load the module and some synthetic example data:

```{python}
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
try:
    from lets_plot import LetsPlot
    _HAS_LETS_PLOT = True
except ImportError:
    _HAS_LETS_PLOT = False

from marginaleffects import slopes, avg_slopes

import pyfixest as pf

if _HAS_LETS_PLOT:
    LetsPlot.setup_html()

plt.style.use("seaborn-v0_8")

%load_ext watermark
%config InlineBackend.figure_format = "retina"
%watermark --iversions

data = pf.get_data()

data.head()
```

```{python}
data.info()
```

We see that some of our columns have missing data.

## OLS Estimation

We are interested in the relation between the dependent variable `Y` and the independent variables `X1` using a fixed effect model for `group_id`. Let's see how the data looks like:

```{python}
ax = data.plot(kind="scatter", x="X1", y="Y", c="group_id", colormap="viridis")
```

We can estimate a fixed effects regression via the `feols()` function. `feols()` has three arguments: a two-sided model formula, the data, and optionally, the type of inference.

```{python}
fit = pf.feols(fml="Y ~ X1 | group_id", data=data, vcov="HC1")
type(fit)
```


The first part of the formula contains the dependent variable and "regular" covariates, while the second part contains fixed effects.

`feols()` returns an instance of the `Fixest` class.

## Inspecting Model Results

To inspect the results, we can use a summary function or method:

```{python}
fit.summary()
```

Or display a formatted regression table:

```{python}
pf.etable(fit)
```

Alternatively, the `.summarize` module contains a `summary` function, which can be applied on instances of regression model objects or lists of regression model objects. For details on how to customize `etable()`, please take a look at the [dedicated vignette](https://py-econometrics.github.io/pyfixest/table-layout.html).

```{python}
pf.summary(fit)
```

You can access individual elements of the summary via dedicated methods: `.tidy()` returns a "tidy" `pd.DataFrame`,
`.coef()` returns estimated parameters, and `se()` estimated standard errors. Other methods include `pvalue()`, `confint()`
and `tstat()`.

```{python}
fit.tidy()
```

```{python}
fit.coef()
```

```{python}
fit.se()
```

```{python}
fit.tstat()
```

```{python}
fit.confint()
```

Last, model results can be visualized via dedicated methods for plotting:

```{python}
fit.coefplot()
# or pf.coefplot([fit])
```

## How to interpret the results?

Let's have a quick d-tour on the intuition behind fixed effects models using the example above. To do so, let us begin by comparing it with a simple OLS model.

```{python}
fit_simple = pf.feols("Y ~ X1", data=data, vcov="HC1")

fit_simple.summary()
```

We can compare both models side by side in a regression table:

```{python}
pf.etable([fit, fit_simple])
```

We see that the `X1` coefficient is `-1.019`, which is less than the value from the OLS model in column (2). Where is the difference coming from?
Well, in the fixed effect model we are interested in controlling for the feature `group_id`. One possibility to do this is by adding a simple dummy variable for each level of `group_id`.

```{python}
fit_dummy = pf.feols("Y ~ X1 + C(group_id) ", data=data, vcov="HC1")

fit_dummy.summary()
```

This is does not scale well! Imagine you have 1000 different levels of `group_id`. You would need to add 1000 dummy variables to your model. This is where fixed effect models come in handy. They allow you to control for these fixed effects without adding all these dummy variables. The way to do it is by a *demeaning procedure*. The idea is to subtract the average value of each level of `group_id` from the respective observations. This way, we control for the fixed effects without adding all these dummy variables. Let's try to do this manually:

```{python}
def _demean_column(df: pd.DataFrame, column: str, by: str) -> pd.Series:
    return df[column] - df.groupby(by)[column].transform("mean")


fit_demeaned = pf.feols(
    fml="Y_demeaned ~ X1_demeaned",
    data=data.assign(
        Y_demeaned=lambda df: _demean_column(df, "Y", "group_id"),
        X1_demeaned=lambda df: _demean_column(df, "X1", "group_id"),
    ),
    vcov="HC1",
)

fit_demeaned.summary()
```

We get the same results as the fixed effect model `Y1 ~ X | group_id` above. The `PyFixest` package uses a more efficient algorithm to estimate the fixed effect model, but the intuition is the same.

## Updating Regression Coefficients

You can update the coefficients of a model object via the `update()` method, which may be useful in an online learning setting where data arrives sequentially.

To see this in action, let us first fit a model on a subset of the data:

```{python}
data_subsample = data.sample(frac=0.5)
m = pf.feols("Y ~ X1 + X2", data=data_subsample)
# current coefficient vector
m._beta_hat
```

Then sample 5 new observations and update the model with the new data. The update rule is

$$
\hat{\beta}_{n+1} = \hat{\beta}_n + (X_{n+1}' X_{n+1})^{-1} x_{n+1} + (y_{n+1} - x_{n+1} \hat{\beta}_n)
$$

for a new observation $(x_{n+1}, y_{n+1})$.

```{python}
new_points_id = np.random.choice(list(set(data.index) - set(data_subsample.index)), 5)
X_new, y_new = (
    np.c_[np.ones(len(new_points_id)), data.loc[new_points_id][["X1", "X2"]].values],
    data.loc[new_points_id]["Y"].values,
)
m.update(X_new, y_new)
```

We verify that we get the same results if we had estimated the model on the appended data.

```{python}
pf.feols(
    "Y ~ X1 + X2", data=data.loc[data_subsample.index.append(pd.Index(new_points_id))]
).coef().values
```

# Standard Errors and Inference

Supported covariance types are "iid", "HC1-3", CRV1 and CRV3 (up to two-way clustering).

**Why do we have so many different types of standard errors?**

The standard errors of the coefficients are crucial for inference. They tell us how certain we can be about the estimated coefficients. In the presence of heteroskedasticity (a situation which typically arises with cross-sectional data), the standard OLS standard errors are biased. The `pyfixest` package provides several types of standard errors that are robust to heteroskedasticity.

- `iid`: assumes that the error variance is spherical, i.e. errors are homoskedastic and not correlated (independent and identically distributed errors have a spherical error variance).
- `HC1-3`: heteroskedasticity-robust standard errors according to White (1980) and MacKinnon and White (1985). See [Econometric Computing with HC and HAC
Covariance Matrix Estimators](https://cran.r-project.org/web/packages/sandwich/vignettes/sandwich.pdf) from the [`sandwich`](https://cran.r-project.org/web/packages/sandwich/) package for more details.
- `CRV1` and `CRV3`: cluster robust standard errors according to Cameron, Gelbach, and Miller (2011). See [A Practitioner's Guide to Cluster-Robust Inference](https://cameron.econ.ucdavis.edu/research/Cameron_Miller_JHR_2015_February.pdf).  For   `CRV1` and `CRV3` one should pass a dictionaty of the form `{"CRV1": "clustervar"}`.

Inference can be adjusted "on-the-fly" via the `.vcov()` method:

```{python}
fit.vcov({"CRV1": "group_id + f2"}).summary()

fit.vcov({"CRV3": "group_id"}).summary()
```

The estimated covariance matrix is available as an attribute of the `Feols` object called `._vcov`.

## Inference via the Wild Bootstrap

It is also possible to run a wild (cluster) bootstrap after estimation (via the [wildboottest module](https://github.com/py-econometrics/wildboottest), see [MacKinnon, J. G., Nielsen, M. Ø., & Webb, M. D. (2023). Fast and reliable jackknife and bootstrap methods for cluster-robust inference. Journal of Applied Econometrics, 38(5), 671–694.](http://qed.econ.queensu.ca/pub/faculty/mackinnon/working-papers/qed_wp_1485.pdf)):

```{python}
fit2 = pf.feols(fml="Y ~ X1", data=data, vcov={"CRV1": "group_id"})
fit2.wildboottest(param="X1", reps=999)
```

## The Causal Cluster Variance Estimator

Additionally, `PyFixest` supports the causal cluster variance estimator following [Abadie et al. (2023)](https://academic.oup.com/qje/article/138/1/1/6750017). Let's look into it with another data set:

```{python}
df = pd.read_stata("http://www.damianclarke.net/stata/census2000_5pc.dta")

df.head()
```


```{python}
axes = df.plot.hist(column=["ln_earnings"], by=["college"])
```

Now we can estimate the model `ln_earnings ~ college` where we cluster the standard errors at the state level:

```{python}
fit3 = pf.feols("ln_earnings ~ college", vcov={"CRV1": "state"}, data=df)
fit3.ccv(treatment="college", pk=0.05, n_splits=2, seed=929)
```

## Randomization Inference

You can also conduct inference via randomization inference [(see Heß, Stata Journal 2017)](https://hesss.org/ritest.pdf).
`PyFixest` supports random and cluster random sampling.

```{python}
fit2.ritest(resampvar="X1=0", reps=1000, cluster="group_id")
```

## Multiple Testing Corrections: Bonferroni and Romano-Wolf

To correct for multiple testing, p-values can be adjusted via either the [Bonferroni](https://en.wikipedia.org/wiki/Bonferroni_correction), the method by Romano and Wolf (2005), see for example [The Romano-Wolf Multiple Hypothesis
Correction in Stata](https://docs.iza.org/dp12845.pdf), and the method by Westfall & Young (see [here](https://www.jstor.org/stable/2532216)).

```{python}
pf.bonferroni([fit, fit2], param="X1").round(3)
```

```{python}
pf.rwolf([fit, fit2], param="X1", reps=9999, seed=1234).round(3)
```

```{python}
pf.wyoung([fit, fit2], param="X1", reps=9999, seed=1234).round(3)
```

## Joint Confidence Intervals

Simultaneous confidence bands for a vector of parameters can be computed via the `joint_confint()` method. See [Simultaneous confidence bands: Theory, implementation, and an application to SVARs](https://onlinelibrary.wiley.com/doi/abs/10.1002/jae.2656) for background.

```{python}
fit_ci = pf.feols("Y ~ X1+ C(f1)", data=data)
fit_ci.confint(joint=True).head()
```

# Panel Data Example: Causal Inference for the Brave and True

In this example we replicate the results of the great (freely available reference!) [Causal Inference for the Brave and True - Chapter 14](https://matheusfacure.github.io/python-causality-handbook/14-Panel-Data-and-Fixed-Effects.html). Please refer to the original text for a detailed explanation of the data.

```{python}
data_path = "https://raw.githubusercontent.com/bashtage/linearmodels/main/linearmodels/datasets/wage_panel/wage_panel.csv.bz2"
data_df = pd.read_csv(data_path)

data_df.head()
```

The objective is to estimate the effect of the variable `married` on the variable `lwage` using a fixed effect model on the entity variable `nr` and the time variable `year`.

```{python}
panel_fit = pf.feols(
    fml="lwage ~ expersq + union + married + hours | nr + year",
    data=data_df,
    vcov={"CRV1": "nr + year"},
)

pf.etable(panel_fit)
```

We obtain the same results as in the book!

# Instrumental Variables (IV) Estimation

It is also possible to estimate [instrumental variable models](https://en.wikipedia.org/wiki/Instrumental_variables_estimation) with *one* endogenous variable and (potentially multiple) instruments.

In general, the syntax for IV is `depvar ~ exog.vars | fixef effects | endog.vars ~ instruments`.

```{python}
iv_fit = pf.feols(fml="Y2 ~ 1 | f1 + f2 | X1 ~ Z1 + Z2", data=data)
iv_fit.summary()
```

If the model does not contain any fixed effects, just drop the second part of the formula above:

```{python}
pf.feols(fml="Y ~ 1 | X1 ~ Z1 + Z2", data=data).summary()
```

You can access the first stage regression object via the `._model_1st_stage` attribute:

```{python}
pf.etable([iv_fit._model_1st_stage, iv_fit])
```

You can access the F-Statistic of the first stage via the `_f_stat_1st_stage` attribute:

```{python}
iv_fit._f_stat_1st_stage
```

Via the `IV_Diag` method, you can compute additional IV Diagnostics, as the **effective F-statistic** following Olea & Pflueger (2013):

```{python}
iv_fit.IV_Diag()
iv_fit._eff_F
```

IV estimation with multiple endogenous variables and multiple estimation syntax is currently not supported.

# Poisson Regression

It is possible to estimate Poisson Regressions (for example, to model count data). We can showcase this feature with another synthetic data set.


```{python}
pois_data = pf.get_data(model="Fepois")

ax = pois_data.plot(
    kind="scatter",
    x="X1",
    y="Y",
    c="group_id",
    colormap="viridis",
    s="f2",
)
```

```{python}
pois_fit = pf.fepois(fml="Y ~ X1 | group_id", data=pois_data, vcov={"CRV1": "group_id"})
pois_fit.summary()
```

# Quantile Regression

You can fit a quantile regression via the `quantreg` function:

```{python}
fit_qr = pf.quantreg("Y ~ X1 + X2", data = data, quantile = [0.1, 0.5, 0.9])
pf.qplot(fit_qr)
```

For details, take a look at the dedicated [quantreg vignette](https://py-econometrics.github.io/pyfixest/quantile-regression.html).

# Tests of Multiple Hypothesis / Wald Tests

You can test multiple hypotheses simultaneously via the `wald_test` method.

```{python}
fit = pf.feols("Y ~ X1 + X2 | f1", data=data)
```

For example, to test the joint null hypothesis of $X_{1} = 0$ and $X_{2} = 0$ vs the alternative that $X_{1} \neq 0$ or $X_{2} \neq 0$, we would run

```{python}
fit.wald_test(R=np.eye(2))
```

Alternatively, suppose we wanted to test a more complicated joint null hypothesis:  $X_{1} + 2X_{2} = 2.0$ and $X_{2} = 1.0$. To do so, we would define $R$ and $q$ as

```{python}
R1 = np.array([[1, 2], [0, 1]])
q1 = np.array([2.0, 1.0])
fit.wald_test(R=R1, q=q1)
```

# Other GLMs (without fixed effects)

`PyFixest` experimentally supports a range of other GLMs without fixed effects (adding fixed effect support is WIP) via the `pf.feglm()` function. Full support with all bells and whistles (in particular, fixed effects demeaning) is planned for PyFixest 0.29.

```{python}
data_glm = pf.get_data(N=100, seed = 170)
data_glm["Y"] = np.where(data_glm["Y"] > 0, 1, 0)

fit_gaussian = pf.feglm(fml = "Y~X1", data = data_glm, family = "gaussian")
fit_logit = pf.feglm(fml = "Y~X1", data = data_glm, family = "logit")
fit_probit = pf.feglm(fml = "Y~X1", data = data_glm, family = "probit")

pf.etable([
    fit_gaussian,
    fit_logit,
    fit_probit,
])
```

You can make predictions on the `response` and `link` scale via the `predict()` method:

```{python}
fit_logit.predict(type = "response")[0:5]
fit_logit.predict(type = "link")[0:5]
```

You can compute the **average marginal effect** via the [marginaleffects package](https://github.com/vincentarelbundock/pymarginaleffects):

```{python}
avg_slopes(fit_logit, variables = "X1")
```

Please take a look at the [marginaleffects book](https://marginaleffects.com/) to learn about other transformations that the `marginaleffects` package supports.

# Multiple Estimation

`PyFixest` supports a range of multiple estimation functionality: `sw`, `sw0`, `csw`, `csw0`, and multiple dependent variables. The meaning of these options is explained in the [Multiple Estimations](https://lrberge.github.io/fixest/articles/multiple_estimations.html) vignette of the `fixest` package:

> - `sw`: this function is replaced sequentially by each of its arguments. For example, `y ~ x1 + sw(x2, x3)` leads to two estimations: `y ~ x1 + x2` and `y ~ x1 + x3`.
> - `sw0`: identical to sw but first adds the empty element. E.g. `y ~ x1 + sw0(x2, x3)` leads to three estimations: `y ~ x1`, `y ~ x1 + x2` and `y ~ x1 + x3`.
> - `csw`: it stands for cumulative stepwise. It adds to the formula each of its arguments sequentially. E.g. `y ~ x1 + csw(x2, x3)` will become `y ~ x1 + x2` and `y ~ x1 + x2 + x3`.
> - `csw0`: identical to csw but first adds the empty element. E.g. `y ~ x1 + csw0(x2, x3)` leads to three estimations: `y ~ x1`, `y ~ x1 + x`2 and `y ~ x1 + x2 + x3`.

Additionally, we support `split` and `fsplit` function arguments.
> - `split` allows to split a sample by a given variable. If specified, `pf.feols()` and `pf.fepois()` will loop through all resulting sample splits.
> - `fsplit` works just as `split`, but fits the model on the full sample as well.

If multiple regression syntax is used,
`feols()` and `fepois` returns an instance of a `FixestMulti` object, which essentially consists of a dicionary of `Fepois` or [Feols](/reference/Feols.qmd) instances.

```{python}
multi_fit = pf.feols(fml="Y ~ X1 | csw0(f1, f2)", data=data, vcov="HC1")
multi_fit
```

```{python}
multi_fit.etable()
```

You can access an individual model by its name - i.e. a formula - via the `all_fitted_models` attribute.

```{python}
multi_fit.all_fitted_models["Y~X1"].tidy()
```

or equivalently via the `fetch_model` method:

```{python}
multi_fit.fetch_model(0).tidy()
```

Here, `0` simply fetches the first model stored in the `all_fitted_models` dictionary, `1` the second etc.

Objects of type `Fixest` come with a range of additional methods: `tidy()`, `coef()`, `vcov()` etc, which
essentially loop over the equivalent methods of all fitted models. E.g. `Fixest.vcov()` updates inference for all
models stored in `Fixest`.


```{python}
multi_fit.vcov("iid").summary()
```

You can summarize multiple models at once via `etable()`. `etable()` has many options to customize the output to obtain publication-ready tables.

```{python}
pf.etable(
    [fit, fit2],
    labels={"Y": "Wage", "X1": "Age", "X2": "Years of Schooling"},
    felabels={"f1": "Industry Fixed Effects"},
    caption="Regression Results",
)
```

You can also visualize multiple estimation results via `iplot()` and `coefplot()`:

```{python}
multi_fit.coefplot().show()
```

# Difference-in-Differences / Event Study Designs

`PyFixest` supports eventy study designs via two-way fixed effects, Gardner's 2-stage estimator, and the linear projections estimator.

```{python}
url = "https://raw.githubusercontent.com/py-econometrics/pyfixest/master/pyfixest/did/data/df_het.csv"
df_het = pd.read_csv(url)

df_het.head()
```


```{python}
fit_did2s = pf.did2s(
    df_het,
    yname="dep_var",
    first_stage="~ 0 | state + year",
    second_stage="~i(rel_year,ref= -1.0)",
    treatment="treat",
    cluster="state",
)


fit_twfe = pf.feols(
    "dep_var ~ i(rel_year,ref = -1.0) | state + year",
    df_het,
    vcov={"CRV1": "state"},
)

from pyfixest.report.utils import rename_categoricals
pf.iplot(
    [fit_did2s, fit_twfe], coord_flip=False, figsize=(900, 400), title="TWFE vs DID2S", rotate_xticks=90,
    labels= rename_categoricals(fit_did2s._coefnames, template="{value_int}")
)
```


The `event_study()` function provides a common API for several event study estimators.

```{python}
fit_twfe = pf.event_study(
    data=df_het,
    yname="dep_var",
    idname="state",
    tname="year",
    gname="g",
    estimator="twfe",
)

fit_did2s = pf.event_study(
    data=df_het,
    yname="dep_var",
    idname="state",
    tname="year",
    gname="g",
    estimator="did2s",
)

pf.etable([fit_twfe, fit_did2s])
```

For more details see the vignette on [Difference-in-Differences Estimation](https://py-econometrics.github.io/pyfixest/difference-in-differences.html).
