Using formulas to specify models

All of the models can be specified using formulas. The formulas used here utilize patsy are similar to those in statsmodels. The basis formula syntax for a single variable regression would be

y ~ 1 + x

The formulas used with BetweenOLS, PooledOLS and RandomEffects are completely standard and are identical to statsmodels. FirstDifferenceOLS is nearly identical with the caveat that the model cannot include an intercept.

PanelOLS, which implements effects (entity, time or other) has a small extension to the formula which allows entity effects or time effects (or both) to be specified as part of the formula. While it is not possible to specify other effects using the formula interface, these can be included as an optional parameter when using a formula.

Loading and preparing data

When using formulas, a MultiIndex pandas dataframe where the index is entity-time is required. Here the Grunfeld data, from “The Determinants of Corporate Investment”, provided by statsmodels, is used to illustrate the use of formulas. This dataset contains data on firm investment, market value and the stock of plant capital.

set_index is used to set the index using variables from the dataset.

[1]:
from statsmodels.datasets import grunfeld

data = grunfeld.load_pandas().data
data = data.set_index(["firm", "year"])
print(data.head())
                       invest   value  capital
firm           year
General Motors 1935.0   317.6  3078.5      2.8
               1936.0   391.8  4661.7     52.6
               1937.0   410.6  5387.1    156.9
               1938.0   257.7  2792.2    209.2
               1939.0   330.8  4313.2    203.4

PanelOLS with Entity Effects

Entity effects are specified using the special command EntityEffects. By default a constant is not included, and so if a constant is desired, 1+ should be included in the formula. When including effects, the model and fit are identical whether a constant is included or not.

[2]:
from linearmodels import PanelOLS

mod = PanelOLS.from_formula("invest ~ value + capital + EntityEffects", data=data)
print(mod.fit())
                          PanelOLS Estimation Summary
================================================================================
Dep. Variable:                 invest   R-squared:                        0.7667
Estimator:                   PanelOLS   R-squared (Between):              0.8223
No. Observations:                 220   R-squared (Within):               0.7667
Date:                Thu, Dec 16 2021   R-squared (Overall):              0.8132
Time:                        11:31:27   Log-likelihood                   -1167.4
Cov. Estimator:            Unadjusted
                                        F-statistic:                      340.08
Entities:                          11   P-value                           0.0000
Avg Obs:                       20.000   Distribution:                   F(2,207)
Min Obs:                       20.000
Max Obs:                       20.000   F-statistic (robust):             340.08
                                        P-value                           0.0000
Time periods:                      20   Distribution:                   F(2,207)
Avg Obs:                       11.000
Min Obs:                       11.000
Max Obs:                       11.000

                             Parameter Estimates
==============================================================================
            Parameter  Std. Err.     T-stat    P-value    Lower CI    Upper CI
------------------------------------------------------------------------------
capital        0.3100     0.0165     18.744     0.0000      0.2774      0.3426
value          0.1101     0.0113     9.7461     0.0000      0.0879      0.1324
==============================================================================

F-test for Poolability: 49.207
P-value: 0.0000
Distribution: F(10,207)

Included effects: Entity

PanelOLS with Entity Effects and a constant

The constant can be explicitly included using the 1 + notation. When a constant is included in the model, and additional constraint is imposed that the number of the effects is 0. This allows the constant to be identified using the grand mean of the dependent and the regressors.

[3]:
mod = PanelOLS.from_formula("invest ~ 1 + value + capital + EntityEffects", data=data)
print(mod.fit())
                          PanelOLS Estimation Summary
================================================================================
Dep. Variable:                 invest   R-squared:                        0.7667
Estimator:                   PanelOLS   R-squared (Between):              0.8193
No. Observations:                 220   R-squared (Within):               0.7667
Date:                Thu, Dec 16 2021   R-squared (Overall):              0.8071
Time:                        11:31:27   Log-likelihood                   -1167.4
Cov. Estimator:            Unadjusted
                                        F-statistic:                      340.08
Entities:                          11   P-value                           0.0000
Avg Obs:                       20.000   Distribution:                   F(2,207)
Min Obs:                       20.000
Max Obs:                       20.000   F-statistic (robust):             340.08
                                        P-value                           0.0000
Time periods:                      20   Distribution:                   F(2,207)
Avg Obs:                       11.000
Min Obs:                       11.000
Max Obs:                       11.000

                             Parameter Estimates
==============================================================================
            Parameter  Std. Err.     T-stat    P-value    Lower CI    Upper CI
------------------------------------------------------------------------------
Intercept     -55.272     10.891    -5.0750     0.0000     -76.743     -33.800
capital        0.3100     0.0165     18.744     0.0000      0.2774      0.3426
value          0.1101     0.0113     9.7461     0.0000      0.0879      0.1324
==============================================================================

F-test for Poolability: 49.207
P-value: 0.0000
Distribution: F(10,207)

Included effects: Entity

Panel with Entity and Time Effects

Time effects can be similarly included using TimeEffect. In many models, time effects can be consistently estimated and so they could be equivalently included in the set of regressors using a categorical variable.

[4]:
mod = PanelOLS.from_formula(
    "invest ~ 1 + value + capital + EntityEffects + TimeEffects", data=data
)
print(mod.fit())
                          PanelOLS Estimation Summary
================================================================================
Dep. Variable:                 invest   R-squared:                        0.7253
Estimator:                   PanelOLS   R-squared (Between):              0.7944
No. Observations:                 220   R-squared (Within):               0.7566
Date:                Thu, Dec 16 2021   R-squared (Overall):              0.7856
Time:                        11:31:27   Log-likelihood                   -1153.0
Cov. Estimator:            Unadjusted
                                        F-statistic:                      248.15
Entities:                          11   P-value                           0.0000
Avg Obs:                       20.000   Distribution:                   F(2,188)
Min Obs:                       20.000
Max Obs:                       20.000   F-statistic (robust):             248.15
                                        P-value                           0.0000
Time periods:                      20   Distribution:                   F(2,188)
Avg Obs:                       11.000
Min Obs:                       11.000
Max Obs:                       11.000

                             Parameter Estimates
==============================================================================
            Parameter  Std. Err.     T-stat    P-value    Lower CI    Upper CI
------------------------------------------------------------------------------
Intercept     -72.394     12.732    -5.6861     0.0000     -97.509     -47.278
capital        0.3514     0.0210     16.696     0.0000      0.3099      0.3930
value          0.1167     0.0129     9.0219     0.0000      0.0912      0.1422
==============================================================================

F-test for Poolability: 18.476
P-value: 0.0000
Distribution: F(29,188)

Included effects: Entity, Time

Between OLS

The other panel models are straight-forward and are included for completeness.

[5]:
from linearmodels import BetweenOLS, FirstDifferenceOLS, PooledOLS

mod = BetweenOLS.from_formula("invest ~ 1 + value + capital", data=data)
print(mod.fit())
                         BetweenOLS Estimation Summary
================================================================================
Dep. Variable:                 invest   R-squared:                        0.8644
Estimator:                 BetweenOLS   R-squared (Between):              0.8644
No. Observations:                  11   R-squared (Within):               0.4195
Date:                Thu, Dec 16 2021   R-squared (Overall):              0.7616
Time:                        11:31:27   Log-likelihood                   -61.997
Cov. Estimator:            Unadjusted
                                        F-statistic:                      25.500
Entities:                          11   P-value                           0.0003
Avg Obs:                       20.000   Distribution:                     F(2,8)
Min Obs:                       20.000
Max Obs:                       20.000   F-statistic (robust):             25.500
                                        P-value                           0.0003
Time periods:                      20   Distribution:                     F(2,8)
Avg Obs:                       11.000
Min Obs:                       11.000
Max Obs:                       11.000

                             Parameter Estimates
==============================================================================
            Parameter  Std. Err.     T-stat    P-value    Lower CI    Upper CI
------------------------------------------------------------------------------
Intercept     -7.3825     40.444    -0.1825     0.8597     -100.65      85.881
capital        0.0297     0.1746     0.1700     0.8692     -0.3730      0.4323
value          0.1346     0.0269     5.0065     0.0010      0.0726      0.1966
==============================================================================

First Difference OLS

The first difference model must never include a constant since this is not identified after differencing.

[6]:
mod = FirstDifferenceOLS.from_formula("invest ~ value + capital", data=data)
print(mod.fit())
                     FirstDifferenceOLS Estimation Summary
================================================================================
Dep. Variable:                 invest   R-squared:                        0.4287
Estimator:         FirstDifferenceOLS   R-squared (Between):              0.8643
No. Observations:                 209   R-squared (Within):               0.7539
Date:                Thu, Dec 16 2021   R-squared (Overall):              0.8461
Time:                        11:31:28   Log-likelihood                   -1071.1
Cov. Estimator:            Unadjusted
                                        F-statistic:                      77.679
Entities:                          11   P-value                           0.0000
Avg Obs:                       20.000   Distribution:                   F(2,207)
Min Obs:                       20.000
Max Obs:                       20.000   F-statistic (robust):             77.679
                                        P-value                           0.0000
Time periods:                      20   Distribution:                   F(2,207)
Avg Obs:                       11.000
Min Obs:                       11.000
Max Obs:                       11.000

                             Parameter Estimates
==============================================================================
            Parameter  Std. Err.     T-stat    P-value    Lower CI    Upper CI
------------------------------------------------------------------------------
capital        0.2786     0.0449     6.1990     0.0000      0.1900      0.3673
value          0.0891     0.0078     11.348     0.0000      0.0736      0.1045
==============================================================================

Pooled OLS

The pooled OLS estimator is a special case of PanelOLS when there are no effects. It is effectively identical to OLS in statsmodels (or WLS) but is included for completeness.

[7]:
mod = PooledOLS.from_formula("invest ~ 1 + value + capital", data=data)
print(mod.fit())
                          PooledOLS Estimation Summary
================================================================================
Dep. Variable:                 invest   R-squared:                        0.8179
Estimator:                  PooledOLS   R-squared (Between):              0.8426
No. Observations:                 220   R-squared (Within):               0.7357
Date:                Thu, Dec 16 2021   R-squared (Overall):              0.8179
Time:                        11:31:28   Log-likelihood                   -1301.3
Cov. Estimator:            Unadjusted
                                        F-statistic:                      487.28
Entities:                          11   P-value                           0.0000
Avg Obs:                       20.000   Distribution:                   F(2,217)
Min Obs:                       20.000
Max Obs:                       20.000   F-statistic (robust):             487.28
                                        P-value                           0.0000
Time periods:                      20   Distribution:                   F(2,217)
Avg Obs:                       11.000
Min Obs:                       11.000
Max Obs:                       11.000

                             Parameter Estimates
==============================================================================
            Parameter  Std. Err.     T-stat    P-value    Lower CI    Upper CI
------------------------------------------------------------------------------
Intercept     -38.410     8.4134    -4.5654     0.0000     -54.992     -21.828
capital        0.2275     0.0242     9.3904     0.0000      0.1798      0.2753
value          0.1145     0.0055     20.753     0.0000      0.1037      0.1254
==============================================================================