Using formulas to specify models

Formulas can be used to specify models using mostly standard formulaic syntax. Since system estimation is more complicated than the specification of a single model, there are two methods available to specify a system:

  • Dictionary of formulas

  • Single formula separated using {}

These examples use data on fringe benefits from F. Vella (1993), “A Simple Estimator for Simultaneous Models with Censored Endogenous Regressors” which appears in Wooldridge (2002). The model consists of two equations, one for hourly wage and the other for hourly benefits. The initial model uses the same regressors in both equations.

[1]:
import numpy as np
import pandas as pd
from linearmodels.datasets import fringe

data = fringe.load()

Dictionary

The dictionary syntax is virtually identical to standard formulaic syntax where each equation is specified in a key-value pair where the key is the equation label and the value is the formula. It is recommended to use an OrderedDict which will preserve equation order in results. Keys must be strings.

[2]:
from collections import OrderedDict

formula = OrderedDict()
formula["benefits"] = (
    "hrbens ~ educ + exper + expersq + union + south + nrtheast + nrthcen + male"
)
formula["earnings"] = "hrearn ~ educ + exper + expersq + nrtheast + married + male"
[3]:
from linearmodels.system import SUR

mod = SUR.from_formula(formula, data)
print(mod.fit(cov_type="unadjusted"))
                           System GLS Estimation Summary
===================================================================================
Estimator:                        GLS   Overall R-squared:                   0.6951
No. Equations.:                     2   McElroy's R-squared:                 0.2197
No. Observations:                 616   Judge's (OLS) R-squared:             0.1873
Date:                Tue, Sep 24 2024   Berndt's R-squared:                  0.3775
Time:                        09:29:36   Dhrymes's R-squared:                 0.6950
                                        Cov. Estimator:                  unadjusted
                                        Num. Constraints:                      None
                Equation: benefits, Dependent Variable: hrbens
==============================================================================
            Parameter  Std. Err.     T-stat    P-value    Lower CI    Upper CI
------------------------------------------------------------------------------
educ           0.0346     0.0050     6.8605     0.0000      0.0247      0.0445
exper          0.0350     0.0063     5.5974     0.0000      0.0228      0.0473
expersq       -0.0006     0.0001    -4.4326     0.0000     -0.0009     -0.0003
union          0.3682     0.0474     7.7659     0.0000      0.2753      0.4612
south         -0.1775     0.0586    -3.0289     0.0025     -0.2923     -0.0626
nrtheast      -0.1224     0.0714    -1.7132     0.0867     -0.2624      0.0176
nrthcen       -0.1433     0.0621    -2.3074     0.0210     -0.2650     -0.0216
male           0.2518     0.0490     5.1435     0.0000      0.1559      0.3478
                Equation: earnings, Dependent Variable: hrearn
==============================================================================
            Parameter  Std. Err.     T-stat    P-value    Lower CI    Upper CI
------------------------------------------------------------------------------
educ           0.3547     0.0357     9.9280     0.0000      0.2847      0.4247
exper         -0.0750     0.0492    -1.5260     0.1270     -0.1714      0.0213
expersq        0.0038     0.0011     3.4600     0.0005      0.0016      0.0059
nrtheast      -0.7127     0.4473    -1.5934     0.1111     -1.5894      0.1640
married        0.4472     0.3920     1.1410     0.2539     -0.3210      1.2155
male           1.8899     0.3904     4.8408     0.0000      1.1247      2.6551
==============================================================================

Covariance Estimator:
Homoskedastic (Unadjusted) Covariance (Debiased: False, GLS: True)

Curly Braces

The same formula can be expressed in a single string by surrounding each equation with braces {}.

[4]:
braces_formula = """
{hrbens ~ educ + exper + expersq + union + south + nrtheast + nrthcen + male}
{hrearn ~ educ + exper + expersq + nrtheast + married + male}
"""
braces_mod = SUR.from_formula(braces_formula, data)
braces_res = braces_mod.fit(cov_type="unadjusted")
print(braces_res)
                           System GLS Estimation Summary
===================================================================================
Estimator:                        GLS   Overall R-squared:                   0.6951
No. Equations.:                     2   McElroy's R-squared:                 0.2197
No. Observations:                 616   Judge's (OLS) R-squared:             0.1873
Date:                Tue, Sep 24 2024   Berndt's R-squared:                  0.3775
Time:                        09:29:36   Dhrymes's R-squared:                 0.6950
                                        Cov. Estimator:                  unadjusted
                                        Num. Constraints:                      None
                 Equation: hrbens, Dependent Variable: hrbens
==============================================================================
            Parameter  Std. Err.     T-stat    P-value    Lower CI    Upper CI
------------------------------------------------------------------------------
educ           0.0346     0.0050     6.8605     0.0000      0.0247      0.0445
exper          0.0350     0.0063     5.5974     0.0000      0.0228      0.0473
expersq       -0.0006     0.0001    -4.4326     0.0000     -0.0009     -0.0003
union          0.3682     0.0474     7.7659     0.0000      0.2753      0.4612
south         -0.1775     0.0586    -3.0289     0.0025     -0.2923     -0.0626
nrtheast      -0.1224     0.0714    -1.7132     0.0867     -0.2624      0.0176
nrthcen       -0.1433     0.0621    -2.3074     0.0210     -0.2650     -0.0216
male           0.2518     0.0490     5.1435     0.0000      0.1559      0.3478
                 Equation: hrearn, Dependent Variable: hrearn
==============================================================================
            Parameter  Std. Err.     T-stat    P-value    Lower CI    Upper CI
------------------------------------------------------------------------------
educ           0.3547     0.0357     9.9280     0.0000      0.2847      0.4247
exper         -0.0750     0.0492    -1.5260     0.1270     -0.1714      0.0213
expersq        0.0038     0.0011     3.4600     0.0005      0.0016      0.0059
nrtheast      -0.7127     0.4473    -1.5934     0.1111     -1.5894      0.1640
married        0.4472     0.3920     1.1410     0.2539     -0.3210      1.2155
male           1.8899     0.3904     4.8408     0.0000      1.1247      2.6551
==============================================================================

Covariance Estimator:
Homoskedastic (Unadjusted) Covariance (Debiased: False, GLS: True)

Labeled Formulas

When using the curly brace formula specification, the equation names are determined by the dependent variable names. When names are repeated as is the case in some datasets (e.g. a SUR on GDP of multiple countries) then the equation labels will be modified until they are unique. This can produce meaningless equation labels, and so it is possible to pass an equation label using the syntax

{label : dep ~ exog}
[5]:
labeled_formula = """
{benefits: hrbens ~ educ + exper + expersq + union + south + nrtheast + nrthcen + male}
{earnings: hrearn ~ educ + exper + expersq + nrtheast + married + male}
"""
labels_mod = SUR.from_formula(labeled_formula, data)
labeled_res = labels_mod.fit(cov_type="unadjusted")

print("Unlabeled")
print(braces_res.equation_labels)
print("Labeled")
print(labeled_res.equation_labels)
Unlabeled
['hrbens', 'hrearn']
Labeled
['benefits', 'earnings']

Other Options

Estimation Weights

SUR supports weights which are assumed to be proportional to the inverse variance of the data so that

\[V(y_i \times w_i) = \sigma^2 \,\,\forall i.\]

Weights can be passed using a DataFrame where each column.

Here the results are printed to ensure that the estimates are different from those in the standard GLS model.

[6]:
random_weights = np.random.chisquare(5, size=(616, 2))
random_weights = pd.DataFrame(random_weights, columns=["benefits", "earnings"])
weighted_mod = SUR.from_formula(formula, data, weights=random_weights)
print(weighted_mod.fit())
                           System GLS Estimation Summary
===================================================================================
Estimator:                        GLS   Overall R-squared:                   0.7002
No. Equations.:                     2   McElroy's R-squared:                 0.2211
No. Observations:                 616   Judge's (OLS) R-squared:             0.1846
Date:                Tue, Sep 24 2024   Berndt's R-squared:                  0.3501
Time:                        09:29:36   Dhrymes's R-squared:                 0.7001
                                        Cov. Estimator:                      robust
                                        Num. Constraints:                      None
                Equation: benefits, Dependent Variable: hrbens
==============================================================================
            Parameter  Std. Err.     T-stat    P-value    Lower CI    Upper CI
------------------------------------------------------------------------------
educ           0.0399     0.0049     8.1546     0.0000      0.0303      0.0495
exper          0.0248     0.0063     3.9521     0.0001      0.0125      0.0371
expersq       -0.0004     0.0002    -2.3692     0.0178     -0.0006  -6.143e-05
union          0.4389     0.0477     9.2064     0.0000      0.3455      0.5324
south         -0.1534     0.0557    -2.7518     0.0059     -0.2627     -0.0441
nrtheast      -0.1443     0.0665    -2.1700     0.0300     -0.2746     -0.0140
nrthcen       -0.1709     0.0548    -3.1184     0.0018     -0.2784     -0.0635
male           0.2281     0.0464     4.9198     0.0000      0.1372      0.3189
                Equation: earnings, Dependent Variable: hrearn
==============================================================================
            Parameter  Std. Err.     T-stat    P-value    Lower CI    Upper CI
------------------------------------------------------------------------------
educ           0.3824     0.1179     3.2427     0.0012      0.1513      0.6135
exper         -0.1327     0.2654    -0.4999     0.6171     -0.6528      0.3875
expersq        0.0052     0.0070     0.7522     0.4519     -0.0084      0.0189
nrtheast      -0.8381     0.7005    -1.1964     0.2315     -2.2111      0.5349
married        0.5310     0.4095     1.2968     0.1947     -0.2716      1.3336
male           1.8749     0.3206     5.8480     0.0000      1.2465      2.5033
==============================================================================

Covariance Estimator:
Heteroskedastic (Robust) Covariance (Debiased: False, GLS: True)

Prespecified Residual Covariance

Like a standard SUR, it is possible to pass a prespecified residual covariance for use in the GLS step. This is done using the keyword argument sigma in the from_formula method, and is otherwise identical to passing one to the standard SUR.