linearmodels.iv.common.kleibergen_paap¶

linearmodels.iv.common.kleibergen_paap(endog: ndarray[tuple[int, ...], dtype[float64]], instr: ndarray[tuple[int, ...], dtype[float64]], exog: ndarray[tuple[int, ...], dtype[float64]], cov_type: str = 'robust', cov_config: dict[str, Any] | None = None) → WaldTestStatistic | InvalidTestStatistic[source]¶

Kleibergen-Paap rk LM test of underidentification

Parameters:¶

endog: ndarray[tuple[int, ...], dtype[float64]]¶: Weighted endogenous regressor array (nobs, nendog)
instr: ndarray[tuple[int, ...], dtype[float64]]¶: Weighted instrument array (nobs, ninstr)
exog: ndarray[tuple[int, ...], dtype[float64]]¶: Weighted exogenous regressor array (nobs, nexog), partialled out before testing. Include a constant column here if the model has one.
cov_type: str = 'robust'¶: Covariance estimator used for the first-stage scores. One of “unadjusted”, “robust”, “clustered” or “kernel”, or any other name that the IV models accept for the same estimator, such as “homoskedastic” or “heteroskedastic”. The default is “robust”.
cov_config: dict[str, Any] | None = None¶: Options for the covariance estimator. “clustered” requires clusters, a 1-d array with one element for each observation. “kernel” uses kernel (default “bartlett”) and bandwidth. If the bandwidth is not provided it is selected automatically.

Returns:¶

Test statistic, distributed chi2(ninstr - nendog + 1) under the null that the first-stage coefficient matrix does not have full column rank, which is the null that the model is underidentified. An InvalidTestStatistic is returned if the statistic is not defined. See cragg_donald(). It is also returned for two-way clustering, which is not supported, for fewer than two clusters, and if the covariance of the statistic is singular.

Return type:¶

linearmodels.shared.hypotheses.WaldTestStatistic or linearmodels.shared.hypotheses.InvalidTestStatistic

Raises:¶

ValueError – If cov_type is not supported, or is “clustered” and cov_config does not contain clusters or clusters does not have an element for each row of the arrays.

Warning

The statistic is asymptotic. It is robust only to the extent that the chosen covariance estimator is appropriate for the data. Use “clustered” when observations are dependent within groups, and “kernel” when the data are ordered in time and the scores are autocorrelated. The data must be sorted in time order when “kernel” is used. If the dependence is not accounted for, including by “unadjusted” and “robust”, the statistic can reject the null of underidentification much too often when the instruments and first-stage errors are clustered or persistent.

The covariance has dimension \(km \times km\) and is poorly estimated unless \(n\) is large relative to \(km\). It is formed as a dense matrix, and so the memory that is needed grows with \((km)^2\). With clustering it needs many clusters, and with few clusters the test can be very conservative. A kernel bandwidth that is too short for the dependence does not remove the over-rejection. If there are too few clusters or observations the covariance is rank deficient. A warning is issued, a generalized inverse is used and the degrees of freedom are reduced, but the result should not be relied on. The chi-square approximation is also less reliable with many instruments.

The null is that the model is underidentified. Rejecting it does not show that the instruments are strong, so this is not a test for weak instruments. With weak instruments the asymptotic distribution can be a poor approximation. Choosing whether to use an instrumental variable estimator based on the outcome of a test like this one distorts the inference that follows.

See also

kleibergen_paap_f: The F form of the statistic.
cragg_donald: The version that requires homoskedastic errors.

Notes

Let \(\hat{\Pi}\) be the \(k \times m\) first-stage coefficient matrix after the exogenous regressors have been partialled out. The test is applied to \(\hat{\Theta}\), the matrix of sample partial canonical correlations between the endogenous regressors and the excluded instruments. The null is that \(\Theta\) has rank \(m-1\). Let \(\hat{\Theta} = USV'\) be a singular value decomposition, and let \(A_\perp\) and \(B_\perp\) contain the singular vectors of the smallest \(m-1\) and \(1\) singular values. The statistic is

\[rk = n \,\mathrm{vec}(\lambda)' \left[(B_\perp \otimes A_\perp)' \hat{V}_{\Theta} (B_\perp \otimes A_\perp)\right]^{-1} \mathrm{vec}(\lambda), \quad \lambda = A_\perp' \hat{\Theta} B_\perp\]

where \(\hat{V}_\Theta\) is the estimated covariance of \(\sqrt{n}\,\mathrm{vec}(\hat{\Theta})\). It is built from the scores \(x_i \otimes z_i\), where \(x_i\) is the orthogonalized endogenous regressor and \(z_i\) is the orthogonalized instrument, using the covariance estimator chosen by cov_type. The statistic is asymptotically \(\chi^2_{k-m+1}\) under the null.

With cov_type “unadjusted” the covariance has a Kronecker structure and the statistic is Anderson’s canonical correlation LM statistic, \(n r^2_{\min}\). No small-sample adjustment is applied to the covariance, so the debiased option of the IV models has no effect on these statistics.

This is the statistic that Stata’s ivreg2 reports as the “Kleibergen-Paap rk LM statistic” in its underidentification test. It is computed following ranktest (Kleibergen and Schaffer). Results can differ in detail from Stata for kernel covariances since the bandwidth conventions differ: the Bartlett weight on lag \(j\) here is \(1 - j/(b+1)\), so a Stata bandwidth of \(b+1\) corresponds to a bandwidth of \(b\).

Weights. In a weighted model, each of the arrays has to be multiplied by the square root of the observation weights, including the constant column of exog, as for cragg_donald(). The scores are then those of the weighted first-stage regressions, and the covariance estimator is applied to them. The statistic does not change if all of the weights are multiplied by the same constant.

Clusters. With “clustered”, the scores are summed within the clusters in cov_config["clusters"]. clusters has one element for each row of the arrays, in the order of the rows. It is a 1-d array, or a 2-d array with a single column. Two-way clustering is not supported. The LM statistic is a quadratic form in a covariance with \(k-m+1\) rows, whose rank is at most the number of clusters, and so it needs at least \(k-m+1\) clusters. The scores of the Wald F statistic of kleibergen_paap_f() sum to zero, and so it needs one more cluster. A warning is issued if there are fewer clusters.

References

Kleibergen, F., & Paap, R. (2006). Generalized reduced rank tests using the singular value decomposition. Journal of Econometrics, 133(1), 97-126.

Kleibergen, F., & Schaffer, M. E. (2007). ranktest: Stata module to test the rank of a matrix using the Kleibergen-Paap rk statistic. Statistical Software Components S456865, Boston College.