Bootstrap Examples

This setup code is required to run in an IPython notebook

[1]:
import warnings
warnings.simplefilter('ignore')

%matplotlib inline
import matplotlib.pyplot as plt
import seaborn

seaborn.set_style('darkgrid')
plt.rc("figure", figsize=(16, 6))
plt.rc("savefig", dpi=90)
plt.rc("font",family="sans-serif")
plt.rc("font",size=14)

Sharpe Ratio

The Sharpe Ratio is an important measure of return per unit of risk. The example shows how to estimate the variance of the Sharpe Ratio and how to construct confidence intervals for the Sharpe Ratio using a long series of U.S. equity data.

[2]:
import numpy as np
import pandas as pd

import arch.data.frenchdata

ff = arch.data.frenchdata.load()

The data set contains the Fama-French factors, including the excess market return.

[3]:
excess_market = ff.iloc[:, 0]
print(ff.describe())
            Mkt-RF          SMB          HML           RF
count  1109.000000  1109.000000  1109.000000  1109.000000
mean      0.659946     0.206555     0.368864     0.274220
std       5.327524     3.191132     3.482352     0.253377
min     -29.130000   -16.870000   -13.280000    -0.060000
25%      -1.970000    -1.560000    -1.320000     0.030000
50%       1.020000     0.070000     0.140000     0.230000
75%       3.610000     1.730000     1.740000     0.430000
max      38.850000    36.700000    35.460000     1.350000

The next step is to construct a function that computes the Sharpe Ratio. This function also return the annualized mean and annualized standard deviation which will allow the covariance matrix of these parameters to be estimated using the bootstrap.

[4]:
def sharpe_ratio(x):
    mu, sigma = 12 * x.mean(), np.sqrt(12 * x.var())
    values = np.array([mu, sigma, mu / sigma]).squeeze()
    index = ['mu', 'sigma', 'SR']
    return pd.Series(values, index=index)

The function can be called directly on the data to show full sample estimates.

[5]:
params = sharpe_ratio(excess_market)
params
[5]:
mu        7.919351
sigma    18.455084
SR        0.429115
dtype: float64

Reproducibility

All bootstraps accept the keyword argument random_state which can contain a NumPy RandomState instance. This allows the same pseudo random values to be used across multiple runs.

Warning

The bootstrap chosen must be appropriate for the data. Squared returns are serially correlated, and so a time-series bootstrap is required.

Bootstraps are initialized with any bootstrap specific parameters and the data to be used in the bootstrap. Here the 12 is the average window length in the Stationary Bootstrap, and the next input is the data to be bootstrapped.

[6]:
from arch.bootstrap import StationaryBootstrap
# Initialize with entropy from random.org
entropy = [877788388, 418255226, 989657335, 69307515]
rs = np.random.RandomState(entropy)

bs = StationaryBootstrap(12, excess_market, random_state=rs)
results = bs.apply(sharpe_ratio, 2500)
SR = pd.DataFrame(results[:, -1:], columns=['SR'])
fig = SR.hist(bins=40)
../_images/bootstrap_bootstrap_examples_15_0.png
[7]:
cov = bs.cov(sharpe_ratio, 1000)
cov = pd.DataFrame(cov, index=params.index, columns=params.index)
print(cov)
se = pd.Series(np.sqrt(np.diag(cov)), index=params.index)
se.name = 'Std Errors'
print('\n')
print(se)
             mu     sigma        SR
mu     3.835234 -0.700250  0.222532
sigma -0.700250  3.517844 -0.119008
SR     0.222532 -0.119008  0.014912


mu       1.958375
sigma    1.875592
SR       0.122114
Name: Std Errors, dtype: float64
[8]:
ci = bs.conf_int(sharpe_ratio, 1000, method='basic')
ci = pd.DataFrame(ci, index=['Lower', 'Upper'], columns=params.index)
print(ci)
              mu      sigma        SR
Lower   4.319805  14.559092  0.186293
Upper  12.004185  21.517835  0.660455

Alternative confidence intervals can be computed using a variety of methods. Setting reuse=True allows the previous bootstrap results to be used when constructing confidence intervals using alternative methods.

[9]:
ci = bs.conf_int(sharpe_ratio, 1000, method='percentile', reuse=True)
ci = pd.DataFrame(ci, index=['Lower', 'Upper'], columns=params.index)
print(ci)
              mu      sigma        SR
Lower   3.834517  15.392333  0.197774
Upper  11.518896  22.351076  0.671937

Optimal Block Length Estimation

The function optimal_block_length can be used to estimate the optimal block lengths for the Stationary and Circular bootstraps. Here we use the squared market return since the Sharpe ratio depends on the mean and the variance, and the autocorrelation in the squares is stronger than in the returns.

[10]:
from arch.bootstrap import optimal_block_length
opt = optimal_block_length(excess_market ** 2)
print(opt)
        stationary   circular
Mkt-RF   47.766787  54.679322

We can repeat the analysis above using the estimated optimal block length. Here we see that the extremes appear to be slightly more extreme.

[11]:
# Reinitialize using the same entropy
rs = np.random.RandomState(entropy)

bs = StationaryBootstrap(opt.loc["Mkt-RF", "stationary"], excess_market, random_state=rs)
results = bs.apply(sharpe_ratio, 2500)
SR = pd.DataFrame(results[:, -1:], columns=['SR'])
fig = SR.hist(bins=40)
../_images/bootstrap_bootstrap_examples_23_0.png

Probit (statsmodels)

The second example makes use of a Probit model from statsmodels. The demo data is university admissions data which contains a binary variable for being admitted, GRE score, GPA score and quartile rank. This data is downloaded from the internet and imported using pandas.

[12]:
import arch.data.binary

binary = arch.data.binary.load()
binary = binary.dropna()
print(binary.describe())
            admit         gre         gpa       rank
count  400.000000  400.000000  400.000000  400.00000
mean     0.317500  587.700000    3.389900    2.48500
std      0.466087  115.516536    0.380567    0.94446
min      0.000000  220.000000    2.260000    1.00000
25%      0.000000  520.000000    3.130000    2.00000
50%      0.000000  580.000000    3.395000    2.00000
75%      1.000000  660.000000    3.670000    3.00000
max      1.000000  800.000000    4.000000    4.00000

Fitting the model directly

The first steps are to build the regressor and the dependent variable arrays. Then, using these arrays, the model can be estimated by calling fit

[13]:
import statsmodels.api as sm

endog = binary[['admit']]
exog = binary[['gre', 'gpa']]
const = pd.Series(np.ones(exog.shape[0]), index=endog.index)
const.name = 'Const'
exog = pd.DataFrame([const, exog.gre, exog.gpa]).T

# Estimate the model
mod = sm.Probit(endog, exog)
fit = mod.fit(disp=0)
params = fit.params
print(params)
Const   -3.003536
gre      0.001643
gpa      0.454575
dtype: float64

The wrapper function

Most models in statsmodels are implemented as classes, require an explicit call to fit and return a class containing parameter estimates and other quantities. These classes cannot be directly used with the bootstrap methods. However, a simple wrapper can be written that takes the data as the only inputs and returns parameters estimated using a Statsmodel model.

[14]:
def probit_wrap(endog, exog):
    return sm.Probit(endog, exog).fit(disp=0).params

A call to this function should return the same parameter values.

[15]:
probit_wrap(endog, exog)
[15]:
Const   -3.003536
gre      0.001643
gpa      0.454575
dtype: float64

The wrapper can be directly used to estimate the parameter covariance or to construct confidence intervals.

[16]:
from arch.bootstrap import IIDBootstrap

bs = IIDBootstrap(endog=endog, exog=exog)
cov = bs.cov(probit_wrap, 1000)
cov = pd.DataFrame(cov, index=exog.columns, columns=exog.columns)
print(cov)
          Const           gre       gpa
Const  0.384854 -5.813301e-05 -0.101077
gre   -0.000058  4.244940e-07 -0.000057
gpa   -0.101077 -5.701083e-05  0.039506
[17]:
se = pd.Series(np.sqrt(np.diag(cov)), index=exog.columns)
print(se)
print('T-stats')
print(params / se)
Const    0.620366
gre      0.000652
gpa      0.198760
dtype: float64
T-stats
Const   -4.841552
gre      2.521038
gpa      2.287054
dtype: float64
[18]:
ci = bs.conf_int(probit_wrap, 1000, method='basic')
ci = pd.DataFrame(ci, index=['Lower', 'Upper'], columns=exog.columns)
print(ci)
          Const       gre       gpa
Lower -4.194772  0.000354  0.020491
Upper -1.582297  0.002943  0.825360

Speeding things up

Starting values can be provided to fit which can save time finding starting values. Since the bootstrap parameter estimates should be close to the original sample estimates, the full sample estimated parameters are reasonable starting values. These can be passed using the extra_kwargs dictionary to a modified wrapper that will accept a keyword argument containing starting values.

[19]:
def probit_wrap_start_params(endog, exog, start_params=None):
    return sm.Probit(endog, exog).fit(start_params=start_params, disp=0).params
[20]:
bs.reset()  # Reset to original state for comparability
cov = bs.cov(
    probit_wrap_start_params,
    1000,
    extra_kwargs={'start_params': params.values})
cov = pd.DataFrame(cov, index=exog.columns, columns=exog.columns)
print(cov)
          Const           gre       gpa
Const  0.384854 -5.813301e-05 -0.101077
gre   -0.000058  4.244940e-07 -0.000057
gpa   -0.101077 -5.701083e-05  0.039506

Bootstrapping Uneven Length Samples

Independent samples of uneven length are common in experiment settings, e.g., A/B testing of a website. The IIDBootstrap allows for arbitrary dependence within an observation index and so cannot be naturally applied to these data sets. The IndependentSamplesBootstrap allows datasets with variables of different lengths to be sampled by exploiting the independence of the values to separately bootstrap each component. Below is an example showing how a confidence interval can be constructed for the difference in means of two groups.

[ ]:
from arch.bootstrap import IndependentSamplesBootstrap


def mean_diff(x, y):
    return x.mean() - y.mean()


rs = np.random.RandomState(0)
treatment = 0.2 + rs.standard_normal(200)
control = rs.standard_normal(800)

bs = IndependentSamplesBootstrap(treatment, control, random_state=rs)
print(bs.conf_int(mean_diff, method='studentized'))