A skeptical appraisal of asset pricing tests

Lewellen, Jonathan, Stefan Nagel, and Jay Shanken. “A skeptical appraisal of asset pricing tests” Journal of Financial Economics Vol. 96 (2010) 175-194.

Summary:

The authors bring to light some problems with empirical asset pricing tests which are common in the literature and then offer up several (partial) solutions as a way to have more confidence in the implications of asset pricing tests moving forward.

Many proposed asset pricing models offer explanations for the size and book-to-market effects. They then test their models, at least in part, using the Fama and French (1993) 25 size and book-to-market portfolios, concluding that the models do a good job of explaining these effects because they generate a high cross-sectional R-squared. The authors, however, suggest that this is a low hurdle to meet and that, while the models may give important economic insights, they may not perform as well as originally thought.

The problem is the strong factor structure of these test portfolios. The Fama and French (1993) factors explain “more than 90% of the time variation in the portfolios’ realized returns and more than 80% of the cross-sectional variation in their average returns.” Thus, a high R-squared for a proposed factor is achieved even when having a weak correlation with SMB or HML but not with the idiosyncratic residuals.

Illustrating the Problem:

The authors randomly generate artificial factors which are correlated with the three factors of Fama and French (1993), but not with the idiosyncratic residuals. They then obtain R-squared from regressions of returns from the 25 size and book-to-market portfolios on 1 to 5 of these randomly generated factors. Their figure 1 gives the results.

As we see, it is easy to obtain a high cross-sectional R-squared in each case when the estimation uses more than one factor. This is true even in the case of Panel C where only randomly drawn factors with an expected return close to zero are kept in the analysis (it should be zero).

Proposed (Partial) Solutions:

The authors suggest four:

1. Add additional test portfolios sorted on some other characteristic (the authors use the Fama and French 30 industry portfolio returns).

2. Impose restrictions on risk premia when guided by theory rather than just allowing it to find the best fit.

3. Report GLS cross-sectional R-squared rather than, or at least in addition to, those for OLS.

4. Report confidence intervals for test statistics. In particular, this will illustrate the (sometimes wide) range of parameters which are consistent with the data.

Empirical Results:

The authors test their suggestions on a total of eight models, which they report in Table 1 and the effectiveness of each of the models is clearly weaker than initial tests suggested.

In summary, the authors have pointed to some problems with how we evaluate the empirical successes of proposed asset pricing models and then offered some (partial) solutions to these problems. In all, they seem to have raised the hurdle which must be cleared in order to declare a proposed model a success in explaining cross-sectional variation in asset returns.