Home > Quantitative Equity > The importance of the benchmark

The importance of the benchmark

February 24th, 2009 Leave a comment Go to comments

Business school classes and other quantitative equity readings spend a lot of time on model construction (the theoretical underpinnings of factor models, how factor models are created, the statistical issues to watch out for) and portfolio construction (mean-variance theory, efficient frontier, optimal portfolio), but hardly ever touch on benchmark selection and it’s importance in equity portfolio management. So, I’m dedicating this post to the under-appreciated benchmark…

(If you haven’t done so, you may want to read my post on Absorelative performance to better understand the performance context of this post, and to fully appreciate the effect of a bad benchmark.)

Imagine this, you’re going into the fourth quarter with an outperforming portfolio – you’ve racked up three times your excess return target for the portfolio (think “Cha-ching! There will be a great bonus this near!”). To lock in your gain, you decide to bring down your tracking error (that is, reduce the differences between your portfolio and your benchmark so that on a relative basis you neither make nor lose money going forward). You do the math and realize that in order for you to fall below the excess return target, your portfolio would have to lost a substantial percentage relative to the benchmark – an event you doubt will happen. You cross your fingers and hope to coast for the rest of the year, or maybe take a nice long vacation. I hear Monaco is nice…

Fast-forward three months: it’s the last week of December, and you’re looking at the (almost) year-end performance numbers. Your portfolio, which had earned three times the excess return target (that is, three time the amount that you were supposed to beat the benchmark by) by June now trails the benchmark by 50 basis points (one-half of a percent). That “Cha ching” you had heard a quarter earlier now sounds like the Lose-A-Turn sound on Wheel of Fortune (or any other bad event on your game show of choice) – there most certainly won’t be a bonus this year…

So what happened?! How did this great story of fortune turn into a big bust? We were entering the fourth quarter of the championship game, up by three touchdowns, and ended up losing the game by a field goal. What happened?!

What happened is that we were playing a running defense in the fourth quarter while the other team was playing a passing offense. In fact, we’d been doing the same thing during the whole game, but had just gotten lucky during the first three quarters. Heck, it was working, so why change it?

This mistake is rarely seen in football, but unfortunately is much more prevalent in portfolio management. It often happens that you start the year with a performance objective relative to some benchmark – let’s say the S&P 500 (a portfolio of the largest 500 firms in the U.S.). Now suppose that your model for selecting stocks ends up picking some firms that are in the S&P 500, but picks more firms that are smaller in size. Let’s further assume that during the first three quarters of the year, small firms on average outperform large firms. Since your portfolio is mostly small firms, it, by definition, outperforms the benchmark (which is all large firms). Finally, suppose that trend reverses in the last quarter of the year, and reverses in a big way. Since small firms perform worse than large firms in the last quarter, your portfolio under-performs the benchmark. Depending on the magnitude of the under-performance, this could cost you all of your excess return and then some (as in my example above). This is clearly a type of volatility (or tracking error to be precise) that the portfolio manager just doesn’t want!

So now you understand the source of the issue – that the benchmark is incorrectly specified – if the portfolio is comprised of smaller firms, then the benchmark should be selected to match. This way there is no upward (or downward) bias to the manager’s performance as a result of differences in firm size (or style, or any other factors). As you see in the above example, this misspecification can negate any return the manager may have earned through his skill (though in our case, the three times out-performance was a combination of skill and luck to the extent that smaller firms outperformed in the first three quarters). Thus we see that correct specification of the benchmark is just as important as alpha, or the skill of your manager.

So the issue seems easy to fix – why not just change the benchmark to match the portfolio’s attributes? There are a couple of reasons for this. If you’re benefitingfrom the misspecified benchmark, you’re not likely to have the benchmark changed since you’ll lose any benefit you may be enjoying (both from a financial bonus perspective, and from a track record perspective). If you’re disadvantaged by the misspecified benchmark, you may request it be changed, but the request is not likely to be approved. Imagine your job was to approve these requests, and you see a request on your desk. The manager, who happens to be under-performing his benchmark, wants you to change his benchmark to something that will result in his performance looking better. An immediate skepticism sets in - is this manager trying to game the system by having us choose a benchmark that favors him more? This skepticism is likely to result in a rejected request (rightfully so too, since I imagine there are many managers out there who would try just such a thing if they could get away with it!). So, changing the benchmark mid-year is just not likely to happen.

The only solution may be to finish the year and take whatever gain or loss that may come with the misspecified benchmark, with the goal of fixing the issue before the start of the next year (so that “gaming the system” becomes irrelevant). Even better, rather than hastily just picking a benchmark, the manager should fully understand his strategy and model, the stocks it typically picks, and the manager’s investment universe. After carefully considering these things, an appropriate benchmark should be selected. This prudence can alleviate a lot of subsequent problems, and results in more accurate performance measurement and compensation – something that we’ll all be better off for in the long run. So next time you’re chasing that alpha and building fascinating models, don’t forget to take a step back and give that under-appreciated benchmark some consideration.

  1. No comments yet.
  1. No trackbacks yet.