Archive for the ‘Quantitative Equity’ Category

Matlab woes…

August 11th, 2010 No comments

WARNING: The following rant is somewhat technical. It’s a departure from the economics/finance of quantitative equity into the actual technical implementation and the challenges that are often faced. If you don’t know what threading, serialization, and parallel computing is you’ll probably want to stop reading right here!

I use Matlab to analyze the gigabytes of equities data I have to put on trades. Last week I’ve run into a frustrating brick wall with Matlab that I can’t seem to get around. Hopefully this helps someone else in a similar situation.

Until a couple of weeks ago, it would take 8 hours for my program to simulate 20 years of trading activity and spit out results. Clearly this is annoying since if I want to change a quantitative factor or two and see how things would look, I have to let it run for 8 hours! At this point, I was fully utilizing Matlab’s parallel computing features (which actually helped considerably – without them I would have seen a run time of 20+ hours!).

I had the idea of implementing some kind of caching scheme to speed up the data retrieval from a heavily normalized database. That took a bit of programming to implement, but is now working like a charm. Run time was reduced from 8 hours to about 3 hours. That’s a great improvement, but if we could get faster still, that’d be great.

To understand where the bottleneck was, I killed of one factor at a time and measured how runtime was affected. In the end, I found that killing all factors (so for each month my program essentially connects to the database and disconnects), made an inconsequential difference. That is, the majority of the time spent is in the database connect/disconnect (something I, as a former programmer, should have known quite well!).

So, the solution: reuse the database connections (in the technical world, known as connection pooling). So, I spent many hours investigating Matlab’s database toolbox and its ability to pool connections. To make a long story short, Matlab’s database connectivity (build on Java JDBC drivers) doesn’t support connection pooling. I suppose if I were running it in an application server (Tomcat, something else…), I may be able to work. But, I have neither the environment, expertise, nor desire to go down that path.

So, I decided to implement my own pooling. To do this, I wrote a Java connection pool manager. That is actually a simple task since all a connection pool manager is, is a vector of connections and the logic to reuse a connection if it is idle. Having never done Matlab/Java integration before, it took me a bit to figure out how to make it all work, but in short order I was there. All set, right? No…

I changed my code to use the connection pooling rather than native Matlab database connections. However, it blew up due to the parallelization. What I didn’t realize going in is that when Matlab parallelizes code, it actually serializes objects, splits up the code across multiple threads, and then hydrates the objects in each thread. Unfortunately, the Matlab database object doesn’t support serialization. Thus, there was no way to pass a connection object into some parallel code. If I choose to eliminate the parallel code, then I can use connection pooling, but everything will run synchronously in a single thread – eliminating the need to even use multiple connections, and much of the performance boost I’ve seen so far.

I also learnt along the way that a Matlab database object isn’t the same as the underlying JDBC database object. Matlab encapsulates the JDBC equivalent in some custom structures. Once I realized this, I thought that maybe I could get around this issue by using JDBC objects directly rather than their Matlab equivalents. However, the JDBC connection objects also don’t support serialization, bringing me back to a dead end.

So, I’m left with a frustrating situation:

  1. The only way to use parallelization is to have a connection pool pass a connection into the parallel code. Having multiple connection pools (one for each thread) is silly and is no better than where I am now. So, in order to have a single pool, but have the parallel code use the connections, the connection must be passed to the parallel code, which requires serialization and is not supported by JDBC connections (and thus Matlab connections).
  2. I could skip parallelization altogether and run everything single threaded: this’ll likely take me back to 20+ hours!
  3. Leave things they way they are: the parallel code creates a connection, works with the data, and disconnects. This totals a 3 hour runtime, but seems to be the lesser of all evils.


  1. Rebuild an environment that uses some kind of application server that provides the connection pooler. I’m not even sure what this would look like, how Matlab would play in the environment, and the necessary support tasks involved in maintaining the environment. This is probably the least attractive alternative.
  2. Use .NET connections in Matlab since .NET provides connection pooling natively when using SQL server. This sounds like a great option, but a .NET connection isn’t a JDBC connection and so I can’t use the already existing Matlab database infrastructure. I’ll have to rewrite all database operations like reading, updating, etc.
  3. Scrap Matlab altogether and switch to a (real) programming language like Java or C# (most like the latter in my case). This will allow me to use connection pooling without having to do a bunch of custom Matlab database connectivity work. Of course, I’m not sure how this will compare in terms of execution speed for large array operations (which Matlab is notoriously good at).

Conclusion: for now I think I’ll leave things as they are. I have “bigger fish to fry” than the details of the simulator implementation. Besides, I’m not sure how much all of the work I put into alternatives 2 or 3 will save me in terms of run time. If I go from 3 hours to 2.5, the exercise seems pointless. If I can get from 3 hours to under an hour (which is what I think would happen), it may make sense. Either way, this’ll have to wait until I have nothing else to do and feel like rewriting all my infrastructure.

End: rant.

Portfolio Page

October 5th, 2009 No comments

I’ve finally got the portfolio page up! On this page you’ll find all kinds of information regarding the quantitative strategy I’ve started investing in, in summer 2009.

To give you an idea of the kinds of things you’ll find there (over time – I’ve got a lot of work to do here!):

  • My to-do list: explore long-short rather than long only, think about a portable alpha structure with a beta overlay, explore leverage using options, look into sector-allocation and how that affects risk, …
  • Continue to dig into historical results: do the numbers make sense? Are there any biases? What could be missed?
  • Continue to dig into the model itself: do the factors make sense individually? Does the method of combination make sense? How are the factors and the model results affected by data availability?
  • Think about risk: what are the appropriate risk measures to consider? How can these risks be mitigated or controlled?

I’ll be updating the portfolio page monthly with realized returns, and will address some of the issues above over time. Every now and then I’ll post a “portfolio update” entry to keep you current on new information regarding the portfolio. If you have comments or questions, please let me know!

Categories: Equity Portfolio Tags: ,

Long, short, or put?

August 12th, 2009 No comments

As you know, I recently started investing in a quantitative equity portfolio. I’m a big fan of long-short strategies (on paper at least!). For folks that aren’t familiar with the term, it refers to strategies where investors borrow stocks they think will do poorly and sell them in the market. They then use these funds to buy stocks they think will do well. Later they sell the stock that hopefully did well to realize a profit, and then buy back the stock that hopefully did poorly at a discount to what they got for it when they sold it, and finally repay the stock loan. For example, suppose both Stock A and Stock B are currently priced at $100. It’s my belief that Stock A will do very well over the coming month and that Stock B will get decimated. In this case I short Stock B – that is, I borrow it and sell it in the market for $100. I then use that $100 to buy Stock A. Note, there is no investment of my own money at this point (ignoring transaction costs, that is). Now, suppose that one month later Stock A is trading at $110 and Stock B has fallen to $25. In this example, I was correct about both stocks. In this case I would sell Stock A for $110 (realizing a profit of $10, since it cost me $100 to buy), and I would purchase Stock B for $25. At this point my net cash position is the $110 I got from selling Stock A less the $25 it cost me to buy back Stock B, for a total of $85. I now return Stock B to the person I borrowed it from, and the transaction is done (again, ignoring interest cost for borrowing Stock B). I invested $0 of my money in the strategy and walked away with $85! Note that I could also have just invested $100 in Stock A since I thought it would go up. In this case I would have made $10. The short transaction in my example simply augments my income if I am correct.

As a side note, note what happens if I’m wrong. What if I though Stock A would tank and Stock B would rise. In this case I’d short Stock A and buy Stock B. One month later I’d sell Stock B for $25 (for a loss of $75, since it cost $100), and I’d buy back Stock A for $110 (for a loss of $10, since I got $100 when I sold it) and repay the stock loan. My net cash position is $25 from selling Stock B less $110 for buying Stock A, or a total loss of $85. I invested nothing and lost $85!

Back to the portfolio… I had intended for it to be a long-short portfolio where I’d buy the names the model thinks are the best, and sell those it thinks are the worst. After looking through some recent data I realized that I could lose as much as 10% of my portfolio in two weeks doing this (if the short stocks take off), and so I decided against that. However, I really wanted to incorporate the model’s views about poor stocks into the trades. One way to do this is to buy put options on the poor stocks. Again, for those not familiar, a put option is basically an option to sell a stock at a pre-determined price. If Stock B is trading at $100 now and I think it’ll be at $50 in a month, I can buy a put option struck at $100. That means that I now have the right to sell the stock at $100 at some time in the future. Now, a month later if the stock does go to $50, I can but it in the open market for $50 and then exercise my option to sell it to the person who sold me the option at the strike price of $100. This would result in a profit of $50. Note, the option is not free, I would have had to pay for it when I bought it. The price depends on many factors, but would have certainly been much less than the $50 profit. On the flip side, if Stock B rallied, I simply choose not to exercise the option. I don’t make any money, and my only loss is the premium I paid to buy the option. You see how this can be more attractive than naked shorting – when stocks really take off, my losses using a put option are limited to the premium whereas my losses from shorting are unlimited.

In case of my portfolio the options would allow me to bet on the downside. The problem here is that a single option contract is for 100 stocks. That is, they are sold in bundles of 100. So, if it costs $5 for a single stock option, I’m forced into buying 100 for $500. This is an issue for two reasons: 1) I need a lot more money up front to buy the options, and 2) if the stocks take off, I have a lot more premium to lose. In my case the total premium for a single contract on each of the stocks I wanted to short exceeds the total amount of my portfolio, so it’s just not doable. Yes, it is true that if I’m right and the stocks in question tank I’ll make a lot more money, but the risk is just too high right now.

So, it seems as if I’ll have to forget about the short side of the equation for now and just place my bets on stocks I think will go up. However, it does leave us with some food for thought: if options limit losses in case the stocks take off, why would I ever short?

The answer lies in the cash. When you short a stock, you get money equal to the value of the shorted stock, and your payoff is inversely related to the stock (for each dollar the stock makes you lose a dollar and for each dollar the stock loses, you make a dollar). With options you still get an inversely related payoff, though it’s generally less than one (that is, you won’t make a dollar for each dollar the stock loses. Rather, your payoff will depend on the delta of the option, a detail I’ll save for a later post). The other key difference is that when shorting a stock you get money, but when buying a put option you pay money (the option premium).

In a long-short portfolio of $10,000, you short $10,000 worth of stock and use that money to buy $10,000 worth of stock, leaving you with $10,000 of cash. You can then either keep this cash and earn interest on it, or you can invest it in a benchmark like the S&P 500, or do whatever else you want with it (a sort of portable alpha strategy). The nice part here is that if you’re wrong about your calls on the long side or the short side, you still have the investment in the benchmark to diversify away some risk (assuming you used the extra $10,000 to but the benchmark). However, if you had used options, you would use part of your $10,000 to buy options and the balance to buy the long stocks. Here you have two negative effects: 1) your total investment is less than the long-short (since you’re long fewer assets and you have less short exposure, ignoring contract size for a moment), and 2) you have lost the diversification benefit of owning the benchmark. Your entire return is based on your long and short calls, which may or may not be correct. So, now you see the cost of going with the put option – yes, your loss is limited if the “short” stocks take off, but the price of that is a smaller position and less diversification.

Categories: Equity Portfolio Tags: ,

Kicking off a new strategy

August 12th, 2009 No comments

After much delay, I’m finally kicking off a new stock investment strategy. Before you ask, no, I won’t divulge the details of the strategy. Suffice it to say that it is a quantitative equity strategy which looks at valuation, momentum, and sentiment/risk. Though I won’t spell out the factors here (hey, a magician never gives away his secrets!), I will be tracking returns on a monthly basis and posting them on the Y-Factor (link to follow). Shortly I’ll also include simulated returns for the strategy for some period in the past.

Actually investing money in this strategy (as opposed to simulation and paper-trading) is pretty nerve-racking – there are real gains and losses to be had now. I’ve also come across a few challenges, which you may find of interest:

  1. Unlike at work, capital is much more constrained and transaction costs are much higher. Building a strategy for work is great because we choose our universe (let’s say the stocks in the S&P 500), apply our strategy and buy the top decile (50 names). If we were a hedge fund, we could also short the bottom decile. A couple of things make this easy: 1) when you have millions to invest, it’s very easy to put on your positions (i.e. if I only have $10,000 to invest and Google is to be 1% of my portfolio, or $100, I’m in trouble because the stock costs $450, meaning the smallest position I can take is 4.5%. However, if you have $100 million to invest, you can easily but $1 million worth of Google), and 2) transaction costs at work are a much smaller percentage of capital. Even though I use a very-low cost broker, I’ll still pay a minimum of $1/trade. If you buy the top decile and short the bottom decile, that’s a minimum of $100 in commission, or 1% of a $10,000 portfolio. Paying that every month (since the strategy will rebalance positions monthly) is very costly – 12% a year! To alleviate this problem, I’ll go long the top 25 names and short the bottom 25 names. This will reduce my diversification, but will also reduce transaction costs and allow me more money per name to put on the positions I want.
  2. The had intended the strategy to be a long-short strategy. However, as I think about this more, taking short positions is very risky. For example, over the last two weeks the long positions in my strategy would have returned 2.6% but the short positions would have cost me 10.5%! This is primarily due to some of the stocks the model shorts being financials, which have rallied over the past two weeks (AIG has doubled!!). To get away from the downside risk to shorting, I could buy put options, what is what I was considering. The problem here is that an equity option contract is 100 in size. That means that I have to buy options in multiples of 100. So, if I want to short 10 shares of a stock, I can’t – I have to take on an exposure similar to short 100. This in itself is not bad because if the stock tanks I’ll make 10 times the money. But, the cost of the option is the limiting factor – the cost of the options for my portfolio would be greater than the equity I have! If I had $100,000 or more to invest, this wouldn’t be an issue, but that, unfortunately, if not my case.

So, after considering these annoyances, I’ve made a few last minute adjustments. I’ll go long only (since I can’t afford the options and don’t want the risk of the shorts), and I’ll buy the top 25 names my model chooses rather than the top 50 (as mentioned earlier). If things work well and I find that I am willing to put more money into the strategy, then I can always try to capitalize on the shorts later.

That’s a lot to digest for now. Keep watching and I’ll put up the simulated returns (soon) and actual returns (at the end of each month). I’ll also write more on the actual model and considerations that went into that as we move forward…

Categories: Equity Portfolio Tags: ,

The importance of the benchmark

February 24th, 2009 No comments

Business school classes and other quantitative equity readings spend a lot of time on model construction (the theoretical underpinnings of factor models, how factor models are created, the statistical issues to watch out for) and portfolio construction (mean-variance theory, efficient frontier, optimal portfolio), but hardly ever touch on benchmark selection and it’s importance in equity portfolio management. So, I’m dedicating this post to the under-appreciated benchmark…

(If you haven’t done so, you may want to read my post on Absorelative performance to better understand the performance context of this post, and to fully appreciate the effect of a bad benchmark.)

Imagine this, you’re going into the fourth quarter with an outperforming portfolio – you’ve racked up three times your excess return target for the portfolio (think “Cha-ching! There will be a great bonus this near!”). To lock in your gain, you decide to bring down your tracking error (that is, reduce the differences between your portfolio and your benchmark so that on a relative basis you neither make nor lose money going forward). You do the math and realize that in order for you to fall below the excess return target, your portfolio would have to lost a substantial percentage relative to the benchmark – an event you doubt will happen. You cross your fingers and hope to coast for the rest of the year, or maybe take a nice long vacation. I hear Monaco is nice…

Fast-forward three months: it’s the last week of December, and you’re looking at the (almost) year-end performance numbers. Your portfolio, which had earned three times the excess return target (that is, three time the amount that you were supposed to beat the benchmark by) by June now trails the benchmark by 50 basis points (one-half of a percent). That “Cha ching” you had heard a quarter earlier now sounds like the Lose-A-Turn sound on Wheel of Fortune (or any other bad event on your game show of choice) – there most certainly won’t be a bonus this year…

So what happened?! How did this great story of fortune turn into a big bust? We were entering the fourth quarter of the championship game, up by three touchdowns, and ended up losing the game by a field goal. What happened?!

What happened is that we were playing a running defense in the fourth quarter while the other team was playing a passing offense. In fact, we’d been doing the same thing during the whole game, but had just gotten lucky during the first three quarters. Heck, it was working, so why change it?

This mistake is rarely seen in football, but unfortunately is much more prevalent in portfolio management. It often happens that you start the year with a performance objective relative to some benchmark – let’s say the S&P 500 (a portfolio of the largest 500 firms in the U.S.). Now suppose that your model for selecting stocks ends up picking some firms that are in the S&P 500, but picks more firms that are smaller in size. Let’s further assume that during the first three quarters of the year, small firms on average outperform large firms. Since your portfolio is mostly small firms, it, by definition, outperforms the benchmark (which is all large firms). Finally, suppose that trend reverses in the last quarter of the year, and reverses in a big way. Since small firms perform worse than large firms in the last quarter, your portfolio under-performs the benchmark. Depending on the magnitude of the under-performance, this could cost you all of your excess return and then some (as in my example above). This is clearly a type of volatility (or tracking error to be precise) that the portfolio manager just doesn’t want!

So now you understand the source of the issue – that the benchmark is incorrectly specified – if the portfolio is comprised of smaller firms, then the benchmark should be selected to match. This way there is no upward (or downward) bias to the manager’s performance as a result of differences in firm size (or style, or any other factors). As you see in the above example, this misspecification can negate any return the manager may have earned through his skill (though in our case, the three times out-performance was a combination of skill and luck to the extent that smaller firms outperformed in the first three quarters). Thus we see that correct specification of the benchmark is just as important as alpha, or the skill of your manager.

So the issue seems easy to fix – why not just change the benchmark to match the portfolio’s attributes? There are a couple of reasons for this. If you’re benefitingfrom the misspecified benchmark, you’re not likely to have the benchmark changed since you’ll lose any benefit you may be enjoying (both from a financial bonus perspective, and from a track record perspective). If you’re disadvantaged by the misspecified benchmark, you may request it be changed, but the request is not likely to be approved. Imagine your job was to approve these requests, and you see a request on your desk. The manager, who happens to be under-performing his benchmark, wants you to change his benchmark to something that will result in his performance looking better. An immediate skepticism sets in - is this manager trying to game the system by having us choose a benchmark that favors him more? This skepticism is likely to result in a rejected request (rightfully so too, since I imagine there are many managers out there who would try just such a thing if they could get away with it!). So, changing the benchmark mid-year is just not likely to happen.

The only solution may be to finish the year and take whatever gain or loss that may come with the misspecified benchmark, with the goal of fixing the issue before the start of the next year (so that “gaming the system” becomes irrelevant). Even better, rather than hastily just picking a benchmark, the manager should fully understand his strategy and model, the stocks it typically picks, and the manager’s investment universe. After carefully considering these things, an appropriate benchmark should be selected. This prudence can alleviate a lot of subsequent problems, and results in more accurate performance measurement and compensation – something that we’ll all be better off for in the long run. So next time you’re chasing that alpha and building fascinating models, don’t forget to take a step back and give that under-appreciated benchmark some consideration.