Archive for the ‘Work’ Category

Matlab woes…

August 11th, 2010 No comments

WARNING: The following rant is somewhat technical. It’s a departure from the economics/finance of quantitative equity into the actual technical implementation and the challenges that are often faced. If you don’t know what threading, serialization, and parallel computing is you’ll probably want to stop reading right here!

I use Matlab to analyze the gigabytes of equities data I have to put on trades. Last week I’ve run into a frustrating brick wall with Matlab that I can’t seem to get around. Hopefully this helps someone else in a similar situation.

Until a couple of weeks ago, it would take 8 hours for my program to simulate 20 years of trading activity and spit out results. Clearly this is annoying since if I want to change a quantitative factor or two and see how things would look, I have to let it run for 8 hours! At this point, I was fully utilizing Matlab’s parallel computing features (which actually helped considerably – without them I would have seen a run time of 20+ hours!).

I had the idea of implementing some kind of caching scheme to speed up the data retrieval from a heavily normalized database. That took a bit of programming to implement, but is now working like a charm. Run time was reduced from 8 hours to about 3 hours. That’s a great improvement, but if we could get faster still, that’d be great.

To understand where the bottleneck was, I killed of one factor at a time and measured how runtime was affected. In the end, I found that killing all factors (so for each month my program essentially connects to the database and disconnects), made an inconsequential difference. That is, the majority of the time spent is in the database connect/disconnect (something I, as a former programmer, should have known quite well!).

So, the solution: reuse the database connections (in the technical world, known as connection pooling). So, I spent many hours investigating Matlab’s database toolbox and its ability to pool connections. To make a long story short, Matlab’s database connectivity (build on Java JDBC drivers) doesn’t support connection pooling. I suppose if I were running it in an application server (Tomcat, something else…), I may be able to work. But, I have neither the environment, expertise, nor desire to go down that path.

So, I decided to implement my own pooling. To do this, I wrote a Java connection pool manager. That is actually a simple task since all a connection pool manager is, is a vector of connections and the logic to reuse a connection if it is idle. Having never done Matlab/Java integration before, it took me a bit to figure out how to make it all work, but in short order I was there. All set, right? No…

I changed my code to use the connection pooling rather than native Matlab database connections. However, it blew up due to the parallelization. What I didn’t realize going in is that when Matlab parallelizes code, it actually serializes objects, splits up the code across multiple threads, and then hydrates the objects in each thread. Unfortunately, the Matlab database object doesn’t support serialization. Thus, there was no way to pass a connection object into some parallel code. If I choose to eliminate the parallel code, then I can use connection pooling, but everything will run synchronously in a single thread – eliminating the need to even use multiple connections, and much of the performance boost I’ve seen so far.

I also learnt along the way that a Matlab database object isn’t the same as the underlying JDBC database object. Matlab encapsulates the JDBC equivalent in some custom structures. Once I realized this, I thought that maybe I could get around this issue by using JDBC objects directly rather than their Matlab equivalents. However, the JDBC connection objects also don’t support serialization, bringing me back to a dead end.

So, I’m left with a frustrating situation:

  1. The only way to use parallelization is to have a connection pool pass a connection into the parallel code. Having multiple connection pools (one for each thread) is silly and is no better than where I am now. So, in order to have a single pool, but have the parallel code use the connections, the connection must be passed to the parallel code, which requires serialization and is not supported by JDBC connections (and thus Matlab connections).
  2. I could skip parallelization altogether and run everything single threaded: this’ll likely take me back to 20+ hours!
  3. Leave things they way they are: the parallel code creates a connection, works with the data, and disconnects. This totals a 3 hour runtime, but seems to be the lesser of all evils.


  1. Rebuild an environment that uses some kind of application server that provides the connection pooler. I’m not even sure what this would look like, how Matlab would play in the environment, and the necessary support tasks involved in maintaining the environment. This is probably the least attractive alternative.
  2. Use .NET connections in Matlab since .NET provides connection pooling natively when using SQL server. This sounds like a great option, but a .NET connection isn’t a JDBC connection and so I can’t use the already existing Matlab database infrastructure. I’ll have to rewrite all database operations like reading, updating, etc.
  3. Scrap Matlab altogether and switch to a (real) programming language like Java or C# (most like the latter in my case). This will allow me to use connection pooling without having to do a bunch of custom Matlab database connectivity work. Of course, I’m not sure how this will compare in terms of execution speed for large array operations (which Matlab is notoriously good at).

Conclusion: for now I think I’ll leave things as they are. I have “bigger fish to fry” than the details of the simulator implementation. Besides, I’m not sure how much all of the work I put into alternatives 2 or 3 will save me in terms of run time. If I go from 3 hours to 2.5, the exercise seems pointless. If I can get from 3 hours to under an hour (which is what I think would happen), it may make sense. Either way, this’ll have to wait until I have nothing else to do and feel like rewriting all my infrastructure.

End: rant.

Incredible India: A Bubble About to Pop?

February 19th, 2010 6 comments

Can you imagine being able to rewind time and watch the U.S. real estate bubble as it’s about to pop? While none of us has the ability to time-travel just yet, I did have the opportunity to witness something remarkably similar to the U.S. real estate bubble while on a recent trip to India with my family.

Just like tech stocks were the talk of the late 1990′s in the U.S. (everyone from your barber to the grocery store clerk had a winning stock pick and was a market expert), real estate is the current hot topic of conversation in India. This in itself, in my opinion, is indication that a growing bubble is afoot – when everyone from the professional investor to the gardener is talking about how to make lots of money in a particular asset class, the prudent investor should start looking for an exit.

Another factor supporting the not-so-distant decline in Indian real estate prices is the excess leverage in personal balance sheets. Just as we saw in the U.S., residents of India are now taking out mortgages that they can just about afford. The availability of credit and the shifting mindset in favor of borrowing (traditionally Indians like to pay for everything, from groceries to housing, with cash; this trend is rapidly changing with the younger, more affluent middle class opening up to mortgages and credit cards) is creating demand for housing, and pushing up prices. The opening up of credit is actually a good thing. The issue is that people can barely afford their mortgages. So, when the Indian economy slows a bit and people lose jobs, their mortgages will immediately be at risk, just as was the case here.

Excess foreign invesment also plays a significant role in Indian real estate prices. It’s a well known fact that investable assets chase returns. In this instance, NRI (Non-Resident Indian) assets that had, until recently, been invested in the U.S. and European stock markets, are migrating to India. This is natural; older Indians living abroad have accumulated a lot of wealth, which was parked in cash, gold, and financial markets. With the collapse of equity markets around the world and the drop in interest rates, there aren’t many good places to park cash. In such an environment it’s no surprise that these assets flow into housing “back home” in India. The older Indian population applies the following logic: “Should I chose to move back home once my kids are settled, I’ll have a place to live. If I choose not to move back home, I can always sell the investment in Indian real estate for a handsome profit since the market is so hot”.

The net effect of these factors is that real estate prices in India have about tripled over the last three years. This is an alarming rate of appreciation given that I see no sustainable demand for real estate. Yes, it is true that many corporations are moving to India and the economy there is growing at a brisk rate. However, the jobs created by these actions aren’t enough, in my opinion, to support the kind of national real estate price appreciation India is experiencing. Rather, I’m concerned that there are too many speculators in the market, which will result in an inevitable, and painful crash. In addition to the foreign speculators I mentioned above, there are also the domestic type – those buying a house with the hope of selling it in the next six months to a year and turning a profit (much like what we saw in the U.S.). In one conversation I learned that about 40% of the purchasers in a new subdivision that was being built were investors; they wouldn’t actually be living in the house. 40%! That’s huge! 4 out of 10 homes will sit empty when built – either awaiting immediate sale by a domestic speculator, or awaiting an NRI family that, in a few years, may live there or, in the more likely scenario, will sell the home to turn a profit.

The risk of this speculation is clear. As soon as the Indian economy slows and those young, affluent borrowers face difficulty paying the mortgages they can barely afford now, there’ll be panic, fueled by memories of what happened in the U.S. This panic will be further exacerbated by the immediate dumping of homes by investors who fear the loss of capital experienced in 2008-2009, creating a snowball effect in the decline of real estate prices. Though I can’t say for certain when this will happen (if I could, I’d have made millions on it), my intuition is that it’ll be within the next 3-5 years.

Though the issue in India is certainly not identical to what happened in the U.S. (high risk mortgages aren’t as prevalent as they were in the U.S. and the secondary mortgage and derivatives market is no where near as large as it was in the U.S.), there are many alarming similarities. That being the case, the prudent investor should be mindful of the similarities and be aware and history can very well repeat itself.

Figuring out what’s next

October 19th, 2009 No comments

I’ve spent the last two weeks doing a lot of thinking; thinking about what the future holds. Without going into a lot of unnecessary detail, I’ll say that my employer notified us two weeks ago that they no longer want to actively manage equities within the asset management division. Our cash and positions were transferred to an external money manager to run a passive, index-based strategy. (In laymen terms, they decided that they didn’t want to pay us to invest in stocks anymore, and gave our money to someone else to effectively buy something that looks like the S&P 500). Needless to say, our entire department (portfolio managers, analysts, traders, and admin support) was eliminated.

The news came as quite a shock – none of us saw it coming. In hindsight it seems as if the decision was based on a few factors, one of which was management’s unfamiliarity with equity management overall as well as what our group specifically was doing. More frustrating is the fact that our quantitative portfolio was doing very well. However we were such a small piece of my employer’s equity pie (though the $200 million we managed is no small number) that we weren’t able to have as much of an impact on the overall portfolio as we would have liked. Anyway, having been through something similar much earlier in my career (Aaron, this went down much like what happened to us in summer 2001 – same surprise, same effect), I took the news better than most on the team. A lot were distraught, wondering what will happen next. I took the news in good stride and determined that rather than a door closing, this was actually the chance for new doors to open. Regardless of how upbeat I was, this was indeed sad news – for me personally, I loved what I did. For other members of the team, that same passion was combined with over 20 years of service to the same employer. Clearly, when you’re let go all of a sudden after working at the same place for 20 years, you’d be absolutely dumbfounded and wondering what to do when you get up the next morning.

I’ve spent most of the last two weeks since we were informed of our fates thinking about where to go from here. I continue to have the same perspective I had on day zero – this may actually be an opportunity to open some exciting doors. However, just like everything else in life, this is good and bad. One one hand there are many doors I could open – I’m grateful that I have choices; on the other hand, the many choices make it very difficult to decide what to do next. The most obvious option is to find another opportunity in quantitative equity management. My education coupled with background in software and experience in running a quantitative equity portfolio make me a good fit for such a position. The problem is that there aren’t very many such positions in the Chicago area right now. Another option would be to move into a quantitative equity research role – I can continue to use my skills and stay on top of the markets and quantitative equity. The clear downside here is that doing research alone means I wouldn’t be managing money. A third option is to try to do something independent – start my own fund. Though exciting, this option poses its own set of complications and questions.

So, I continue to contemplate what’s next. This decision, like that of getting married or having children, is one that obviously shouldn’t be rushed into. Swift decisions made with brief consideration lead to mediocre careers. A little more time thinking, doing homework, and strategizing can make all the difference between mediocre and something much, much more.

Categories: Life, Work Tags:

Portfolio Page

October 5th, 2009 No comments

I’ve finally got the portfolio page up! On this page you’ll find all kinds of information regarding the quantitative strategy I’ve started investing in, in summer 2009.

To give you an idea of the kinds of things you’ll find there (over time – I’ve got a lot of work to do here!):

  • My to-do list: explore long-short rather than long only, think about a portable alpha structure with a beta overlay, explore leverage using options, look into sector-allocation and how that affects risk, …
  • Continue to dig into historical results: do the numbers make sense? Are there any biases? What could be missed?
  • Continue to dig into the model itself: do the factors make sense individually? Does the method of combination make sense? How are the factors and the model results affected by data availability?
  • Think about risk: what are the appropriate risk measures to consider? How can these risks be mitigated or controlled?

I’ll be updating the portfolio page monthly with realized returns, and will address some of the issues above over time. Every now and then I’ll post a “portfolio update” entry to keep you current on new information regarding the portfolio. If you have comments or questions, please let me know!

Categories: Equity Portfolio Tags: ,

Cash for Clunkers

September 23rd, 2009 No comments

By now the Cash for Clunkers program is well behind us. Before the chaos of this financial crisis, I had never thought that the government would introduce such a program, and I’m not sure I’ll ever see another one like it in my remaining years. Given that, I’m glad that we (specifically, my parents) were able to take advantage of the program. Based on my experience, a few thoughts:

The government’s objective was the lift vehicle sales; there’s no doubt that they were successful. About 700,000 vehicles were sold under the program, resulting in a total of $2.87 billion paid out by the government. In addition to this, we should add the cost of processing the applications and any additional infrastructure needed to execute the program. The cost still comes in under the $3 billion the government had budgeted. This was a great thing for car dealers, for people who were able to take advantage of the program, and for the auto industry in general (production was ramped up and thousands of people were put back to work). However, my concern is about the short-lived nature of the program. I’m afraid that in the books of history, this will be but a blip. Now that the program is finished, we still have extremely high unemployment, we still have lack of desire for American cars, and we still have no evidence of improved American cars (in terms of quality and reliability). So, what will happen to the auto industry and all the workers who were put back to work over the next six months? I hate to say it, but it isn’t looking pretty – auto sales are down again, and I feel like it’s only a matter of time before production stops again. We need a more permanent solution to this problem…

Buying a car through the Cash for Clunkers program was quite an experience. Firstly, finding a car was an issue. Many dealer lots were empty and inventory was scarce. Dealers couldn’t really get cars from other dealers since the problem was systematic. We pulled the trigger pretty quickly – I spent a couple of days reviewing candidate cars, did my usual email blitz, and then ended up deciding where to go. We bought on the Saturday of the final weekend (the program was to end on Monday and Illinois law requires dealers to be closed on Sunday, so we effectively bought on the last day of the program). We visited two dealerships and I’d never seen dealerships so busy. Both dealerships were swarming with people – in the showrooms, on the lots, and in the tiny cubes doing paperwork. Sales people were juggling 4-5 deals at a time. Normally a sales person is asking what they can help you with within minutes of your entering a car dealership. Now you could be looking at a car or trying to get someone’s attention for over a half an hour before you finally got to talk to someone. It was taking about 5 hours to get a customer from agreeing on a price to out the door with a detailed and inspected vehicle (when normally it shouldn’t take much more than an hour or two). I asked the sales guy at the dealership what time they close and how their day yesterday went. He told me that the were supposed to close at 9pm the previous day (a Friday), but couldn’t. They had new customers walking in at 8:55, and ended up staying at the dealership until 1am. They returned to work at 5pm the following day. There was an immense buzz and excitement in the dealerships – something I’d never seen before, and unfortunately am not likely to see in the near future.

The Cash for Clunkers program created interesting distortions in the auto industry. Prior to the program dealers would do anything to get a car off the lot. In addition to rebates, the dealers were willing to pass along any marketing incentives they were given by the manufacturer, as well as part of any volume discounts they got. Basically, if they could make a couple of hundred dollars on a deal (or less!), they’d sell you the car. Heck, it was difficult to understand the quotes you’d get for a car – they’d be below the invoice less all available discounts – a price that just made no sense! As far as choice goes – you could get almost any color with any options – there was so much inventory either on the lot or with nearby dealers that you got whatever you wanted. After the program, things couldn’t be any more different. Cars were scarce – the only color available at all the Chicagoland Honda dealerships we spoke to but one was black. You’d be lucky if you could get pricing any lower than $50 below MSRP or so (I even received a couple of quotes OVER MSRP!). From an economics point of view the government incentive created a surplus for the consumer. However, much of that surplus was consumed by the dealers. As incentives disappeared and demand went up, prices went up by thousands. Of the $3,500 or $4,500 the government gave you, it wasn’t unusual for the dealership to take more than half (as a result of the higher prices). At the end of the day, this gives a double bonus to the dealers, and only a slight incentive to consumers. Dealers win because they reduce inventory and get MSRP for the vehicles sold. Consumers are told they’re getting $3,500 or $4,500, but in reality are getting much less since they paid much more for the vehicle than they would have paid before or after the program.

Another interesting dimension of the program is that the government didn’t put any restrictions on re-selling vehicles bought under Cash for Clunkers. What prevents someone from buying a new car at a steep discount and then turning around and selling it? Granted that after factoring in taxes and the dealerships’ higher prices this idea isn’t too lucrative, but depending on the situation (getting the full $4,500 and buying one of the less popular vehicles), one could make $1,000 or more doing this. Not preventing this from happening results in car prices (both new and used) falling (since there would be several new clunkers cars hitting the used market with less than 1,000 miles). Practically this may not be doable for most people since they’d lose the use of the vehicle (i.e. if I used my clunker to get to work everyday and then traded it for a new car, I can’t exactly sell my new car since I will no longer be able to get to work!), but I do find it interesting that the government didn’t limit this behavior.

Finally, one unfortunate side effect of this program is the cost to people who were in the market to buy a vehicle but don’t have a clunker to trade in. Because of the spike in demand, car prices shot up. If you had a clunker to trade, that was fine since the government rebate more than accounted for the increase in prices. However, if you didn’t, that $13,000 car you were looking at just jumped to $16,000 with no recourse for you. If you can delay your purchase until the program is over, inventory is sufficiently replenished, and demand has slowed again, you may be able to buy that car for $13,000 again. However, hopefully manufacturers learn from their mistakes and don’t create so much supply this time, keeping prices high. Even if prices to fall to the pre-clunkers level, the annoyance and inconvenience of not being able to buy for months is still significant.

I’m glad that I was able to be involved in a Cash for Clunkers transaction – it was great to see a buzz in dealerships that I’d never seen before. Getting $4,500 for our old van (which was probably worth no more than $500) didn’t hurt either. However, I’m not sure that in the long term the program will be as effective as the administration had hoped…

Categories: Economics Tags: , ,