Sunday, February 20, 2011

Market Ecology

The erudite and very readable RT Leuchtkafer has posted yet another comment for the Securities and Exchange Commission to digest. This one was prompted by a paper by Andrei Kirilenko, Albert Kyle, Mehrdad Samadi and Tugkan Tuzun that provides a fascinating glimpse into the kinds of trading strategies that are common in asset markets today and the manner in which they interact to determine the dynamics of asset prices.

As I have argued on a couple of earlier occasions, the stability of a market depends on the composition of trading strategies, which in turn evolves over time under pressure of differential performance. Since performance itself depends on market stability, and destabilizing strategies prosper most when they are rare, this process can give rise to switching regimes: the market alternates between periods of stability and instability, giving rise to empirical patterns such as fat tails and clustered volatility in asset returns.

But the underlying strategies that are at the heart of this evolutionary process are generally unobservable. Since traders have no incentive to reveal successful strategies, these can only be inferred if individual orders can be traced to specific accounts.

This is what Kirilenko and his co-authors have been able to do, on the basis of "audit-trail, transaction-level data for all regular transactions in the June 2010 E-mini S&P 500 futures contract (E-mini) during May 3-6, 2010 between 8:30 a.m. CT and 3:15 p.m. CT." While their primary concern is with the flash crash that materialized on the afternoon of the 6th, their analysis also sheds light on the composition and behavior of strategies over the period that led up to this event. Their analysis accordingly provides broader insight into the ecology of financial markets.

The authors classify accounts into six categories based on patterns exhibited in their trading behavior, such as horizon length, order size, and the willingness to accumulate significant net positions.  The categories are High Frequency Traders (HFTs), Intermediaries, Fundamental Buyers, Fundamental Sellers, Opportunistic Traders and Small Traders:
[Different] categories of traders occupy quite distinct, albeit overlapping, positions in the “ecosystem” of a liquid, fully electronic market. HFTs, while very small in number, account for a large share of total transactions and trading volume. Intermediaries leave a market footprint qualitatively similar, but smaller to that of HFTs. Opportunistic Traders at times act like Intermediaries (buying a selling around a given inventory target) and at other times act like Fundamental Traders (accumulating a directional position). Some Fundamental Traders accumulate directional positions by executing many small-size orders, while others execute a few larger-size orders. Fundamental Traders which accumulate net positions by executing just a few orders look like Small Traders, while Fundamental Traders who trade a lot resemble Opportunistic Traders. In fact, it is quite possible that in order not to be taken advantage of by the market, some Fundamental Traders deliberately pursue execution strategies that make them appear as though they are Small or Opportunistic Traders. In contrast, HFTs appear to play a very distinct role in the market and do not disguise their market activity.
Based on this taxonomy, the authors examine the manner in which the strategies vary with respect to trading volume, liquidity provision, directional exposure, and profitability. Although high-frequency traders constitute a minuscule proportion (about one-tenth of one percent) of total accounts, they are responsible for more than a third of aggregate trading volume in this market. They have extremely short trading horizons and maintain low levels of directional exposure. Under normal market conditions they are net providers of liquidity but their desire to avoid significant exposure means that they can become liquidity takers very quickly and on a large scale.

The extent to which different trading strategies provide liquidity to the market is assessed by the authors on the basis of a measure of order aggression. An order is said to be aggressive if it is marketable against a resting order in the limit order book (and is therefore executed immediately.) The resting order with which it is matched is said to be passive:
From a liquidity standpoint, a passive order (either to buy or to sell) has provided visible liquidity to the market and an aggressive order has taken liquidity from the market. Aggressiveness ratio is the ratio of aggressive trade executions to total trade executions... weighted either by the number of transactions or trading volume... HFTs and Intermediaries have aggressiveness ratios of 45.68% and 41.62%, respectively. In contrast, Fundamental Buyers and Sellers have aggressiveness ratios of 64.09% and 61.13%, respectively.
This is consistent with a view that HFTs and Intermediaries generally provide liquidity while Fundamental Traders generally take liquidity. The aggressiveness ratio of High Frequency Traders, however, is higher than what a conventional definition of passive liquidity provision would predict.
Moreover, the aggressiveness ratio of HFTs is not stable over time and can spike in times of market stress as they compete for liquidity with other market participants:
During the Flash Crash, the trading behavior of HFTs, appears to have exacerbated the downward move in prices. High Frequency Traders who initially bought contracts from Fundamental Sellers, proceeded to sell contracts and compete for liquidity with Fundamental Sellers. In addition, HFTs appeared to rapidly buy and [sell] contracts from one another many times, generating a “hot potato” effect before Opportunistic or Fundamental Buyers were attracted by the rapidly falling prices to step in and take these contracts off the market.
To my mind, the most revealing findings in the paper pertain to the profitability of the various strategies, and the ability of some traders to anticipate price movements over very short horizons (emphasis added):
High Frequency Traders effectively predict and react to price changes... [they] are consistently profitable although they never accumulate a large net position. This does not change on May 6 as they appear to have been even more successful despite the market volatility observed on that day... Intermediaries appear to be relatively less profitable than HFTs. During the Flash Crash, Intermediaries also appeared to have incurred significant losses... consistent with the notion that the relatively slower Intermediaries were unable to liquidate their position immediately, and were subsequently run over by the decrease in price...

HFTs appear to trade in the same direction as the contemporaneous price and prices of the past five seconds. In other words, they buy... if the immediate prices are rising. However, after about ten seconds, they appear to reverse the direction of their trading... possibly due to their speed advantage or superior ability to predict price changes, HFTs are able to buy right as the prices are about to increase... In marked contrast... Intermediaries buy when the prices are already falling and sell when the prices are already rising...

We consider Intermediaries and HFTs to be very short term investors. They do not hold positions over long periods of time and revert to their target inventory level quickly... HFTs very quickly reduce their inventories by submitting marketable orders. They also aggressively trade when prices are about to change. Over slightly longer time horizons, however, HFTs sometimes act as providers of liquidity. In contrast... unlike HFTs, Intermediaries provide liquidity over very short horizons and rebalance their portfolios over longer horizons.
What appears to have happened during the crash is that the fastest moving market makers with the most effective algorithms for short run price prediction were able to trade ahead of their slower and less effective brethren, imposing significant losses on the latter. In Leuchtkafer's colorful language, this was a case of interdealer panic and market maker fratricide.

But regardless of how the gains or losses were distributed in this instance, the fact remains that an overwhelming share of trading activity is based short-run price forecasts rather than fundamental research. Under these conditions, how can one expect prices to track changes in the fundamental values of the income streams to which the assets give title?

Markets have always been based on a shifting balance between information augmenting and information extracting strategies, but a computational arms race coupled with changes in institutions and regulation seem to have shifted the balance markedly towards the latter. Unless the structure of incentives is altered to favor longer holding periods, I suspect that we shall continue to see major market disruptions and spikes in volatility.

This is not just a matter of academic interest. To the extent that changes in the perceived volatility of stocks gives rise to changes in asset allocations by institutional and retail investors, there will be consequences for the extent and distribution of risk-bearing, and ultimately for rates of job creation and economic growth.


Update (2/21). Yves Smith has generously allowed me to crosspost freely on naked capitalism, where this entry has attracted a couple of interesting comments. Here is Peripheral Visionary:
With respect to May 6... the faster algorithms may have caused the damage, but I think they also suffered from it. From the data I reviewed, the traditional market makers had huge numbers of buys at the bottom and huge numbers of sells through the recovery, and so may have come out net positive on the day, while the faster algorithms panicked when the market moved outside the range of expected behavior, and many were shut down, effectively locking in losses. In fact, I suspect that losses for HFT algorithms would have been much larger had not the exchanges canceled so many trades, with many, even most, of the sells at the bottom being algorithm trades.
This was also my initial reaction to the crash, which is why I argued against the cancellation of trades on grounds of stability. The Kirilenko paper does not really settle the question because it focuses only on the E-mini futures market where no trades were broken.

The comment by financial matters is also worth a look; this one links to a CNBC interview with Jim McCaughan in which the exit of institutional and retail investors from the market is documented.

Saturday, February 12, 2011

Belief Heterogeneity

There was an interesting conference at Columbia yesterday (though not nearly as interesting as the momentous events unfolding elsewhere at the time). The theme was "Heterogeneous Expectations and Economic Stability" and this is how the organizers (Ricardo Reis and Mike Woodford) described the goal of the meeting:
Conventional models in both macroeconomics and finance are based on the hypothesis of rational expectations, under which all agents are assumed to have common expectations, corresponding to the probabilities implied by the economist’s model. The adequacy of this familiar hypothesis has been called into question by recent events, however, notably the instability resulting from the boom and bust in real estate prices. The purpose of this conference is to bring together researchers exploring alternative approaches to modeling the dynamics of expectations, with particular attention to applications in macroeconomics and finance. We have sought to bring together proponents of a variety of approaches, who may not frequently engage one another, in the hope of reaching conclusions about which directions are most promising at this time.
And, indeed, the collection of papers presented were methodologically diverse. Although any such classification is bound to be coarse and imperfect, there seem to be four different directions in which research on expectations is proceeding. First, there is the approach of near-rational expectations, in which intertemporal optimization and Bayesian rationality are maintained but allowance is made for heterogeneous prior beliefs. Then there is the behavioral approach, which endows agents with heuristics based on regularities identified in laboratory experiments. Third, there is the evolutionary approach, which allows for a broad range of competing forecasting rules with the population composition shifting over time under pressure of performance differentials. And finally, the empirical approach, which treats expectations as a state variable to be measured using survey or market data and explained just as one would explain output or inflation. Each of these perspectives was on prominent display at the conference.

Regular readers of this blog (if there are any left, given the recent decline in my rate of posting) will know that I am deeply skeptical of the behavioral approach to trading strategies, for the simple reason that behavior in high stakes environments with strong selection pressures driving entry and exit is unlikely to be psychologically typical in the sense of reflecting outcomes of lab experiments with standard subject pools. What might be a common behavioral trait in the population at large could be extremely rare among traders, especially if such traits can be exploited with ease by other market participants. By the same token, behavior that is pathological in the lab could well become widespread in financial markets from time to time. As a result my favored approach to trading strategies in general and forecasting rules in particular is ecological.

Not surprisingly, then, the presentation I found most appealing was that of Blake LeBaron. Blake is a pioneer in the development of agent-based computational models of financial markets, and the paper he presented belonged to this class. A large number of different forecasting strategies, some based on fundamental information and others on technical data analysis, compete with each other and with a traditional buy-and-hold strategy in his model. The resulting trading dynamics give rise to asset price returns that exhibit both moderate levels of short-run momentum as well as mean reversion over longer horizons. Moreover, the long run population of forecasting rules is ecologically diverse, with both passive and active strategies well represented.

During the panel discussion at the end of the conference, Albert Marcet observed that the conference itself was symptomatic of a revolution in economic thought that is currently underway, prompted in large measure by the global financial crisis. If methodologies such as agent-based computational economics start to be published in major journals and attract attention from the most promising graduate students, then there really will be a revolution underway. But I'm not convinced that we're there yet.

One final thought. The conference organizers described the rational expectations hypothesis as one "under which all agents are assumed to have common expectations, corresponding to the probabilities implied by the economist’s model." This is an accurate characterization as far as the contemporary implementation of the hypothesis is concerned, but it is important to note that this is not the hypothesis originally advanced by John Muth in his classic paper. In fact, Muth cited survey data exhibiting "considerable cross-sectional differences of opinion" and was quite explicit in stating that his hypothesis "does not assert... that predictions of entrepreneurs are perfect or that their expectations are all the same.'' In Muth's version of rational expectations, each individual holds beliefs that are model inconsistent, although the distribution of these diverse beliefs is unbiased relative to the data generated by the actions resulting from these expectations. It is a wisdom of crowds argument, rather than one based on individual rationality.

Viewed in this manner, there a sense in which the heterogeneous prior models (with diverse beliefs centered on a model consistent mean) represent both a departure from the rational expectations hypothesis as currently understood, as well as a return to the original rational expectations hypothesis as formulated by Muth. The history of economic thought is full of such rather strange twists and turns.