Guest Post: Review of Pablo Triana’s "Lecturing Birds on Flying"

Posted on by

Submitted by Richard Smith:

This is the Black Swan gospel according to Triana. Taleb endorses it in a characteristically incendiary and intemperate foreword. He does come out all guns blazing, and you just have to go with that. Or chuck a glass of water over him, if he’s in range, I suppose.

A quick recap for anyone who has spent the last two years in a coma: Taleb put together the beginnings of a rap sheet for modern mathematical finance theory in his book “The Black Swan”, and rapidly attained worldwide celebrity when his criticisms appeared to be borne out by the recent financial crisis. The main tenet of Black Swan theory, rather dry sounding, but with dramatic consequences, is that price changes are not normally distributed (in the way that, say, human weight or height are), but follow a power law (‘fat tails’). This implies much greater extremes of price movement than those predicted under the assumption of a normal distribution. The events that cause such price moves may be perfectly intelligible in hindsight, but are not necessarily predictable: like the existence of black swans.

The point about price distributions is actually quite an old one. Paul Levy made the same observation in the 1900s; Mandelbrot’s studies of cotton prices, in the 60s, reached similar conclusions. What gives it contemporary relevance is that the finance theory underlying current regulatory practice, risk management, fund management and derivatives pricing all overwhelmingly assume that price changes are normally distributed. And they all failed at once in the recent financial crisis, when price changes were indeed far more extreme than a normal distribution implies. It doesn’t look so good for orthodox financial theory just now.

So it is a good time for Triana to review modern finance theory’s rap sheet, add items, and add more detail to the existing charges. It goes like this.

Chapter 2: modern finance theory is a crock, peddled by charlatans at business schools who have managed to seal themselves off from the usual empirical tests of a theory.

I’ll admit I don’t see what logical point there is in attacking the character of business school teachers in this manner, whether it is a correct assessment or not. However the empirical criticism really does stack up. Consider GS CFO David Viniar’s notorious comments from August 2007 when the ABS meltdown got into full swing (Ch1, p12): “We were seeing things that were 25-standard-deviation moves, several days in a row”. To which the rejoinder from an empirically-minded observer simply has to be “No you weren’t, imbecile: those observations actually mean that your models are hopelessly wrong”. There are several reasons why one can so insouciantly cheek such an august figure. If we assume Viniar means daily observations and a normal distribution, then (if the numbers I am cribbing are correct: I haven’t gone back to the equations) one should expect to wait quite a lot longer than the age of the universe to see even a 16-standard deviation event, with a 25-standard deviation event taking many, many times longer than that. I suppose I should work out the exact number of years, just to see how big of a number it is: exercise for any readers with access to an arbitrary-precision mathematical engine.

You can find an old post by Yves on the subject that helped kick off some blogosphere chat.

Even if you assume (very charitably, I grimly suspect) that Viniar is not just parroting his VaR model outputs (more on that later), and is a bit more sophisticated about his distributions, he is still . And if Mandelbrot, and Taleb, his follower, and Triana, his follower, are right about the kind of distribution that underlies financial market price movements, there just ain’t sech a thing as a standard deviation of price movements, nor no correlation neither. Both standard deviation and correlation are defined in terms of variance. Since variance is infinite for stable distributions (other than the normal distribution), neither standard deviation nor correlation is defined for the distribution of market prices (a Levy skew alpha stable distribution, if you want the full geeky glory). On this theory, Viniar is talking about things that just don’t exist. Not encouraging behaviour in a CFO.

So here is the bleedin’ obvious: given its track record of ultra-wild underestimates of the frequency of sharp price moves, the assumption of normal distributions in stock price changes must be among the most lavishly disconfirmed scientific hypotheses of all time. No wonder, then, that Taleb and Triana are somewhat ratty with its various obstinately blithe proponents.

Chapter 3: Is a bit of a digression. According to Triana, the quants who work at banks work mostly on bits of IT dealing infrastructure, which is useful, and less often than you might think on mathematical models used in trading. The quants tend to be physicists and engineers rather than business school graduates. Models are used in a much more sceptical, provisional way on the trading floor than they are in academia.

I’ll take his word for it. Evidently, scepticism of models doesn’t extend to the risk management department. And, uh, actually it doesn’t look as if that trading floor scepticism managed to avert 2007’s monster trading screwups, either. Except, perhaps in the case of GS, who famously hedged a lot of MBS exposure starting late in 2006, to the great indignation of folk who don’t understand where fiduciary duties stop and start for broker-dealers.

Now we get into the meaty detail chapters. The non-normalness of price distributions means that a whole bunch of financial orthodoxies are dubious on theoretical grounds, and, post meltdown, there are some nasty data points to back up the theory.

First up is the Gaussian copula (Chapter 4). This is a modelling device which was used to calculate default correlations, for MBS and other bonds, and thus to structure, price, rate, and hedge CDOs. I think we already know how well that went overall– but the detail of how the behaviour of various tranches of CDOs diverged from predicted paths during the ’07 meltdown is instructive. Triana leaves open the question of whether the Gaussian copula was adopted out of blind faith in its efficacy, or precisely because it underrated extreme events, and thus gave an excuse for assigning a high rating, and getting a good price. Were the ratings agencies knaves or fools in this respect? I doubt we’ll find out any time soon. Anyhow, from the data and testimony Triana assembles, it looks as if the Gaussian copula is dead in the water as a structured finance tool. One wonders how Remics and re-Remics are to be priced and rated. Any NC readers want to buy one?

Chapter 5: Now we are into VaR, the risk management methodology that JP Morgan gave to the world back in the early 90’s, in the sadly mistaken belief that being able to generate a firm wide “risk number” daily would be a useful contribution to financial risk management. Back then you were reasonably smug about your bank if central management actually knew what the firm’s positions were at all (vide Barings, Sumitomo, then fast forward again to SocGen in – oh dear – 2007), so VaR was pretty cool. It was later endorsed by the Basel regulatory framework. Then the paint started to flake off.

The shortcomings of VaR have been a regular topic at NC. That pesky normal distribution assumption again. Note the reminder from practitioner Irene in the discussion thread though – the officially sanctioned VaR model may use a rolling 2 year price history rather than a normal distribution. This desperate kludge has its own perverse side effects: in times of increased volatility, the models all tell banks to stay on the sidelines at the same time. Once the volatile part of the price history rolls out, the models are all happy again. This is not a commonsense way to run banking businesses.

The other perversity of that approach to VaR is that it encourages herd behaviour in volatile markets, before the banks have even made it to the sidelines. In other words, since all the models in all the banks are essentially the same model of the same data, they all start screaming ‘fire’ at the same time, with predictable consequences at the exits. All this and more is well covered by Triana: particularly the way that a long period of low volatility before 2007 meant that VaR endorsed massive positions in assets that were suddenly big loss makers, when things went sour.

Banks were Gadarene enough without VaR. VaR makes it worse.

Oh, one thing that bugs me about VaR as used is this: if price histories tell you nothing about future prices (EMH), why is it that price volatility histories tell you something about future price volatility (VaR)? I’m just asking.

Anyhow, Triana makes the challenging points, with persuasive evidence: first, VaR is perfectly useless (it works until you need it, and at that point, it packs up: it is the chocolate teapot of risk management); second, like MTM, it is actively procyclical.

Chapter 6 is a brisk injunction to business schools (specifically, Sloane) to snap out of it and start teaching useful stuff.

In Chapter 7, we get to another polemic, against the Black-Scholes option pricing model. One can’t fault the reasoning or evidence, but somehow this is the weakest part of the meat. It is built around a recent paper by Taleb and Haug in which they review the historical record on options market making and option pricing theory and announce that a) the parts of Black-Scholes theory that are correct are not original, having been long anticipated by Thorp-Bachelier option pricing b) the parts that are original are not correct (normal distributions are again assumed, and the model simply can’t accommodate non-normal ones, unlike Thorp-Bachelier) c) no practitioners actually use Black-Scholes. The key item of evidence for (c) is the ‘volatility smile’ by which options traders systematically adjust option prices, so that the implied volatility (calculated according to Black Scholes methods) of options actually increases progressively for deeper and deeper out-of-the-money options. Under Black-Scholes pricing theory the implied volatility should be constant across all option strike prices. Traders don’t do it that way: they are compensating for the way the BS model fails to accommodate fat tails. QED. And by the way, Triana adds, there’s no such thing as implied volatility anyway, just supply and demand pushing prices around.

Well, OK to all that, so call implied volatility “demand premium” or something, and concede that Black-Scholes is a roundabout way to prices that can be reached more directly under other theories. So now what? Black-Scholes has become part of banking’s infrastructure. Do we strip out all the Black Scholes models and replace them with Thorp-Bachelier models? Will it make enough of a difference to options pricing or risk management to be worth it? Triana doesn’t try to determine the ROI. Instead (in the Finale) he asks whether Merton and Scholes should be stripped of their Economic prizes, and eventually concludes that instead the RiksBank prize should be given a silly name, so that people know it is a bit of a crock. It is an amazingly lightweight way to round off an otherwise enlightening discussion. It doesn’t come off like a joke fallen flat either: just cheesy.

While I’m carping, I’ll add a comment on the style. When I started reading the book, I kept stumbling over awesome quasi-English barbarisms, such as “qualification-inundated resumes”, “dangerously faulty mathematically charged steering”; also horrific neologisms like “analyticization”, “nonenthusiastically”, “impacting” (adj., I kid you not, and repeatedly), and “scientification aroma” (my favourite – I want some – either in a spray dispenser or roll-on form, not fussy). To my relief Triana (or his copy editor) gets more of a grip in later chapters and it’s not such a terrible read in the end. Doubtless the same relief is reflected in the generous verdicts of Taleb (“lucid”), Tett (“readable”) and Skypala (“a treat”). So, do not despair if you find yourself entangled in some pretty strange thickets of verbiage early in the piece: it does get better if you plough on.

Back to Chapters 8, 9 and 10 so that we can end on a note more favourable to the book (just skip the Finale).

Chapter 8 is a good one on the way models can be used as alibis or excuses by the lazy, reckless, or incompetent. Good reading for head traders, risk managers and regulators, I’d say; and buy-siders and pension fund trustees, come to that. Chapter 9 is a quick round up of how seductive the spurious certainty of mathematical models can be, largely illustrated by LTCM and by the confusion surrounding the meaning of the VIX.

In the end, the message of the book is that quantitative finance is a delusion, and that common sense is a better starting point for risk management. Accordingly Chapter 10 is a paean to Fat Tony, the street smart invention of Taleb in “The Black Swan”, and a call to reverse the quantification of finance. The negative leg of the case is argued persuasively. It is discomfiting to recognise just how little there was to quantitative finance.

On the positive side of Traiana’s recommendation: well, you are welcome to make your own mind up about the reserves of common sense to be found in the banking industry just now.

Print Friendly, PDF & Email

15 comments

  1. skippy

    @Mr. Smith….Well put and throughly injoyed.

    But what does all this say about us and our activitys, just a tale of a crooked man with a crooked cane walking down a crooked road?

    @Yves…I for one see potential in this review format.

    Skippy…going to have a go at this enchilada tomorrow.

  2. Brick

    Having had arguments that climate models are just atmospheric models and don’t include the carbon cycle perhaps there is a lesson to be learnt by economists from climate scientists. Namely that you should use multiple models run multiple times with different parameters and slowly combine models to build a big picture. My main concern with risk models is that they don’t look at the big picture and are not regularly reviewed for effectiveness. When I see risk modelling which accurately predicts the events that unfolded over the last year I will have some confidence that we are on the mend. One problem will be incentives at banks who are quite rightly motivated towards profit rather than the health of the general economy . I think we are beginning to see this rearing its ugly head again as wider risks to the economy are not properly assessed in the need to repair balance sheets. Hedging everything as risk management is not quite the same as mitigating risks to the economy.

  3. Andrew Foland

    Thanks for the word "Gadarene"!

    Your exposition of the math was quite clear.

    Personally, I've always suspected that most of the model makers understood that their various gaussian models were inadequate, but that most of them were thinking, "how wrong can it be?"

    I also bet the "same models on the same data" aspects weren't given the attention they deserved in the answering of that question.

    Also, to be honest, I've trained physicists (in physics) for years, and even at the best schools, only the truly best of the best understand what it looks like, in detail, when you sample from a distribution of infinite variance. (The merely good ones would have a vague sense of unease about sampling from such a distribution; the bad ones probably think such a distribution can't exist.) I know that Wall Street hired many more of my students than the number who understood it, and feel confident I'm not alone in that.

  4. Blissex

    «The main tenet of Black Swan theory, rather dry sounding, but with dramatic consequences, is that price changes are not normally distributed (in the way that, say, human weight or height are), but follow a power law (‘fat tails’).»

    That to me seems a big fatal misunderstanding of what Taleb is saying, it mistakes a tree for the forest.

    The main message is that we have little positive knowledge about complex systems, and overestimating such knowledge is very bad for trading and policy alike.

    The fat-tail story is not at all Taleb's main message — it is just an example, even if a very important one, of the general principle above.

  5. Richard Kline

    Common sense, the ol' analog modeling device: if it's hot, don't pick it up. Hmmm. . . . There's a thought.

  6. lambert_strether_openid

    I dated a model once.

    But I'm not sure if they were normally distributed.

  7. Ben Ross

    Hate to nitpick, but plenty of distributions with fatter tails than the normal distribution have a variance. Basically, any distribution whose tail falls off faster than 1/z^3 has a finite variance.

    You refer to a theorem for "stable" distributions. Stable distributions are not something I've ever worked with, but I see that they are limits that arise when you sum a lot of independent random processes. Surely, in the processes that led to the current financial crisis, the behavior of market participants was highly correlated. So I suspect that it would be a mistake to assume that price distributions must be "stable".

    I've never read Mandelbrot on finance, but a long time ago I studied his fractals book very closely. His basic message was to avoid tempting assumptions about probability distributions – I understood the distributions he applied to data as counterexamples to the normal distribution, rather than replacements for it.

  8. Richard Smith

    Blissex & lambert – the idea of Black Swan certainly has applications outside finance, and I definitely glossed that over in the above. But as I understand it, the Black Swan symbolizes not so much the intractability of complex systems, but rather, human blindness to the frequency of supposedly rare & improbable events…I'd point you to the quotes but inconveniently my copy of the damn book happens to be 60 miles away!

    Ben Ross – you are right, there is the fact that a distribution has fat tails doesn't of itself imply that it has infinite variance too. But Mandelbrot really does mean infinite variance. You can find a summary of his arguments & some tech references for infinite variance in "The Fractal Geometry of Nature" Chap 37(which I hope is still in print…).

    Incidentally the normal distribution actually is a stable distribution, but a rather special one with some obliging mathematical characteristics. There is a handy little summary here which I've just noticed has a sentence or two on Mandelbrot.

    I'll have a bit of a chew on your point about market participants.

  9. Richard Smith

    Oh and thx to Andrew Foland and ʎddıʞs, (I'm very proud of my Antipodean typing).

  10. step314

    Suppose it is true that recent events suggest that sense, logic, etc., are more important than statistical results. Probably a clear majority of mathematicians would agree that proofs are more important than theorems. So how does one best improve one's logical reasoning? A huge help is to learn the most elegant and fundamental parts of mathematics with logical rigor. Since the mathematics of the normal distribution is elegant and fundamental, while the mathematics of ad-hoc statistical distributions is less so, it seems to me that by concentrating on normal distributions and a few other elegantly behaved ones, one can better teach thinking skills than otherwise. Personally, I once was thinking about becoming an actuary, but when I got to time series analysis, the subject was such a motley collection of ad hoc inelegant barbarisms, I quit (perhaps I couldn't have forced my brain to do it even if I wanted to).

    Not that one shouldn't try to teach things in the right generality. Results that don't have to do with distributions being normal should be taught in appropriate generality.

    I feel there are (at least) two different ways of discovering mathematical truth. One is to choose a result one wants and that one thinks one can prove and start proving like crazy anything one thinks might help give the result one wants, until one has shot what one is aiming at. Another is to understand mathematics one finds beautiful very carefully, and to always be on the lookout for an impression that these arguments can be combined and extended, which impressions, being mostly just impressions of similarity between previous arguments one has worked through and present possibilities, rely heavily on past experiences of rational thoughts and the logical patterns underpinning the latter. Counterexamples play different roles depending on the method chosen.

    In aim-and-shoot mathematics, examples and counterexamples are essential. One doesn't want to aim to prove something that is wrong, and so knowing counterexamples is a great help in determining what exactly to aim for, and what generality one should choose in aiming for it. (But Godel's incompleteness theorem says that in any consistent mathematical theory strong enough for arithmetic, there will always be statements neither provable nor disprovable, so one shouldn't be too stubborn if adopting this approach, or one can waste one's life trying to prove an unprovable result that has no counterexample.)

    Examples and counterexamples are much less useful otherwise. Examples are interesting if they are interesting in their own right (if the mathematics dealing with them is useful and elegant). And counterexamples are somewhat useful quite generally when one makes mistakes in logical arguments. When one makes a logical mistake (as is inevitable), it sometimes takes a while before one proves something ridiculous (at which point one realizes one has made a mistake), and this time can be reduced by having a supply of counterexamples.

    Anway, though I have not read the book, I am skeptical of mathematicians that say what is needed is more knowledge of arcane statistical objects and counterexamples, since it's aim-and-shoot mathematicians who find these sorts of things most useful, and an aim-and-shoot approach to mathematics is not the best sort for teaching one how to think, which knowing how to think in the long-run probably actually gives better results than specific training.

  11. Hugh

    My view is that the modelers thought they could model the performance of a system based on its past performance without any practical knowledge of the system, even if it were a system, or how it had changed structurally over time.

    "And by the way, Triana adds, there’s no such thing as implied volatility anyway, just supply and demand pushing prices around."

    I find supply and demand arguments way too simplistic. As we have seen repeatedly in the last several years, speculation can push prices around with little or no reference to supply and demand. Look at the oil bubble last year and this year, for example.

  12. Doc Merlin

    The power law point is a very good one, I don't think i've actually ever seen a gaussian distribution in good economic data. Its all power laws and kappa distributions. Really the only time you see a lot of gaussian distributions is when the "particles" in your system don't really interact, once they start interacting everything goes power law or kappa, although in really systems you can approximate a gaussian.

  13. skippy

    [email protected] Smith said… ʎddıʞs, (I'm very proud of my Antipodean typing).

    The Antipodeans were a group of Australian modern artists who asserted the importance of figurative art, and protested against abstract … (in my case the truth) and polar opposite.

    Skippy…thanks for the chuckle…BTW I can't look up, bloody NH skirts and all.

  14. NotZed

    I know this is somewhat off-topic, but I wish you guys would stop using the 'black swan' terminology.

    The only swans around here – for thousands of miles in any direction – are black.

    He should've called it a 'platypus event' – now those things are both fairly rare and strange beyond normal reckoning.

Comments are closed.