In the previous post we argued that the starting point for managing risk in IT project delivery should be a description of the distribution and frequency of project success: you can’t manage something if you don’t know what it looks like. However, we saw that project success in real terms – i.e. of maintaining or increasing the long-term viability of the organisation – is not obviously measurable. We therefore proposed a triangulation approach to infer its distribution from a number of key indicators. These indicators all display power law behaviour. We will now examine what this means..
First however, some historical context. The history of ideas within our culture has its roots in the Renaissance and before that Persia and Ancient Greece. And as we should expect of any people starting to explore the unknown workings of the world they inhabit, the first relationships they discovered were the simplest. Mathematical descriptions of simple, independent observable events were formulated in the natural philosophy of Newton and Descartes, out of which evolved the classical physical sciences. The apparently objective, predictive and repeatable nature of these relationships was hailed as a sign of their exactitude (as opposed to their simplicity) and as a result the physical sciences became the benchmark by which the validity of other areas of inquiry were judged. At the same time, their core tenets of predictability and causal interaction were used as the foundations on which fields ranging from financial mathematics to the social sciences and management theory have been built.
This world of classical physics is one of Bell Curves (also known as the Normal or Gaussian distribution), stable averages and meaningful standard deviations. It is easily demonstrated by example of a coin toss: if I repeatedly toss 10 unbiased coins then the distribution of heads will tend towards a bell curve with an average/peak at 5 heads.
Fig 1. example bell curves (courtesy of Wikipedia):
The first challenge to this world view came from quantum mechanics at the turn of the last century, where discrete causal interaction was replaced by the fuzziness of probability distribution functions and the uncertainty principle. More recently it was then challenged at the macro level by the study of the chaotic behaviour of complex systems. These systems are characterised by interdependence between events which can result in both positive and negative feedback loops. On the one hand seemingly large causal triggers can be absorbed without apparent impact whilst on the other, large effects can be spun up from trivial and essentially untraceable root causes. The result is pseudo-random behaviour, and something that follows the same mathematical description the economist Pareto discovered eighty years earlier in his studies of income distribution (succintly summarised as the 80:20 rule) and that Bradford discovered thirty years earlier in textual index analysis: namely the power law. Since then examples have been found everywhere from epidemiology, stock price variations, fractals and premature birth frequencies through to coastline structure, word usage in language, movie profits and job vacancies.
Fig 2. example Power Law Curves (courtesy of Wikipedia):
The power law derives its name from the dependence or inverse dependence of one variable on the squared, cubed, etc power of the other. (Plot the log of one against the other, and the gradient of the straight line will give you the exponent – i.e. whether it is a square or cube relationship). For example, Pareto discovered that income distributions across populations often followed a roughly inverse square law: for a given income band, roughly one quarter of the amount of people will receive double that income and one ninth will receive triple. The fact that this holds true whether you are looking at the lowest or highest income brackets denotes a signature characteristic of power law phenomena. It is known as scale-invariance or self-similarity, and is most widely recognised in another power law field: fractals.
Other key characteristics of power laws are an unstable mean and variance (i.e. they are statistically irregular, hence unpredictable), and they have a fat/long tail in comparison to bell curves (i.e. extreme events are a lot more frequent):
“The dream of social science [JE: project methodologies??], of building robust frameworks that allow prediction, is shattered by the absence of statistical regularity in phenomena dominated by persistent interconnectivity.” (Sornette, 2003)
“Paretian tails decay more slowly than those of normal distributions. These fat tails affect system behaviour in significant ways. Extreme events, that in a Gaussian world could be safely ignored, are not only more common than expected but also of vastly larger magnitude and consequence. For instance, standard theory suggests that over that time [JE: 1916 – 2003] there should be 58 days when the Dow moved more than 3.4 percent; in fact there were 1001″ (Mandelbrot and Hudson, 2004)
The fundamental message here can be read as follows. The apparently objective world of simple, independent events, normal distributions and classical physical/economic sciences is not actually the norm. Being the domain of the most simple events, it’s just that we discovered it earlier than everything else. In fact it is the limiting edge case along a sliding scale of much more commonly occurring complex and/or chaotic systems through to truly random or stochastic processes, all of which exhibit intrinsically unpredictable and more extreme power law behaviour. And the critically important point as it affects us in the delivery of IT projects? – that we need a risk management model tailored to the complex world of generating business value rather than the vastly over-simplistic world of basic mechanics. The most spectacular/shocking example of what happens when someone attempts to model such power law systems using the normal distributions of classical methodologies is given by the collapse of the Long Term Capital Management hedge fund. As regards the implications for us within the realms of risk management of IT project deliveries, that will be the subject of the next post.