The phrase “Agile in the Large” is one I’ve heard used a number of times over the last year in discussions about scaling up agile delivery. I have to say that I’m not a fan, primarily because it entails some pretty significant ambiguity. That ambiguity arises from the implied question: Agile What in the Large? So far I have encountered two flavours of answer:

1.) Agile Practices in the Large
This is the common flavour. It involves the deployment of some kind of overarching programme container, e.g. RUP, which is basically used to facilitate the concurrent rollout of standard (or more often advanced) agile development practices.

2.) Agile Principles in the Large
This is the less common, but I believe much more valuable, flavour. It involves taking the principles for managing complexity that have been proven over the last ten years within the domain of software delivery and re-applying them to manage complexity in wider domains, in particular the generation of return from technology investment. That means:

  • No more Big Upfront Design: putting an end to fixed five year plans and big-spend technology programmes, and instead adopting an incremental approach to both budgeting and investment (or even better, inspirationally recognising that budgeting is a form of waste and doing without it altogether – thanks to Dan North for the pointer)
  • Incremental Delivery: in order to ensure investment liability (i.e. code that has yet to ship) is continually minimised
  • Frequent, Rapid Feedback: treating analytics integration, A/B testing capabilites, instrumentation and alerting as a first order design concern
  • Retrospectives and Adaptation: a test-and-learn product management culture aligned with an iterative, evolutionary approach to commercial and technical strategy

When it comes down to it, it seems to me that deploying cutting-edge agile development practices without addressing the associated complexities of the wider business context is really just showing off. It makes me think back to being ten years old and that kid at the swimming pool who was always getting told off by his parents “Yes Johnny I know you can do a double piked backflip, but forget all that for now – all I need you to do is enter the water without belly-flopping and emptying half the pool”.

Towards the end of last year I was lucky enough to meet up with Chris Matts for lunch on a fairly regular basis. To my great surprise, he did me the amazing honour of capturing some of what we discussed concerning memes in one of his legendary comics. They have just been published on InfoQ together with a discussion piece here: http://www.infoq.com/articles/meme-lifecycle

In previous posts we saw that the generation of business value via IT projects essentially follows a power law distribution. By examining the nature of power law systems, we went on to conclude that adaptive strategies offer the most effective way of managing risk in such environments. We will now begin to explore what a fully adaptive risk management strategy might look like, using as our starting point an overview of the key principles underlying nature’s great adaptive risk management engine: Natural Selection..

Evolutionary ideas have recently been gaining prominence in studies of organisational behaviour and efficiency from two directions:

  • Evolutionary Micro-economics (top down), in response to the limitations of traditional rationalist supply/demand models based on Game Theory.
  • Adaptive Project Methodologies (bottom up), focussing on evolutionary design and iterative delivery to mitigate the inherent unpredictability of requirements and market conditions.

The most fundamental principle on which these ideas are based is the notion of a replicator. A replicator can be defined as any entity of which copies are made, where that entity has some causal influence on its own probability of being propagated. The classic biological example is a gene, which is copied during cell division and which influences its probability of being propagated via the environmental effects of the proteins it encodes (and in turn, the effects of the composite structures out of which those proteins are built). The specific DNA sequence of a gene is known as its genotype, and the corresponding expression of that genotype is its phenotype.

In the Extended Phenotype, Richard Dawkins switched the primary focus of evolutionary studies away from the organism. He showed that “organism” is ultimately just an arbitrary point along the scale of phenotypes: from specific proteins at one end, up through more complex protein structures to organs, organisms and social groups at the other. The fundamental unit driving natural selection forwards across the generations is the replicator or Selfish Gene – everything else from protein to social group is just artefactual byproduct (that impacts the probability of further replicator propagation).

Other instances of replicators include memes. A meme is “any unit of cultural information, such as a practice or idea, that gets transmitted verbally or by repeated action from one mind to another. Examples include thoughts, ideas, theories, practices..” When we consider the field of IT project delivery within this context, we can spot obvious correlations. Business cases are memes which, when ratified, result in the generation of a suite of phenotypic artefacts ranging from marketing strategies to IT delivery teams to unit tests, SCM repositories and deployed production systems to new revenue streams. These artefacts end up shaping their business division, organisation and industry sector, and in doing so determine the probability of the business case propagating and spawning further system releases, new marketing campaigns, etc.

There is a key lesson for us as IT practioners to take from this, one that evolutionary biologists have already learnt. It is that artefacts (be they organisms, social groups, IT projects or marketing campaigns) don’t ultimately matter. The thing that matters is the replicator: the business case or gene. We need to follow evolutionary biology’s re-orientation towards the gene, and shift our focus away from IT projects and create practices centred solely on the business case. I now believe that “projects” can actually be an impediment to the efficient generation of real business value from IT. They act as an inflexible body of emotional and financial investment that creates resistance to both a.) change and b.) termination where such change makes the business case no longer viable in real terms (which is when real damage is then inflicted). We will discuss more on this topic in subsequent posts. Before that however, we need to examine the nature of selective environments – which will be the subject of the next post. In doing so we will hopefully shed some light on the factors that have led to our current project-orientated IT world view.