Complexity
Complex systems is at essence the study of systems composed of many parts that interact to produce behaviour that is not easily determinable from the constituents themselves. The majority of real world systems contain parts that are not linearly decomposable and hence often intractable to analytical approaches. It is, as such, the science of dealing with system properties that do not yield to reductionist approaches. In order to deal with these problems we adopt the technique of building computational models that ignore finer details and attempt to capture qualitative behaviour.
Complex Systems has also turned out to be a science that can describe abstract kinds of behaviour that are in some senses independent of the particulars of the system in which they are embedded. In this way it constitutes the coming together of various scientific disciplines to describe the properties of systems in general.
These initial definitions are somewhat vague and also misleading, because complex systems is really an amalgamation of various insights from a range of disciplines that indicate that the world behaves in ways that are very different to the assumptions of classical models. In a very strong sense complex systems is best understood by identifying the ways in which our previous attempts to understand the world have failed. The following foray into the historical emergence of complex systems will help clarify the picture by defining the kinds of systems with which we are concerned.
History and Physics Perspective
Thermodynamics and statistical mechanics were the first historical attempt to deal analytically with systems composed of many interacting parts. Thermodynamics describes the evolution of global properties of a system and statistical mechanics covers the detailed interaction between components from which thermodynamical behaviour emerges.
Inherent in the statistical mechanical approach is that the system can be described as a point in a state space that moves over time. In order to make the transition to global properties of the system, we require the ergodic hypothesis, which states that given enough time the system will explore all states with a given energy from any starting point. The ergodic hypothesis allows us to make predictions of global properties by averaging across an ensemble consisting of multiple possible states of the system. In effect the ergodic hypothesis allows us to treat properties of a system evolving over time using a collection of parallel snapshots.
This approach to studying systems involves studying the system in its environment by separating behaviour that is occurring at different time scales. Slow behaviour results in physcial constants and very rapid behaviour can be treated thermodynamically to result in averages across the fast processes. This leaves the dynamics of the system to be studied, which can be then done in the context of the constants provided by the slow and fast processes.
In order for this approach to work, we require not only the ergodic hypothesis for our fast systems, but also a relatively neat separation of scales, so that we can isolate the desired behaviour to a particular scale for the sake of modelling. Unfortunately these requirements break down for many systems of interest.
The ergodic hypothesis breaks down for many of the systems we call complex, because the system has access to only a small subset of states with the requisite energy. Meaning that in reality it explores only a relatively small subset of its state space.
When we study phase transitions in materials we discover that second order phase transitions involve fluctuations in systems properties happening across all spatial scales. This is one example where behaviour under investigation cannot be isolated at a particular scale. We can also observe systems for which we have dynamical behaviour occurring across numerous time scales that cannot be turned into an ensemble average.
This discussion of behaviour occurring at different scales then leads naturally into the overview of fractal geometry and the consequent examination of the scaling behaviour of system properties. It also raised the central theme in complex systems of how we understand the relationships between processes occurring at different scales.
The Modelling Approach
The essence of the problems which complex systems engages with are systems that have dynamical structure across a range of spatial and temporal scales. We are thus unable to use averaging techniques to look at the behaviour of subsystems in context.
Even though we may be able to build models of these systems, we will not find neat equilibrium analytical solutions to them. Instead we must simulate them and analyse their scaling behaviour and look for relationships between behaviour at varying scales that conform to what we observe in the world.
From this persepective complex systems has grown as a discipline of building and analysing models. It in some sense abandons the hope of finding complete solutions and hence predictions, in favour of understanding the limitations of models and the qualitative behaviours that emerge from different kinds of models.
Curiously, the process of building and analysing models has resulted in the discovery that there are universal behaviours that emerge from many different models. For example, the discovery of the constant ratio betwteen bifurcations in a system that is descending toward chaos with variation in a key parameter. These universal properties allow us to abstract from the real world implementation of a particular system and find commonalities across diverse disciplines, from economics to physics. This has lead to the discovery that there are entire classes of models that all exhibit the same larger scale behaviour even though they have many differences in the particulars of the model, this is called Universality.
Another important concept in the study of complex systems is the notion of emergence, which in its simplest form describes the manner in which certain properties of a system emerge from the interactions of the parts. There are further distinction made to clarify this fuzzy notion. Strong Emergence is defined as system properties that cannot in principle be predicted from knowledge of the parts, whereas weak emergence describes merely behaviour that is surprising or counter intuitive.
There is also a more fundamental distinction between local and global emergence. Local emergence refers to global properties of a system which would be difficult or impossible to describe in terms of all of the constituent parts, but can be understood or estimated using almost any subset of parts from the system. For example the temperature of a gas, which emerges from the energy of the individual molecules, can be understood using subsets of the gas.
Global emergence on the other hand describes system wide phenomena that cannot be described using any subset of the parts. This is key to systems that we would call complex adaptive systems, homeostasis is one such property, the entire organisation of the system is such that it maintains itself in the face of environmental fluctuations.
Despite the philosophical distinctions and debates, the notion of emergence is important simply because it expresses the fact that simple models like cellular automata have produced complex patterns that people in general were not expecting. Combined with Universality we gain a sense that not only do we find unexpected pattern formation at higher scales than which our model is implemented but also these patterns do not depend heavily on the specifics of the model itself.
Self-organisation, Pattern Formation and Phase Transitions
A significant component of the study of complex systems involves understanding the spontaneous formation of patterns (self organisation) and the conditions under which they break down or change completely (phase transitions).
As mentioned earlier this topic has a history in the physical study of materials. Phase diagrams captured the relationship between a number of parameters which influenced the organisational form that a system settles into. The common phase diagram of water, shows the three forms, solid liquid and gas with the temperature and pressure conditions conducive to each. The second order phase transition that occurs between liquid and gas at high temperature and pressure is an archetype of a dynamical process without a characteristic scale.
Insights about complex systems have also been drawn from the study of fluid dynamics. The Rayleigh-Bernard flow in convection cells is a spontaneous process where fluid trapped between two sheets with a temperature gradient will organise into hexagonal cells. The Rayleigh number is composed of several relevant parameters including temperature gradient and viscosity. Above the value 1708 the cells will form a stable pattern.
Similarly in the study of turbulence the Reynolds number allows the prediction of the qualitative kinds of patterns that will form when a liquid flows around and behind and obstruction. As the Reynolds number increases more fine grained patterns emerge. The patterns in fluid dynamics result from conflicting constraints bouyancy vs viscosity or misalignment of density and pressure gradients.
We can take from these specific examples a number of lessons about pattern formation in complex systems.
- Tension between conflicting constraints provides a crucial source of order.
- Transitions between patterns can be found to be sensitive a single (often composite) parameter.
- The reason why transition occur at the critical values is not intuitively obvious and may not be known.
A substantial part of the study of complex system involves observing the manner in which these patterns form in systems as emergent properties. As well as these examples from physical systems we have observed the spontaneous formation of patterns in simulations like cellular automata. The key point is that sometimes we observe ordered patterns emerging from a dynamically evolving system, but where the patterns themselves are not in any way written into the system. They emerge from the interactions between components and the environmental constraints.
This idea is central to most of the talk about modelling, because the a key insight from complex systems is that the introduction of models that include spatial constraints results in complex pattern formation and predictions contrary to models which average across a population.
Describing Complexity
How complex is something? There are numerous competing methods of quantification such as algorithmic complexity, information theory, negentropy.
In information theoretic terms complexity may be defined as the amount of information needed to describe a system. This is the length of the shortest bit string, which is equal to the log base two of the number of states that the system can hold. If we chose to use a linguistic description we need to use Shannon formalisation in which we incorporate the fact that all letters do not occur with equal frequency. This results in approx 0.6-1.2 bits of information in each English character.
If we perform such an analysis of a variety of systems at an identical level of description (i.e. Atomic scale) then a paradox emerges. A human being in a blender is more complex than an unblended human being. This is due to the fact that the unblended human has greater constraints on where particular atoms can be, this constraint reduces the possibilities and hence in information theoretic terms reduces the length of the minimal description.
The blender paradox inspired the development of a conceptual tool for thinking about what is happening in the kinds of complex systems we are interested in.
Instead of providing a minimal description of the system at the most fine grained scale available to us, we can provide a range of descriptions at varying scales. This results in a graph called the complexity profile of the system [1][2][3][4]. As we observe the human in a blender at larger scales the complexity drops off rapidly because of the homogeneity in the system. However, the unblended human has patterns of organisation across a range of scales that always require some description.
The complexity profile is a monotonically decreasing function with a constant area for a system with a given number of particles and volume. It demonstrates that the kinds of systems we think of as 'complex' are actually complex across a range of scales.
Systems and environments
Ashby's Law of Requisite Variety [5] is a cybernetic principle which states that in order to regulate a system a controller must have an internal variety at least as great as the variation in the system it seeks to regulate. This principle can be related to any system which needs to sustain itself in the face of environmental change. In order to maintain homeostasis or perform acts of adequate self-preservation an organism must have an internal complexity on par with the crucial elements of its environment.
This allows an estimation of the probability that a system will fail
-log2(P) = Ce - Cs
We can use this notion to make estimates of the average mismatch between and organism and its environment based on the breeding rate [1][3]. We first assume that the breeding rate has an inverse relationship to the probability that an individual of the species will fail:
P(failure) = 1 - 1/(Average number of offspring)
Then we plug this into the previous formula to get an estimate of the average mismatch between the an individual of that species and the environment the inhabit. The result is a curious re-interpretation of the K and r breeding strategies from evolutionary biology.
Human Civilization
Human social structures are themselves complex systems that deal with a broader environment than the ones that individual operate in. Looking at the historical record we can observe a gradual change in the arrangement of these structures. Early civilizations tended to have hierarchies, which have been extended as societies grew. Even though hierarchical organisations have dominated the post industrial political and corporate era, a rapid change occurred during the 1980s. Suddenly a great many dictatorships across the world voluntarily dissolved. In the private sector the Total Quality Management movement shifted the focus of decision making from centralised management directives passed down the chain to the creation and support of independant business units.
The perspective put forward by Yaneer Bar-Yam is that the hierarchical organisation was littered with information bottlenecks, most significantly with the CEO or dictator[1][2][3][4][6]. This placed an upper limit on the complexity of the environment in which such a structure could operate. When a large number of political and business communities began to surpass this upper limit in the 1980s, these structures became unviable. The result was a trend toward decentralised social organisations. This trend will continue toward the idealised networked social structure as the world becomes more complex.
As well as providing a framework for understanding this shift in social organisation, the complex systems perspective offers some heuristics for management.
- In order to function effectively the individuals within an organisation must be effectively shielded from the full complexity of the environment in which the organisation operates.
- Conversely, no individual will be capable of fully comprehending the organisation.
- One should regularly analyse the operating environment for evidence of activities occurring at different scales. If a single business unit is attempting to operate at numerous scales it is likely to fail, and should be separated into units specialised for each of the tasks individually.
References
[1] Y. Bar-Yam, Dynamics of Complex Systems. Perseus, 1997. Available Online
[2] Y. Bar-Yam, Multiscale Complexity / Entropy, Advances in Complex Systems 7, pp. 47-63, 2004.
[3] Y. Bar-Yam, Unifying Principles in Complex Systems, in Converging Technology (NBIC) for Improving Human Performance, M. C. Roco and W. S. Bainbridge eds, Kluwer, 2003.
[4] Y. Bar-Yam, Multiscale Variety in Complex Systems, Complexity 9, 37-45, 2004.
[5] W.R. Ashby, An Introduction to Cybernetics; Chapman & Hall, 1956,
[6] Y. Bar-Yam, Complexity rising: From human beings to human civilization, a complexity profile, in Encyclopedia of Life Support Systems (EOLSS), developed under the Auspices of the UNESCO, EOLSS Publishers, Oxford, UK, 2002.
[7] H. Sayama, L. Kaufman and Y. Bar-Yam: Symmetry breaking and coarsening in spatially distributed evolutionary processes including sexual reproduction and disruptive selection, Physical Review E 62, pp.7065-7069, 2000;
[8] Y. Bar-Yam: Formalizing the gene centered view of evolution, Advances in Complex Systems 2, pp.277-281, 1999; and my texbook (chapter 6).