Tool: Essential Complexity

Essential Complexity is a term used by Frederick Brooks in his cornerstone paper “No Silver Bullet – Essence and Accident in Software Engineering”, The term refers to the actual or natural complexity that exists in the system and cannot be substantially reduced (as opposed to Accidental Complexity which can in principle be reduced by usage of better tooling and method). The origin of the term goes back to Aristotle’s ontological notion of essence (as opposed to accident), something that makes an entity or phenomenon what it is. Brooks definition, however, was significantly oriented around the complexity of the System Under Development as well as human cognition, but generally did not include the complexity of social interaction and collaboration within the enterprise.

In a modern product development organization, the following aspects significantly determine essential complexity:

  1. Dynamic nature of business demand that incorporates varying and often unclear customer needs, user goals and market context.
  2. Complexity of the System Under Development (SUD), underlying technology, supporting infrastructure and how the SUD enables business/user scenarios.
  3. People, their interaction and collaboration in realizing SUD capabilities in support of the business demand.

This definition basically reflects upon the three main components of the Demand-Portfolio-Supply model.

The importance of Essential Complexity as a thinking tool comes from the unfortunate fact that most change agents de facto over-attend Accidental Complexity at the cost of what really matters and therefore fail to build an adequate model of the enterprise. A typical example of this is a lot of attention to formal ceremonies and events, while overlooking such things as spontaneous human interaction, deep intellectual states (“flow”), exchange of tacit knowledge, etc. Another example of overlooking essential complexity is often associated with underestimating the complexity and uniqueness of the SUD. Sometimes the biggest impediment to improving the enterprise’s performance is in restructuring the hairball of fragile dependencies across various across a bunch of core SUD functions.

See also: Artificial Complexity, Inattention to SUD and Cargo-Cult Agile anti-patterns.

Note: The author is grateful to Saif Islam for pointing out the importance of Essential and Accidental Complexity to enterprise considerations.