Designing for Emergence

I was reading a fascinating website (23 years old!) of University of York Professor Susan Stepney, and this call to action caught my eye.

If we are ever to advance beyond simple computational artefacts, we need a science of emergence. ... Not only do we need to engineer the initially desired emergent properties, we need to engineer ability to change in desired, if unanticipated, ways.

— Susan Stepney, Non-Standard Computation: an overview (2001)

Stepney is—still, 23 years later—an expert of "non-standard computation", which I suspect to be a euphemism for "contrarian computer science". She works on how to compute stuff, but without using all the tried-and-true techniques, algorithms, and hardware that we've perfected over a century. The page linked above explains it better than I could ever do: non-Turing, non-von-Neumann machines, cellular automata, and cool things like that. It's worth a read despite the age.

In this passage, she argues that, while emergence is fundamental in everything, we have no theory for how to design and create emergence as we like.

Here, as is usually the case among scientists, she uses "emergence" to refer to what I call "context emergence"—the most interesting kind. In the broader sense of the word, we do know how to engineer for emergence, and very well. A pair of scissors is engineered (feature) emergence: the ability to make straight cuts in paper that would be nearly impossible with the same two blades handled separately. And the same holds for every single product of technology. In this sense, engineering is nothing but the design of emergence.

But Stepney, of course, is thinking about something much more sophisticated than that. She wants a theory that makes new physical laws emerge, as the laws of thermodynamics emerge from the assemblage of simple atoms, or as the self-organized mechanisms of embryonic development lead to a grown adult from a single cell, or the "laws" of social behavior, and so on. Stepney wants us to learn how to engineer complexity.

That's a whole different level of difficulty. She admits that there are high barriers against that dream, like the computational irreducibility of certain processes, and the huge role that contingency plays in complex systems. So how do we do that? Stepney's essay proposes a full research program to tackle that problem, which makes a lot of sense. I'm especially interested in one point that she makes:

Complex systems should be "grown" rather than "switched on".

It immediately reminds me of "Gall's Law":

A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over with a working simple system.

— John Gall, Systemantics (1977)

This is a very different kind of engineering from what we're used to. You can't make a "blueprint" for a complex system, nor a step-by-step instruction plan that can be replicated accurately to mass-produce it. You have to begin from a seed, and tend to it like a tree, letting it do its own thing while finding gentle, focused ways to influence its growth.

Perhaps the closest thing to an engineering of emergence that we've come up with are the latest advances in artificial intelligence. We don't know exactly why a specific language model does what it does, and we have no hope of "planning" its behavior beforehand. We are forced to "grow" (train) it, feeding it the nourishment (data) it needs, and discover what it does after the fact. We can try to steer it a little by selecting the inputs we give it, pushing it away from its biases and towards the kind of knowledge that we would want it to talk about, but it's at best a trial-and-error process.

My gut feeling is that we're already good enough—as good as we can hope to get—at at the "growing" part. We have enough experience of that with all the natural complex systems we're surrounded with, from gardens to groups of people. The skill that we really miss is creating that initial seed, Gall's "simple sistem that works". Deep neural networks are one example of that, began as an imitation of brain structure and improved over decades of experimentation. That, then, is one path: copy nature, and see where it leads you. Is there any other, much faster way? ●