Patterns, Textures and Physical Processes
Neural Cellular Automata (NCA) are capable of learning a diverse set of behaviors: from generating stable, regenerating, static images to segmenting images to learning to “self-classify” shapes. The inductive bias imposed by using cellular automata is powerful. A system of individual agents running the same learned local rule can solve surprisingly complex tasks. Moreover, individual agents, or cells, can learn to coordinate their behavior even when separated by large distances.
Patterns, Textures and Physical Processes
Zebra stripes are an iconic texture. Ask almost anyone to identify zebra stripes in a set of images, and they will have no trouble doing so. Ask them to describe what zebra stripes look like, and they will gladly tell you that they are parallel stripes of slightly varying width, alternating in black and white. And yet, they may also tell you that no two zebra have the same set of stripes.
Patterns and textures are ill-defined concepts. The Cambridge English Dictionary defines a pattern as “any regularly repeated arrangement, especially a design made from repeated lines, shapes, or colors on a surface”. This definition falls apart rather quickly when looking at patterns and textures that impart a feeling or quality, rather than a specific repeating property.
As a result, when having any model learn to produce textures or patterns, we want it to learn a generative process for the pattern. We can think of such a process as a means of sampling from the distribution governing this pattern.
Hidden States
When biological cells communicate with each other, they do so through a multitude of available communication channels. Cells can emit or absorb different ions and proteins, sense physical motion or “stiffness” of other cells, and even emit different chemical signals to diffuse over the local substrate.
In the principal components of this coral-like texture, we see a pattern which is similar to the visible channels. However, the “threads” pointing in each diagonal direction have different colors – one diagonal is green and the other is a pale blue. This suggests that one of the things encoded into the hidden states is the direction of a “thread”, likely to allow cells that are inside one of these threads to keep track of which direction the thread is growing, or moving, in.
Conclusion
In this work, we selected texture templates and individual neurons as targets and then optimized NCA populations so as to produce similar excitations in a pre-trained neural network. This procedure yielded NCAs that could render nuanced and hypnotic textures. During our analysis, we found that these NCAs have interesting and unexpected properties.
Hidden States
We hypothesize that the hidden states are a way for cells to communicate with each other, allowing them to learn distributed, local, algorithms.
