From parameter scaling to memory scaling
What is the information bottleneck in biological neural networks, and how does it relate to neural plasticity?
The information bottleneck is an important characteristic of biological neural networks. During biological evolution, the amount of information carried by genes is extremely limited compared with the complexity of the brains of higher animals such as humans. This observation offers an important lesson for artificial intelligence: because of this information bottleneck, a biological neural network cannot directly inherit a large collection of skills and knowledge from the previous generation. It must learn them again after birth. Genes primarily encode the capacity for memory and learning, rather than memories and skills themselves.
One important mechanism behind human learning and memory is neural plasticity. This differs from current artificial neural networks, whose weights are difficult to change after pretraining and whose updates require an objective function, large amounts of data, and gradient descent. Plasticity allows a neural network to modify its own weights during inference, exhibiting some properties of gradient-free in-context learning and meta-learning. However, most manually designed plasticity rules do not satisfy the information bottleneck: they often require more learning rules than the number of model parameters.
We attempt to design plasticity rules that conform to the information bottleneck. Models governed by these rules are not highly capable at the beginning, but their performance improves progressively as contextual information accumulates. Their defining characteristics are a small parameter count and a large memory capacity. This learning pattern more closely resembles biological intelligence: an infant starts from a lower baseline but has much greater potential for in-context learning.
Biological evolution under an information bottleneck. Our genes are inherited from the previous generation and contain a relatively small amount of information, represented by the genotype. The memories in our brains are acquired after birth, and the amount they can store is at least three orders of magnitude larger than the information in our genes, represented by the phenotype.
Evolving a plastic recurrent neural network with few rules
We use a plastic recurrent neural network called the Decomposed Plastic RNN (DecPRNN). It has very few parameters, with only hundreds of learning rules, but a large memory space containing thousands of memory states. We train and evaluate it for in-context reinforcement learning in maze environments. The experiments lead to several important conclusions:
- Fewer parameters mean a lower starting point.
- A larger memory space means a higher endpoint.
- Parameter count is not directly related to final capability. Models with fewer parameters can even generalize better.
A recurrent neural network with plastic weight parameters uses interaction trajectories for in-context reinforcement learning. At every step, the model receives the state, action, and reward as input and predicts the next action.
Parameter scale, shown as meta-parameters, and memory scale, shown as adaptive components, for different neural networks. We find that models with fewer parameters generalize better, while models with larger memory spaces have stronger learning capabilities.
Final performance of different neural networks. Architectures and algorithms that conform to the genomic bottleneck start lower but ultimately perform better.
Implications for general intelligence
This work suggests a possible path toward AGI that does not overemphasize enormous parameter counts. A moderate number of parameters combined with a vast number of readable and writable memory units may offer greater promise for building artificial general intelligence. These memory units include both explicit memories acquired through in-context learning, such as memory states, and implicit memories, such as plastic connection weights. We also believe that combining many repeated units can expand the memory space while reducing the parameter count, making collective intelligence another highly promising research direction.