Self-organization toward 1/f noise in deep neural networks

Abstract

In biological neural networks, it has been well recognized that a healthy brain exhibits 1/f noise patterns. However, in artificial neural networks that are increasingly matching or even out-performing human cognition, this phenomenon has yet to be established. In this work, we found that similar to that of their biological counterparts, 1/f noise exists in artificial neural networks when trained on time series classification tasks. Additionally, we found that the activations of the neurons are the closest to 1/f noise when the neurons are highly utilized. Conversely, if the network is too large and many neurons are underutilized, the neuron activations deviate from 1/f noise patterns toward that of white noise.