Chong NJL, Feng L. Self-organization toward 1/f noise in deep neural networks.
CHAOS (WOODBURY, N.Y.) 2024;
34:081101. [PMID:
39088349 DOI:
10.1063/5.0224138]
[Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/19/2024] [Accepted: 07/12/2024] [Indexed: 08/03/2024]
Abstract
In biological neural networks, it has been well recognized that a healthy brain exhibits 1/f noise patterns. However, in artificial neural networks that are increasingly matching or even out-performing human cognition, this phenomenon has yet to be established. In this work, we found that similar to that of their biological counterparts, 1/f noise exists in artificial neural networks when trained on time series classification tasks. Additionally, we found that the activations of the neurons are the closest to 1/f noise when the neurons are highly utilized. Conversely, if the network is too large and many neurons are underutilized, the neuron activations deviate from 1/f noise patterns toward that of white noise.
Collapse