知の物理学研究センター / Institute for Physics of Intelligence (ipi)

【Date】
3:00-4:30 pm JST, May 29
【Speaker】
Keiichi TAMAI /The University of Tokyo
【Title】
Non-equilibrium Phase Transitions in Artificial Deep Neural Networks
【Abstract】
Despite the rapid progress in applying the so-called deep learning in various fields, the behaviors of artificial deep neural networks are not yet fully understood from a theoretical perspective, even for the classical ones (such as the fully-connected feedforward multilayer perceptrons and the convolutional neural networks). Improvement of the theoretical understanding is desirable for a more principled design of efficient networks. In particular, given the energy efficiency of human brains in learning and generalizing, it is natural to ask what is different between artificial and biological neural networks.
In this talk, I will demonstrate that the theoretical framework of non-equilibrium phase transitions in statistical mechanics is a promising tool for describing the behavior of classical artificial deep neural networks (as well as biological ones). After briefly recalling the recent progress in understanding biological/artificial neural networks, I will combine theoretical and numerical approaches to uncover the universal scaling properties in the initialized artificial networks. The universality of the critical phenomena in non-equilibrium phase transitions allows for an intuitive understanding of the behaviors of the artificial networks across the different architectures.
【Venue】
Room 512, 5F, Bldg. #1, Faculty of Science