{"id":2943,"date":"2021-06-15T10:40:22","date_gmt":"2021-06-15T01:40:22","guid":{"rendered":"https:\/\/www.phys.s.u-tokyo.ac.jp\/en\/?p=2943"},"modified":"2021-10-13T10:00:18","modified_gmt":"2021-10-13T01:00:18","slug":"ipi-seminar-1700-1830-wednesday-june-9-2021-2","status":"publish","type":"post","link":"https:\/\/www.phys.s.u-tokyo.ac.jp\/en\/news\/2943\/","title":{"rendered":"[finished] ipi seminar 17:00-18:30, Tuesday July 13, 2021"},"content":{"rendered":"\n<h3>\u77e5\u306e\u7269\u7406\u5b66\u7814\u7a76\u30bb\u30f3\u30bf\u30fc \/ Institute for Physics of Intelligence (ipi)<\/h3>\n<p><br \/>\u3010Speaker\u3011Masaaki IMAIZUMI<span class=\"x_author-d-1gg9uz65z1iz85zgdz68zmqkz84zo2qowz82z5anr4b6129z90zz66zz74zwjicsz79zjz86zz81zz66zcz77zxz72z1gs\"> (The University of Tokyo\uff09<br \/><\/span><\/p>\n<p>\u3010Date\u3011July 13 (Tuesday), 17:00-18:30JST<br data-rich-text-line-break=\"true\" \/><br data-rich-text-line-break=\"true\" \/>\u3010Title\u3011\"Generalization Analysis of Deep Learning: Implicit Regularization and Over-parameterization\"<br data-rich-text-line-break=\"true\" \/><br data-rich-text-line-break=\"true\" \/>\u3010Abstract\u3011Deep learning achieves high generalization performance, but a theoretical understanding of its principles is still a developing topic. In this talk, I will present two theoretical results on this topic: (i) loss surface-oriented implicit regularization, and (ii) double descent for deep models.<\/p>\n<div>(i) Implicit regularization argues that a learning algorithm implicitly constrains the degrees of freedom of neural networks. However, a specific implicit regularization achieved by deep neural networks has not been clarified. In this paper, we theoretically show that when a loss surface has many local minima satisfying certain assumptions, its shape constrains a learning algorithm to achieve regularization. In this case, we also show that a generalization error of deep neural networks has an upper bound independent of the number of parameters.<\/div>\n<div>\u00a0<\/div>\n<div>(ii) Asymptotic risk analysis, including double descent, is a theoretical framework to analyze the generalization error of models with excessive parameters. Although it has attracted strong attention, it can analyze linear models in features such as random feature models. We show that, for a family of models without linearity constraints, the upper bound of the generalization error follows the theory of asymptotic risk. By investigating our regularity condition, we show that specific nonlinear models, such as parallelized deep neural networks, obey our result.<\/div>\n<div>\u00a0<\/div>\n<div>\n<div><span class=\"x_author-d-iz88z86z86za0dz67zz78zz78zz74zz68zjz80zz71z9iz90z8z86zdnagz83zz74zikpz70zz72zkz79zkfbz71z2z88zz74zm9gxc1z74zz79zk\">\u203bTo receive the Zoom invitation and monthly reminders,<span class=\"x_Apple-converted-space\">\u00a0<\/span><\/span><span class=\"x_author-d-iz88z86z86za0dz67zz78zz78zz74zz68zjz80zz71z9iz90z8z86zdnagz83zz74zikpz70zz72zkz79zkfbz71z2z88zz74zm9gxc1z74zz79zk\"><b>please register via this google form<\/b><\/span><span class=\"x_author-d-iz88z86z86za0dz67zz78zz78zz74zz68zjz80zz71z9iz90z8z86zdnagz83zz74zikpz70zz72zkz79zkfbz71z2z88zz74zm9gxc1z74zz79zk\">:<span class=\"x_Apple-converted-space\">\u00a0<\/span><\/span><span class=\"x_attrlink x_url x_author-d-iz88z86z86za0dz67zz78zz78zz74zz68zjz80zz71z9iz90z8z86zdnagz83zz74zikpz70zz72zkz79zkfbz71z2z88zz74zm9gxc1z74zz79zk x_url\"><a class=\"x_attrlink\" href=\"https:\/\/forms.gle\/dqxhpsZXLNYvbSB38\" target=\"_blank\" rel=\"noreferrer nofollow noopener\" data-auth=\"NotApplicable\"><u>https:\/\/forms.gle\/dqxhpsZXLNYvbSB38<\/u><\/a><\/span><\/div>\n<div><span class=\"x_author-d-iz88z86z86za0dz67zz78zz78zz74zz68zjz80zz71z9iz90z8z86zdnagz83zz74zikpz70zz72zkz79zkfbz71z2z88zz74zm9gxc1z74zz79zk\">Your e-mail addresses will be used for this purpose only, you can unsubscribe anytime, and we will not send more than three e-mails per month.<\/span><\/div>\n<div>\u00a0<\/div>\n<div>\u00a0<\/div>\n<div style=\"text-align: right\"><span class=\"x_author-d-iz88z86z86za0dz67zz78zz78zz74zz68zjz80zz71z9iz90z8z86zdnagz83zz74zikpz70zz72zkz79zkfbz71z2z88zz74zm9gxc1z74zz79zk\">Tilman HARTWIG, Takashi TAKAHASHI &amp; Ken NAKANISHI<\/span><\/div>\n<div>\u00a0<\/div>\n<div style=\"text-align: left\"><span class=\"x_author-d-iz88z86z86za0dz67zz78zz78zz74zz68zjz80zz71z9iz90z8z86zdnagz83zz74zikpz70zz72zkz79zkfbz71z2z88zz74zm9gxc1z74zz79zk\"><a href=\"https:\/\/www.phys.s.u-tokyo.ac.jp\/en\/lp\/ipi\/\" data-rich-text-format-boundary=\"true\">\u21e6Top page<\/a><\/span><\/div>\n<div>\u00a0<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"\u77e5\u306e\u7269\u7406\u5b66\u7814\u7a76\u30bb\u30f3\u30bf\u30fc \/ Institute for Physics of Intelligence (ipi) \u3010Speaker\u3011Masaaki IMAIZUMI (The University of Tokyo\uff09  [&hellip;]","protected":false},"author":13,"featured_media":3277,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[3],"tags":[36],"_links":{"self":[{"href":"https:\/\/www.phys.s.u-tokyo.ac.jp\/en\/wp-json\/wp\/v2\/posts\/2943"}],"collection":[{"href":"https:\/\/www.phys.s.u-tokyo.ac.jp\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.phys.s.u-tokyo.ac.jp\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.phys.s.u-tokyo.ac.jp\/en\/wp-json\/wp\/v2\/users\/13"}],"replies":[{"embeddable":true,"href":"https:\/\/www.phys.s.u-tokyo.ac.jp\/en\/wp-json\/wp\/v2\/comments?post=2943"}],"version-history":[{"count":9,"href":"https:\/\/www.phys.s.u-tokyo.ac.jp\/en\/wp-json\/wp\/v2\/posts\/2943\/revisions"}],"predecessor-version":[{"id":3279,"href":"https:\/\/www.phys.s.u-tokyo.ac.jp\/en\/wp-json\/wp\/v2\/posts\/2943\/revisions\/3279"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.phys.s.u-tokyo.ac.jp\/en\/wp-json\/wp\/v2\/media\/3277"}],"wp:attachment":[{"href":"https:\/\/www.phys.s.u-tokyo.ac.jp\/en\/wp-json\/wp\/v2\/media?parent=2943"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.phys.s.u-tokyo.ac.jp\/en\/wp-json\/wp\/v2\/categories?post=2943"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.phys.s.u-tokyo.ac.jp\/en\/wp-json\/wp\/v2\/tags?post=2943"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}