{"id":4593,"date":"2023-11-24T10:29:40","date_gmt":"2023-11-24T01:29:40","guid":{"rendered":"https:\/\/www.phys.s.u-tokyo.ac.jp\/en\/?p=4593"},"modified":"2023-12-27T15:39:43","modified_gmt":"2023-12-27T06:39:43","slug":"ipi-seminar-1030-1200-tuesday-dec-5-2023","status":"publish","type":"post","link":"https:\/\/www.phys.s.u-tokyo.ac.jp\/en\/about\/4593\/","title":{"rendered":"[finished] ipi seminar 10:30-12:00, Tuesday Dec. 5, 2023"},"content":{"rendered":"\n<div class=\"article\">\n<h3>\u77e5\u306e\u7269\u7406\u5b66\u7814\u7a76\u30bb\u30f3\u30bf\u30fc \/ Institute for Physics of Intelligence (ipi)<\/h3>\n<p>\u3010Date\u3011<br \/>Dec. 5, 2023 10:30 - 12:00 (JST)<br \/>via zoom &amp; #415, #1 Faculty of Science Building <\/p>\n<p>\u3010Speaker\u3011<br \/>Toshihiro Ota, Cyber Agent AI Lab<\/p>\n<p>\u3010Title\u3011<br \/>\u201cHopfield\/Mixer correspondence: towards a better understanding of MetaFormers architecture design\u201d<br \/><br \/>\u3010Abstract\u3011<br \/>In the last few years, the success of Transformers in computer vision has stimulated the discovery of many alternative models (MetaFormers) that compete with Transformers, such as the MLP-Mixer. Despite their weak induced bias, these models have achieved performance comparable to well-studied convolutional neural networks. Recent studies on modern Hopfield networks suggest the correspondence between certain energy-based associative memory models and Transformers or MLP-Mixer, and shed some light on the theoretical background of the Transformer-type architectures design. In this talk, we discuss how the modern Hopfield networks may be useful for a unified understanding of MetaFormer. In particular, we propose Hopfield\/Mixer correspondence as a new direction for MetaFormers architecture design, and introduce a novel MetaFormer model theoretically derived from the correspondence. Finally, possible extensions and their prospects will be discussed.<\/p>\n<p>*To receive the Zoom invitation and monthly reminders,\u00a0<strong>please register via this google form<\/strong>:\u00a0<a href=\"https:\/\/forms.gle\/dqxhpsZXLNYvbSB38\" target=\"_blank\" rel=\"noreferrer noopener\"><u>https:\/\/forms.gle\/dqxhpsZXLNYvbSB38<\/u><\/a><\/p>\n<p>Your e-mail addresses will be used for this purpose only, you can unsubscribe anytime, and we will not send more than three e-mails per month.<\/p>\n<p class=\"has-text-align-right\">Takashi Takahashi and Ken Nakanishi<\/p>\n<p><a href=\"https:\/\/www.phys.s.u-tokyo.ac.jp\/en\/lp\/ipi\/\">\u21e6Top page<\/a><\/p>\n<\/div>\n\n\n\n\n","protected":false},"excerpt":{"rendered":"\u77e5\u306e\u7269\u7406\u5b66\u7814\u7a76\u30bb\u30f3\u30bf\u30fc \/ Institute for Physics of Intelligence (ipi) \u3010Date\u3011Dec. 5, 2023 10:30 - 12:00 (JST)via zoom &amp; [&hellip;]","protected":false},"author":13,"featured_media":4596,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[22],"tags":[],"_links":{"self":[{"href":"https:\/\/www.phys.s.u-tokyo.ac.jp\/en\/wp-json\/wp\/v2\/posts\/4593"}],"collection":[{"href":"https:\/\/www.phys.s.u-tokyo.ac.jp\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.phys.s.u-tokyo.ac.jp\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.phys.s.u-tokyo.ac.jp\/en\/wp-json\/wp\/v2\/users\/13"}],"replies":[{"embeddable":true,"href":"https:\/\/www.phys.s.u-tokyo.ac.jp\/en\/wp-json\/wp\/v2\/comments?post=4593"}],"version-history":[{"count":3,"href":"https:\/\/www.phys.s.u-tokyo.ac.jp\/en\/wp-json\/wp\/v2\/posts\/4593\/revisions"}],"predecessor-version":[{"id":4685,"href":"https:\/\/www.phys.s.u-tokyo.ac.jp\/en\/wp-json\/wp\/v2\/posts\/4593\/revisions\/4685"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.phys.s.u-tokyo.ac.jp\/en\/wp-json\/wp\/v2\/media\/4596"}],"wp:attachment":[{"href":"https:\/\/www.phys.s.u-tokyo.ac.jp\/en\/wp-json\/wp\/v2\/media?parent=4593"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.phys.s.u-tokyo.ac.jp\/en\/wp-json\/wp\/v2\/categories?post=4593"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.phys.s.u-tokyo.ac.jp\/en\/wp-json\/wp\/v2\/tags?post=4593"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}