{"id":113,"date":"2021-02-05T19:18:50","date_gmt":"2021-02-05T19:18:50","guid":{"rendered":"http:\/\/www.aarondefazio.com\/tangentially\/?p=113"},"modified":"2021-02-05T19:18:51","modified_gmt":"2021-02-05T19:18:51","slug":"madgrad-a-high-performance-deep-learning-optimizer","status":"publish","type":"post","link":"https:\/\/www.aarondefazio.com\/tangentially\/?p=113","title":{"rendered":"MADGRAD: A high performance deep learning optimizer"},"content":{"rendered":"<p>I&#8217;ve just open sourced an implementation of the MADGRAD optimizer that I developed together with Samy Jelassi. It out-performs Adam on every problem I&#8217;ve tried it on, and it has generalization performance comparable to SGD, avoiding the overfitting problems of adaptive methods entirely!<\/p>\n<p>Check it out here: <a href=\"https:\/\/github.com\/facebookresearch\/madgrad\">https:\/\/github.com\/facebookresearch\/madgrad<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>I&#8217;ve just open sourced an implementation of the MADGRAD optimizer that I developed together with Samy Jelassi. It out-performs Adam on every problem I&#8217;ve tried it on, and it has generalization performance comparable to SGD, avoiding the overfitting problems of adaptive methods entirely! Check it out here: https:\/\/github.com\/facebookresearch\/madgrad<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"_links":{"self":[{"href":"https:\/\/www.aarondefazio.com\/tangentially\/index.php?rest_route=\/wp\/v2\/posts\/113"}],"collection":[{"href":"https:\/\/www.aarondefazio.com\/tangentially\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.aarondefazio.com\/tangentially\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.aarondefazio.com\/tangentially\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.aarondefazio.com\/tangentially\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=113"}],"version-history":[{"count":2,"href":"https:\/\/www.aarondefazio.com\/tangentially\/index.php?rest_route=\/wp\/v2\/posts\/113\/revisions"}],"predecessor-version":[{"id":115,"href":"https:\/\/www.aarondefazio.com\/tangentially\/index.php?rest_route=\/wp\/v2\/posts\/113\/revisions\/115"}],"wp:attachment":[{"href":"https:\/\/www.aarondefazio.com\/tangentially\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=113"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.aarondefazio.com\/tangentially\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=113"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.aarondefazio.com\/tangentially\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=113"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}