The report explores the ideas presented in Deep Ensembles: A Loss Landscape Perspective by Stanislav Fort, Huiyi Hu, and Balaji Lakshminarayanan.
In the paper, the authors investigate the question - why do deep ensembles work better than single deep neural networks?
In their investigation, the authors figure out:
Inspired by their findings, in this report, we present several different insights that are useful for understanding the dynamics of deep neural networks in general.