Approximate Fisher Information Matrix to Characterize the Training of Deep Neural Networks (WITH Zhibin Liao, Ian Reid and Gustavo Carneiro)

This paper looks at how to measure the condition number of the training problem for deep networks, how this is affected by batch size and learning rate, and how this characterises performance in terms of convergence and generalisation.

[PAMI 2018 paper]