Improving Denoising Diffusion Models via Simultaneous Estimation of Image and Noise (with Zhenkai Zhang and Krista Ehinger)

This paper shows how to improve denoising diffusion models by having the network predict the image and the noise jointly, rather than predicting just one and recovering the other algebraically. The dual prediction provides a richer training signal and a more stable sampling trajectory, Reformulating the noise schedule in terms of the arc on the unit circle between pure-image and pure-noise states removes singularities and enables the use of higher order ODE solvers such as RK4.

[ACML 2023 paper]