Wasserstein loss for image synthesis and restoration

This paper presents a novel variational approach to imposing statistical constraints on the output of both image generation (typically to perform texture synthesis) and image restoration (for instance, to achieve denoising and inpainting) methods. The empirical distributions of linear or nonlinear image descriptors are imposed to be close to some input distributions by minimizing a Wasserstein loss, i.e., the optimal transport distance between the distributions. We advocate the use of a Wasserstein distance because it is robust when using discrete distributions without the need to resort to kernel estimators. We showcase the use of different descriptors to tackle various image processing applications. These descriptors include linear wavelet-based filtering to account for simple textures, nonlinear sparse coding coefficients for more complicated patterns, and the image gradient to restore sharper contents. For applications to texture synthesis, the input distributions are the empirical distributions computed from an exemplar image. For image denoising and inpainting, the estimation process is more difficult; we propose making use of parametric models, and we show results using generalized Gaussian distributions.