Dual Back-Projection-Based Internal Learning for Blind Super-Resolution

Abstract

State-of-the-art super-resolution (SR) methods commonly assume that the downscaling kernel (the point spread function of the camera) is a Gaussian kernel. Therefore, the methods are particularly vulnerable to solving the problem of blind SR, which deals with a real low-resolution (LR) image that does not follow the assumption. Recently, to address this issue, several internal learning-based methods, which train an image-specific network using a single input image, have been introduced. In this approach, the blind SR is modeled as a two-stage optimization problem, which conducts downscaling kernel estimation followed by SR network training with the estimated kernel. In this letter, we assume that not only the estimated kernel can be employed for SR network training, but also the super-resolved image can contribute to downscaling kernel estimation. To that end, we propose a unified internal learning-based blind SR method that jointly trains two image-specific networks for 1) downscaling together with kernel estimation and 2) SR. More specifically, we train the two networks to minimize a dual back-projection loss; the SR network is trained to reconstruct a given input image from the LR image generated by the downscaling network, and the downscaling network is trained to downscale the high-resolution image generated by the SR network to be as close as possible to the given input image. Through the complementary training, the downscaling kernel estimation becomes more accurate, resulting in better SR performance. In the experiment, we show that the proposed method outperforms previous two-stage internal learning-based methods in terms of both SR performance and efficiency.