It has been argued that for AI to fundamentally understand the world around us, it must learn to identify and disentangle the underlying explanatory factors hidden in the observed environment of low-level sensory data. In this talk, I will first provide an overview of the recent developments in disentangled representation learning and identify some major trends. I will then present some applications of this powerful concept for robust media processing and understanding in tasks such as image restoration, super-resolution, classification, person re-ID, depth estimation, etc. I will also discuss some future directions.

More often than not, practical application scenarios call for systems to be capable of dealing with input visual signals with low resolution/quality or environmental illumination. This talk will introduce related recent research in super-resolution reconstruction, signal quality assessment, content enhancement, and person re-identification for low-resolution or varying illumination. We will also discuss possible new research attempts to advance the relevant techniques.

Computational imaging refers to sensing the real world with optimally designed, task-specific, multi-modality image sensors and optics which actively probes key visual information. Together with advances in AI algorithms and powerful hardware, computational imaging has significantly improved the image and video quality of mobile phones in many aspects, which not only benefits computer vision tasks but also results in novel hardware, such as AI image sensors, AI ISP (Image Signal Processing) chips, and AI camera systems. In this talk, I will present several latest research results including high quality image restoration and accurate depth estimation from time-of-flight sensors or monocular videos, as well as some latest computational photography products in smart phones including under-display cameras, AI image sensors and AI ISP chips. I will also layout several open challenges and future research directions in this area.

Full-Reference Image Quality Assessment (FR-IQA) metrics such as PSNR, SSIM, and LPIPS have been widely adopted for evaluating image restoration (IR) methods. However, pristine-quality images are usually not available, making inferior No-Reference Image Quality Assessment (NR-IQA) metrics seem to be the only solutions in practical applications. Fortunately, when evaluating image restoration methods, paired degraded and restoration images are generally available. Thus, this paper takes a step forward to develop a Degraded-Reference IQA (DR-IQA) model while respecting its correspondence with FR-IQA metrics. To this end, we adopt a simple encoder-decoder as DR-IQA model, and take paired degraded and restoration images as the input to predict distortion maps guided by FR-IQA metrics. More importantly, due to the diversity and continuous development of image restoration models, it is difficult to make the DR-IQA model learned based on a specific restoration model generalize well to other ones. To address this issue, we augment the DR-IQA training samples by adding the results produced by in-training restoration models. Benefiting from the diversity of training samples, our learned DR-IQA model generalizes well to unseen restoration models. We respectively test our DR-IQA models on various image restoration tasks,e.g., denoising, super-resolution, JPEG deblocking, and complicated degradations, where our method can further close the performance gap between FR-IQA metrics and the state-of-the-art NR-IQA methods. Moreover, experiments also show the effectiveness of our method in performance comparison and model selection of image restoration models without ground-truth clean images. Source code will be made publicly available.