Research

High-Quality Visual Data Acquisition

Handling Real-World Data

While synthetically generated datasets have been widely used for training neural networks for image restoration, such networks do not perform well on real-world images due to the significant domain gap between naively synthesized datasets and real-world degradations. To better handle real-world degraded images, we have explored different possibilities such as real-world datasets and realistic degradation synthesis. Our RealBlur dataset, which is a dataset consisting of real-world blurred images and their ground-truth sharp images, is now utilized as a standard benchmark dataset by numerous works.

Rim et al., Real-WorldBlur Dataset for Learning and Benchmarking Deblurring Algorithms, ECCV 2020

Priors and Neural Network Frameworks for Image Restoration

Lee et al., UGPNet: Universal Generative Prior for Image Restoration, WACV 2024

Camera ISPs

Camera ISPs produce visually-pleasing images from RAW data captured by camera sensors. Specifically, camera ISPs perform two tasks: image restoration and enhancement. The goals of our research are twofold. 1) We aim at accurate modeling of real-world camera ISPs so that we can use the model to more accurately model real-world image degradations. 2) We aim at replacing traditional camera ISPs with learnable ISPs to overcome the limitations of traditional camera ISPs.

Kim et al., ParamISP: Learned Forward and Inverse ISPs using Camera Parameters, CVPR 2024

Beyond Single Images

Traditional image restoration tasks mainly focus on restoring a single image that is already captured by using a single camera. On the other hand, recent smartphones provide more than one cameras and additional sensors such as gyro sensors. Moreover, we may also program the way to capture images such as modulating the exposure time. We seek to exploit such possibilities.

Rim et al., Deep Hybrid Camera Deblurring for Smartphone Cameras, SIGGRAPH 2024

Visual Recognition from Low-Quality Images

While many recent visual recognition algorithms perform highly accurately on high-quality images, their performance severely degrade on real-world low-quality images captured in extreme environments. We also work on improving visual recognition performance on such low-quality images.

Lee et al., Human Pose Estimation in Extremely Low-Light Conditions, CVPR 2023

Image Tone/Color Manipulation

Lee et al., CLIPtone: Unsupervised Learning for Text-based Image Tone Adjustment, CVPR 2024

Videos

Past Projects

I have also worked on many other projects on low-level vision that are not based on deep learning. Some of them are listed below.

Cho and Lee, Fast Motion Deblurring, SIGGRAPH Asia 2009

Tech Transfer

Since my Ph.D. study, I’ve been trying to make image deblurring more practical solution for consumer photography. Specifically, I’ve worked on blind deconvolution, non-blind deconvolution, noise & outlier handling, and non-uniform deblurring. Especially, my efficient blind deconvolution method, which was presented at SIGGRAPH Asia 2009, has been proven to be one of the fastest and most reliable methods by several following works by other researchers.

When I was at Adobe Research, I worked on a tech transfer project, which aimed at shipping deblurring technology with Adobe Photoshop. The first version of “Shake Reduction”, which was based on my research, was first released in the summer of 2013 as a new feature of Photoshop CC.

Visual Content Synthesis & Editing

Real-World Image Editing

Artistic Image Synthesis

3D Synthesis

Ryu et al., 360° Reconstruction From a Single Image Using Space Carved Outpainting, SIGGRAPH Asia 2023

Understanding 3D World


Lee et al., Generalizable Novel-View Synthesis using a Stereo Camera, CVPR 2024