Research
High-Quality Visual Data Acquisition
Handling Real-World Data
While synthetically generated datasets have been widely used for training neural networks for image restoration, such networks do not perform well on real-world images due to the significant domain gap between naively synthesized datasets and real-world degradations. To better handle real-world degraded images, we have explored different possibilities such as real-world datasets and realistic degradation synthesis. Our RealBlur dataset, which is a dataset consisting of real-world blurred images and their ground-truth sharp images, is now utilized as a standard benchmark dataset by numerous works.
ParamISP: Learned Forward and Inverse ISPs using Camera Parameters
Woohyeok Kim, Geonu Kim, Junyong Lee, Seungyong Lee, Seung-Hwan Baek, Sunghyun Cho
CVPR 2024 [Project]
Realistic Blur Synthesis for Learning Image Deblurring
Jaesung Rim, Geonung Kim, Jungeon Kim, Junyong Lee, Seungyong Lee, Sunghyun Cho
ECCV 2022 [Project]
RSBlur dataset [Benchmarks]
Real-World Blur Dataset for Learning and Benchmarking Deblurring Algorithms
Jaesung Rim, Haeyun Lee, Jucheol Won, Sunghyun Cho
ECCV 2020 [Project]
RealBlur dataset [Benchmarks]
Rim et al., Real-WorldBlur Dataset for Learning and Benchmarking Deblurring Algorithms, ECCV 2020
Priors and Neural Network Frameworks for Image Restoration
UGPNet: Universal Generative Prior for Image Restoration
Hwayoon Lee, Kyoungkook Kang, Hyeongmin Lee, Seung-Hwan Baek, Sunghyun Cho
WACV 2024 [Project]
BigColor: Colorization using a Generative Color Prior for Natural Images
Single Image Defocus Deblurring Using Kernel-Sharing Parallel Atrous Convolutions
Iterative Filter Adaptive Network for Single Image Defocus Deblurring
Junyong Lee, Hyeongseok Son, Jaesung Rim, Sunghyun Cho, Seungyong Lee
CVPR 2021 [PDF] [Supp] [Presentation Video/Poster/Slides] [Code & Data]
RealDOF dataset [Benchmark]
Locally Adaptive Channel Attention-based Network for Denoising Images
Haeyun Lee, Sunghyun Cho
IEEE Access, 2020 [PROJECT]
Lee et al., UGPNet: Universal Generative Prior for Image Restoration, WACV 2024
Camera ISPs
Camera ISPs produce visually-pleasing images from RAW data captured by camera sensors. Specifically, camera ISPs perform two tasks: image restoration and enhancement. The goals of our research are twofold. 1) We aim at accurate modeling of real-world camera ISPs so that we can more accurately model real-world image degradations. 2) We aim at replacing traditional camera ISPs with learnable ISPs for higher-quality imaging.
ParamISP: Learned Forward and Inverse ISPs using Camera Parameters
Woohyeok Kim, Geonu Kim, Junyong Lee, Seungyong Lee, Seung-Hwan Baek, Sunghyun Cho
CVPR 2024 [Project]
CLIPtone: Unsupervised Learning for Text-based Image Tone Adjustment
Hyeongmin Lee, Kyoungkook Kang, Jungseul Ok, Sunghyun Cho
CVPR 2024 [Project]
Kim et al., ParamISP: Learned Forward and Inverse ISPs using Camera Parameters, CVPR 2024
Beyond Single Images
Traditional image restoration tasks mainly focus on restoring a single image that is already captured by using a single camera. On the other hand, recent smartphones provide more than one cameras and additional sensors such as gyro sensors. Moreover, we may also program the way to capture images such as modulating the exposure time. We seek to exploit such possibilities.
Deep Hybrid Camera Deblurring for Smartphone Cameras
Burst Image Super-Resolution with Base Frame Selection
Sanghyun Kim, Min Jung Lee, Woohyeok Kim, Deunsol Jung, Jaesung Rim, Sunghyun Cho, Minsu Cho
NTIRE 2024 (CVPR 2024 Workshop)
Reference-based Video Super-Resolution Using Multi-Camera Video Triplets
Junyong Lee, Myeonghee Lee, Sunghyun Cho, Seungyong Lee
CVPR 2022 [Project]
Rim et al., Deep Hybrid Camera Deblurring for Smartphone Cameras, SIGGRAPH 2024
Visual Recognition from Low-Quality Images
While many recent visual recognition algorithms perform highly accurately on high-quality images, their performance severely degrade on real-world low-quality images captured in extreme environments. We also work on improving visual recognition performance on such low-quality images.
Human Pose Estimation in Extremely Low-Light Conditions
URIE: Universal Image Enhancement for Visual Recognition in the Wild
Taeyoung Son, Juwon Kang, Namyup Kim, Sunghyun Cho, Suha Kwak
ECCV 2020 [ArXiv]
Lee et al., Human Pose Estimation in Extremely Low-Light Conditions, CVPR 2023
Image Tone/Color Manipulation
CLIPtone: Unsupervised Learning for Text-based Image Tone Adjustment
Hyeongmin Lee, Kyoungkook Kang, Jungseul Ok, Sunghyun Cho
CVPR 2024 [Project]
Deep Color Transfer using Histogram Analogy
Junyong Lee, Hyeongseok Son, Gunhee Lee, Jonghyeop Lee, Sunghyun Cho, Seungyong Lee
The Visual Computer Journal (CGI 2020)
Naturalness-Preserving Image Tone Enhancement Using Generative Adversarial Networks
Hyeongseok Son, Gunhee Lee, Sunghyun Cho, Seungyong Lee
Computer Graphics Forum (special issue on Pacific Graphics 2019), 2019 [Paper]
Best Paper Award
Lee et al., CLIPtone: Unsupervised Learning for Text-based Image Tone Adjustment, CVPR 2024
Videos
Real-Time Video Deblurring via Lightweight Motion Compensation
Hyeongseok Son, Junyong Lee, Sunghyun Cho, Seungyong Lee
Computer Graphics Forum (special issue on Pacific Graphics 2022), 2022 [Project]
Reference-based Video Super-Resolution Using Multi-Camera Video Triplets
Recurrent Video Deblurring with Blur-Invariant Motion Estimation and Pixel Volumes
Video Upright Adjustment and Stabilization
Jucheol Won, Sunghyun Cho
BMVC 2019 [PROJECT]
Video Deblurring for Hand-held Cameras Using Patch-based Synthesis
Sunghyun Cho, Jue Wang, Seungyong Lee
ACM Transactions on Graphics (SIGGRAPH 2012) vol. 31, no. 4, article no. 64, July 2012 [PROJECT]
Past Projects
I have also worked on many other projects on low-level vision that are not based on deep learning. Some of them are listed below.
Convergence Analysis of MAP based Blur Kernel Estimation
Sunghyun Cho, Seungyong Lee
ICCV 2017 [PDF] [Supplementary]
Good Image Priors for Non-blind Deconvolution: Generic vs Specific
Libin Sun, Sunghyun Cho, Jue Wang, James Hays
ECCV 2014 [PROJECT]
Deblurring Low-light Images with Light Streaks
Zhe Hu, Sunghyun Cho, Jue Wang, Ming-Hsuan Yang
CVPR 2014 [PROJECT]
Edge-based Blur Kernel Estimation Using Patch Priors
Libin Sun, Sunghyun Cho, Jue Wang, James Hays
ICCP 2013 [PROJECT]
Handling Outliers in Non-blind Image Deconvolution
Fast Motion Deblurring
Sunghyun Cho, Seungyong Lee
ACM Transactions on Graphics (SIGGRAPH ASIA 2009)
vol. 28, no. 5, article no. 145, Dec. 2009 [PROJECT]
Cho and Lee, Fast Motion Deblurring, SIGGRAPH Asia 2009
Tech Transfer
Since my Ph.D. study, I’ve been trying to make image deblurring more practical solution for consumer photography. Specifically, I’ve worked on blind deconvolution, non-blind deconvolution, noise & outlier handling, and non-uniform deblurring. Especially, my efficient blind deconvolution method, which was presented at SIGGRAPH Asia 2009, has been proven to be one of the fastest and most reliable methods by several following works by other researchers.
When I was at Adobe Research, I worked on a tech transfer project, which aimed at shipping deblurring technology with Adobe Photoshop. The first version of “Shake Reduction”, which was based on my research, was first released in the summer of 2013 as a new feature of Photoshop CC.
Visual Content Synthesis & Editing
Real-World Image/Video Editing
RNA:Video Editing with ROI-based Neural Atlas
Jaekyeong Lee, Geonung Kim, Sunghyun Cho
ACCV 2024 [Project]
GAN Inversion for Out-of-Range Images with Geometric Transformations
Image Synthesis
Diffusion Model Compression for Image-to-Image Translation
Geonung Kim, Beomsu Kim, Eunhyeok Park, Sunghyun Cho
ACCV 2024 [Project]
DynaGAN: Dynamic Few-shot Adaptation of GANs to Multiple Domains
Seongtae Kim, Kyoungkook Kang, Geonung Kim, Seung-Hwan Baek, Sunghyun Cho
SIGGRAPH Asia 2022 (Conference Track) [Project]
Dr.3D: Adapting 3D GANs to Artistic Drawings
Wonjoon Jin, Nuri Ryu, Geonung Kim, Seung-Hwan Baek, Sunghyun Cho
SIGGRAPH Asia 2022 (Conference Track) [Project]
3D Synthesis
360° Reconstruction From a Single Image Using Space Carved Outpainting
Nuri Ryu, Minsu Gong, Geonung Kim, Joo-Haeng Lee, Sunghyun Cho
SIGGRAPH Asia 2023 [Project]
3D-Aware Generative Model for Improved Side-View Image Synthesis
Kyungmin Jo, Wonjoon Jin, Jaegul Choo, Hyunjoon Lee, Sunghyun Cho
ICCV 2023 [Project]
Exp-GAN: 3D-Aware Facial Image Generation with Expression Control
Yeonkyeong Lee, Taeho Choi, Hyunsung Go, Hyunjoon Lee, Sunghyun Cho, Junho Kim
ACCV 2022 (Oral presentation) [Paper]
Dr.3D: Adapting 3D GANs to Artistic Drawings
Wonjoon Jin, Nuri Ryu, Geonung Kim, Seung-Hwan Baek, Sunghyun Cho
SIGGRAPH Asia 2022 (Conference Track) [Project]
3D Scene Painting via Semantic Image Synthesis
Jaebong Jeong, Janghun Jo, Sunghyun Cho, Jaesik Park
CVPR 2022 [Project]
Ryu et al., 360° Reconstruction From a Single Image Using Space Carved Outpainting, SIGGRAPH Asia 2023
Understanding 3D World
Generalizable Novel-View Synthesis using a Stereo Camera
Neural Spectro-polarimetric Fields
Youngchan Kim, Wonjoon Jin, Sunghyun Cho, Seung-Hwan Baek
SIGGRAPH Asia 2023 [Project]
ExBluRF: Efficient Radiance Fields for Extreme Motion Blurred Images
CTRL-C: Camera calibration TRansformer with Line-Classification
Lee et al., Generalizable Novel-View Synthesis using a Stereo Camera, CVPR 2024