Hassan Ahmed Sial PhD defense. Estimating Light Effects from a Single Image: Deep Architectures and Ground-Truth Generation

September 27, 2021 at 11:00 am by
When: 27th September at 11am
Where: Online
Thesis Director: Dr. Ramon Baldrich & Dr. Maria Vanrell
Thesis Tutor: Dr. Maria Vanrell
More info: “Estimating light effects from a single image: Deep architectures and grouns-truth generation“, de Hassan Ahmed Sial – Universitat Autònoma de Barcelona

Abstract: In this thesis, we explore how to estimate the effects of the light interacting with the scene objects from a single image. To achieve this goal, we focus on recovering intrinsic components like reflectance, shading, or light properties such as color and position using deep architectures. The success of these approaches relies on training on large and diversified image datasets. Therefore, we present several contributions on this such as: (a) a data-augmentation technique; (b) a ground-truth for an existing multi-illuminant dataset; (c) a family of synthetic datasets, SID for Surreal Intrinsic Datasets, with diversified backgrounds and coherent light conditions; and (d) a practical pipeline to create hybrid ground-truths to overcome the complexity of acquiring realistic light conditions in a massive way. In parallel with the creation of datasets, we trained different flexible encoder-decoder deep architectures incorporating physical constraints from the image formation models.

In the last part of the thesis, we apply all the previous experiences to two different problems. Firstly, we create a large hybrid Doc3DShade dataset with real shading and synthetic reflectance under complex illumination conditions, that is used to train a two-stage architecture that improves the character recognition task in complex lighting conditions of unwrapped documents.  Secondly, we tackle the problem of single image scene relighting by extending both, the SID dataset to present stronger shading and shadows effects, and the deep architectures to use intrinsic components to estimate new relit images.

�