Empirical Bayesian Light-Field Stereo Matching by Robust Pseudo Random Field Modeling

Chao-Tsung Huang

National Tsing Hua University, Department of Electrical Engineering




Light-field stereo matching problems are commonly modeled by Markov Random Fields (MRFs) for statistical inference of depth maps. Nevertheless, most previous approaches did not adapt to image statistics but instead adopted fixed model parameters. They explored explicit vision cues, such as depth consistency and occlusion, to provide local adaptability and enhance depth quality. However, such additional assumptions could end up confining their applicability, e.g. algorithms designed for dense view sampling are not suitable for sparse one. In this paper, we get back to MRF fundamentals and develop an empirical Bayesian framework—Robust Pseudo Random Field—to explore intrinsic statistical cues for broad applicability. Based on pseudo-likelihoods with hidden soft-decision priors, we apply soft expectation-maximization (EM) for good model fitting and perform hard EM for robust depth estimation. We introduce novel pixel difference models to enable such adaptability and robustness simultaneously. Accordingly, we devise a stereo matching algorithm to employ this framework on dense, sparse, and even denoised light fields. It can be applied to both true-color and grey-scale pixels. Experimental results show that it estimates scene-dependent parameters robustly and converges quickly. In terms of depth accuracy and computation speed, it also outperforms state-of-the-art algorithms constantly.


C.-T. Huang, "Empirical Bayesian Light-Field Stereo Matching by Robust Pseudo Random Field Modeling," IEEE Transactions on Pattern Analysis and Machine Intelligence, accepted.
[preprint (24MB)] [appendices (0.2MB)]

C.-T. Huang, "Robust Pseudo Random Fields for Light-Field Stereo Matching," IEEE ICCV 2017 (oral presentation).
[preprint (6MB)]


Executable program
- [rprf_matlab2017a.zip (13MB)]: Compiled by Visual Studio 2012 with MATLAB R2017a dynamic-link library


3x3 test light fields converted from raw light-field data in EPFL dataset  by Lytro Power Tools Beta
[dataset.zip (11MB)]

Supplementary Experimental Results

- Results on Five Crosshair Views with Grey-Scale Intensity [link]

- Results on 17x17 Light Fields [link]


This work was supported by the Ministry of Science and Technology, Taiwan, R.O.C. under Grant No. MOST 103-2218-E-007-008-MY3.



Last update on Feb 22, 2018