Deep Learning MRF for Image Segmentation

Deep Learning Markov Random Field for Semantic Segmentation

We address semantic image segmentation by incorporating rich information into Markov Random Field (MRF), including high-order relations and mixture of label contexts. Unlike previous works that optimized MRFs using iterative algorithm, we solve MRF by proposing a Convolutional Neural Network (CNN), namely Deep Parsing Network (DPN), which enables deterministic end-to-end computation in a single forward pass. Specifically, DPN extends a contemporary CNN architecture to model unary terms and additional layers are carefully devised to approximate the mean field algorithm (MF) for pairwise terms. It has several appealing properties. First, different from the recent works that combined CNN and MRF, where many iterations of MF were required for each training image during back-propagation, DPN is able to achieve high performance by approximating one iteration of MF. Second, DPN represents various types of pairwise terms, making many existing works as its special cases. Third, DPN makes MF easier to be parallelized and speeded up in Graphical Processing Unit (GPU).

An oral talk can be found here.

Citation

@inproceedings{liu2015dpn,
 author = {Ziwei Liu and Xiaoxiao Li and Ping Luo and Chen Change Loy and Xiaoou Tang},
 title = {Semantic Image Segmentation via Deep Parsing Network},
 booktitle = {Proceedings of International Conference on Computer Vision (ICCV)},
 month = {December},
 year = {2015} 
}

@article{liu2017deep,
  author={Ziwei Liu and Xiaoxiao Li and Ping Luo and Chen Change Loy and Xiaoou Tang},
  title={Deep Learning Markov Random Field for Semantic Segmentation},
  journal={TPAMI},
  year={2017}
}