DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs. In this work we address the task of semantic image segmentation with Deep Learning and make three main contributions that are experimentally shown to have substantial practical merit. First, we highlight convolution with upsampled filters, or ’atrous convolution’, as a powerful tool in dense prediction tasks. Atrous convolution allows us to explicitly control the resolution at which feature responses are computed within Deep Convolutional Neural Networks. It also allows us to effectively enlarge the field of view of filters to incorporate larger context without increasing the number of parameters or the amount of computation. Second, we propose atrous spatial pyramid pooling (ASPP) to robustly segment objects at multiple scales. ASPP probes an incoming convolutional feature layer with filters at multiple sampling rates and effective fields-of-views, thus capturing objects as well as image context at multiple scales. Third, we improve the localization of object boundaries by combining methods from DCNNs and probabilistic graphical models. The commonly deployed combination of max-pooling and downsampling in DCNNs achieves invariance but has a toll on localization accuracy. We overcome this by combining the responses at the final DCNN layer with a fully connected Conditional Random Field (CRF), which is shown both qualitatively and quantitatively to improve localization performance. Our proposed ”DeepLab” system sets the new state-of-art at the PASCAL VOC-2012 semantic image segmentation task, reaching 79.7% mIOU in the test set, and advances the results on three other datasets: PASCAL-Context, PASCAL-Person-Part, and Cityscapes. All of our code is made publicly available online.

References in zbMATH (referenced in 13 articles )

Showing results 1 to 13 of 13.
Sorted by year (citations)

  1. Kalsi, Jaspreet Singh; Azam, Muhammad; Bouguila, Nizar: Color image segmentation using semi-bounded finite mixture models by incorporating mean templates (2020)
  2. Lindeberg, Tony: Provably scale-covariant continuous hierarchical networks based on scale-normalized differential expressions coupled in cascade (2020)
  3. Liu, Peng; Song, Yan: Segmentation of sonar imagery using convolutional neural networks and Markov random field (2020)
  4. Wang, Faqiang; Zhao, Cuicui; Liu, Jun; Huang, Haiyang: A variational image segmentation model based on normalized cut with adaptive similarity and spatial regularization (2020)
  5. Zhuang, Huiping; Lin, Zhiping; Toh, Kar-Ann: Training a multilayer network with low-memory kernel-and-range projection (2020)
  6. Falcão, Alexandre; Bragantini, Jordão: The role of optimum connectivity in image segmentation: can the algorithm learn object information during the process? (2019)
  7. Fan, Yuwei; Feliu-Fabà, Jordi; Lin, Lin; Ying, Lexing; Zepeda-Núñez, Leonardo: A multiscale neural network based on hierarchical nested bases (2019)
  8. Fan, Yuwei; Lin, Lin; Ying, Lexing; Zepeda-Núñez, Leonardo: A multiscale neural network based on hierarchical matrices (2019)
  9. Joy, Thomas; Desmaison, Alban; Ajanthan, Thalaiyasingam; Bunel, Rudy; Salzmann, Mathieu; Kohli, Pushmeet; Torr, Philip H. S.; Kumar, M. Pawan: Efficient relaxations for dense CRFs with sparse higher-order potentials (2019)
  10. Mondal, Ranjan; Purkait, Pulak; Santra, Sanchayan; Chanda, Bhabatosh: Morphological networks for image de-raining (2019)
  11. Tripp, Bryan: Approximating the architecture of visual cortex in a convolutional network (2019)
  12. Zhang, Yuanping; Tang, Yuanyan; Fang, Bin; Shang, Zhaowei: Multi-object tracking using deformable convolution networks with tracklets updating (2019)
  13. Larsson, Måns; Arnab, Anurag; Zheng, Shuai; Torr, Philip; Kahl, Fredrik: Revisiting deep structured models for pixel-level labeling with gradient-based inference (2018)