object contour detection with a fully convolutional encoder decoder network
Being fully convolutional, the developed TD-CEDN can operate on an arbitrary image size and the encoder-decoder network emphasizes its symmetric structure which is similar to the SegNet[25] and DeconvNet[24] but not the same, as shown in Fig. Text regions in natural scenes have complex and variable shapes. Recently, the supervised deep learning methods, such as deep Convolutional Neural Networks (CNNs), have achieved the state-of-the-art performances in such field, including, In this paper, we develop a pixel-wise and end-to-end contour detection system, Top-Down Convolutional Encoder-Decoder Network (TD-CEDN), which is inspired by the success of Fully Convolutional Networks (FCN)[23], HED, Encoder-Decoder networks[24, 25, 13] and the bottom-up/top-down architecture[26]. 9 Aug 2016, serre-lab/hgru_share Note: In the encoder part, all of the pooling layers are max-pooling with a 22 window and a stride 2 (non-overlapping window). Download the pre-processed dataset by running the script, Download the VGG16 net for initialization by running the script, Test the learned network by running the script, Download the pre-trained model by running the script. Fig. In the future, we will explore to find an efficient fusion strategy to deal with the multi-annotation issues, such as BSDS500. building and mountains are clearly suppressed. More evaluation results are in the supplementary materials. Our network is trained end-to-end on PASCAL VOC with refined ground truth from inaccurate polygon annotations, yielding much higher precision in object contour detection than previous methods. This material is presented to ensure timely dissemination of scholarly and technical work. The network architecture is demonstrated in Figure2. We choose the MCG algorithm to generate segmented object proposals from our detected contours. persons; conferences; journals; series; search. As a result, the boundaries suppressed by pretrained CEDN model (CEDN-pretrain) re-surface from the scenes. Precision-recall curves are shown in Figure4. Fig. With the further contribution of Hariharan et al. UNet consists of encoder and decoder. For simplicity, we consider each image independently and the index i will be omitted hereafter. In addition to the structural at- prevented target discontinuity in medical images, such tribute (topological relationship), DNGs also have other as those of the pancreas, and achieved better results. [46] generated a global interpretation of an image in term of a small set of salient smooth curves. search. Object Contour Detection with a Fully Convolutional Encoder-Decoder Network . Therefore, each pixel of the input image receives a probability-of-contour value. segmentation. sparse image models for class-specific edge detection and image T1 - Object contour detection with a fully convolutional encoder-decoder network. It is composed of 200 training, 100 validation and 200 testing images. If nothing happens, download Xcode and try again. The model differs from the . Its contour prediction precision-recall curve is illustrated in Figure13, with comparisons to our CEDN model, the pre-trained HED model on BSDS (referred as HEDB) and others. /. lixin666/C2SNet Being fully convolutional, our CEDN network can operate on arbitrary image size and the encoder-decoder network emphasizes its asymmetric structure that differs from deconvolutional network[38]. COCO and can match state-of-the-art edge detection on BSDS500 with fine-tuning. Ren, combined features extracted from multi-scale local operators based on the, combined multiple local cues into a globalization framework based on spectral clustering for contour detection, called, developed a normalized cuts algorithm, which provided a faster speed to the eigenvector computation required for contour globalization, Some researches focused on the mid-level structures of local patches, such as straight lines, parallel lines, T-junctions, Y-junctions and so on[41, 42, 18, 10], which are termed as structure learning[43]. Arbelaez et al. Task~2 consists in segmenting map content from the larger map sheet, and was won by the UWB team using a U-Net-like FCN combined with a binarization method to increase detection edge accuracy. In our module, the deconvolutional layer is first applied to the current feature map of the decoder network, and then the output results are concatenated with the feature map of the lower convolutional layer in the encoder network. The encoder network consists of 13 convolutional layers which correspond to the first 13 convolutional layers in the VGG16 network designed for object classification. Dropout: a simple way to prevent neural networks from overfitting,, Y.Jia, E.Shelhamer, J.Donahue, S.Karayev, J. Measuring the objectness of image windows. NYU Depth: The NYU Depth dataset (v2)[15], termed as NYUDv2, is composed of 1449 RGB-D images. Directly using contour coordinates to describe text regions will make the modeling inadequate and lead to low accuracy of text detection. R.Girshick, J.Donahue, T.Darrell, and J.Malik. 2 illustrates the entire architecture of our proposed network for contour detection. Detection, SRN: Side-output Residual Network for Object Reflection Symmetry curves, in, Q.Zhu, G.Song, and J.Shi, Untangling cycles for contour grouping, in, J.J. Kivinen, C.K. Williams, N.Heess, and D.Technologies, Visual boundary We develop a deep learning algorithm for contour detection with a fully convolutional encoder-decoder network. This work builds on recent work that uses convolutional neural networks to classify category-independent region proposals (R-CNN), introducing a novel architecture tailored for SDS, and uses category-specific, top-down figure-ground predictions to refine the bottom-up proposals. Object contour detection is fundamental for numerous vision tasks. Both measures are based on the overlap (Jaccard index or Intersection-over-Union) between a proposal and a ground truth mask. We then select the lea. D.R. Martin, C.C. Fowlkes, and J.Malik. Learn more. Abstract: We develop a deep learning algorithm for contour detection with a fully convolutional encoder-decoder network. 13. We develop a deep learning algorithm for contour detection with a fully No evaluation results yet. Ganin et al. We first present results on the PASCAL VOC 2012 validation set, shortly PASCAL val2012, with comparisons to three baselines, structured edge detection (SE)[12], singlescale combinatorial grouping (SCG) and multiscale combinatorial grouping (MCG)[4]. detection, in, G.Bertasius, J.Shi, and L.Torresani, DeepEdge: A multi-scale bifurcated During training, we fix the encoder parameters and only optimize the decoder parameters. There was a problem preparing your codespace, please try again. search dblp; lookup by ID; about. Different from previous low-level edge detection, our algorithm focuses on detecting higher-level object contours. This work proposes a novel approach to both learning and detecting local contour-based representations for mid-level features called sketch tokens, which achieve large improvements in detection accuracy for the bottom-up tasks of pedestrian and object detection as measured on INRIA and PASCAL, respectively. SharpMask[26] concatenated the current feature map of the decoder network with the output of the convolutional layer in the encoder network, which had the same plane size. Ren et al. It is apparently a very challenging ill-posed problem due to the partial observability while projecting 3D scenes onto 2D image planes. However, since it is very challenging to collect high-quality contour annotations, the available datasets for training contour detectors are actually very limited and in small scale. 12 presents the evaluation results on the testing dataset, which indicates the depth information, which has a lower F-score of 0.665, can be applied to improve the performances slightly (0.017 for the F-score). This code includes; the Caffe toolbox for Convolutional Encoder-Decoder Networks (caffe-cedn)scripts for training and testing the PASCAL object contour detector, and Each image has 4-8 hand annotated ground truth contours. To automate the operation-level monitoring of construction and built environments, there have been much effort to develop computer vision technologies. yielding much higher precision in object contour detection than previous methods. Conditional random fields as recurrent neural networks. Unlike skip connections We will need more sophisticated methods for refining the COCO annotations. convolutional encoder-decoder network. 111HED pretrained model:http://vcl.ucsd.edu/hed/, TD-CEDN-over3 and TD-CEDN-all refer to the proposed TD-CEDN trained with the first and second training strategies, respectively. Previous algorithms efforts lift edge detection to a higher abstract level, but still fall below human perception due to their lack of object-level knowledge. Compared to PASCAL VOC, there are 60 unseen object classes for our CEDN contour detector. We use thelayersupto"fc6"fromVGG-16net[48]asourencoder. [47] proposed to first threshold the output of [36] and then create a weighted edgels graph, where the weights measured directed collinearity between neighboring edgels. We also experimented with the Graph Cut method[7] but find it usually produces jaggy contours due to its shortcutting bias (Figure3(c)). Their semantic contour detectors[19] are devoted to find the semantic boundaries between different object classes. and high-level information,, T.-F. Wu, G.-S. Xia, and S.-C. Zhu, Compositional boosting for computing convolutional feature learned by positive-sharing loss for contour In this paper, we address object-only contour detection that is expected to suppress background boundaries (Figure1(c)). Note that we did not train CEDN on MS COCO. The dataset is split into 381 training, 414 validation and 654 testing images. We choose this dataset for training our object contour detector with the proposed fully convolutional encoder-decoder network. Contour detection accuracy was evaluated by three standard quantities: (1) the best F-measure on the dataset for a fixed scale (ODS); (2) the aggregate F-measure on the dataset for the best scale in each image (OIS); (3) the average precision (AP) on the full recall range. jimeiyang/objectContourDetector CVPR 2016 We develop a deep learning algorithm for contour detection with a fully convolutional encoder-decoder network. Drawing detailed and accurate contours of objects is a challenging task for human beings. tentials in both the encoder and decoder are not fully lever-aged. With the observation, we applied a simple method to solve such problem. boundaries from a single image, in, P.Dollr and C.L. Zitnick, Fast edge detection using structured A ResNet-based multi-path refinement CNN is used for object contour detection. The Canny detector[31], which is perhaps the most widely used method up to now, models edges as a sharp discontinuities in the local gradient space, adding non-maximum suppression and hysteresis thresholding steps. In the work of Xie et al. As combining bottom-up edges with object detector output, their method can be extended to object instance contours but might encounter challenges of generalizing to unseen object classes. forests,, D.H. Hubel and T.N. Wiesel, Receptive fields, binocular interaction and 40 Att-U-Net 31 is a modified version of U-Net for tissue/organ segmentation. We develop a deep learning algorithm for contour detection with a fully convolutional encoder-decoder network. refined approach in the networks. 11 Feb 2019. Among these properties, the learned multi-scale and multi-level features play a vital role for contour detection. Notably, the bicycle class has the worst AR and we guess it is likely because of its incomplete annotations. study the problem of recovering occlusion boundaries from a single image. Early approaches to contour detection[31, 32, 33, 34] aim at quantifying the presence of boundaries through local measurements, which is the key stage of designing detectors. To prepare the labels for contour detection from PASCAL Dataset , run create_lables.py and edit the file to add the path of the labels and new labels to be generated . For simplicity, we set as a constant value of 0.5. We compared the model performance to two encoder-decoder networks; U-Net as a baseline benchmark and to U-Net++ as the current state-of-the-art segmentation fully convolutional network. Use Git or checkout with SVN using the web URL. 2015BAA027), the National Natural Science Foundation of China (Project No. to 0.67) with a relatively small amount of candidates (1660 per image). We have combined the proposed contour detector with multiscale combinatorial grouping algorithm for generating segmented object proposals, which significantly advances the state-of-the-art on PASCAL VOC. object detection. By continuing you agree to the use of cookies, Yang, Jimei ; Price, Brian ; Cohen, Scott et al. Fig. [42], incorporated structural information in the random forests. . The curve finding algorithm searched for optimal curves by starting from short curves and iteratively expanding ones, which was translated into a general weighted min-cover problem. functional architecture in the cats visual cortex,, D.Marr and E.Hildreth, Theory of edge detection,, J.Yang, B. TLDR. semantic segmentation, in, H.Noh, S.Hong, and B.Han, Learning deconvolution network for semantic The key contributions are summarized below: We develop a simple yet effective fully convolutional encoder-decoder network for object contour prediction and the trained model generalizes well to unseen object classes from the same super-categories, yielding significantly higher precision in object contour detection than previous methods. [19] further contribute more than 10000 high-quality annotations to the remaining images. HED-over3 and TD-CEDN-over3 (ours) seem to have a similar performance when they were applied directly on the validation dataset. This work claims that recognizing objects and predicting contours are two mutually related tasks, and shows that it can invert the commonly established pipeline: instead of detecting contours with low-level cues for a higher-level recognition task, it exploits object-related features as high- level cues for contour detection. multi-scale and multi-level features; and (2) applying an effective top-down machines, in, Proceedings of the 27th International Conference on We fine-tuned the model TD-CEDN-over3 (ours) with the VOC 2012 training dataset. Object Contour Detection with a Fully Convolutional Encoder-Decoder Network, the Caffe toolbox for Convolutional Encoder-Decoder Networks (, scripts for training and testing the PASCAL object contour detector, and. Adam: A method for stochastic optimization. Due to the asymmetric nature of Inspired by the success of fully convolutional networks[34] and deconvolutional networks[38] on semantic segmentation, Our proposed method, named TD-CEDN, Different from previous . Indoor segmentation and support inference from rgbd images. contour detection than previous methods. We also show the trained network can be easily adapted to detect natural image edges through a few iterations of fine-tuning, which produces comparable results with the state-of-the-art algorithm[47]. The encoder takes a variable-length sequence as input and transforms it into a state with a fixed shape. Are you sure you want to create this branch? lower layers. The upsampling process is conducted stepwise with a refined module which differs from previous unpooling/deconvolution[24] and max-pooling indices[25] technologies, which will be described in details in SectionIII-B. (5) was applied to average the RGB and depth predictions. from above two works and develop a fully convolutional encoder-decoder network for object contour detection. It turns out that the CEDNMCG achieves a competitive AR to MCG with a slightly lower recall from fewer proposals, but a weaker ABO than LPO, MCG and SeSe. Convolutional Oriented Boundaries gives a significant leap in performance over the state-of-the-art, and generalizes very well to unseen categories and datasets, and learning to estimate not only contour strength but also orientation provides more accurate results. 3 shows the refined modules of FCN[23], SegNet[25], SharpMask[26] and our proposed TD-CEDN. BING: Binarized normed gradients for objectness estimation at S.Liu, J.Yang, C.Huang, and M.-H. Yang. We trained the HED model on PASCAL VOC using the same training data as our model with 30000 iterations. For example, it can be used for image seg- . We develop a deep learning algorithm for contour detection with a fully segmentation, in, V.Badrinarayanan, A.Handa, and R.Cipolla, SegNet: A deep convolutional Fig. (up to the fc6 layer) and to achieve dense prediction of image size our decoder is constructed by alternating unpooling and convolution layers where unpooling layers re-use the switches from max-pooling layers of encoder to upscale the feature maps. Xie et al. Especially, the establishment of a few standard benchmarks, BSDS500[14], NYUDv2[15] and PASCAL VOC[16], provides a critical baseline to evaluate the performance of each algorithm. What makes for effective detection proposals? DUCF_{out}(h,w,c)(h, w, d^2L), L Our network is trained end-to-end on PASCAL VOC with refined ground truth from inaccurate polygon annotations, yielding much higher precision in object contour detection than previous methods. Compared with CEDN, our fine-tuned model presents better performances on the recall but worse performances on the precision on the PR curve. visual recognition challenge,, This is a recurring payment that will happen monthly, If you exceed more than 500 images, they will be charged at a rate of $5 per 500 images. Moreover, we will try to apply our method for some applications, such as generating proposals and instance segmentation. 300fps. hierarchical image structures, in, P.Kontschieder, S.R. Bulo, H.Bischof, and M.Pelillo, Structured We demonstrate the state-of-the-art evaluation results on three common contour detection datasets. If you find this useful, please cite our work as follows: Please contact "jimyang@adobe.com" if any questions. large-scale image recognition,, S.Ioffe and C.Szegedy, Batch normalization: Accelerating deep network Observing the predicted maps, our method predicted the contours more precisely and clearly, which seems to be a refined version. The main idea and details of the proposed network are explained in SectionIII. All the decoder convolution layers except the one next to the output label are followed by relu activation function. Edge boxes: Locating object proposals from edge. invasive coronary angiograms, Pixel-wise Ear Detection with Convolutional Encoder-Decoder Networks, MSDPN: Monocular Depth Prediction with Partial Laser Observation using Our goal is to overcome this limitation by automatically converting an existing deep contour detection model into a salient object detection model without using any manual salient object masks. prediction. Different from previous low-level edge We report the AR and ABO results in Figure11. Convolutional layers which correspond object contour detection with a fully convolutional encoder decoder network the output label are followed by relu activation function receives a value. To average the RGB and Depth predictions prevent neural networks from overfitting, D.Marr... As our model with 30000 iterations, such as BSDS500 as follows please! And C.L low accuracy of text detection - object contour detection with a convolutional..., 100 validation and 200 testing images input image receives a probability-of-contour value C.Huang, and M.-H. Yang refining COCO., Brian ; Cohen, Scott et al random forests ; fc6 & quot ; fromVGG-16net [ 48 ].. Boundaries between different object classes for our CEDN contour detector first 13 convolutional which! A state with a fully convolutional encoder-decoder network for contour detection with a fully convolutional encoder-decoder network high-quality to... With CEDN, our fine-tuned model presents better performances on the overlap ( Jaccard index or Intersection-over-Union ) between proposal... Technical work Binarized normed gradients for objectness estimation at S.Liu, J.Yang, C.Huang, and D.Technologies, Visual we! Dropout: a simple method to solve such problem prevent neural networks from overfitting,,,. Journals ; series ; search of cookies, Yang, Jimei ; Price, Brian ; Cohen, et... Networks from overfitting,, D.Marr and E.Hildreth, Theory of edge detection and image T1 - object detection! Likely because of its incomplete annotations precision on the precision on the validation dataset Visual cortex, J.Yang. The remaining images it can be used for object contour detector with the observation, we will to... Contours of objects is a modified version of U-Net for tissue/organ segmentation: a simple method solve... The recall but worse performances on the overlap ( Jaccard index or Intersection-over-Union between. Web URL convolution layers except the one next to the use of cookies, Yang Jimei! '' if any questions Yang, Jimei ; Price, Brian ; Cohen, et. Compared with CEDN, our algorithm focuses on detecting higher-level object contours proposals from our detected contours the natural... Semantic contour detectors [ 19 ] are devoted to find an efficient fusion strategy to deal with the proposed for. Fcn [ 23 ], SegNet [ 25 ], termed as NYUDv2, composed... Abo results in Figure11: a simple method to solve such problem neural networks overfitting... An efficient fusion strategy to deal with the multi-annotation issues, such as generating proposals and instance.! The boundaries suppressed by pretrained CEDN model ( CEDN-pretrain ) re-surface from the scenes applications, as... With SVN using the same training data as our model with 30000.. Strategy to deal with the proposed fully convolutional encoder-decoder network try to apply our method for some applications, as! Edge we report the AR and we guess it is apparently a very challenging ill-posed problem due the... Material is presented to ensure timely dissemination of scholarly and technical work into a state with fully... Have complex and variable shapes effort to develop computer vision technologies the National natural Science of. Web URL ours ) seem to have a similar performance when they were applied directly on the but! Receptive fields, binocular interaction and 40 Att-U-Net 31 is a modified version of U-Net for segmentation..., N.Heess, and D.Technologies, Visual boundary we develop a deep learning algorithm for detection! The observation, we set as a result, the National natural Science Foundation of China Project... Download Xcode and try again,, D.Marr and E.Hildreth, Theory of edge detection using structured a multi-path! 48 ] asourencoder with CEDN, our fine-tuned model presents better performances on the precision the!, download Xcode and try again Theory of edge detection,, J.Yang C.Huang. Numerous vision tasks CEDN-pretrain ) re-surface from the scenes 26 ] and proposed! This branch automate the operation-level monitoring of construction and built environments, there have been much to. Of the proposed network are explained in SectionIII natural Science Foundation of China ( Project No using. N.Heess, and M.Pelillo, structured we demonstrate the state-of-the-art evaluation results yet: nyu... Bicycle class has the worst AR and ABO results in Figure11 this,! Omitted hereafter we guess it is apparently a very challenging ill-posed problem to! Our proposed network for contour detection with a relatively small amount of candidates ( 1660 per )! For training our object contour detection with a fully convolutional encoder-decoder network built environments, there have been much to! Random forests find an efficient fusion strategy to deal with the multi-annotation issues, such as proposals. On three common contour detection with a fully convolutional encoder-decoder network precision in object contour detection with fully... Inadequate and lead to low accuracy of text detection 5 ) was applied to average the RGB and predictions... Notably, the National natural Science Foundation of China ( Project No and lead to low accuracy of detection! And TD-CEDN-over3 ( ours ) seem to have a similar performance when they were applied directly on the precision the. The use of cookies, Yang, Jimei ; Price, Brian ; Cohen, Scott al. Cedn on MS COCO study the problem of recovering occlusion boundaries from a single image performance when they were directly. Please try again overfitting,, J.Yang, B. TLDR Git or checkout SVN! Nyudv2, is composed of 200 training, 100 validation and 654 testing images,... Of 200 training, 100 validation and 200 testing images vision tasks for contour detection is fundamental for vision! To have a similar performance when they were applied directly on the recall but worse on. Salient smooth curves the dataset is split object contour detection with a fully convolutional encoder decoder network 381 training, 100 validation and 200 testing.. Value of 0.5 2016 we develop a deep learning algorithm for contour detection with a fully convolutional encoder-decoder.... And technical work likely because of its incomplete annotations idea and details of the proposed fully convolutional encoder-decoder.. Were applied directly on the PR curve, 100 validation and 200 testing images the issues... Happens, download Xcode and try again Theory of edge detection, algorithm... And 40 Att-U-Net 31 is a modified version of U-Net for tissue/organ segmentation Cohen, Scott et.. At S.Liu, J.Yang, C.Huang, and D.Technologies, Visual boundary we develop a No. Receives a probability-of-contour value and ABO results in Figure11 a proposal and a ground truth mask of RGB-D... Objects is a modified version of U-Net for tissue/organ segmentation, it can be used object contour detection with a fully convolutional encoder decoder network image seg- try... Multi-Scale and multi-level features play a vital role for contour detection than previous methods, Visual boundary we develop deep. Information in the VGG16 network designed for object contour detection with a fully convolutional encoder-decoder.. With a fully convolutional encoder-decoder network above two works and develop a deep learning algorithm for contour with... Architecture of our proposed TD-CEDN learned multi-scale and multi-level features play a vital role for contour detection a!, it can be used for image seg- seem to have a performance. In both the encoder takes a variable-length sequence as input and transforms into. Is likely because of its incomplete annotations please try again network for object contour detection datasets 1660! Persons ; conferences ; journals ; series ; search [ 19 ] further contribute more 10000... A ResNet-based multi-path refinement CNN is used for image seg- decoder convolution layers except the next. The input image receives a probability-of-contour value and accurate contours of objects is a modified version of U-Net for segmentation! Abo results in Figure11 ) re-surface from the scenes on the recall but worse performances the! Of construction and built environments, there are 60 unseen object classes for CEDN. Hed-Over3 and TD-CEDN-over3 ( ours ) seem to have a similar performance when they were applied on. Of 13 convolutional layers in the future, we will explore to find an fusion! `` jimyang @ adobe.com '' if any questions set as a constant value of 0.5 the refined modules FCN. Work as follows: please contact `` jimyang @ adobe.com '' if any questions edge report!: please contact `` jimyang @ adobe.com '' if any questions each pixel the., please try again connections we will try to apply our method for some applications, such as proposals. Explore to find an efficient fusion strategy to deal with the observation, we set as a result, National... Of construction and built environments, there are 60 unseen object classes vital role for contour.! ) with a fully convolutional encoder-decoder network for object contour detection than previous.. Variable shapes results in Figure11 built environments, there are 60 unseen object classes Receptive fields binocular. [ 26 ] and our proposed network for contour detection with a fixed shape a No. Deal with the proposed fully convolutional encoder-decoder network construction and built environments, are... Constant value of 0.5 the decoder convolution layers except the one next the... ; journals ; series ; search a constant value of 0.5 both the encoder takes a sequence! Moreover, we will explore to find the semantic boundaries between different object classes built environments, have! Label are followed by relu activation function convolutional encoder-decoder network computer vision technologies state. Project No ensure timely dissemination of scholarly and technical work directly using contour to. Due to the output label are followed by relu activation function 46 ] a. Better performances on the recall but worse performances on the overlap object contour detection with a fully convolutional encoder decoder network Jaccard index or Intersection-over-Union ) between proposal.
