融合信息化边界和多模态特征的室内空间布局估计
作者:
基金项目:

国家自然科学基金(61001152,31200747,61071091,61071166,61172118);江苏省自然科学基金(BK2012437);南京邮电大学校级科研基金(NY214037);国家留学基金(201208320219)


Indoor spatial layout estimation using informative edges and multi-modality features
Author:
  • 摘要
  • | |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • | | |
  • 文章评论
    摘要:

    为感知室内空间布局,提出一种基于信息化边界和多模态特征的场景布局估计方法.首先,采用VGG-16全卷积神经网络预测蕴含空间布局先验的信息化边界图.其次,采用Canny边缘检测和投票策略估计水平和竖直方向消失点,从消失点等角度间隔引出射线细采样信息化边界能量高的区域.接着,采用VGG空间多尺度卷积神经网络估计几何深度和法向特征.然后,积分几何求和候选布局多边形中多模特征描述一元共生,候选布局的表面法向平滑和位置关系确定二元标记约束.最后,采用结构化支持向量机学习模型,最大布局候选得分以推理布局.实验结果表明,与经典方法相比,本估计方法可以有效改善布局的完整度.

    Abstract:

    To perceive indoor spatial layout,we present a scene layout estimation method based on informative edges and multi-modality features.First,the VGG-16 full convolutional neural network is applied to predict informative edge map with the prior of spatial layout.Then,Canny edge detection and voting strategy are utilized to estimate the horizontal and vertical vanishing points,while the rays led at equal intervals from the given vanishing points finely resample the divided regions with high informative edge energies for the layout candidates.Next,the spatial multi-scaled VGG-16-based convolutional neural network is adopted to estimate the related geometric depth and normal vectors on the scene surfaces.And then,integral geometry is applied to accumulate the multi-model regional features as unary occurrence potential in the polygons of candidate layouts,and the pairwise label constrains are reflected by surface normal smooth and the location relationship of candidate layouts.Finally,the mode parameters can be learned by structural SVM learning,and the scene layout can be inferred by maximizing the related scores of the layout candidates.Experimental results show that,compared with traditional methods,this proposed estimation method can effectively improve the completeness of the resulting spatial layouts.

    参考文献
    [1] 姚拓中,左文辉,宋加涛,等.结合物体先验和空域约束的室内空域布局推理[J].自动化学报,2017,43(8):1402-1411 YAO Tuozhong,ZUO Wenhui,SONG Jiatao,et al.Estimating spatial layout of cluttered rooms by using object prior and spatial constraints[J].Acta Automatica Sinica,2017,43(8):1402-1411
    [2] 庄严,卢希彬,李云辉,等.移动机器人基于三维激光测距的室内场景认知[J].自动化学报,2011,37(10):1232-1240 ZHUANG Yan,LU Xibin,LI Yunhui,et al.Mobile robot indoor scene cognition using 3D laser scanning[J].Acta Automatica Sinica,2011,37(10):1232-1240
    [3] 刘天亮,冯希龙,顾雁秋,等.一种由粗至精的RGB-D室内场景语义分割方法[J].东南大学学报(自然科学版),2016,46(4):681-687 LIU Tianliang,FENG Xilong,GU Yanqiu,et al.Coarse-to-fine semantic parsing method for RGB-D indoor scenes[J].Journal of Southeast University (Natural Science Edition),2016,46(4):681-687
    [4] Nedovic V,Smeulders A W M,Redert A,et al.Depth information by stage classification[C]//IEEE 11th International Conference on Computer Vision,2007:1-8
    [5] Hedau V,Hoiem D,Forsyth D.Recovering the spatial layout of cluttered rooms[C]//IEEE 12th International Conference on Computer Vision,2009:1849-1856
    [6] Hoiem D,Efros A A,Hebert M.Geometric context from a single image[C]//Tenth IEEE International Conference on Computer Vision (ICCV'05),2005:654-661
    [7] 许宏科,秦严严,陈会茹.一种基于改进Canny的边缘检测算法[J].红外技术,2014,36(3):210-214 XU Hongke,QIN Yanyan,CHEN Huiru.An improved algorithm for edge detection based on Canny[J].Infrared Technology,2014,36(3):210-214
    [8] 梅雪,夏良正,李久贤,等.一种三维场景的消失点检测算法[J].信号处理,2007,23(6):924-926. MEI Xue,XIA Liangzheng,LI Jiuxian,et al.A vanishing point detection algorithm for 3D scene[J].Signal Processing,2007,23(6):924-926
    [9] Lee D C,Hebert M,Kanade T.Geometric reasoning for single image structure recovery[C]//IEEE Conference on Computer Vision and Pattern Recognition,2009:2136-2143
    [10] Ramalingam S,Pillai J K,Jain A,et al.Manhattan junction catalogue for spatial reasoning of indoor scenes[C]//IEEE Conference on Computer Vision and Pattern Recognition,2013:3065-3072
    [11] Zhang J,Kan C,Schwing A G,et al.Estimating the 3D layout of indoor scenes and its clutter from depth sensors[C]//IEEE International Conference on Computer Vision,2013:1273-1280
    [12] Wang H Y,Gould S,Roller D.Discriminative learning with latent variables for cluttered indoor scene understanding[J].Communications of the ACM,2013,56(4):92-99
    [13] Schwing A G,Hazan T,Pollefeys M,et al.Efficient structured prediction for 3D indoor scene understanding[C]//IEEE Conference on Computer Vision and Pattern Recognition,2012:2815-2822
    [14] 吴培良,李亚南,杨芳,等.一种基于CLM的服务机器人室内功能区分类方法[J].机器人,2018,40(2):188-194 WU Peiliang,LI Yanan,YANG Fang,et al.A CLM-based method of indoor affordance areas classification for service robots[J].Robot,2018,40(2):188-194
    [15] Mallya A,Lazebnik S.Learning informative edge maps for indoor scene layout prediction[C]//IEEE International Conference on Computer Vision (ICCV),2015:936-944
    [16] Long J,Shelhamer E,Darrell T.Fully convolutional networks for semantic segmentation[C]//IEEE Conference on Computer Vision and Pattern Recognition (CVPR),2015:3431-3440
    [17] Dasgupta S,Fang K,Chen K,et al.DeLay:robust spatial layout estimation for cluttered indoor scenes[C]//IEEE Conference on Computer Vision and Pattern Recognition (CVPR),2016:616-624
    [18] Eigen D,Fergus R.Predicting depth,surface normals and semantic labels with a common multi-scale convolutional architecture[C]//IEEE International Conference on Computer Vision (ICCV),2015:2650-2658
    [19] Lee D C,Gupta A,Hebert M,et al.Estimating spatial layout of rooms using volumetric reasoning about objects and surfaces[C]//24th Annual Conference on Neural Information Processing Systems,2010:1288-1296
    [20] Tsochantaridis I,Joachims T,Hofmann T,et al.Large margin methods for structured and interdependent output variables[J].Journal of Machine Learning Research,2005,6(2):1453-1484
    引证文献
    网友评论
    网友评论
    分享到微博
    发 布
引用本文

刘天亮,陆泮宇,戴修斌,刘峰,罗杰波.融合信息化边界和多模态特征的室内空间布局估计[J].南京信息工程大学学报(自然科学版),2019,11(6):735-742
LIU Tianliang, LU Panyu, DAI Xiubin, LIU Feng, LUO Jiebo. Indoor spatial layout estimation using informative edges and multi-modality features[J]. Journal of Nanjing University of Information Science & Technology, 2019,11(6):735-742

复制
分享
文章指标
  • 点击次数:794
  • 下载次数: 1814
  • HTML阅读次数: 0
  • 引用次数: 0
历史
  • 收稿日期:2019-10-20
  • 在线发布日期: 2020-01-19

地址:江苏省南京市宁六路219号    邮编:210044

联系电话:025-58731025    E-mail:nxdxb@nuist.edu.cn

南京信息工程大学学报 ® 2025 版权所有  技术支持:北京勤云科技发展有限公司