面向室外移动机器人的雷达-图像跨模态检索

Radar-image cross-modal retrieval for outdoor mobile robots

中文关键词  移动机器人  点云  图像  特征提取  深度学习  聚类典型相关分析

英文关键词  mobile robots  point clouds  image  feature extraction  deep learning  cluster-canonical correlation analysis

中文摘要

      移动机器人主要依靠激光雷达采集的点云和摄像机采集的图像信息来感知周围环境. 在极端天气或夜晚的情况下, 摄像机采集图像会受到极大干扰; 本文基于聚类典型相关分析（cluster-CCA）提出一种面向室外移动机器人的雷达图像跨模态检索技术, 首先利用深度学习网络提取点云和图像的特征, 然后使用聚类典型相关分析将两种模态的特征映射到子空间, 最后计算欧氏距离进行检索, 可以从图像数据库中检索得出与点云最相似的图像文件. 本文所提出的方法在KITTI 数据集上进行了验证, 实现了从点云到图像的跨模态检索, 结果验证了cluster-CCA在室外移动机器人雷达图像检索方面应用的有效性.

英文摘要

      Mobile robots mainly rely on point clouds and image information collected by laser radar and cameras to sense the surrounding environment. In extreme weather or at night, the camera captures images will be greatly disturbed; this paper proposes a radar image cross-modal retrieval technique for outdoor mobile robots based on cluster-canonical correlation analysis (cluster-CCA). Firstly, deep learning networks are used to extract the characteristics of the point cloud and the image, then using clustering canonical correlation analysis to map the features of the two modes to the subspace, and finally calculate the Euclidean distance to retrieve and get images, and the image files most similar to the point cloud can be obtained from the image database. The method proposed in this paper is tested on the KITTI data set, and achieves cross-modal retrieval from point cloud to image. The result validates the effectiveness of cluster-CCA in radar image retrieval for outdoor mobile robots.