ubuntu18.04部署DXSLAM，CNN+VSLAM，CPU实时运行

一、下载源代码

打开终端输入命令克隆仓库

git clone https://github.com/raulmur/DXSLAM.git DXSLAM

二、配置环境

We have tested the library in Ubuntu 16.04 and Ubuntu 18.04, but it should be easy to compile in other platforms.

• C++11 or C++0x Compiler
• Pangolin
• OpenCV
• Eigen3
• Dbow、Fbow and g2o (Included in Thirdparty folder)
• tensorflow(1.12)

作者提供了一个脚本build.sh来编译Thirdparty目录下的库以及DXSLAM库libDXSLAM.so。像Pangolin、OpenCV和Eigen3这些必备库直接点链接跳转到安装教程进行安装这里不再赘述。所以我们需要安装的环境只有tensorflow1.12其他所需的库都在Thirdparty目录下。

首先你需要安装深度学习工具anacondaubuntu安装anaconda

安装好anaconda后输入下面命令回车然后输入y回车下载安装python3.6的环境

conda create -n tf112 python=3.6

python环境安装完成后输入conda activate tf112激活环境

输入下面命令安装tensorflow1.12

pip install tensorflow==1.12

安装过程中可能出现下面错误

protobuf requires Python '>=3.7' but the running Python is 3.6.2

原因是pip版本太低升级pip

pip install --upgrade pip

再次输入命令安装tensorflow1.12

pip install tensorflow==1.12

可能会遇到下面错误

Collecting tensorflow==1.12
  Using cached tensorflow-1.12.0-cp36-cp36m-manylinux1_x86_64.whl (83.1 MB)
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='pypi.org', port=443): Read timed out. (read timeout=15)",)': /simple/grpcio/

ERROR: Could not find a version that satisfies the requirement grpcio>=1.8.6 (from tensorflow) (from versions: none)
ERROR: No matching distribution found for grpcio>=1.8.6

原因是未安装numpy或者numpy版本不是最新–upgrade代表如果你没有安装则安装安装则更新

pip install --upgrade numpy

在pip install xxx的过程中你可能会遇到下面的报错不是因为你网速慢而是需要使用国内的镜像源最近墙加厚了导致正常的外网都快链接不上了。解决方法就是指定中科大镜像源pip install xxx -i https://pypi.mirrors.ustc.edu.cn/simple/

pip._vendor.urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Read timed out.

安装好numpy之后再次输入命令安装tensorflow1.12

pip install tensorflow==1.12

这里我指定了N个镜像源要么连接超时要么就没有1.12版本的资源。最后不指定源了挂梯子都3.4KB/s真的蚌埠住了那就耐心等待吧。

终于能正常下载好所有依赖时报错无法安装tensorflow原因是还有依赖库未安装解决方法就是pip install h5py -i https://pypi.tuna.tsinghua.edu.cn/simple/

ERROR: Cannot install tensorflow because these package versions have conflicting dependencies.

The conflict is caused by:
    keras-applications 1.0.8 depends on h5py
    keras-applications 1.0.7 depends on h5py
    keras-applications 1.0.6 depends on h5py

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/user_guide/#fixing-conflicting-dependencies

终于历经千辛万苦几十次install安装成功了我只能说tensorflow你迟早被淘汰怪不得我们都用pytorch人家pytorch安装多神速。

三、编译源代码

cd dxslam
chmod +x build.sh
./build.sh

这里第三方库DBoW2、g2o、cnpy、fbow安装非常快脚本中编译都改成make -j4这没什么好讲的。主要是词袋和神经网络模型这两个需要从github下载这里就没有什么镜像源了挂上梯子都得碰运气国外节点连接不稳定比如我现在就是6.4KB/s。

Download and Uncompress vocabulary

Download and Uncompress hf-net

下载好之后按照脚本中的指令解压到指定目录。然后注释掉build.sh中下载&解压词袋和模型的这两段bash再执行./build.sh

编译真的太快了30s不到就OK了生成目标可执行文件rgbd_tum。

四、下载数据集

从TUM RGB-D Dataset Download下载一个序列并解压。在DXSLAM论文的Evolution章节中做完整系统实验时得出如下一张表格

所以我们选择fr3下的Sequence下载即可这里我选择fr3_walking_xyz序列。

用python脚本 associate.py关联RGB图像和depth图像PATH_TO_SEQUENCE是序列所在目录

python associate.py PATH_TO_SEQUENCE/rgb.txt PATH_TO_SEQUENCE/depth.txt > PATH_TO_SEQUENCE/associations.txt

下面我们要先从数据集的rgb图像中获得关键点、局部描述子和全局描述子使用HF_Net神经网络命令格式如下参数1是rgb的目录参数2是输出目录。确保是在tensorflow1.12的环境下conda activate tf112

cd hf-net
python3 getFeature.py image/path/to/rgb output/feature/path

过程中出现模块未找到的错误所以pip install opencv-python安装下opencv2

ModuleNotFoundError: No module named 'cv2'

当然安装opencv-python绝对不会一帆风顺遇到一直卡在build wheel的问题

Building wheels for collected packages: opencv-python
Building wheel for opencv-python (pyproject.toml) 

搜索了30多个链接后确定解决方法从清华源上手动下载wheel注意是python3.6版本的然后cd到wheel所在目录

 pip install opencv_python-3.2.0.8-cp36-cp36m-manylinux1_x86_64.whl 

只需要1s就安装好了但前面的准备工作花了1h。

Processing ./opencv_python-3.2.0.8-cp36-cp36m-manylinux1_x86_64.whl
Requirement already satisfied: numpy>=1.11.3 in /home/dzh/anaconda3/envs/tf112/lib/python3.6/site-packages (from opencv-python==3.2.0.8) (1.19.5)
Installing collected packages: opencv-python
Successfully installed opencv-python-3.2.0.8

再次使用HF-Net输出信息

cd hf-net
python3 getFeature.py image/path/to/rgb output/feature/path

程序执行完成后在feature目录下生成了3个目录des、glb、point-txt分别是local_descriptors、global_descriptor、 keypoints。

在程序结束后你的光标变成了十字架而且不能动了对不对
知道为什么吗是因为将terminal当成了python运行环境直接输入import命令就会卡死。解决方法就是新建终端找到import的进程号然后kill掉即可。

(base) dzh@dzh-Lenovo-Legion-Y7000:~$ ps -A|grep import
15124 pts/0    00:00:00 import
(base) dzh@dzh-Lenovo-Legion-Y7000:~$ kill -9 15124

至此所有准备工作算是做完了你花了how many hours

五、运行系统

执行下面的命令参数TUMX.yaml是我们下载的数据集的相机参数文件PATH_TO_SEQUENCE_FOLDER是数据集目录ASSOCIATIONS_FILE是关联的文件目录OUTPUT/FEATURE/PATH是我们刚刚生成的HF-Net信息目录。

./Examples/RGB-D/rgbd_tum Vocabulary/DXSLAM.fbow Examples/RGB-D/TUMX.yaml PATH_TO_SEQUENCE_FOLDER ASSOCIATIONS_FILE OUTPUT/FEATURE/PATH

下面这是我根据我的目录所执行的指令包括我所在的python环境和所在目录都一目了然。

(tf112) dzh@dzh-Lenovo-Legion-Y7000:~/slambook/dxslam$ ./Examples/RGB-D/rgbd_tum  Vocabulary/DXSLAM.fbow ../TUM_DataSet/TUM3.yaml ../TUM_DataSet/rgbd_dataset_freiburg3_walking_xyz/ ../TUM_DataSet/rgbd_dataset_freiburg3_walking_xyz/associations.txt ./hf-net/feature/

运行效果如图所示其实录制视频效果更好这里就展示运行时的截图使用的还是ORB-SLAM2的Map Viewer

这是运行时的终端日志默认是SLAM模式可以看到在跟踪相机位姿丢失后会重置局部地图、闭环和数据库每次会输出与局部地图匹配的关键点数量。每帧89ms也就是10fps使用CPU跑的话还是满足要求的。

match numbers: 453
nmatchesMap: 280
match numbers: 411
nmatchesMap: 258
virtual int g2o::SparseOptimizer::optimize(int, bool): 0 vertices to optimize, maybe forgot to call initializeOptimization()
match numbers: 474
nmatchesMap: 0
Track lost soon after initialisation, reseting...
System Reseting
Reseting Local Mapper... done
Reseting Loop Closing... done
Reseting Database... done
New map created with 338 points
match numbers: 452
nmatchesMap: 278
match numbers: 430
nmatchesMap: 260
match numbers: 422
nmatchesMap: 255
match numbers: 400
nmatchesMap: 239
match numbers: 369
nmatchesMap: 219
-------

median tracking time: 0.0919971
mean tracking time: 0.0890905

Saving keyframe trajectory to KeyFrameTrajectory.txt ...

trajectory saved!

可以处理你自己的相机序列

You will need to create a settings file with the calibration of your camera. See the settings file provided for the TUM RGB-D cameras. We use the calibration model of OpenCV. RGB-D input must be synchronized and depth registered.

SLAM 和 Localization 模式

You can change between the SLAM and Localization mode using the GUI of the map viewer.

• SLAM 模式
  This is the default mode. The system runs in parallal three threads: Tracking, Local Mapping and Loop Closing. The system localizes the camera, builds new map and tries to close loops.

• Localization 模式
  This mode can be used when you have a good map of your working area. In this mode the Local Mapping and Loop Closing are deactivated. The system localizes the camera in the map (which is no longer updated), using relocalization if needed.

配置的过程正常都会遇到各种各样的问题而解决问题的过程是最有价值的。