Technology Sharing

Conflict between opencv4.6 and opencv4.2 that cv_bridge depends on when running vins-fusion

2024-07-12

한어Русский языкEnglishFrançaisIndonesianSanskrit日本語DeutschPortuguêsΕλληνικάespañolItalianoSuomalainenLatina

The bug that took more than a day to fix was finally solved... I posted it for your reference

1. The debugging process for the conflict between running opencv4.6 and cv_bridge calling 4.2 versions in vins-mono and vins-fusion is recorded as follows:

Thanks to this big guy's code, all the contents can be applied correctly: ubuntu20.04 configuration vins-fusion+opencv4.6
This article is also great, you can refer to: Ubuntu 20.04 running lvi-sam error! Opencv library and cv_bridge package conflict

1. Install OpenCV 4.6.0

1. Download the opencv source code, select the required version opencv 4.6.0, the corresponding extension opencv_contrib 4.6.0, and cv_bridge for bridging ROS and opencv

Release OpenCV 4.6.0 · opencv/opencv · GitHub
Release 4.6.0 · opencv/opencv_contrib · GitHub
https://github.com/ros-perception/vision_opencv/tree/

1.1 Install opencv in /usr/local/opencv-4.6.0, move opencv and its extension modules and enter the working directory
sudo mv ./opencv-4.6.0/ /usr/local/
sudo mv ./opencv_contirb-4.6.0/ /usr/local/
cd /usr/local/opencv-4.6.0/

  • 1
  • 2
  • 3

insert image description here

1.2 Before compiling, determine the installation path of opencv. My installation path is CMAKE_INSTALL_PREFIX=/usr/local/opencv-4.6.0
1.3 Compile opencv 4.6.0, in which I selected off all cuda related functions and temporarily do not use GPU
sudo mkdir build && cd build
cmake -D CMAKE_BUILD_TYPE=RELEASE 
        -D CMAKE_INSTALL_PREFIX=/usr/local/opencv-4.6.0 
        -D OPENCV_EXTRA_MODULES_PATH=../../opencv_contrib-4.6.0/modules 
        -D WITH_CUDA=OFF 
        -D ENABLE_FAST_MATH=OFF 
        -D CUDA_FAST_MATH=OFF 
        -D WITH_CUBLAS=ON 
        -D WITH_LIBV4L=ON 
        -D WITH_GSTREAMER=ON 
        -D WITH_GSTREAMER_0_10=OFF 
        -D WITH_QT=ON 
        -D WITH_OPENGL=ON 
        -D CUDA_NVCC_FLAGS="--expt-relaxed-constexpr" 
        -D WITH_TBB=ON 
        ..

  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16
1.4 If the computer performance is good, you can compile with multiple cores using make -j8. If compatibility errors occur, it is recommended to use make, which is slower.
make -j8
sudo make install

  • 1
  • 2

2. Configure the cv_bridge package

1 Because the data of opencv and ROS need to be converted, the bridge tool cv_bridge is needed. Next, we will introduce how to download, install and configure your own cv_bridge function package.

2. First create a ros workspace

mkdir -r ~/third/cv_bridge_ws/src/
cd ~/cv_bridge_ws/src/

  • 1
  • 2

3. Then, download the source code of the corresponding version. Since it is Ubuntu 20.04, download the noetic version

GitHub - ros-perception/vision_opencv at noetic

4. After downloading, unzip it to the ~/third/cv_bridge_ws/src/ directory. Since we only need cv_bridge, we only need to keep the cv_bridge folder and put it in ~/third/cv_bridge_ws/src/ and then modify the cv_bridge configuration file.insert image description here

insert image description here

4.1 Modify the CMakeLists.txt file: add your opencv installation path before set(_opencv_version 4), probably in line 20
set(Opencv_DIR /usr/local/opencv-4.6.0/build)
set(_opencv_version 4)
find_package(OpenCV 4 QUIET)

  • 1
  • 2
  • 3
4.2 In addition, since the cv_bridge I made myself corresponds to the function package of opencv version 4.6.0, it is best to change the package name. I changed it to my_cv_bridge here.
4.3 The content of the entire cmakelists file is as follows;
cmake_minimum_required(VERSION 3.0.2)
project(my_cv_bridge)

find_package(catkin REQUIRED COMPONENTS rosconsole sensor_msgs)

if(NOT ANDROID)
  find_package(PythonLibs)

  if(PYTHONLIBS_VERSION_STRING VERSION_LESS "3.8")
    # Debian Buster
    find_package(Boost REQUIRED python37)
  else()
    # Ubuntu Focal
    find_package(Boost REQUIRED python)
  endif()
else()
find_package(Boost REQUIRED)
endif()

set(Opencv_DIR /usr/local/opencv-4.6.0/build)
set(_opencv_version 4)
find_package(OpenCV 4 QUIET)
if(NOT OpenCV_FOUND)
  message(STATUS "Did not find OpenCV 4, trying OpenCV 3")
  set(_opencv_version 3)
endif()

find_package(OpenCV ${_opencv_version} REQUIRED
  COMPONENTS
    opencv_core
    opencv_imgproc
    opencv_imgcodecs
  CONFIG
)

catkin_package(
  INCLUDE_DIRS include
  LIBRARIES ${PROJECT_NAME}
  CATKIN_DEPENDS rosconsole sensor_msgs
  DEPENDS OpenCV
  CFG_EXTRAS cv_bridge-extras.cmake
)

catkin_python_setup()

include_directories(include ${Boost_INCLUDE_DIRS} ${OpenCV_INCLUDE_DIRS} ${catkin_INCLUDE_DIRS})
link_directories(${Boost_LIBRARY_DIRS})

if(NOT ANDROID)
add_subdirectory(python)
endif()
add_subdirectory(src)
if(CATKIN_ENABLE_TESTING)
  add_subdirectory(test)
endif()
# install the include folder
install(
  DIRECTORY include/${PROJECT_NAME}/
  DESTINATION ${CATKIN_PACKAGE_INCLUDE_DESTINATION}
)

  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16
  • 17
  • 18
  • 19
  • 20
  • 21
  • 22
  • 23
  • 24
  • 25
  • 26
  • 27
  • 28
  • 29
  • 30
  • 31
  • 32
  • 33
  • 34
  • 35
  • 36
  • 37
  • 38
  • 39
  • 40
  • 41
  • 42
  • 43
  • 44
  • 45
  • 46
  • 47
  • 48
  • 49
  • 50
  • 51
  • 52
  • 53
  • 54
  • 55
  • 56
  • 57
  • 58
  • 59
  • 60
4.5 Modify package.xml and change the function package name
<package format="2">
#  <name>cv_bridge</name>
  <name>my_cv_bridge</name>

  • 1
  • 2
  • 3
4.6 Compile cv_bridge package
cd ~/cv_bridge_ws/
catkin_make

  • 1
  • 2
4.7 If the compilation is successful, the my_cv_bridge package is created. This also means that opencv 4.6.0 has been correctly installed, and the my_cv_bridge package can be correctly connected to the installed opencv version. Finally, we add the path of this package to the ~/.bashrc folder, so that we can call this my_cv_bridge package when compiling vins-fusion later. Open the ~/.bashrc folder and add the following command at the end:
source ~/cv_bridge_ws/devel/setup.bash

  • 1
4.8 Finally, source the current path in the terminal

source ~/.bas在这里插入代码片hrc

3. Compile vins-fusion

1. Enter the vins-fusion workspace cd ~/vins-fusion

2. Modify the vins_estimator/CMakeLists.txt file

2.1 Change cv_bridge to my_cv_bridge
find_package(catkin REQUIRED COMPONENTS
    roscpp
    std_msgs
    geometry_msgs
    nav_msgs
    tf
    cv_bridge_460
    camera_models
    image_transport)

  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
2.2 Add the path of cv_bridge_460 package and the path of opencv-4.6.0
set(my_cv_bridge_DIR "/home/lemontech/thirdlib/cv_bridge_ws/devel/share/my_cv_bridge/cmake")

set(OpenCV_DIR "/usr/local/opencv-4.6.0/bulid/")

  • 1
  • 2
  • 3
2.3 Ensure that the cv_bridge library is correctly installed and linked
include_directories(${catkin_INCLUDE_DIRS} ${CERES_INCLUDE_DIRS} ${my_cv_bridge_INCLUDE_DIRS} ${OpenCV_INCLUDE_DIRS})

  • 1

3 Modify loop_fusion

3.1 CMakeLists.txt file
find_package(catkin REQUIRED COMPONENTS
    roscpp
    std_msgs
    nav_msgs
    camera_models
    my_cv_bridge
    roslib
    )

  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
3.2 Add the path of cv_bridge_460 package and the path of opencv-4.6.0
set(CMAKE_PREFIX_PATH ${CMAKE_PREFIX_PATH} "/home/lemontech/thirdlib/cv_bridge_ws/devel")
set(my_cv_bridge_DIR "/home/lemontech/thirdlib/cv_bridge_ws/devel/share/my_cv_bridge/cmake")
#-DEIGEN_USE_MKL_ALL")
set(CMAKE_CXX_FLAGS_RELEASE "-O3 -Wall -g")

set(OpenCV_DIR "/usr/local/opencv-4.6.0/bulid/")

  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
3.3 Ensure that the cv_bridge library is correctly installed and linked
include_directories(${catkin_INCLUDE_DIRS} ${CERES_INCLUDE_DIRS}  ${EIGEN3_INCLUDE_DIR} ${my_cv_bridge_INCLUDE_DIRS} ${OpenCV_INCLUDE_DIRS})

  • 1
3.4 Complete cmakelist file
cmake_minimum_required(VERSION 2.8.3)
project(loop_fusion)

set(CMAKE_BUILD_TYPE "Release")
set(CMAKE_CXX_FLAGS "-std=c++17")
set(CMAKE_PREFIX_PATH ${CMAKE_PREFIX_PATH} "/home/lemontech/thirdlib/cv_bridge_ws/devel")
set(my_cv_bridge_DIR "/home/lemontech/thirdlib/cv_bridge_ws/devel/share/my_cv_bridge/cmake")
#-DEIGEN_USE_MKL_ALL")
set(CMAKE_CXX_FLAGS_RELEASE "-O3 -Wall -g")

set(OpenCV_DIR "/usr/local/opencv-4.6.0/bulid/")

find_package(catkin REQUIRED COMPONENTS
    roscpp
    std_msgs
    nav_msgs
    camera_models
    my_cv_bridge
    roslib
    )

find_package(OpenCV)


find_package(Ceres REQUIRED)

set(CMAKE_MODULE_PATH ${PROJECT_SOURCE_DIR}/cmake)
find_package(Eigen3)

include_directories(${catkin_INCLUDE_DIRS} ${CERES_INCLUDE_DIRS}  ${EIGEN3_INCLUDE_DIR} ${my_cv_bridge_INCLUDE_DIRS} ${OpenCV_INCLUDE_DIRS})
catkin_package()

add_executable(loop_fusion_node
    src/pose_graph_node.cpp
    src/pose_graph.cpp
    src/keyframe.cpp
    src/utility/CameraPoseVisualization.cpp
    src/ThirdParty/DBoW/BowVector.cpp
    src/ThirdParty/DBoW/FBrief.cpp
    src/ThirdParty/DBoW/FeatureVector.cpp
    src/ThirdParty/DBoW/QueryResults.cpp
    src/ThirdParty/DBoW/ScoringObject.cpp
    src/ThirdParty/DUtils/Random.cpp
    src/ThirdParty/DUtils/Timestamp.cpp
    src/ThirdParty/DVision/BRIEF.cpp
    src/ThirdParty/VocabularyBinary.cpp
    )

target_link_libraries(loop_fusion_node ${catkin_LIBRARIES}  ${OpenCV_LIBS} ${CERES_LIBRARIES}) 

  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16
  • 17
  • 18
  • 19
  • 20
  • 21
  • 22
  • 23
  • 24
  • 25
  • 26
  • 27
  • 28
  • 29
  • 30
  • 31
  • 32
  • 33
  • 34
  • 35
  • 36
  • 37
  • 38
  • 39
  • 40
  • 41
  • 42
  • 43
  • 44
  • 45
  • 46
  • 47
  • 48
  • 49

4. Compilation successful

If the compilation is successful and the my_cv_bridge package in find_package is correctly identified, the above three files can be obtained in the devel file of vins-fusion. It is the same as installing the official cv_bridge, except that the official package is in opt/ros/noetic/…
insert image description here

5. But successful compilation does not mean successful operation...

insert image description here

This error occurs, well, it is because opencv4.2 and 4.6 conflict, but I have clearly replaced cv_bridge and opencv with my own!! Then... I checked cv_bridge-extras.cmake under my_cv_bridge/cmake and found that it still says 4.2!!!

  1. The following is the correct version after modification
    a. (1) Change the version to 4.6
    b. （2）opencv_config_path设置为/usr/local/opencv-4.6.0/lib/cmake/opencv4
    insert image description here

  2. Then, I opened /usr/local/opencv-4.6.0/lib/cmake/opencv4 and found that the set (OpenCV_VERSION 4.2.0) in the figure below was written as 4.2, not 4.6!!! Correct it and recompile

rm -rf build
mkdir build && cd build
cmake ..
make -j8

  • 1
  • 2
  • 3
  • 4

insert image description here

  1. Then recompile the above cv_bridge_ws, and run vins-fusion again to run normally
6. About the uninstallation of opencv4.2 version
  1. To ensure that other versions of the files are not deleted by mistake, you can use a wildcard to delete all 4.2 version library files:
sudo rm /usr/local/lib/libopencv_*4.2*

  • 1

Then there may be related files in /var, and the same deletion method is used.
2. Check the path related to the opencv library. The library link files are all under this file:

sudo find / -iname "*opencv*"

  • 1
  1. Update dynamic link library cache
sudo ldconfig

  • 1
  1. Uninstall the original cv_bridge sudo apt-get remove ros-melodic-cv-bridge, download your own ros corresponding version of cv_bridge and link it to the specified opencv [Is this uninstallable or not? ? I uninstalled it]

7. About cmakelist file

set(CMAKE_CXX_FLAGS "-std=c++17")指定了c++17标准进行编译
set(CMAKE_PREFIX_PATH ${CMAKE_PREFIX_PATH} "/home/lemontech/thirdlib/cv_bridge_ws/devel") 这个设置将/home/lemontech/thirdlib/cv_bridge_ws/devel路径添加到cmake的前缀路径中，以便CMake可以在路径中找到依赖项和包
set(my_cv_bridge_DIR "/home/lemontech/thirdlib/cv_bridge_ws/devel/share/my_cv_bridge/cmake")设置置顶了my_cv_bridge包的CMake配置文件路径，以便CMake可以找到并使用这个包
find_package(catkin REQUIRED COMPONENTS
my_cv_bridge
roscpp
)用于查找并配置指定的catkin包和组件，catkin是ros的构建系统，REQUIRED表示这些组件是必须的，如果缺少，会报错)

  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7

Personal profile

I have devoted myself to the research of technology for more than 30 years. I am proficient in various languages ​​such as Java, Linux, JavaScript, PHP, CSS, etc. I have made many contributions in the field of open source. I have established a developer documentation site to share some problems in technology development for everyone to read.

my contact information

Mail