代码收藏家 FoundationPose复现指南
目录
intro
论文我就不多说了
code: https://github.com/NVlabs/FoundationPose
paper: https://arxiv.org/abs/2312.08344
1 电脑配置
- Linux 环境 ：ubuntu 22.04
- cuda 版本： 11.8
- 显卡：GeForce RTX 4080
- IDE：pycharm
2 环境配置
2.1 资源下载
git clone https://github.com/NVlabs/FoundationPose.git
必须下载
权重
主目录下新建./weights/，放在该路径下
测试数据
主目录下新建./demo_data/，放在该路径下
可选下载
数据集
太大了，我没下载
model-free few-shot version
2.2 conda环境
这里参考blog
cd $HOME && wget -q https://gitlab.com/libeigen/eigen/-/archive/3.4.0/eigen-3.4.0.tar.gz && \
tar -xzf eigen-3.4.0.tar.gz && \
cd eigen-3.4.0 && mkdir build && cd build
cmake .. -Wno-dev -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_FLAGS=-std=c++14 ..
sudo make install
cd $HOME && rm -rf eigen-3.4.0 eigen-3.4.0.tar.gz
建议不要使用官方github上的，我用下面的路径一直配置不对
# Install Eigen3 3.4.0 under conda environment
conda install conda-forge::eigen=3.4.0
export CMAKE_PREFIX_PATH="$CMAKE_PREFIX_PATH:/eigen/path/under/conda"
我这里都是在pycharm的终端中进行（个人习惯）
# create conda environment
conda create -n foundationpose python=3.9
# activate conda environment
conda activate foundationpose
# install dependencies
python -m pip install -r requirements.txt
下面的貌似自动给安装好了
# Install NVDiffRast
python -m pip install --quiet --no-cache-dir git+https://github.com/NVlabs/nvdiffrast.git
# Kaolin (Optional, needed if running model-free setup)
python -m pip install --quiet --no-cache-dir kaolin==0.15.0 -f https://nvidia-kaolin.s3.us-east-2.amazonaws.com/torch-2.0.0_cu118.html
# PyTorch3D
python -m pip install --quiet --no-index --no-cache-dir pytorch3d -f https://dl.fbaipublicfiles.com/pytorch3d/packaging/wheels/py39_cu118_pyt200/download.html
这里可以检查下自己的torch、pytorch3d版本，终端输入python
Python 3.9.19 (main, May 6 2024, 19:43:03)
[GCC 11.2.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch,torchvision,pytorch3d
>>> torch.__version__
'2.0.0+cu118'
>>> torchvision.__version__
'0.15.1+cu118'
>>> pytorch3d.__version__
'0.7.3'
>>> exit()
-构建C++扩展
# Build extensions
CMAKE_PREFIX_PATH=$CONDA_PREFIX/lib/python3.9/site-packages/pybind11/share/cmake/pybind11 bash build_all_conda.sh
这里会花费比较长时间，并且可能会有各种Warning，只要最后能到100%，无视
最后成功的输出
[100%] Linking CXX shared module mycpp.cpython-39-x86_64-linux-gnu.so
lto-wrapper: warning: using serial compilation of 2 LTRANS jobs
[100%] Built target mycpp
Obtaining file:///home/user/code/embodied_AI/FoundationPose/bundlesdf/mycuda
Preparing metadata (setup.py) ... done
Installing collected packages: common
Running setup.py develop for common
Successfully installed common-0.0.0
3 测试
3.1 初始demo
python run_demo.py
成功弹出
4 补充
4.1 cuda版本问题
我之前的所有代码都是在cuda11.6基础上的，所以这里需要进行一个cuda版本切换，否则会很麻烦（连环报错）
按照blog一步步来
官方下载11.8
wget https://developer.download.nvidia.com/compute/cuda/11.8.0/local_installers/cuda_11.8.0_520.61.05_linux.run
sudo sh cuda_11.8.0_520.61.05_linux.run
记得再下载cudnn
更换~/.bashrc里的cuda路径并且source
在虚拟环境终端下nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Wed_Sep_21_10:33:58_PDT_2022
Cuda compilation tools, release 11.8, V11.8.89
Build cuda_11.8.r11.8/compiler.31833905_
这里我还在conda中进行了下面操作
conda install cudatoolkit=11.8
成功
4.2 Build extensions报错
bash: build_all_conda.sh: 没有那个文件或目录
原因：路径没有在FoundationPose下
在终端使用下面代码重新安装
cd $HOME && wget -q https://gitlab.com/libeigen/eigen/-/archive/3.4.0/eigen-3.4.0.tar.gz && \
tar -xzf eigen-3.4.0.tar.gz && \
cd eigen-3.4.0 && mkdir build && cd build
cmake .. -Wno-dev -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_FLAGS=-std=c++14 ..
sudo make install
cd $HOME && rm -rf eigen-3.4.0 eigen-3.4.0.tar.gz
如果cuda与torch会pytorch3d的版本不对应，也会报错，所以强烈建议按照官方的版本来，为自己11.6就是遇到各种问题换成11.8直接都解决
4.3 run_demo.py报错
packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/home/zqy/anaconda3/envs/foundationpose/lib/python3.9/site-packages/torch/nn/modules/transformer.py", line 437, in forward
return torch._transformer_encoder_layer_fwd(
RuntimeError: expected scalar type Half but found Float
原因：torch的版本太低，切换成2.0以上就不会遇到了
作者：AN小Y
