跳到主要内容

安装与快速开始

安装​

环境要求​

  • Python ≥ 3.9
  • PyTorch ≥ 2.0.0
  • 支持 CUDA 的 GPU（推荐）

从源码安装​

git clone https://github.com/robbyant/lingbot-depth
cd lingbot-depth

# Install the package (use 'python -m pip' to ensure correct environment)
conda create -n lingbot-depth python=3.9
conda activate lingbot-depth
python -m pip install -e .

快速开始​

推理：

import torch
import cv2
import numpy as np
from mdm.model.v2 import MDMModel

# Load model
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = MDMModel.from_pretrained('robbyant/lingbot-depth-pretrain-vitl-14-v0.5').to(device)

# Load and prepare inputs
image = cv2.cvtColor(cv2.imread('examples/0/rgb.png'), cv2.COLOR_BGR2RGB)
h, w = image.shape[:2]
image = torch.tensor(image / 255, dtype=torch.float32, device=device).permute(2, 0, 1)[None]

depth = cv2.imread('examples/0/raw_depth.png', cv2.IMREAD_UNCHANGED).astype(np.float32) / 1000.0
depth = torch.tensor(depth, dtype=torch.float32, device=device)[None]

intrinsics = np.loadtxt('examples/0/intrinsics.txt')
intrinsics[0] /= w  # Normalize fx and cx by width
intrinsics[1] /= h  # Normalize fy and cy by height
intrinsics = torch.tensor(intrinsics, dtype=torch.float32, device=device)[None]

# Run inference
output = model.infer(
    image,
    depth_in=depth,
    intrinsics=intrinsics)

depth_pred = output['depth']  # Refined depth map
points = output['points']      # 3D point cloud

运行示例：

模型在首次运行时会自动从 Hugging Face 下载（无需手动下载）：

# Basic usage - processes example 0
python example.py

# Use a different example (0-7 available)
python example.py --example 1

# Use depth completion optimized model
python example.py --model robbyant/lingbot-depth-postrain-dc-vitl14-v0.5

# Custom output directory
python example.py --output my_results

# See all options
python example.py --help

该过程会处理示例数据，并将结果保存至 result/（或自定义目录）：

result/
├── rgb.png                 # Input RGB image
├── depth_input.npy        # Input depth (float32, meters)
├── depth_refined.npy      # Refined depth (float32, meters)
├── depth_input.png        # Input depth visualization
├── depth_refined.png      # Refined depth visualization
├── depth_comparison.png   # Side-by-side comparison
└── point_cloud.ply       # 3D point cloud

可用示例： examples/ 目录中包含 8 个示例场景（0-7）。