Daniellesry/DKT-Depth-14B

Diffusion Knows Transparency (DKT)

DKT is a foundation model for transparent-object, in-the-wild, and arbitrary-length video depth and normal estimation. It repurposes video diffusion priors to achieve robust and temporally coherent perception for challenging real-world scenarios involving glass, plastic, and metal materials.

Paper | Project Page | GitHub

Introduction

Transparent objects pose a significant challenge for traditional perception systems due to refraction and reflection. DKT leverages modern video diffusion models that have internalized optical rules, using lightweight LoRA adapters to predict depth and normals. It yields temporally consistent predictions for arbitrary-length input videos and achieves state-of-the-art results on benchmarks like ClearPose and DREDS.

Usage

To use this model, please follow the installation instructions in the official GitHub repository.

from dkt.pipelines.pipelines import DKTPipeline
import os
from tools.common_utils import save_video

# Initialize the pipeline
pipe = DKTPipeline()

# Define input path
demo_path = 'examples/1.mp4'

# Run inference
prediction = pipe(demo_path)

# Save result
save_dir = 'logs'
os.makedirs(save_dir, exist_ok=True)
output_path = os.path.join(save_dir, 'demo.mp4')
save_video(prediction['colored_depth_map'], output_path, fps=25)

Citation

@article{dkt2025,
  title   = {Diffusion Knows Transparency: Repurposing Video Diffusion for Transparent Object Depth and Normal Estimation},
  author  = {Shaocong Xu and Songlin Wei and Qizhe Wei and Zheng Geng and Hong Li and Licheng Shen and Qianpu Sun and Shu Han and Bin Ma and Bohan Li and Chongjie Ye and Yuhang Zheng and Nan Wang and Saining Zhang and Hao Zhao},
  journal = {https://arxiv.org/abs/2512.23705},
  year    = {2025}
}