suvadityamuk/TRELLIS.2-4B-diffusers-3d

TRELLIS.2-4B for diffusers-3d

microsoft/TRELLIS.2-4B converted into a diffusers-3d pipeline.

The DINOv3 image encoder is not included. Its weights are gated under Meta's DINOv3 License, so this repository ships every TRELLIS.2 component except image_encoder/; you accept the license on facebook/dinov3-vitl16-pretrain-lvd1689m and pass the image encoder in when loading.

Install

pip install git+https://github.com/suvadityamuk/diffusers-3d.git
pip install "git+https://github.com/suvadityamuk/diffusers-3d.git#subdirectory=packages/diffusers-3d"

diffusers-3d runs every network in plain PyTorch on CPU or GPU. Rendering Gaussian splats needs the optional gsplat backend; the PBR-textured mesh for TRELLIS.2 needs CuMesh (see the package docs).

Usage

import torch
from transformers import DINOv3ViTModel

from diffusers_3d import AutoPipelineForImageTo3D, ImageCondition

image_encoder = DINOv3ViTModel.from_pretrained("facebook/dinov3-vitl16-pretrain-lvd1689m")
pipeline = AutoPipelineForImageTo3D.from_pretrained(
    "suvadityamuk/TRELLIS.2-4B-diffusers-3d", image_encoder=image_encoder, dtype=torch.bfloat16
).to("cuda")
output = pipeline(ImageCondition(image=rgba), pipeline_type="512")
ovoxel = output.objects[0]  # OVoxelAsset: dual-grid surface with PBR channels

pipeline_type selects the released preset (512, 1024, 1024_cascade, 1536_cascade). formats=("mesh",) returns a vertex-coloured mesh on any device; baking the PBR-textured mesh (pipeline.postprocess_ovoxel(..., output_format="textured_mesh")) needs CuMesh, which bundles cubvh under NVIDIA's non-commercial license; see the package docs.

Components

Folder	Class	Released file
image_encoder	transformers.DINOv3ViTModel	not included (gated facebook/dinov3-vitl16-pretrain-lvd1689m)
sparse_structure_flow_model	Trellis2SparseStructureFlowModel	ss_flow_img_dit_1_3B_64_bf16
sparse_structure_decoder	Trellis2SparseStructureDecoder	ss_dec_conv3d_16l8_fp16 (from TRELLIS-image-large)
shape_slat_flow_model	Trellis2SLatFlowModel	slat_flow_img2shape_dit_1_3B_512_bf16
shape_slat_flow_model_1024	Trellis2SLatFlowModel	slat_flow_img2shape_dit_1_3B_1024_bf16
shape_slat_decoder	Trellis2ShapeDualGridDecoder	shape_dec_next_dc_f16c32_fp16
texture_slat_flow_model	Trellis2SLatFlowModel	slat_flow_imgshape2tex_dit_1_3B_512_bf16
texture_slat_flow_model_1024	Trellis2SLatFlowModel	slat_flow_imgshape2tex_dit_1_3B_1024_bf16
pbr_decoder	Trellis2PBRSparseDecoder	tex_dec_next_dc_f16c32_fp16

Provenance

Converted with diffusers-3d-convert-trellis2 from diffusers-3d 0.1.0.dev0 against TRELLIS.2 revision 75fbf0183001ed9876c8dbb35de6b68552ee08bd. Weight values are unchanged.

License and attribution

TRELLIS.2 weights and architecture: MIT License, Copyright (c) Microsoft Corporation. The sparse-structure decoder comes from TRELLIS (MIT). No DINOv3 weights are redistributed. Not affiliated with or endorsed by Microsoft or Meta.