NVIDIA just dropped a gigantic multimodal model called NVLM 72B 🦖 
text
NVIDIA just dropped a gigantic multimodal model called NVLM 72B 🦖 
new_line
https://huggingface.co/nvidia/NVLM-D-72B
nvidia/NVLM-D-72B
model
resource
https://huggingface.co/nvidia/NVLM-D-72B
new_line
Paper page 
text
Paper page 
NVLM: Open Frontier-Class Multimodal LLMs (2409.11402)
https://huggingface.co/papers/2409.11402
2409.11402
paper
resource
https://huggingface.co/papers/2409.11402
new_line
new_line
The paper contains many ablation studies on various ways to use the LLM backbone 👇🏻
text
The paper contains many ablation studies on various ways to use the LLM backbone 👇🏻
new_line
new_line
🦩 Flamingo-like cross-attention (NVLM-X)
text
🦩 Flamingo-like cross-attention (NVLM-X)
new_line
🌋 Llava-like concatenation of image and text embeddings to a decoder-only model (NVLM-D)
text
🌋 Llava-like concatenation of image and text embeddings to a decoder-only model (NVLM-D)
new_line
✨ a hybrid architecture (NVLM-H)
text
✨ a hybrid architecture (NVLM-H)
new_line
new_line
Checking evaluations, NVLM-D and NVLM-H are best or second best compared to other models 👏
text
Checking evaluations, NVLM-D and NVLM-H are best or second best compared to other models 👏
new_line
new_line
The released model is NVLM-D based on Qwen-2 Instruct, aligned with InternViT-6B using a huge mixture of different datasets
text
The released model is NVLM-D based on Qwen-2 Instruct, aligned with InternViT-6B using a huge mixture of different datasets
new_line
new_line
You can easily use this model by loading it through transformers' AutoModel 😍
text
You can easily use this model by loading it through transformers' AutoModel 😍
NVIDIA just dropped a gigantic multimodal model called NVLM 72B 🦖 
https://huggingface.co/nvidia/NVLM-D-72B
Paper page https://huggingface.co/papers/2409.11402

The paper contains many ablation studies on various ways to use the LLM backbone 👇🏻

🦩 Flamingo-like cross-attention (NVLM-X)
🌋 Llava-like concatenation of image and text embeddings to a decoder-only model (NVLM-D)
✨ a hybrid architecture (NVLM-H)

Checking evaluations, NVLM-D and NVLM-H are best or second best compared to other models 👏

The released model is NVLM-D based on Qwen-2 Instruct, aligned with InternViT-6B using a huge mixture of different datasets

You can easily use this model by loading it through transformers' AutoModel 😍
🔥
louisbrulenaudet iojvsuynv KingNish DmitryRyumin namelessai AtAndDev John6666 garrethlee osanseviero djuna gizemsarsinlar