EPFL and Apple (at 
text
EPFL and Apple (at 
@EPFL-VILAB
mention
EPFL-VILAB
)  just released 4M-21: single any-to-any model that can do anything from text-to-image generation to generating depth masks! 🙀
text
)  just released 4M-21: single any-to-any model that can do anything from text-to-image generation to generating depth masks! 🙀
new_line
4M is a multimodal training framework introduced by Apple and EPFL.
text
4M is a multimodal training framework introduced by Apple and EPFL.
new_line
Resulting model takes image and text and output image and text 🤩
text
Resulting model takes image and text and output image and text 🤩
new_line
new_line
Models: 
text
Models: 
https://huggingface.co/collections/EPFL-VILAB/4m-models-660193abe3faf4b4d98a2742
EPFL-VILAB/4m-models-660193abe3faf4b4d98a2742
collection
resource
https://huggingface.co/collections/EPFL-VILAB/4m-models-660193abe3faf4b4d98a2742
new_line
Demo: 
text
Demo: 
https://huggingface.co/spaces/EPFL-VILAB/4M
EPFL-VILAB/4M
space
resource
https://huggingface.co/spaces/EPFL-VILAB/4M
new_line
Paper: 
text
Paper: 
4M-21: An Any-to-Any Vision Model for Tens of Tasks and Modalities (2406.09406)
https://huggingface.co/papers/2406.09406
2406.09406
paper
resource
https://huggingface.co/papers/2406.09406
new_line
new_line
This model consists of transformer encoder and decoder, where the key to multimodality lies in input and output data:
text
This model consists of transformer encoder and decoder, where the key to multimodality lies in input and output data:
new_line
new_line
input and output tokens are decoded to generate bounding boxes, generated image's pixels, captions and more!
text
input and output tokens are decoded to generate bounding boxes, generated image's pixels, captions and more!
new_line
new_line
This model also learnt to generate canny maps, SAM edges and other things for steerable text-to-image generation 🖼️
text
This model also learnt to generate canny maps, SAM edges and other things for steerable text-to-image generation 🖼️
new_line
new_line
The authors only added image-to-all capabilities for the demo, but you can try to use this model for text-to-image generation as well ☺️
text
The authors only added image-to-all capabilities for the demo, but you can try to use this model for text-to-image generation as well ☺️
new_line
EPFL and Apple (at @EPFL-VILAB)  just released 4M-21: single any-to-any model that can do anything from text-to-image generation to generating depth masks! 🙀
4M is a multimodal training framework introduced by Apple and EPFL.
Resulting model takes image and text and output image and text 🤩

Models: https://huggingface.co/collections/EPFL-VILAB/4m-models-660193abe3faf4b4d98a2742
Demo: https://huggingface.co/spaces/EPFL-VILAB/4M
Paper: https://huggingface.co/papers/2406.09406

This model consists of transformer encoder and decoder, where the key to multimodality lies in input and output data:

input and output tokens are decoded to generate bounding boxes, generated image's pixels, captions and more!

This model also learnt to generate canny maps, SAM edges and other things for steerable text-to-image generation 🖼️

The authors only added image-to-all capabilities for the demo, but you can try to use this model for text-to-image generation as well ☺️

🤯
Tonic YaTharThShaRma999 radames KingNish LeroyDyer Jebadiah Ramikan-BR louisbrulenaudet
👀
Tonic YaTharThShaRma999 Taylor658 radames LeroyDyer Jebadiah Ukdah Ramikan-BR
👍
Tonic YaTharThShaRma999 LeroyDyer Jebadiah Norod78 Ramikan-BR
😎
Tonic YaTharThShaRma999 LeroyDyer Jebadiah Ramikan-BR
🧠
Tonic YaTharThShaRma999 Jebadiah Ramikan-BR
🤝
Ukdah Ramikan-BR
🔥
Ramikan-BR