The Stable Diffusion 3 research paper broken down, including some overlooked details! 📝
text
The Stable Diffusion 3 research paper broken down, including some overlooked details! 📝
new_line
new_line
Model
text
Model
new_line
📏 2 base model variants mentioned: 2B and 8B sizes
text
📏 2 base model variants mentioned: 2B and 8B sizes
new_line
new_line
📐 New architecture in all abstraction levels: 
text
📐 New architecture in all abstraction levels: 
new_line
- 🔽 UNet; ⬆️ Multimodal Diffusion Transformer, bye cross attention 👋
text
- 🔽 UNet; ⬆️ Multimodal Diffusion Transformer, bye cross attention 👋
new_line
- 🆕 Rectified flows for the diffusion process
text
- 🆕 Rectified flows for the diffusion process
new_line
- 🧩 Still a Latent Diffusion Model
text
- 🧩 Still a Latent Diffusion Model
new_line
new_line
📄 3 text-encoders: 2 CLIPs, one T5-XXL; plug-and-play: removing the larger one maintains competitiveness
text
📄 3 text-encoders: 2 CLIPs, one T5-XXL; plug-and-play: removing the larger one maintains competitiveness
new_line
new_line
🗃️ Dataset was deduplicated with SSCD which helped with memorization (no more details about the dataset tho) 
text
🗃️ Dataset was deduplicated with SSCD which helped with memorization (no more details about the dataset tho) 
new_line
new_line
Variants
text
Variants
new_line
🔁 A DPO fine-tuned model showed great improvement in prompt understanding and aesthetics
text
🔁 A DPO fine-tuned model showed great improvement in prompt understanding and aesthetics
new_line
✏️ An Instruct Edit 2B model was trained, and learned how to do text-replacement 
text
✏️ An Instruct Edit 2B model was trained, and learned how to do text-replacement 
new_line
new_line
Results
text
Results
new_line
✅ State of the art in automated evals for composition and prompt understanding 
text
✅ State of the art in automated evals for composition and prompt understanding 
new_line
✅ Best win rate in human preference evaluation for prompt understanding, aesthetics and typography (missing some details on how many participants and the design of the experiment)
text
✅ Best win rate in human preference evaluation for prompt understanding, aesthetics and typography (missing some details on how many participants and the design of the experiment)
new_line
new_line
Paper: 
text
Paper: 
https://stabilityai-public-packages.s3.us-west-2.amazonaws.com/Stable+Diffusion+3+Paper.pdf
https://stabilityai-public-packages.s3.us-west-2.amazonaws.com/Stable+Diffusion+3+Paper.pdf
link
text
The Stable Diffusion 3 research paper broken down, including some overlooked details! 📝

Model
📏 2 base model variants mentioned: 2B and 8B sizes

📐 New architecture in all abstraction levels: 
- 🔽 UNet; ⬆️ Multimodal Diffusion Transformer, bye cross attention 👋
- 🆕 Rectified flows for the diffusion process
- 🧩 Still a Latent Diffusion Model

📄 3 text-encoders: 2 CLIPs, one T5-XXL; plug-and-play: removing the larger one maintains competitiveness

🗃️ Dataset was deduplicated with SSCD which helped with memorization (no more details about the dataset tho) 

Variants
🔁 A DPO fine-tuned model showed great improvement in prompt understanding and aesthetics
✏️ An Instruct Edit 2B model was trained, and learned how to do text-replacement 

Results
✅ State of the art in automated evals for composition and prompt understanding 
✅ Best win rate in human preference evaluation for prompt understanding, aesthetics and typography (missing some details on how many participants and the design of the experiment)

Paper: https://stabilityai-public-packages.s3.us-west-2.amazonaws.com/Stable+Diffusion+3+Paper.pdf 
👍
osanseviero fffiloni victor mvaloatto clem BrigitteTousi WaveCut EtienneDosSantos samusenps Tonic chethan62 Eungbean MrForExample Obhee GPT007
❤️
clem BrigitteTousi samusenps leegao19 Tonic Bils gonduras
61f4d468587c793cdf55b4dd
/avatars/ce597d8d2640c726473dd85ae8c5cdc7.svg
Lee Gao
leegao19
user
6303613d5c70c21d0ea75c72
https://cdn-avatars.huggingface.co/v1/production/uploads/1661165878439-noauth.jpeg
Valeriy Selitskiy
WaveCut
user
638a1b3cc7f3cc0d220e94ca
/avatars/38495afec0e0dc4c23308bb517bbc409.svg
NibbyNobNibber
MoveScores18
user