🚧 Reproducing LBM-Eraser… in progress! [1]
text
🚧 Reproducing LBM-Eraser… in progress! [1]
new_line
new_line
When repurposing a T2I model into a pure I2I model, there’s always that orphaned text path — what do we do with it? 🤔
text
When repurposing a T2I model into a pure I2I model, there’s always that orphaned text path — what do we do with it? 🤔
new_line
new_line
You can reuse it as learnable embeddings in multi-task setups [2], freeze an empty text prompt, distillate or prune the corresponding part.
text
You can reuse it as learnable embeddings in multi-task setups [2], freeze an empty text prompt, distillate or prune the corresponding part.
new_line
new_line
In LBM, they take a clever route — zeroing [3] and reshaping [4] the text-related cross-attentions into self-attentions.
text
In LBM, they take a clever route — zeroing [3] and reshaping [4] the text-related cross-attentions into self-attentions.
new_line
This gives you fresh weights for I2I computation, nicely integrated into your SD architecture.
text
This gives you fresh weights for I2I computation, nicely integrated into your SD architecture.
new_line
new_line
📎 References
text
📎 References
new_line
[1] Our LBM Fork: 
text
[1] Our LBM Fork: 
https://github.com/finegrain-ai/LBM
https://github.com/finegrain-ai/LBM
link
new_line
[2] OmniPaint: 
text
[2] OmniPaint: 
OmniPaint: Mastering Object-Oriented Editing via Disentangled
  Insertion-Removal Inpainting (2503.08677)
https://huggingface.co/papers/2503.08677
2503.08677
paper
resource
https://huggingface.co/papers/2503.08677
new_line
[3] LBM Zeroing: 
text
[3] LBM Zeroing: 
https://github.com/gojasper/LBM/blob/cafebc46a9ac16dcc61691d289cc4676b5c75380/examples/training/train_lbm_surface.py#L147-L148
https://github.com/gojasper/LBM/blob/cafebc46a9ac16dcc61691d289cc4676b5c75380/examples/training/train_lbm_surface.py#L147-L148
link
new_line
[4] LBM Reshaping: 
text
[4] LBM Reshaping: 
https://github.com/gojasper/LBM/blob/cafebc46a9ac16dcc61691d289cc4676b5c75380/examples/training/train_lbm_surface.py#L100
https://github.com/gojasper/LBM/blob/cafebc46a9ac16dcc61691d289cc4676b5c75380/examples/training/train_lbm_surface.py#L100
link
🚧 Reproducing LBM-Eraser… in progress! [1]

When repurposing a T2I model into a pure I2I model, there’s always that orphaned text path — what do we do with it? 🤔

You can reuse it as learnable embeddings in multi-task setups [2], freeze an empty text prompt, distillate or prune the corresponding part.

In LBM, they take a clever route — zeroing [3] and reshaping [4] the text-related cross-attentions into self-attentions.
This gives you fresh weights for I2I computation, nicely integrated into your SD architecture.

📎 References
[1] Our LBM Fork: https://github.com/finegrain-ai/LBM
[2] OmniPaint: https://huggingface.co/papers/2503.08677
[3] LBM Zeroing: https://github.com/gojasper/LBM/blob/cafebc46a9ac16dcc61691d289cc4676b5c75380/examples/training/train_lbm_surface.py#L147-L148
[4] LBM Reshaping: https://github.com/gojasper/LBM/blob/cafebc46a9ac16dcc61691d289cc4676b5c75380/examples/training/train_lbm_surface.py#L100
🧠
deltheil John6666 SkyNotion
👀
dnsbrl
632334c533a1e1cf9deebf37
https://cdn-avatars.huggingface.co/v1/production/uploads/632334c533a1e1cf9deebf37/Ioa4n2ir-XZlDmcxMUYE3.png
Cédric
deltheil
user