This is great! Florence 2 is an amazing model for its size but it hallucinates a lot in terms of positioning and color, that's reduced with this model but I think it still persists
Yatharth Gupta
Warlord-K
AI & ML interests
Generative AI, Natural Language Processing, Reinforcement Learning
Recent Activity
updated a model 3 days ago
Kanha-AI/7b5309ac-d3b5-49ed-9e95-be7032d1a202-medium published a model 3 days ago
Kanha-AI/7b5309ac-d3b5-49ed-9e95-be7032d1a202-medium updated a model 12 days ago
Kanha-AI/dae07326-a05a-4cf2-932d-7b78ac9a437d-largeOrganizations
replied to gokaygokay's post over 2 years ago
replied to gokaygokay's post over 2 years ago
What are your thoughts on Florence2? Do you think finetuning it on these datasets will help on the captioning task?
reacted to gokaygokay's post with ❤️ over 2 years ago
Post
6068
I've fine-tuned three types of PaliGemma image captioner models for generating prompts for Text2Image models. They generate captions similar to prompts we give to the image generation models. I used google/docci and google/imageinwords datasets for fine-tuning.
This one gives you longer captions.
gokaygokay/SD3-Long-Captioner
This one gives you middle size captions.
https://huggingface.co/spaces/gokaygokay/SD3-Long-Captioner-V2
And this one gives you shorter captions.
https://huggingface.co/spaces/gokaygokay/SDXL-Captioner
This one gives you longer captions.
gokaygokay/SD3-Long-Captioner
This one gives you middle size captions.
https://huggingface.co/spaces/gokaygokay/SD3-Long-Captioner-V2
And this one gives you shorter captions.
https://huggingface.co/spaces/gokaygokay/SDXL-Captioner
- 10 replies
replied to their post over 2 years ago
Do you mean the consistency between generations? Could you elaborate a little?
replied to their post over 2 years ago
That'd be really cool indeed!
