Can we train a VLM to ๐ฉ๐ซ๐ž๐Ÿ๐ž๐ซ?This is now possible,... | Can we train a VLM to ๐ฉ๐ซ๐ž๐Ÿ๐ž๐ซ?This is now possible,...
Can we train a VLM to ๐ฉ๐ซ๐ž๐Ÿ๐ž๐ซ?

This is now possible, thanks to the new TRL/DPO support for VLMs! ๐ŸŽ‰

As an example, we've trained a model to reduce hallucinations.

Check out:
๐Ÿ“ฐ Blog post: https://huggingface.co/blog/dpo_vlm
๐Ÿ™ TRL: https://github.com/huggingface/trl

Thanks to
@mervenoyann
,
@vwxyzjn
and
@krasul
who helped me with this work!