Can we train a VLM to 𝐩𝐫𝐞𝐟𝐞𝐫?

This is now possible, thanks to the new TRL/DPO support for VLMs! 🎉

As an example, we've trained a model to reduce hallucinations.

Check out:
📰 Blog post: https://huggingface.co/blog/dpo_vlm
🐙 TRL: https://github.com/huggingface/trl

Thanks to
@mervenoyann
,
@vwxyzjn
and
@krasul
who helped me with this work!