You can now use DoRA for your embedding layers!

PR: https://github.com/huggingface/peft/pull/2006

I have documented my journey of this specific PR in a blog post for everyone to read. The highlight of the PR was when the first author of DoRA reviewed my code.

Blog Post: https://huggingface.co/blog/ariG23498/peft-dora

Huge thanks to @BenjaminB for all the help I needed.

GitHub

[Add] DoRA Embedding by ariG23498 · Pull Request #2006 · huggingface/peft

This PR adds DoRA for the embedding layer.
TODOs:

Check whether this implementation is correct
Write tests for the implementation

@BenjaminBossan can you give me an initial review to set the to...