microsoft/Phi-3.5-MoE-instruct
Model Summary
Phi-3.5-MoE is a lightweight, state-of-the-art open model built upon datasets used for Phi-3 - synthetic data and filtered publicly available documents - with a focus on very high-quality, reasoning dense data. The model supports multilingual and comes with 128K context length (in tokens). The model underwent a rigorous enhancement process, incorporating supervised fine-tuning, proximal policy optimization, and direct preference optimization to ensure precise instruction adherence and robust safety measures.
๐ก Phi-3 Portal ๐ฐ Phi-3 Microsoft Blog ๐ Phi-3 Technical Report ๐ฉโ๐ณ Phi-3 Cookbook ๐ฅ๏ธ Try It
Model Summary
Phi-3.5-MoE is a lightweight, state-of-the-art open model built upon datasets used for Phi-3 - synthetic data and filtered publicly available documents - with a focus on very high-quality, reasoning dense data. The model supports multilingual and comes with 128K context length (in tokens). The model underwent a rigorous enhancement process, incorporating supervised fine-tuning, proximal policy optimization, and direct preference optimization to ensure precise instruction adherence and robust safety measures.
๐ก Phi-3 Portal ๐ฐ Phi-3 Microsoft Blog ๐ Phi-3 Technical Report ๐ฉโ๐ณ Phi-3 Cookbook ๐ฅ๏ธ Try It