I can't believe this... Phi-3.5-mini (3.8B) running in-browser at ~90 tokens/second on WebGPU w/ Transformers.js and ONNX Runtime Web! 🤯 Since everything runs 100% locally, no messages are sent to a server — a huge win for privacy!
- 🤗 Demo:
webml-community/phi-3.5-webgpu
- 🧑💻 Source code: https://github.com/huggingface/transformers.js-examples/tree/main/phi-3.5-webgpu
- 🤗 Demo:
webml-community/phi-3.5-webgpu
- 🧑💻 Source code: https://github.com/huggingface/transformers.js-examples/tree/main/phi-3.5-webgpu
GitHub
transformers.js-examples/phi-3.5-webgpu at main · huggingface/transformers.js-examples
A collection of 🤗 Transformers.js demos and example applications - huggingface/transformers.js-examples