I can't believe this... Phi-3.5-mini (3.8B) running in-browser at ~90 tokens/second on WebGPU w/ Transformers.js and ONNX Runtime Web! ๐คฏ Since everything runs 100% locally, no messages are sent to a server โ a huge win for privacy!
- ๐ค Demo:
webml-community/phi-3.5-webgpu
- ๐งโ๐ป Source code: https://github.com/huggingface/transformers.js-examples/tree/main/phi-3.5-webgpu
- ๐ค Demo:
webml-community/phi-3.5-webgpu
- ๐งโ๐ป Source code: https://github.com/huggingface/transformers.js-examples/tree/main/phi-3.5-webgpu