miolinif087a7499852451c/llama2_wasm32_wasi_wasix

LLaMA2 inference as Rust code compiled to WebAssembly WASIX.

0. Repeat instructions from README.md to download binary models.

1. Compile

cargo wasix build --release

2. Inference

wasmer run --enable-threads --enable-simd --mapdir .:. llama2-rs.wasi.wasm -- stories110M.bin 0.9

WASIX

WebAssembly WASI extension website

Credits

Cloned and modified from here:

https://github.com/gaxler/llama2.rs