mistral.rs

Fast, flexible LLM inference

curl -fsSL https://mistralrs.dev/install.sh | sh

Go from install to inference in one command.

Serve mistralrs serve -m Qwen/Qwen3.8-27B --quant 4

Run mistralrs run -m google/gemma-4-E4B-it --quant 4