On-premise AI
Planned — later. No inference engine ships yet. x0ne Engine has no release yet.
Run models on your own machines, with x0ne Engine. Your prompts and your weights stay with you.
x0ne Engine, the local engine
x0ne Engine (x0ne-infer) is the inference engine of x0ne OS. It is written in Rust, on Hugging Face candle.
- Model families: Qwen3 (default) and ModernBERT, available. Mistral, in progress.
- Weights in safetensors, with Hugging Face tokenizers.
- CPU, and NVIDIA GPUs with
--device cuda. - It serves only inside a verified trusted execution environment: Intel TDX with NVIDIA GPU attestation.
For local use outside a TEE, a separate lab build exists, x0ne-infer-lab. It also loads LoRA adapters per request. It is in no release yet. Planned
Optional fallback
llama.cpp and Ollama serve model families the Engine does not support yet. They are optional and never installed by default. Planned
Today x0ne-ai-local pulls ggml from backports when recommends are on. llama.cpp is not in trixie or its backports. Ollama is not in Debian.
What x0ne OS adds around the engine
- One model store, shared read-only by every service, container and VM. Models are never inside the image.
- Import checks: checksum against a signed manifest; safetensors by default; pickle files refused.
- Hardened services: no outbound network, a sandboxed systemd unit, local authentication.
- A tuning profile, where each setting must win a benchmark to stay.
Where each workload runs
| Confidential, CPU or GPU | the qcow2 VM as a TDX guest, see Confidential computing and attestation |
| CPU, fast start, not confidential | microVM, lab build, model on a read-only disk |
| GPU, one tenant | bare metal, or a container with CDI |
| GPU, isolated | a VM with VFIO |
Benchmarks
The system under test is x0ne Engine. All values are targets. Each release will publish the measured numbers.
| Measure | Target |
|---|---|
| Tokens per second, against the previous release | no regression |
| microVM boot to first token, small Qwen3 model, warm cache | under 2 s |
| GPU in a container, against bare metal | 98 % or more |
| Confidential VM, against a plain VM | overhead measured and published |
Runs are repeated five times. The median is published with the hardware.
How to verify
Once released, the Engine answers its version and refuses to serve outside a verified TEE. The test suite checks this and expects a failure today: no binary yet.
How to undo
sudo apt purge x0ne-engine x0ne-ai-local. The model store is never removed by a package: delete it yourself.