Flide:以精確狀態分叉加速決策推理
Flide: Accelerating Decision Inference with Exact State Forking
Flide 是一款開源推理引擎,面向 Jev-like 決策模型,透過精確狀態分叉,讓多個問題共用運算,同時保留精確的決策語義。測試中,Flide 的推理速度最高比 vLLM 和 SGLang 快 9.5×,並支援 Decision Index 0.4.1 上的 30+ 款開源決策模型。
Hi everyone! I'm one of the authors of Flide.
We built Flide from scratch, an open-source inference engine for Jev-like decision models.
The key idea is exact state forking, which allows multiple questions to share computation while preserving exact decision semantics.
In our testing, Flide achieves up to 9.5× faster inference than vLLM and SGLang.
We support 30+ open-source decision models on Decision Index 0.4.1, with support for more models on the way.
Get started with one line:pip install flide
Code: https://github.com/OmniJev/Flide
Docs: https://docs.flide.org/
Arxiv: https://arxiv.org/abs/2610.42148
We'd love to hear your feedback, questions, and ideas!
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co