跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 23 天前AI 評分42

Flide:以精確狀態分叉加速決策推理

Flide: Accelerating Decision Inference with Exact State Forking

AI 導讀

Flide 是一款開源推理引擎,面向 Jev-like 決策模型,透過精確狀態分叉,讓多個問題共用運算,同時保留精確的決策語義。測試中,Flide 的推理速度最高比 vLLM 和 SGLang 快 9.5×,並支援 Decision Index 0.4.1 上的 30+ 款開源決策模型。

正文

Hi everyone! I'm one of the authors of Flide.

We built Flide from scratch, an open-source inference engine for Jev-like decision models.

The key idea is exact state forking, which allows multiple questions to share computation while preserving exact decision semantics.

In our testing, Flide achieves up to 9.5× faster inference than vLLM and SGLang.
We support 30+ open-source decision models on Decision Index 0.4.1, with support for more models on the way.

Get started with one line:
pip install flide

Code: https://github.com/OmniJev/Flide
Docs: https://docs.flide.org/
Arxiv: https://arxiv.org/abs/2610.42148

We'd love to hear your feedback, questions, and ideas!

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co