LLM-to-Flow Distillation: Teaching Natural Language Goals to Scientific Generators
LLM-to-Flow Distillation: Teaching Natural Language Goals to Scientific Generators
LLM-to-Flow Distillation(LFD)以凍結的 LLM 作為自適應教師,將高層次自然語言科研目標逐步轉化為領域專用生成器的子目標。教師評判候選樣本,並透過 DPO 迭代蒸餾偏好,無須明確代理獎勵。治療肽和小分子結構設計實驗展示了這種方法。
Published on Oct 12
Authors:
,
,
,
,
,
,
,
Abstract
Generative discovery in science requires coupling high-level scientific reasoning with the ability to generate samples respecting the structure of a design space. Yet these capabilities remain largely separated: foundation LLMs encode broad chemical and biological knowledge, while domain-specific flow- and diffusion-based generators capture the priors needed to produce valid sequences and geometries. We study how to align such scientific generators to high-level natural-language discovery goals without requiring an explicit proxy reward, which is often unavailable or poorly captured by learned predictors in experimental sciences. We introduce LLM-to-Flow Distillation (LFD), where a frozen LLM acts as an adaptive teacher for a domain-specific generator. Given a goal such as 'generate peptides with promising anticancer activity', the teacher constructs a curriculum of intermediate sub-goals aware of the flow's current capabilities, and progressively directs them toward the original discovery goal. At each round, the LLM judges sampled candidates under the current objective. These judgments induce preferences that are iteratively distilled into the generator through DPO. This procedure provides dense learning signals even when the initial model places negligible probability on designs satisfying the final goal, enabling adaptation toward poorly covered, out-of-distribution regions. Experiments on therapeutic peptides and small-molecule structure design demonstrate that LFD enables a new form of generative control: aligning arbitrary domain-specific scientific generators to scientific discovery tasks expressed via high-level natural-language goals.
View arXiv page View PDF Project page GitHub 13 Add to collection
Models citing this paper 0
No model linking this paper
Cite arxiv.org/abs/2610.17246 in a model README.md to link it from this page.
Datasets citing this paper 0
No dataset linking this paper
Cite arxiv.org/abs/2610.17246 in a dataset README.md to link it from this page.
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2610.17246 in a Space README.md to link it from this page.
Collections including this paper 0
No Collection including this paper
Add this paper to a collection to link it from this page.
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co