跳到正文
DEV Community· mycute·· 13 天前AI 評分34

模型共生演化:一個 LoRA adapter 抹除另一個

Model symbiogenesis: one LoRA adapter erases another

AI 導讀

在 4-bit Qwen3B 上,AG News LoRA adapter 在評分測試中令 SST-2 準確率降至 0.0%,而 0.5/0.5 合併兩個 adapter 後,SST-2 準確率回升至 98.5%。

正文

Skip to content

Image 1: DEV Community

Powered by Algolia

Log inCreate account

DEV Community

Image 20 Add reaction

Image 30 Like Image 40 Unicorn Image 50 Exploding Head Image 60 Raised Hands Image 70 Fire

0 Jump to Comments 0 Save Boost

Pick as gem

Copy link

Copied to Clipboard

Share to XShare to LinkedInShare to FacebookShare to Mastodon

Share Post via...Report Abuse

Image 8: mycute

mycute Posted on Nov 9

Model symbiogenesis: one LoRA adapter erases another

#ai#showdev#machinelearning#opensource

Model symbiogenesis: one adapter erases another, the merge restores it

Two LoRA adapters share one 4-bit Qwen3B host. One is a sentiment adapter (SST-2), the
other a topic adapter (AG News). This is not a container experiment; it asks whether two
learned behaviors coexist when their weights are merged.

Reproduce:

python research/symbiogenesis.py   # -> research/results/symbiogenesis.json

Logit-scored accuracy (200 test rows each)

state SST-2 AG News
base 100.0% 80.5%
SST-2 adapter 96.0% 88.0%
AG News adapter 0.0% 70.5%
0.5/0.5 merge 98.5% 84.0%

The AG News adapter alone does not just lower sentiment accuracy; it erases it under
this scoring. The 0.5 merge restores sentiment to 98.5% and recovers AG News above the AG
adapter (84.0 vs 70.5), but not above the SST-2 adapter's 88.0.

Generation check (8 SST-2 rows)

state correct example output
base 7/8 negative
AG News adapter 0/8 ulative business business
0.5 merge 8/8 negative

The generation check confirms the logit result: the topic adapter turns sentiment prompts
into topic-like text. The merge restores the sentiment behavior.

What this looks like

  • Cross-task interference is not additive. A topic adapter can overwrite a sentiment behavior completely under this recipe.
  • A merge can rescue a behavior. The sentiment adapter is a symbiotic partner: the merged organism regains sentiment while keeping most of the topic ability.
  • No hybrid vigor. The merge does not beat the best parent on either task; it is a compromise (SST-2 98.5 vs base 100; AG 84.0 vs SST-2 adapter 88.0).
  • The SST-2 adapter improves AG News over base by 7.5 pp under logit scoring. That is either cross-task transfer or a scoring artifact; 200 rows is not enough to tell.

Caveats

  • 200 rows per task, one merge ratio (0.5), one seed.
  • Logit scoring is not generation accuracy; the 8-row generation check only confirms the direction, not the exact numbers. The base SST-2 logit score is a ceiling.
  • No 4-bit/8-bit merged adapter yet; no third task; no statistical test.
  • The AG adapter's AG accuracy (70.5%) is below base (80.5%), so the adapter is not a clean "topic expert" under this prompt protocol.

Next experiments

  • Sweep merge weights (0.25 / 0.5 / 0.75) and quantize the merged adapter.
  • Add a third adapter (Dolly QA or watermark) and build a behavioral epistasis matrix: which pairs rescue, dominate, or kill each other?
  • Test whether a merge can be synthetic lethal: two adapters that are each fine alone but fail when combined.
  • Compare merge rules: linear, TIES, DARE, task-arithmetic.

This is a first probe of model symbiogenesis, not a result about all LoRA pairs. The
interesting part is that the behavior is not a simple additive quantity.

Image 9: Google Cloud Google CloudPromoted

Image 10: Build Agents and Models on One Platform

Build Agents and Models on One Platform

Rapidly deploy, scale, govern and optimize production-ready agents grounded in your organization’s data. Choose to build with Google’s latest multimodal models like Gemini 4 Argon, third-party models like Anthropic's Claude Model Family, and open models like Gemma in Model Garden.

Learn More →

Top comments (0)

Subscribe

Image 11: pic

Submit PreviewDismiss

Code of Conduct•Report abuse

Image 12: Google Cloud Google CloudPromoted

Image 13: Save Up to 91% on Cloud Compute With Spot VMs

Save Up to 91% on Cloud Compute With Spot VMs

Run VMs on high-performance and reliable cloud infrastructure. Choose from preset or custom machine types for web servers, databases, and applications that fuel your agents.

Learn More →

Image 14 mycute

Follow

  • Joined Oct 22, 2026

More from mycute

4-bit Naive Bayes: symmetric fails on AG News, zero-point rescues it #ai#machinelearning#opensource#showdevAI in a MIDI file: the model is the notes #ai#webdev#opensource#showdev

Image 15: Google Cloud Google CloudPromoted

Image 16: Build Agents and Models on One Platform

Build Agents and Models on One Platform

Rapidly deploy, scale, govern and optimize production-ready agents grounded in your organization’s data. Choose to build with Google’s latest multimodal models like Gemini 4 Argon, third-party models like Anthropic's Claude Model Family, and open models like Gemma in Model Garden.

Learn More →

👋 Kindness is contagious


Discover fresh viewpoints in this insightful post, supported by our vibrant DEV Community. Every developer’s experience matters—add your thoughts and help us grow together.

A simple “thank you” can uplift the author and spark new discussions—leave yours below!

On DEV, knowledge-sharing connects us and drives innovation. Found this useful? A quick note of appreciation makes a real impact.

Okay

DEV Community — A space to discuss and keep up software development and manage your software career

Built on Forem — the open source software that powers DEV and other inclusive communities.

Made with love and Ruby on Rails. DEV Community © 2016 - 2026.

Image 17: DEV Community

We're a place where coders share, stay up-to-date and grow their careers.

Log inCreate account

Image 18Image 19Image 20Image 21Image 22

來源:DEV Community · dev.to