跳到正文
DEV Community· Gan·· 7 天前AI 評分39

pbwire 的 20% 效能差距真的無法縮小嗎?

AI Says This 20% Gap Can Never Be Closed. Can You Prove It Wrong?

AI 導讀

開發者將動態 protobuf 設計移植至 pbwire,以一個 C 核心支援 23 種語言綁定,並與各自的官方實作比較效能。Java 在 74KB 級別的編碼和解碼分別達到 1.13x 和 1.95x,但在 136B 級別的編碼僅約 0.7x;8 次針對性優化嘗試均未能改善。開發者徵求維持動態設計並附有 A/B 數據的解法,也接受以充分證據證明差距無法消除。

正文

Gan

Gan

Posted on Nov 16 AI-assisted

Ten years ago I worked at a mobile game company. To speed up our gateway, I wrote enif_protobuf — Erlang has no official protobuf runtime, so I built a C core + NIF that raised gateway throughput several times over pure Erlang code (today's Erlang table still shows ~3.9x encode / ~2.3x decode vs pure-Erlang gpb).

Recently, with AI help, I ported that same dynamic design into pbwire: one C core, 23 language bindings, each benchmarked against its official counterpart. All numbers are in BENCHMARKS.md.

One rule: stay dynamic. Load a standard FileDescriptorSet at runtime, no per-message generated code, and bytes must match protoc exactly before anything is timed.

It won almost everywhere. Java (pure Java, no JNI) does 1.13x / 1.95x on the 74KB tier. The single loss: 136B encode at only ~0.7x (~75ns vs ~58ns against protobuf-java).

Why insist on dynamic?

  1. Generated code pins the schema at compile time — every .proto change means regenerate, recompile, restart. No hot updates for a gateway.
  2. The output is big, unreadable, slow to compile, and pollutes the repo.
  3. Business needs annotations and custom field behavior; generated classes are hard to extend.
  4. Some languages have no official runtime at all (Erlang back then) — a shared dynamic core is the only reusable answer.

So my codegen emits only thin models, and all wire encoding goes through the same core. Winning this cell with a generated writer would mean losing the product.

The last 10 days: small wins, no breakthrough

I ran many intensive rounds over the past 10 days. A few small optimizations landed, but nothing fundamental — the 136B cell never moved.

The ledger is in perf-attempts: 8 targeted attempts all failed, and JNI is rejected too (small-tier encode at just 0.54x of pure Java). AI's verdict is a structural floor: stripping all byte writes still leaves 23.3ns of per-field skeleton (22 fields, ~1.06ns/field), while the official writeTo() is straight-line code with no such loop. The second wall is HotSpot's inlining ceiling.

Help wanted

Full context is in the help doc — please don't retry what's already there. I have no preset direction in mind; any idea is welcome as long as it keeps the dynamic constraint and comes with A/B numbers.

Repro (needs JDK 17+ and protoc; the baseline is stock protobuf-java 4.36.2, auto-downloaded from Maven on first run, matching the pinned protoc 36.2):

make java-bench
# A/B: build two class trees with ./build.sh, then run alternately
bindings/java/benchmarks/ab.sh /tmp/base out/classes
# Only if you want a newer-than-Maven baseline, build it yourself:
# export PROTOBUF_JAVA_JAR=/path/to/protobuf-java.jar

A well-evidenced "0.7x is the floor" is equally welcome — it closes the issue. AI says this 20% can never be won back. Can you prove it wrong?

來源:DEV Community · dev.to