跳到正文
DEV Community· devs·· 9 小時前AI 評分54

FunASR MCP 相關的 modelscope/FunASR 約有 21k 星標,雙音軌 WASAPI 擷取仍主導即時 STT 延遲

FunASR MCP Crosses 21k Stars — Dual-Track WASAPI Capture Still Owns Real-Time STT Latency

AI 導讀

文章指出,FunASR MCP 可讓 Agent 呼叫本地語音轉錄,但 Windows 即時語音延遲仍主要取決於 WASAPI 擷取路徑。modelscope/FunASR 約有 21k stars;文章建議分別擷取麥克風與 loopback 音訊,並以 lock-free SPSC ring 傳送樣本。

正文

FunASR's MCP server just became one of the most-starred speech tools in the ecosystem. modelscope/FunASR sits at roughly 21k stars with a ready-to-run MCP endpoint that hands any agent a transcribe_audio tool backed by SenseVoiceSmall (or larger Fun-ASR models). The Glama ranking and the v1.3.3 release notes make the pitch clear: drop a local, private, OpenAI-compatible or MCP transcription backend into Claude Code, Cursor, or any agent loop without shipping audio to the cloud.

FunASR GitHub

That is useful. It is also not the hard part if you are building a real-time overlay for calls or interviews on Windows.

The agent-facing half is solved

The MCP server is intentionally simple. A minimal config for Claude Code looks like this:

{
  "mcpServers": {
    "funasr": {
      "command": "python",
      "args": ["/path/to/examples/mcp_server/funasr_mcp.py"],
      "env": {"FUNASR_DEVICE": "cuda", "FUNASR_MODEL": "iic/SenseVoiceSmall"}
    }
  }
}

Or the OpenAI-compatible path:

pip install funasr fastapi uvicorn python-multipart
funasr-server --model sensevoice --device cuda
# POST /v1/audio/transcriptions

The model does VAD, recognition, punctuation, and optional timestamps in one call. On a decent GPU it is comfortably faster than realtime for file-based work. For an agent that needs to "listen to this recording and summarize," the problem is solved.

Real-time overlays live in a different regime

An interview or call assistant has different constraints:

  • Two independent sources (your mic + the remote party via loopback).
  • Continuous 20–40 ms chunks, not whole files.
  • End-to-end latency budget that is usually under 300–500 ms before the overlay feels laggy.
  • The model never sees clean audio if the capture path is wrong.

The dominant term is still the Windows audio path. Shared-mode WASAPI goes through the audio engine (audiodg.exe + audioeng.dll), APOs, and a fixed period. Exclusive mode (or a carefully tuned shared mode) is the only reliable way to keep the buffer period in the single-digit to low-teens milliseconds.

A practical dual-track shape that actually ships:

  1. Mic stream: IAudioClient on the capture endpoint, preferably exclusive or with a short period.
  2. Loopback stream: IAudioClient on the render endpoint with AUDCLNT_STREAMFLAGS_LOOPBACK.
  3. Each callback only copies samples into a lock-free SPSC ring (power-of-two capacity). No model work in the audio thread.
  4. Consumer threads resample to 16 kHz mono, run a cheap VAD, and feed a streaming session (or chunk the FunASR model if you are willing to accept higher latency).

WASAPI bridge style routing

Typical additive costs once samples leave the callback:

Stage Typical cost
WASAPI shared period 10 ms (5–8 ms if requested)
Callback → ring copy + format convert 1–3 ms
Resample to 16 kHz mono + lightweight VAD 2–5 ms
Local model first partial (or network RTT) 40–150 ms
Overlay paint <16 ms

FunASR (or any local model) only affects the last model stage. If your capture path is a 40 ms Bluetooth shared-mode period plus engine enhancements, the recognizer is already starting late.

Wiring the two halves together

You can still use the FunASR MCP or OpenAI-compatible server for the model side. The difference is the feeder:

  • File mode: agent or script drops a WAV and calls the tool. Great for post-call notes.
  • Streaming mode: your capture process writes short PCM chunks (or uses a local streaming endpoint if the model supports it) and pushes partials to the overlay.

A minimal consumer sketch (Python, after the rings are filled):

import numpy as np
from funasr import AutoModel

model = AutoModel(model="iic/SenseVoiceSmall", vad_model="fsmn-vad", device="cuda")

def on_chunk(pcm_f32_16k: np.ndarray):
    # FunASR generate expects a path or array depending on version;
    # many people write a short temp WAV or use the streaming API when available.
    result = model.generate(input=pcm_f32_16k, batch_size=1, language="auto")
    text = result[0]["text"]
    # push to overlay / translation layer
    return text

Tradeoffs are the usual ones:

  • File-oriented MCP is simpler and more accurate on long context; it is not low-latency.
  • Streaming local models (or chunked calls) cut latency but raise WER on short fragments and require careful VAD + overlap.
  • Exclusive mode gives the best period numbers and fights with other audio apps; shared mode is friendlier and adds engine latency.
  • GPU vs CPU: SenseVoiceSmall is usable on CPU for short chunks; larger models want CUDA or a fast NPU.

What actually moves the needle

Agent integrations (MCP, OpenAI-compatible servers, tool-calling) make local ASR easy to call. They do not shrink the WASAPI period, remove APO processing, or give you clean dual-track separation. Those remain Windows-specific engineering problems.

If you are building a real-time translation or interview overlay, spend the first week on the capture path (two independent clients, lock-free rings, exclusive or short-period shared, VAD before the model). Only then plug in FunASR, Whisper, or whatever local model is currently fastest on your hardware. The MCP server is a convenient way to expose that model to agents for offline analysis; the live path still has to be hand-rolled.

I'm building Talkaside, a Windows app for job interviews and calls in another language.
Drafted with AI assistance from this project's code and logs.

來源:DEV Community · dev.to