跳到正文
DEV Community· Enrique Velasquez·· 7 天前AI 評分57

53ON AI Server:面向 AMD Ryzen AI Max 的 OpenAI API 兼容伺服器

Show HN: 53ON AI Server – OpenAI-compatible server for AMD Ryzen AI Max

AI 導讀

Enrique Velasquez 展示 53ON AI Server,一個面向 AMD Ryzen AI Max、基於 llama.cpp 的單一 Node.js 伺服器,支援 OpenAI 和 Anthropic API。

正文

Enrique Velasquez

Hi HN. I run my own AI at home on a Ryzen AI Max+ 395 mini PC with 128 GB of unified memory, and this is the server it talks to. It's a single Node.js process with no dependencies, on top of llama.cpp. It speaks the OpenAI and Anthropic APIs, and it has a panel to load models, give each client an API key with a token quota, and relay cloud providers when a local model isn't enough.

We also publish our own llama.cpp build. The main change is that the KV cache takes memory as tokens arrive instead of reserving it all at load (sparse buffers on Vulkan). A 1B model with a 1M-token window starts with 0.86 GiB instead of 8.24 GiB.

It's MIT. I've only tested it on Strix Halo with Vulkan (RADV) on Ubuntu 24.04. I built it with a lot of help from Claude. Feedback and bug reports are very welcome.

來源:DEV Community · dev.to