53ON AI Server:面向 AMD Ryzen AI Max 的 OpenAI API 兼容伺服器
Show HN: 53ON AI Server – OpenAI-compatible server for AMD Ryzen AI Max
Enrique Velasquez 展示 53ON AI Server,一個面向 AMD Ryzen AI Max、基於 llama.cpp 的單一 Node.js 伺服器,支援 OpenAI 和 Anthropic API。
Hi HN. I run my own AI at home on a Ryzen AI Max+ 395 mini PC with 128 GB of unified memory, and this is the server it talks to. It's a single Node.js process with no dependencies, on top of llama.cpp. It speaks the OpenAI and Anthropic APIs, and it has a panel to load models, give each client an API key with a token quota, and relay cloud providers when a local model isn't enough.
We also publish our own llama.cpp build. The main change is that the KV cache takes memory as tokens arrive instead of reserving it all at load (sparse buffers on Vulkan). A 1B model with a 1M-token window starts with 0.86 GiB instead of 8.24 GiB.
It's MIT. I've only tested it on Strix Halo with Vulkan (RADV) on Ubuntu 24.04. I built it with a lot of help from Claude. Feedback and bug reports are very welcome.
來源:DEV Community · dev.to