跳到正文
Data Center Knowledge· Sean Michael Kerner·· 16 天前AI 評分50

光學 AI 能否降低數據中心推理的耗電量?

Can Optical AI Cut Data Center Inference Power Use?

AI 導讀

Lumai 正在 Scaling Inference Lab 部署 Iris Nova,測試光學運算能否降低數據中心 AI 推理的電力負擔。Iris Nova 以光學運算處理矩陣乘法,主攻推理的 prefill 階段,並以電子元件處理記憶體、控制及非線性功能。部署將用於驗證系統在商業數據中心環境中的表現,並收集數據以支援下一款產品的開發。

正文

Extreme Close-up of illuminated servers

The trial will examine whether optical compute can reduce the power strain AI inference is placing on data centers with fixed power budgets.Getty Images

Power is always a limiting factor for data centers, and AI inference is increasing that strain more than ever.

Lumai, an optical compute company spun out of the University of Oxford in 2021, says it can ease that pressure by using light rather than transistors to perform the core math of inference.

The company is now putting that claim to a real-world test. Lumai is deploying its Iris Nova servers in the Scaling Inference Lab, a UK facility that evaluates new AI hardware on real workloads in a live commercial setting. Iris Nova is Lumai’s first server, an optical AI accelerator built for data center inference, and it will be the first optical inference platform deployed in the lab.

The UK’s Advanced Research and Invention Agency (ARIA), a government-backed R&D funding agency, supports the lab, while CommonAI CIC, a non-profit focused on collaborative AI development, operates it. Researchers will use the deployment to evaluate how optical compute performs in a commercial data center environment.

Related:Building AI Kill Switches in Data Centers

The problem the hardware is designed to address is power.

“Power is the biggest challenge,” Phillip Burr, director of product at Lumai, told Data Center Knowledge. “Data centers have a fixed power budget, and inference demand is growing faster than the available power.”

How Optical Inference Could Ease Data Center Power Constraints

Matrix multiplication, the process of multiplying large grids of numbers together, sits at the center of AI inference. According to Burr, that is an area where optical compute can operate very efficiently.

To understand why optical hardware may have an advantage, it helps to look at how GPUs and optical systems perform that arithmetic differently.

“GPUs are silicon chips that use transistors to perform the matrix multiplications at the heart of AI inference,” Burr explained. “Optical compute uses the properties of light to perform the same matrix multiplications, completing millions of operations in a single optical cycle.”

Lumai said Iris encodes numbers as light intensity and uses passive optical components, such as lenses, to perform the math. The company said the computation itself uses negligible amounts of energy, with power use rising only minimally as performance scales.

That does not mean the system abandons traditional electronic components altogether. Burr said Lumai’s architecture is hybrid: optics handle matrix multiplication, while memory, control, and nonlinear functions are performed electronically.

There are two main phases of inference, and optical compute targets one of them. Burr said the prefill phase is where the prompt and context are processed. The second phase, decode, is where the output tokens are generated.

Related:Nvidia Unveils DSX POD Amid Greenpeace Protest

“Prefill is compute-bound; each model weight is reused across every token in the prompt, so arithmetic dominates,” Burr said. “This is where optical compute boosts performance.”

In practical terms, compute-bound means the speed of the arithmetic determines overall performance. A token is a small unit of text, such as a word or part of a word.

Decode works differently because tokens are generated one at a time. Burr said decode is limited by memory bandwidth, so faster arithmetic does not provide the same benefit. He added that model training is also not a strong fit for optical compute today, as it requires higher precision and continuous weight updates.

“GPUs remain a good fit for training and decode,” Burr said.

The Lumai Iris Nova server is designed to handle the prefill phase and leave decode to other hardware.

“Iris is designed for the prefill stage, while GPUs or other decode-optimized hardware are used for decode,” Burr said. “Moving prefill off GPUs also frees that GPU capacity for decode.”

How Iris Nova Fits into Existing Data Center Infrastructure

For operators that split inference by running prefill on Iris Nova and decode on GPUs, software is needed to route those tasks to separate hardware. That approach is known as disaggregated serving, and Burr said Iris Nova is designed to work with the frameworks that support it.

Related:Nvidia Unveils DSX Pod for Faster AI Data Center Builds

“For integration, Iris works with standard software frameworks such as PyTorch, standard networking interfaces, and disaggregated inference serving frameworks such as vLLM and SGLang,” Burr said.

Whether that split pays off depends on the work a data center is handling.

“The deciding factor for an operator is how much of their compute goes to prefill,” Burr said. “Prefill dominates with long prompts and agentic workloads, where models repeatedly process large contexts. This is where optical compute delivers the most.”

Another practical consideration is facility requirements, and Burr said the hardware is built from familiar components.

“Lumai Iris servers are built from components already manufactured at high volume for data center communications, rather than novel materials or bespoke fabrication,” Burr said.

The cooling requirements are similarly conventional. The Scaling Inference Lab, where Iris Nova is being deployed, tests hardware in a live commercial setting with standard power, cooling, and rack integration, giving Lumai a trial under typical data center conditions.

“Iris Nova operates in standard air-cooled racks and does not need liquid cooling,” Burr said.

Lumai said the goal of the ARIA-backed deployment is to validate the system further and generate data to help shape its next product for production deployment.

“The ARIA deployment will put a cluster of Iris Nova servers into an independently run lab, ARIA’s Scaling Inference Lab,” Burr said. “It will show that optical compute can run real AI workloads outside a research setting, and it will provide system-level data on performance.”

About the Author

Sean Michael Kerner

Contributor

Sean Michael Kerner is an IT consultant, technology enthusiast and tinkerer. He consults to industry and media organizations on technology issues.

https://www.linkedin.com/in/seanmkerner/

來源:Data Center Knowledge · datacenterknowledge.com