跳到正文
The Elec· Hyun-Seon, Park·· 18 天前AI 評分58

Anthropic 公開 FlashProve 開源技術,開發 AI 輸出來源核驗系統

Anthropic Unveils Open-Source Technology to Verify AI Outputs

AI 導讀

Anthropic 公開 FlashProve 開源技術,旨在以密碼學證明核驗 AI 輸出是否由指定模型生成。系統要求模型開發者在模型發佈前公開指紋,供日後核驗輸出來源,而且不會披露模型權重。研究人員已公開技術規格及可運作程式碼;技術仍在開發中,Anthropic 未公佈在其服務或其他 AI 公司服務部署的計劃。

正文

As enterprise AI adoption expands and environments increasingly combine multiple models, tools such as FlashProve could support verification of AI service contracts and service-level agreements (SLAs), improve the reliability of AI performance evaluations and validate the execution histories of AI agents.

As enterprise AI adoption expands and environments increasingly combine multiple models, tools such as FlashProve could support verification of AI service contracts and service-level agreements (SLAs), improve the reliability of AI performance evaluations and validate the execution histories of AI agents.

As enterprise AI adoption expands and environments increasingly combine multiple models, tools such as FlashProve could support verification of AI service contracts and service-level agreements (SLAs), improve the reliability of AI performance evaluations and validate the execution histories of AI agents.
As enterprise AI adoption expands and environments increasingly combine multiple models, tools such as FlashProve could support verification of AI service contracts and service-level agreements (SLAs), improve the reliability of AI performance evaluations and validate the execution histories of AI agents.

Anthropic is developing technology to prove which artificial intelligence (AI) model actually generated a given output. Acting as a form of digital certification for AI-generated content, the technology mathematically verifies that a specific model produced a particular result.

Anthropic announced on Nov. 3 (local time) through its official blog that it is developing a system called FlashProve. Its researchers describe the technology as "provable inference." The central idea is to attach cryptographic proofs to AI-generated outputs, allowing users to verify whether a response was actually produced by the model they believed they were using.

The approach resembles cryptographic digital signatures used to verify email senders. When an email carries a digital signature, recipients can confirm whether the sender is who they claim to be. Similarly, FlashProve attaches provenance information to AI model outputs. It is specifically designed to address situations in which even the company developing and operating an AI model attempts to pass off another model as the claimed one.

If commercialized, the technology could prevent AI service providers from secretly substituting cheaper, less capable models for the high-performance models requested by users to reduce operating costs. Conversely, it could also verify claims that a more expensive, higher-performance model was used when companies attempt to inflate performance evaluations or benchmark results.

The technology could also be used when third parties claim to have employed a particular AI model. For example, a software developer might claim that its library was checked for vulnerabilities using a state-of-the-art cybersecurity AI model, or a company might advertise that its business plan was reviewed and approved using a suitable high-performance model. FlashProve would allow users to verify whether the claimed model actually performed the task.

Implementing FlashProve requires AI model developers to publish a "fingerprint" for each model before its release. The fingerprint serves as a reference that can later be compared with model outputs to cryptographically verify whether a particular model generated a given result.

Importantly, the model fingerprint does not reveal the model's weights. Weights are internal numerical parameters representing the results of AI training and are critical to determining how a model operates and performs. Companies such as Anthropic and OpenAI, which do not publicly release their proprietary models, treat these parameters as valuable intellectual property.

FlashProve is designed to generate and verify fingerprints without disclosing model weights. Anthropic says this allows the technology to be applied not only to open-source models but also to commercial AI models whose weights remain private.

Anthropic's FlashProve has four key properties: completeness, which accepts valid proofs; soundness, which rejects forged proofs; succinctness, which enables verification on ordinary PCs using small proof files; and zero knowledge, which proves output provenance without revealing model weights. (Source: Anthropic, FlashProve: An Open-Source Library for Provable Inference)
Anthropic's FlashProve has four key properties: completeness, which accepts valid proofs; soundness, which rejects forged proofs; succinctness, which enables verification on ordinary PCs using small proof files; and zero knowledge, which proves output provenance without revealing model weights. (Source: Anthropic, FlashProve: An Open-Source Library for Provable Inference)

The verification process is also designed to be lightweight enough to run directly on users' devices without requiring large-scale computing infrastructure. This would enable users to verify the provenance of AI outputs independently, without relying on the AI service provider.

Alongside the announcement, Anthropic researchers released detailed technical specifications and working code for the system's components. The code is available on GitHub under an open-source license, allowing external developers and researchers to review and improve it.

However, the technology remains under development, and no plans have been announced to deploy it in Anthropic's services or those of other AI companies. The researchers emphasized that the release represents a progress update rather than completed research. They said the technology was made public to enable other researchers to examine, use and further develop it.

In current enterprise AI services, users often cannot independently verify which model generated a response, even when accessing models through APIs or cloud platforms. If a service provider changes models or routes requests to different models, users may have difficulty detecting the change.

來源:The Elec · thelec.net