A voluntary AI accord needs comparable audit evidence, not signatures

A one-page accord reported by Reuters asks frontier AI companies to organise safety around four layers: internal controls, a dedicated internal team, an external auditor or evaluator, and oversight by an independent board committee. The White House event gathered executives who signed the voluntary commitment. Associated Press reporting also describes the initiative as a pledge rather than a regulation.

The architecture is directionally sensible. It distributes responsibility across operators, specialists, outsiders and directors instead of pretending that one model card can carry the entire burden. But the document is too short to make results comparable. It does not define a shared test set, severity scale, minimum disclosure, auditor independence rule or consequence when a layer fails.

Turn four layers into one evidence trail

For every covered model or deployment, require a versioned assurance record. It should name the system boundary, release candidate, enabled tools, access privileges, languages, user groups and excluded conditions. Each material risk claim should link to a test method, sample, threshold, result, uncertainty and owner. A board committee should see the same failed cases and unresolved exceptions that operators see, not a compressed green dashboard.

External review only adds assurance when its scope and incentives are visible. Record who selected and paid the evaluator, what data and infrastructure it could inspect, whether tests were announced, which findings the company disputed, and whether retesting used a materially changed system. “Audited” is not a portable result if one firm tests model behaviour in a sandbox while another tests an agent with production tools.

Compare evidence, not labels

Procurement teams should define a small common evidence table before accepting the accord as a supplier control. Useful fields include the risk scenario, measurement unit, pass threshold, observed distribution, known blind spots, remediation, residual risk and release decision. Preserve raw evidence under appropriate access controls so an independent reviewer can reproduce a sample without revealing sensitive capability details publicly.

Comparability also requires a denominator. Report how many evaluations were attempted, completed, invalidated and repeated, and how many material failures remained open at the decision date. Separate a model-level result from the deployment configuration actually offered to users. Without those fields, a supplier can highlight one strong test while omitting a weak or inapplicable part of the evidence portfolio.

The counterargument is that common disclosure can create a security risk or freeze fast-moving evaluation practice. That is credible. The answer is tiered access and a stable reporting envelope, not identical public test cases. Companies can protect exploit details while still reporting the boundary, method class, severity rubric, aggregate results and whether a failure changed release scope. A schema can remain stable even as individual evaluations evolve.

Board oversight also needs a decision rule. The committee should receive a named recommendation for ship, limit, remediate or stop, plus dissent and expiration dates. A voluntary accord that never records a blocked release may reflect excellent systems, weak thresholds or selective reporting; observers cannot distinguish those explanations from signatures alone.

For workforce leaders, the relevance is practical. The same discipline should govern high-consequence internal agents. Use the Skills Intelligence Role Dictionary to identify accountable outcomes and affected roles, then attach assurance evidence to the actual workflow and authority boundary. Do not import a frontier-model pledge as proof that a local hiring, finance or customer-service deployment is safe.

The immediate move is to translate the accord into a comparable evidence contract. Keep the four layers, but require every layer to produce a dated artifact and every material failure to trigger a visible decision. That turns a moral commitment into an assurance process without pretending a voluntary signature is enforcement.