A Critique of Transparency Provisions in NY’s RAISE Act (1.0)

Following in California’s footsteps, New York’s RAISE Act attempts to mandate AI transparency

The US has a couple of state laws—from California, and now New York—that address the risk of frontier AI models. Both broadly operate by specifying some information about frontier AI models that must be disclosed for the purposes of oversight. In this post I’ll review New York’s “Responsible AI Safety and Education” (RAISE) act through the lens of the quality of transparency information called for in the law. (Note: I examine the act as signed into law, and will likely write another post when the chapter amendments proposed by Governor Hochul are passed by the NY legislature, likely in early 2026).

Probably the biggest issue I see with the law is in the definitions. The RAISE act is geared towards regulating “frontier models” which it defines as: “an artificial intelligence model trained using greater than 10º26 computational operations (e.g., integer or floating-point operations), the compute cost of which exceeds one hundred million dollars” (or a different definition that applies to models produced through knowledge distillation). This is a bad definition because it merges two criteria that are arbitrary, shifting over time, and, most importantly, which model developers are not required to disclose. They are arbitrary because there’s no reason to think that 10^26 computing operations is a magical threshold at which danger suddenly materializes. Based on estimates from Epoch, none of the current breed of frontier models surpasses this threshold, so it’s not clear that the law applies to anything in the real world. And, again, even if OpenAI, Google, or Anthropic has exceeded this threshold in its training there’s no way for us to know because the law doesn’t make them tell anyone. It’s a sort of scouts honor, opt-in system. Also, the definition of compute is in conjunction with a cost greater than $100 million. Because the definition has to match both criteria, a model could use more than the compute threshold but be done for less than $100 million and then the law wouldn’t apply. But compute costs are always getting cheaper, and some model developers like Google control the market price of their computing and so can game this, not to mention that the value of the dollar could change.

The main mechanism for specifying transparency in the law is that large developers of models create a “safety and security protocol” —a form of transparency report—before deployment of the model. There are also provisions to require the reporting of “safety incidents” that might be a case of or increased risk of critical harm. The protocol report is shared with administrative accountability forums such as the attorney general and division of homeland security, as well as being made publicly available in redacted form for media or social forums. The unredacted protocol plus additional information about the tests and test results that inform the protocol need to be maintained for however long the model is deployed plus five years, presumably so that those records are potentially available for discovery by legal forums in the event they are needed. In this sense, the law does pretty well in providing for accessibility of the safety and security protocol to various accountability forums.

The overall aim of the law towards “frontier models” and “critical harm” scopes and sets limits on the relevance of the information in the protocol. Critical harm is defined as causing $1 billion or more in damage or loss of 100+ human lives. But with that scope in mind the definition of the protocol is reasonable as it specifies what should be included, including organizational procedures and sociotechnical measures meant to mitigate the potential for critical harm, as well as the testing procedures used to “evaluate if the frontier model poses an unreasonable risk of critical harm”. The protocol must also designate a person that is responsible for compliance — this is a critical component that ensures accountability for overseeing the protocol. The timeliness of the report is also referenced and calls for the developer to update the protocol on an annual basis as per any changes.

An area where the protocol falls short is in either specifying or auditing the accuracy of the information in the protocol. An earlier version of the law had provisions requiring 3rd party auditing, but those were removed from the final version signed into law. That would have strengthened the law considerably by having an independent entity checking the validity of procedures and the accuracy of provided information in the protocol. What’s left is the comparatively weaker request that large developers not lie, i.e. “shall not knowingly make false or materially misleading statements or omissions.” We can’t really assess whether the information in protocols would be understandable and fit for the purpose of accountability. A stronger law would have created a standard for the protocol that would be considered adequate.

The law provides reasonable carve outs to address typical criticisms and stakeholder pushback about transparency, including that disclosures might undermine privacy, confidentiality, trade secrets, or be used to game the system. Redactions to public safety and security protocols can be undertaken to protect these other interests. The law also protects fundamental innovation by not applying to academic research done at accredited colleges and universities. In addressing the tensions between transparency and other interests at stake, the law probably does about as well as it could, especially because administrative forums like the attorney general can gain access to copies of the protocol that are less redacted, i.e. where redactions only need to respect federal law, and fully unredacted reports must be maintained for possible discovery in legal forums.

Overall, much like its Californian counterpart, New York’s RAISE act is geared towards prospective accountability — trying to prevent future harm. Its scope is narrow around “critical harms”. While it does well to specify the accessibility of the transparency information it calls for, and align that information so it is relevant and timely to its scope, it lacks provisions for ensuring the accuracy of the information, and leaves the understandability of that information up to the large developers who’ll be creating the reports. But it’s not a powerful law because it doesn’t apply to anything in the real world (yet), and it’s unclear whether model developers will ever raise their hand and say that the law actually applies to them. It does provide an example of AI governance through transparency that can inform future legislation. The next version of the law, proposed by the governor’s office and under consideration by the state legislature, is already drastically different in many ways.