AI companies typically approach principles top-down. For instance, Anthropic’s constitution for Claude, published in January 2026, is a detailed statement of values promulgated by the company and encoded into how Claude behaves. But bottom-up approaches drawing on public input are also possible. A new report from the Collective Intelligence Project (CIP) done in collaboration with Anthropic tests one such alternative, drawing on surveys of about 2,200 people in the US, UK and India on how AI should handle free expression issues. It then compared their answers with free-expression-related principles drawn from Anthropic’s constitution.

Subscribe now

The report sets a high bar for what counts as a shared norm. A principle only achieves bridging consensus if more than 85% of participants in every demographic segment agree with it. So it’s not enough to just have a simple majority—it’s got to be a demographically shared agreement. Segments span country, gender and age, plus four cultural quadrants drawn from the Inglehart-Welzel World Values Survey dimensions: traditional versus secular, and survival versus self-expression. Because these values cut across borders, the metric tests agreement across worldviews and not just across nations.

Participant input first produced 100 candidate principles. Of these, 44 made it into the validated framework and 25 “core principles” met the bridging threshold, while 19 met some version of a relaxed criterion that the authors felt still warranted inclusion. The strongest cluster included eight principles at 90–98% agreement. These principles reflect an interest in focusing on evidence-based information, stating uncertainty clearly, citing sources, distinguishing established from uncertain information, identifying disputed claims, and highlighting expert agreement on debated topics.

Other clusters of consensus reflect an interest in explanation around omissions of information (e.g. if content restrictions apply, sensitive information was excluded, or responses are incomplete for some reason), inclusion of warnings (e.g. important unconsidered details, safety notices around potential misuse of information), respect for different cultures (e.g. don’t use stereotypes), elaboration upon request, and provide reasoning support (e.g. facts not opinions, help structuring arguments). Mostly this translates to AI systems providing answers with context rather than refusing to answer. The few exceptions where refusal was accepted were when a response could enable serious harm, create an imminent threat to life, or have clear serious physical harm. In those situations participants were willing to accept some moderation by the system.

Perhaps more illuminating are those principles that didn’t reach much agreement at all, such as requiring government ID to use AI (35% agreement). Others that didn’t reach consensus actually drew majority support but fell short of the author’s threshold, including deferring to government policy (73%), investigating user intent for harm (75%), and withholding answers to suspicious prompts (77%). In fact it does seem there is substantive support for these three principles amongst the participants, even if they don’t reach the authors’ preferred threshold of 85%. This calls into question a key issue: what degree of agreement on this metric is sufficient to provide legitimacy to any given policy?

The study then extracted 30 free expression principles from the roughly 29,700-word Anthropic Constitution and tested them with 353 new participants. Only five reached bridging consensus: (1) not volunteering political opinions, (2) presenting the strongest version of each viewpoint, (3) using neutral terminology, (4) treating access to information as the default, and (5) recognizing that an AI talking to millions shouldn’t homogenize opinion. The other 25 principles extracted from the constitution didn’t reach the consensus threshold, though some still had substantive consensus. Some of the disagreements with the constitution had to do with the boundary between safety and expression. For instance, seven principles about weighing harm against access failed to reach consensus, reflecting the idea that participants tended to err on the cautious side when there’s a safety concern. Other discrepancies had to do with how the principles were framed in the constitution and interpreted by participants.

Overall, this is a promising method for how to surface more legitimate norms and standards to be embedded in AI constitutions. The results show that many of the principles devised by Anthropic for its constitution didn’t find widespread consensus among participants. The “right” threshold for that consensus metric is still an open question. Then again, there were a number of principles surfaced from participants that did reach the authors’ definition consensus and could inform the Anthropic constitution and shape model behavior in ways consistent with participant expectations (e.g. source citation, explicit omission disclosure).

We should take this all with a grain of salt: participants here were recruited through Prolific from only 3 countries. Responses were only in English. A more globally legitimate sample would derive principles from many more countries and across more cultures and worldviews. Testing an AI’s constitution against this kind of consensus finding method is a way to understand divergences with the public. It also creates an opening for accountability insofar as model providers feel they need to explain principles in their constitution which don’t reach consensus. But also we shouldn’t forget that the interests of individuals may not always capture what’s best for society, the information system as a whole, or AI models in particular, such as if there is an incentive mismatch.

Despite these limitations, policy could call for the application of this kind of more participatory method for helping to inform, augment, and critique AI constitutions given their role in training and guiding AI response behavior. It could serve as a democratic legitimating process for the principles in an AI constitution. Factors that policy would need to articulate include what level of consensus reflects a minimal reasonable standard and what the sampling method should be to reflect legitimacy within the societies where the AI is operating.

Note: The author used Claude Opus 5.5 to assist with initial drafting, but then re-wrote the entire draft. There could be some fragments of AI-generated sentences that survived.

References

Collective Intelligence Project (2026) The Free Expression Evaluation: Democratic Principles for AI Free Expression: Findings from a Three-Stage Collective Intelligence Process. https://free-expression.weval.org

One of the challenges they raise is the self-preference bias of LLMs: “When an LLM evaluator shares architectural or training foundations with the evaluated model, the assessment risks systematic distortion”. They propose a principle of independence drawing on the International Ethics Standards Board for Accountants (IESBA) standards which include (1) independence of mind and (2) independence in appearance. Moreover, common law says that an entity with an interest in the outcome of a determination should be disqualified from making it. They argue from these ideas that the AI system used for monitoring should not come from the same model (or model family) as the AI being monitored, and that there should be financial independence in the assessing entity.

The authors further consider what the law in the EU demands in terms of human oversight and monitoring and what that entails for automated oversight. Full human oversight of all interactions is not economically viable, but they argue that a risk-based sampling approach would help fulfill the legal requirements for high-risk use cases. In other words, they still think it would be consistent with the law if a person didn’t oversee and sign-off on every interaction of the agent. This opens the door to a scenario where AI both proactively samples a subset of cases for human attention and also monitors, flags, and escalates problem cases to a human overseer. They argue that this process should be risk-cognizant. So, for instance, in a situation where an action is irreversible or where harm occurs more quickly than monitoring could help intervene, this might require prospective sign-off by a person. On the other hand, “A retrospective regime is adequate where the monitored actions are reversible or where their residual risk to health, safety, or fundamental rights remains acceptable without prior review.”

AI watching AI also means making automated decisions about what to log and retain that might be relevant to an accountability process later. Logging everything isn’t viable from a resource perspective (do we really want data centers everywhere?) nor from a privacy perspective (the more you log, the more likely you sweep in personal data). As Holweg and Hacker write, “a balance needs to be found that enables selective audits but disables surveillance-style universal recording”. Logs need to be retained longer if a potential harm is irreversible or where it would go undetected or take an impacted person longer to recognize the harm. Logging similarly needs to be grounded in an understanding of risk.

All of this means that we shouldn’t expect a one-size-fits-all approach to AI systems monitoring AI agents. Even if a human can’t look at everything (i.e. human on-the-loop rather than in-the-loop), the degree of human involvement in monitoring should align with the risk of the agent’s actions. Oversight of low-risk agent interactions (e.g. reading information from a trusted website) can be more fully automated than high-risk agent interactions (e.g. writing data in a medical context). The use of AI for oversight of AI systems is yet another layer of AI governance that needs more explicit guidance in policy.

References

Holweg M and Hacker P (2026) Should We Allow AI to Police Itself? Assessing the Technical Feasibility and Legal Implications of LLM-Based Agent Evaluation. DOI: 10.2139/ssrn.7378338.

Contestation is an argument between a stakeholder and the AI system. Illustrated in the next figure, the steps of an idealized version of the process go something like this: (1) Harm Discovery: The AI system outputs a harmful decision and a stakeholder learns of it, (2) System Explanation: The stakeholder may request an explanation of the decision provided by the AI system to provide additional understanding for why the system produced the output, (3) Challenge Consideration: The stakeholder considers whether a challenge is warranted based on the system-provided explanation as well as any other available information (e.g. transparency, documentation, audits, etc), (4) Challenge Argument: The stakeholder disagrees with the decision and presents an argument for changing it, grounded in their own information and interpretation, (5) AI System Argument: The AI system may provide a justification to further ground the system output in normative reasons and argue against the stakeholder’s challenge, (6) Resolution: Adjudication of the argument for either the system (the decision stands) or the stakeholder (some change is warranted in the decision or system).

In reality this isn’t a neat or tight process, as research examining dozens of real AI contestation cases has shown, it is an “interactive, dynamic, and often prolonged process” (Pi et al, 2026). It can also vary if the object of contestation is instead the system design itself, or institutional practices that govern the system. The power to compel explanations and justifications and ultimately adjudicate arguments towards resolution is something that legal, political, media or other forums provide in different ways. Power is the key here. In the idealized process the contester is deemed to have the power or authority to expect an explanation or justification. But in reality, operators might not be so cooperative and instead deny or deflect responsibility for the outcome, downplay the severity of the impact, restrict communication channels that enable contestation, blame human operators, withhold information, or strategically concede or delay the process (Pi et al, 2026).

Contestation is supported by available information about the system, as provided through various transparency mechanisms (e.g. auditing, documentation, etc.). It hinges on both explanation and justification. Explanation is about helping someone understand a decision and the facts underlying it, whereas justification is about offering the reasons for why the output is appropriate or acceptable (Henin and Le Métayer, 2022). Explanations offer descriptions of the process leading to the output, whereas justifications draw on norms like law, corporate policy, or ethical principles to argue that the norm supports the outcome provided. The back and forth of explanation and justification needs to be supported by some communication channel that facilitates clear dialog such as a natural language or interactive interface (Yurrita et al, 2025).

To see the distinction between explanation and justification and how they relate to contestation, consider an AI system used by a bank to deny a loan application. The explanation might be that the loan was declined because the applicant’s debt-to-income ratio is above some threshold, whereas the justification is that when debt-to-income is above that threshold it violates the bank’s responsible lending policy and protects the applicant from unaffordable debt. Contestation comes in when arguing against either the explanation (on epistemic grounds) or justification (on normative grounds): perhaps the bank’s data on the applicant’s debt-to-income is inaccurate, or perhaps there is a fair lending norm that could be argued to override bank policy that only narrowly considers that threshold.

Contestation is often framed as something an individual does retrospectively perhaps as an individual right one has as an end-user impacted by the system (Kaminski and Urban, 2021) — an idea reflected in Article 22 of the EU’s GDPR regulation. However, there are also different types of individual stakeholders, from operators to auditors that might also contest a system or its output (Henin and Le Métayer, 2022). Beyond individual contestation there is also interest in enabling groups or coalitions to collectively contest an outcome (Lyons et al, 2021; Schmude et al, 2026; Yurrita et al, 2025). Contestation can be an expensive activity and coalitions or groups (e.g. public-interest, professional, or union organizations) can often marshal more resources, information, and sensemaking capacity while sustaining pressure for change over time (Yurrita et al, 2025). As Upadhyay et al write, “successful contestation can be deeply relational, relying on situated knowledge and coalitions to determine which information is most compelling for a given contestation strategy, how to obtain it, and what actions can achieve accountability while mitigating risks.” (2026). A simple way to put this is that groups of people are more powerful than individuals, so there’s an incentive to work together to better address the power differential with many AI systems.

Contestation through Design

Contestation need not be something invoked only retrospectively. Research has considered how contestation can be supported in the design of a system (Hirsch et al, 2017). Various system design features may enable contestation, such as including a human review process where operators can adjudicate challenges, enabling individuals to find others similarly impacted so they can create coalitions, providing adequate transparency such as appropriate documentation, and incorporating participatory design to better address aspects of the system that might be contested upfront (Alfrink et al, 2022). The general idea is to make AI systems “open and responsive to human intervention throughout their lifecycle: not only after an automated decision has been made, but also during its design and development” (Alfrink et al, 2022).

Veluwenkamp and Buijsman articulate two design features needed to enable operator contestation: epistemic access to the relevant normative reasons (i.e. justifications) for outputs, and authority to act on these reasons (2025). Basically if the operator is going to act as adjudicator of a contestation they need access to system justifications and the authority to consider and potentially override them based on a challenger’s counter argument. Some of the valid reasons they consider for why a system explanation or justification might be challenged include (1) inaccurate system data, (2) data drift such that the model no longer fits reality well, (3) missing data features, (4) system bias, and (5) spurious correlation. The epistemic and normative grounds for contestation based on inaccurate system data and missing data features have recently been formalized (Freiesleben et al, 2026). A further opportunity for epistemic contestation is when there is predictive multiplicity — when different models derived from the same model building process produce different outputs (Freiesleben et al, 2026).

Can AI Argue Back?

Some research considers the possibility that AI systems are able to engage in the contestation process automatically. This requires a system not only to be able to render explanations which are faithful, but also justifications, to interactively argue with a person (or other AI) about an output via a communication channel like natural language, and ultimately to potentially update itself if the contest is deemed valid (Leofante et al, 2024). Fully automating these processes is not currently plausible given that systems often lack access to the reason giving capacity needed for justification (Jahromi et al, 2026).

Henin and Le Métayer are perhaps furthest along in prototyping the possibility here with their Algocate system (2021). The system has access to evidence and norms to logically back an output decision and is equipped with a capacity to compare and rank arguments based on strength. Still, to successfully navigate a resolution, such a system would need to be provided with the authority to overturn a decision, something which adversarial operators with something to lose are unlikely to want to do. People also don’t want to argue with AI systems towards resolution, preferring to have a human reviewer with the requisite capacity to evaluate the challenge and the authority to override the system if needed (Yurrita et al, 2025) though there is also sensitivity to the higher costs this might entail (Lyons et al, 2025).

What Policy Can Do

A primary role of policy to support contestation should be to set the ground rules for fair contestation arguments to take place, with the ultimate goal of enabling better AI system decisions. This means deciding who can contest (e.g. individuals, collectives, etc) and what can be contested (not just outputs but also systemic issues), making sure both sides of the argument have access to relevant information, explanations, and justifications (e.g. through system design features), ensuring communication and interaction channels are available for enabling the argument, and authorizing some entity with the appropriate authority for adjudication. In addition, policy can support harm discovery through more thorough transparency rules and enabling forums such as the media or civil society that engage in monitoring activities.

Ultimately a contestation reflects a power struggle between two entities and policy needs to work to equalize power differentials. To counteract powerful AI system operators, policy might insist on timely disclosure of relevant information, persistent channels for raising challenges, and protection of contesters against retaliation. AI systems should not be authorized to adjudicate challenges not only because the technology to support such computational argumentation is still nascent, but also because it reflects a conflict of interest that undermines fairness. Finally, since coalitions can marshal more resources, information, and sustained pressure than individuals can, policy should enable collective contestation. Policy can create infrastructure that helps affected individuals find and learn from each other such as by establishing public registers of deployed systems and contestation outcomes.

References

Alfrink K, Keller I, Kortuem G, et al. (2022) Contestable AI by Design: Towards a Framework. Minds and Machines 33(4): 613–639.

Freiesleben T, Meding K and König G (2026) Explainable AI Isn’t Enough! Rethinking Algorithmic Contestability. Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society 9(2): 961–973.

Henin C and Métayer DL (2021) A framework to contest and justify algorithmic decisions. AI and Ethics 1(4): 463–476.

Henin C and Métayer DL (2022) Beyond explainability: justifiability and contestability of algorithmic decision systems. AI & SOCIETY 37(4): 1397–1410.

Hirsch T, Merced K, Narayanan S, et al. (2017) Designing Contestability: Interaction Design, Machine Learning, and Mental Health. DIS. Designing Interactive Systems (Conference) 2017: 95–99.

Jahromi MNS, Holm S, Moeslund TB, et al. (2026) Operationalizing AI contestability through technical feasibility analysis. Discover Artificial Intelligence 6(1): 648.

Kaminski M and Urban J (2021) The Right to Contest AI. Columbia Law Review. 121, 7: 1957–2048

Leofante F, Ayoobi H, Dejl A, et al. (2024) Contestable AI needs Computational Argumentation. Proceedings of the International Conference on Principles of Knowledge Representation and Reasoning.

Lyons, H., Velloso, E., & Miller, T. (2021). Conceptualising contestability: Perspectives on contesting algorithmic decisions. Proceedings of the ACM on Human-Computer Interaction, 5(CSCW1), 1–25

Lyons H, Miller T and Velloso E (2025) Diametrically Opposed Attributes? Review Process Preferences and Priorities When Contesting Algorithmic Decisions. Proceedings of the ACM on Human-Computer Interaction 9(7): 1–32.

Pi Y, Lichner L, Lee JW, et al. (2026) Push and Pushback in Contesting AI: Demands for and Resistance to Accountability. Proceedings of the 2026 ACM Conference on Fairness, Accountability, and Transparency: 3590–3625.

Schmude T, Yurrita M, Alfrink K, et al. (2026) Two Means to an End Goal: Connecting Explainability and Contestability in the Regulation of Public Sector AI. In: Conference on Fairness, Accountability, and Transparency, 2026, pp. 3078–3104.

Upadhyay S, Pruss D, DeVrio A, et al. (2026) “It just requires so much more creativity”: Barriers and Workarounds to Gathering Information for AI Contestation. Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems: 1–18.

Veluwenkamp H and Buijsman S (2025) Design for operator contestability: control over autonomous systems by introducing defeaters. AI and Ethics 5(4): 3699–3711.

Yurrita M, Verma H, Balayn A, et al. (2025) Identifying Algorithmic Decision Subjects’ Needs for Meaningful Contestability. Proceedings of the ACM on Human-Computer Interaction 9(7): 1–29.

California’s AI whistleblower protections may even already be yielding benefits as they offer protection to the likes of Jacob Coxon, who recently resigned from Anthropic over AI fears. Perhaps others will also be emboldened to come forward. The laws from California, New York, and Illinois create obligations for publishing frontier AI safety frameworks (tracked here), as well as other bits of transparency. Just the fact that Anthropic and OpenAI have made explicit disclosures of their safety frameworks in compliance with California’s law suggests that their models were trained with more than 10^26 compute operations — otherwise they wouldn’t need to make that disclosure. Some are already raising questions about OpenAI’s compliance with the California law — do recent cybersecurity incidents signal a real loss of control?

The California, New York, and Illinois laws are largely similar. But they also seem to be building on each other. Illinois, for instance, includes provisions for independent third-party audits of compliance with the law. That’s a powerful addition that enhances accountability and was something that was in an early version of the New York law, but got stripped from the final. An advance in both the New York and Illinois laws is the addition of registration and ownership disclosure requirements. For the sake of human responsibility and accountability, it’s critical that society know who the beneficial owners of these large companies are.

These laws aren’t perfect. They are narrowly scoped and don’t address many of the current harms already being enacted by AI systems in society. The degree of similarity in them suggests they are probably being influenced by the tech lobby. They also aren’t responsive enough to deal with the rapid pace of AI advances, and there’s a lag with when they are signed into law and come into force which creates a gap just when we need information the most. But they’re something to build on, and they give us some transparency in the form of safety frameworks (i.e. prevention plans) that the frontier AI companies can and should be held to until the federal government decides to act seriously to regulate AI.

In cybersecurity a “Zero-Day” is a security flaw, unknown to the developer, that when exposed leaves them zero days to fix it before it’s exploited. In their new paper in AI Ethics, Sadler and Sherburn explore the notion of a Legal Zero-Day (2026). The idea is that agents might be able to find vulnerabilities in a society’s legal system that they could exploit in order to facilitate legally dubious and potentially harmful actions that the legal system is not able to reign in through litigation or legislation before impact. It’s bad enough that frontier agents can now hack computer systems, but what if they did it in a way that was immune from legal action against it?

The authors first examine the formal requirements of a Legal Zero-Day, before going on to measure the capabilities of frontier models for finding them using an expert-informed benchmark. They consider five defining characteristics of a Legal Zero-Day: (1) that it is a genuinely novel discovery about the law, (2) that is has immediate effects before any litigation or legal processes could occur, (3) that it doesn’t stem internally from the legal system such as through an executive action, (4) that is can cause significant societal impacts, and (5) that it is time-consuming to rectify and can’t just be waived away through administrative rule-making. In sum, these are new exploits with immediate impacts that can’t be easily rectified.

To create a benchmark to evaluate model performance in finding Legal Zero-Days, the authors worked with legal experts to develop legal puzzles. These puzzles take existing law and introduce a tweak that might go unnoticed but which can be exploited. For instance, shifting the scope of a definition in a law just a little bit might drastically change who is subject to enforcement or make it easy for actors to side-step the underlying intent of the law. Legal experts created nine of these puzzles for the benchmark covering instances from Australia, the UK, India, and Brazil. Each puzzle describes an original law or legal framework, a modified version of it which introduces the vulnerability, and an explanation of the legal logic of how the modification creates issues.

They then tested six models (a frontier model and a smaller version each from OpenAI, Anthropic, and Google) on their ability to find the loophole in each legal puzzle. The legal experts graded 25 responses to create a ground truth and then a prompt was developed that aligned with the legal expert opinion so that the evaluation could be automated. Important to add that they were testing just the model here, not tools or access to the internet. The results show that no model achieved more than 10% detection of exploits introduced in the puzzles. Four of the puzzles were entirely unsolved, while the other five had detection rates spanning 3% to 30%.

The results show that, while there is some evidence that models can find some legal exploits, there is a considerable capability gap in their ability to do so reliably and for a wide variety of puzzles. Still, I find the implications here worth paying attention to, especially since these models were tested without access to tools or search. I assume any model would do much much better at this task if they could access any number of legal interpretations or critiques available on the internet.

Even with limited capability, now that we know this is a possibility, governments should be scanning LLMs over their current legal code as well as any new legislation drafted to identify potential exploits that can be remedied. Our research on using LLMs to simulate policy in light of various changes suggests that such methods can identify policy options that help mitigate risk (Barnett et al, 2024; Barnett et al, 2026), but the method could just as well be flipped around to simulate which policy changes introduce new risks, such as Legal Zero-Days. We need to start making our systems more robust now before agent capability improves.

Subscribe now

References

Barnett J, Kieslich K, Helberger N, et al. (2026) Informing AI Policy Assessment using Large-Scale Simulation of Interventions. Proceedings of the 2026 ACM Conference on Fairness, Accountability, and Transparency: 3942–3978.

Barnett J, Kieslich K and Diakopoulos N (2024) Simulating Policy Impacts: Developing a Generative Scenario Writing Method to Evaluate the Perceived Effects of Regulation. In: Proc. AI, Ethics, and Society, 2024.

Sadler G and Sherburn N (2026) Legal Zero-Days: a novel risk vector for advanced AI systems. AI and Ethics 6(3): 330.

The most basic form of transparency is just knowing whether an AI system is being used at all. There’s no chance to contest or audit an AI system’s decisions or impacts without that fundamental awareness. All of the more complex transparency information that might diagnose an AI system’s failures is downstream of that initial trigger. And so AI accountability hinges on building awareness in society and amongst the relevant forums about where AI is actually deployed within complex systems.

AI registers are one approach that might help build that necessary awareness by making publicly available an inventory of AI systems with societal impact. Since the earliest public registers were launched in Amsterdam and Helsinki in 2020 (Floridi, 2020), there’s been a proliferation of them around the world, with one recent report clocking 83 (Gutiérrez and Muñoz-Cadena, 2024). New York City has one, as does San Jose. An executive order in the US in late 2020 called for federal agencies to publish public inventories of AI use which was later codified into law (which sunsets in 2027) and elaborated by the executive branch, while in the EU the AI Act calls for a register of high-risk AI systems. A number of states have also enacted laws which legislate AI registers to varying degrees (Anex-Ries, 2024).

Nieuwenhuizen’s interview study of oversight authorities and societal watchdogs in the Netherlands found that “one of the most significant positive implications is that the register serves as a crucial starting point for further investigations. The information, or its absence, acts as a point of departure for in-depth examinations.” (2024). By making some basic information public, AI registers provide a foothold for further scrutiny. Research on the Canadian federal public AI register has further demonstrated the downstream analytic value of having a national AI register, namely to provide an overview of the various ways AI is used across government, to track the framing of those uses, and to understand gaps or “bureaucratic silences” (Das et al, 2026).

AI registers have been critiqued over the quality of information included in them, echoing previous observations about AI transparency in general. Issues brought up include accuracy, inconsistency, and understandability of metadata as well as the relevance and comprehensiveness of what systems are selected for inclusion (Meijen and Gujela, 2025; Cath and Jansen, 2022). In the absence of a legal mandate which dictates these elements, authorities can selectively include or exclude AI systems or information about them or inconsistently provide that information (Das et al, 2026; Nieuwenhuizen, 2024). Discretion in the disclosure process is seen as problematic for realizing the full value of registers (Popa, 2025; Pi et al, 2026). As Das notes in their critique of the Canadian AI register, “even when systems are disclosed, they often omit governance-critical information, including degrees of uncertainty, training arrangements, and discretion configuration, while departmental latitude to reinterpret central directives produces heterogeneous reporting standards” (Das et al, 2026). Cath and Jansen further critique AI registers on the grounds that they normalize the use of AI in public bodies (2022), though given the proliferation of AI throughout society it seems society is well beyond debating whether to use AI, and is now more concerned with using it responsibly.

Nieuwenhuizen elaborates several design considerations for effective AI registers, including specifying the audience for the information, the unit of disclosure, disclosure formats, and what details to include (2024). Structural best practices outlined by the Center for Democracy and Technology include collating registries across the government, standardizing information reported across agencies, making registries fully public, updating registries on a set periodic schedule, ensuring inventory language is accessible and understandable (and also machine readable), and making sure there is accountable leadership for the registry process (Anex-Reis, 2024). These ideas help push toward addressing at least some of the information quality issues outlined above.

There is also the question of what exactly to include in an AI register. An analysis of 16 different inventories from around the world identified a harmonized set of 23 distinct fields with a descriptive core set included in most inventories (Das and Guha, 2026). This core included name, responsible organization, description, technology type, and lifecycle status, while fewer registers included information about risks and legal bases, external evaluations, or appeal processes (See Figure). Some standards have begun to emerge which outline what exactly should go in an AI register. In the UK, the Algorithmic Transparency Recording Standard (ATRS) “establishes a standardised way for public sector organisations to publish information about how and why they are using algorithmic tools”. And a consortium of European cities has developed an Algorithmic Transparency Standard, which is meant to help cities “provide clear information about the algorithmic tools they use, and why they’re using them.” These standards articulate tiers of information and define and help standardize what to include.

The development of standards is critical for specifying which bits of data belong in a register, but those standards also need a governance layer to enforce them and to verify information quality. As Das and Guhu show, just because there is a robust scheme for information disclosure doesn’t mean it will always (or even often) be mostly filled out (2026). One approach recently developed is to establish a checklist to audit whether a register fulfills expectations set out in the standard (Peljto et al, 2026). Ideally policy would require such an audit against established standard criteria for a register. Auditing needs to be done by an entity with enough access to the process to accurately evaluate whether certain criteria were met.

The US federal example is surprisingly responsive to the issues of information quality raised in the above critiques, with the Office for Management and Budget (OMB) guidance helping to set the scope for inclusion, articulate exclusions or consolidate common cases (like using generative AI to summarize a document), encourage understandability of text, designate high risk cases, and even normalize metadata that was reported differently by different agencies. Still, it remains unclear what the protocol should be to rigorously assess the accuracy, comprehensiveness, bias and other aspects of information quality provided in the register. Perhaps OMB should audit those things as part of an administrative chain using a checklist akin to what (Peljto et al, 2026) proposes, but perhaps that is also where the work of accountability begins for external forums that might solicit more information on any of the cases disclosed using a freedom of information request. And, while it’s a great first step to make the data available, it’s still not terribly accessible in terms of ease of searching, filtering, or visualizing what’s there. Though forums can now leverage AI to build interactive analytic dashboards to support this last step. It only took me about fifteen minutes to create one for the 3611 federal use cases for AI reported in 2025.

References

Anex-Ries Q (2024) Best Practices for Public Sector AI Use Case Inventories. Center for Democracy and Technology.

Cath C and Jansen F (2022) Dutch Comfort: The Limits of AI Governance through Municipal Registers. Techné: Research in Philosophy and Technology 26(3): 395–412.

Das D, Tessono C, Ahmed SI, et al. (2026) Bureaucratic Silences: What the Canadian AI Register Reveals, Omits, and Obscures. Proceedings of the 2026 ACM Conference on Fairness, Accountability, and Transparency: 4003–4026.

Das D and Guha S (2026) A Global Comparison of Schemas, Transparency, and Interoperability in Public-Sector AI Registers and Inventories. ACM Conference on Digital Transformation.

Floridi L (2020) Artificial Intelligence as a Public Service: Learning from Amsterdam and Helsinki. Philosophy & Technology 33(4): 541–546.

Gutiérrez JD and Sarah Muñoz-Cadena (2024) Algorithmic Transparency in the Public Sector: A state-of-the-art report of algorithmic transparency instruments. Global Partnership on AI (GPAI).

Meijen J and Gujela N (2025) Empowering Citizens through Responsible AI Governance: Policy Recommendations for Public Algorithm Registers. Advancing Responsible AI in Public Sector Application, GPAI Edition. 51–61.

Nieuwenhuizen E (2024) Algorithm Registers: A Box-Ticking Exercise or Meaningful Tool for Transparency? Information Polity 29(4): 415–433.

Peljto I, Heilmann X and Cerrato M (2026) Are Algorithm Registers Transparent? Perspectives from Germany. arXiv. Epub ahead of print 2026. DOI: 10.48550/arxiv.2606.02347.

Pi Y, Singh J and Li W (2026) Understanding the Role of Algorithm Registers in AI Governance Through Comparative Analysis of China and the UK. Proceedings of the 2026 ACM Conference on Fairness, Accountability, and Transparency: 3367–3391.

Popa DM (2025) Frontrunner model for responsible AI governance in the public sector: the Dutch perspective. AI and Ethics 5(3): 2789–2799.

A new research paper published at the 2026 Conference on Fairness, Accountability, and Transparency (Pelloth and Zweig, 2026) takes a crack at evaluating the quality of AI guidelines documents. As the scope of their work the authors define an AI guideline as “a document containing recommendations aimed at aiding individuals, organizations, or society in the development of AI systems which have no legal binding.” Their specific focus is on AI development guidelines rather than application or use guidelines, which are important areas for future work.

To develop their evaluation rubric they adopt and adapt material from a medical clinical practice guideline evaluation called AGREE II. The final rubric includes 28 items across 7 domains: scope and purpose, stakeholder involvement, rigor of development, completeness, clarity, applicability, and editorial independence.

To apply the rubric to an AI guidelines document, the authors use a 7-point scale to grade each item. Rather than just average everything together, each domain gets its own normalized score based on the items in that domain. You can see the appendix of the paper for the full ratings manual on how to use it and how they define and rate each of the items. Based on their tests in applying the rubric, they recommend that at least two independent raters be used.

The authors applied the tool to four prominent guidelines — the OECD’s Recommendation of the Council on Artificial Intelligence, UNESCO’s Recommendation on the Ethics of Artificial Intelligence, the EU’s HLEG Ethics Guidelines for Trustworthy AI, and IEEE’s algorithmic bias standard (7003-2024). Each scored poorly on Rigor of Development because they don’t disclose how their recommendations were actually derived.

This is a useful step towards helping to understand the quality of AI guidelines. But what’s lacking is a full-scale evaluation to connect the dots between a low or high-rated guidelines document and its efficacy in practice. Still, it can be a useful tool, not only retrospectively, but also for orienting guideline writers towards aspects that should improve quality.

Subscribe now

References

Pelloth JM and Zweig KA (2026) Developing an Assessment Tool for AI Guidelines. Proceedings of the 2026 ACM Conference on Fairness, Accountability, and Transparency: 4511–4551.

Cowork is a desktop tool that lets Anthropic models act agentically on your computer, working with local files and using the Chrome browser to access information on the web. The basic workflow is that I first create a new project in Cowork for the conference. I then prompt the system to scrape the papers for the conference by directing it to the proceedings or digital library page. For some mega-conferences like ICLR I point it at a subsection like “Social Aspects” as that already cuts down the volume. The agent is usually pretty good at figuring out how to download the papers into a structured file including titles, authors, abstracts, and DOIs or links. Sometimes there’s some back and forth to help it navigate specific issues. For instance, in one case the agent got blocked when accessing the ACM Digital Library because it was working too quickly and triggered an IP block. A university VPN is helpful for getting around that.

Once you’ve got a list of papers and their abstracts in a structured JSON file, the next step is to screen them according to a relevance filter. Because AIAR is a living literature review, the premise is that I have a set of articles I’ve written on sub-topics, like responsibility, transparency, or explanation, and then want to find literature to advance, update, or elaborate those articles. In my workflow, I use the text of my articles to define the relevance of new papers. Each of my articles is included in the Cowork project as a markdown file. Then I use something like the following prompt to rate each candidate paper against each of my articles:

“Assess the relevance of each of the abstracts in the iclr2026 social aspects papers json file on a three point scale: low=1, medium=2, and high=3 in terms of how relevant the information is to the ideas reflected in each of the blog posts contained in the .md files (except for About_AIAR.md). So there should be 10 relevance scores for each abstract in all. If ANY of the 10 relevance scores equals 3 (i.e. high) then include that abstract as well as other metadata about the paper (title, authors, DOI, link) into a new rated.rss output file. If a paper makes it into rated.rss, its description should also include a list indicating which of the 10 blog posts it’s highly relevant to. If rated.rss contains more than 100 items, then split it into subsets of no more than 100 items.”

Claude will typically spin up a number of sub-agents (e.g. it used 9 in the above case) and tear through the ratings process in a matter of minutes. I output to RSS feeds because it integrates easily with Inoreader for manually reviewing and deciding whether to read it and include it in the AIAR.

The potential risks from using a tool like Claude to help with this screening work is that I potentially miss papers that are relevant to AIAR (false negative), or that some papers are classified as relevant but I decide after manual review that they’re really not-so-relevant (false positive). False positive errors add a minimal amount of work to my manual screening pass, so are not a big deal. False negatives are potentially an issue though, since there could be papers being published that would be highly relevant to AIAR but that I miss. Still, this type of error is not catastrophic for a narrative review like mine (which makes no claim to have found every paper on a topic). And if a paper was really resonating on social media or via other network channels I might still become aware of it and include it in the review.

For systematic literature reviews it’s typically important to evaluate the false positive and false negative rates for screening support tools to show both what might be missed, and also indicate how much of an efficiency gain is created. Cowork can also help with this by integrating Artifacts which are live interactive apps that can pull in the data from the project and be used to assess relevance ratings against my own judgement. When I was screening abstracts from FAccT 2026, I prompted Claude to create an artifact interface that showed a sample of 10 papers marked as high relevance and 10 marked as low or medium relevance. Then I spent about 5-10 minutes manually (and blindly) rating those 20, and was able to measure the false positive and false negative rates (for the record, 1 false positive and 1 false negative). The interface looks like this:

The workflow above has been hugely valuable for refining large sets of articles into more rarified sets that save me time. Instead of wading through more than 1500 papers from the CHI conference this year, I looked at about 150. One area of unease I have, though, is that because relevance is defined in terms of what I’ve already written, the screening might miss newly emerging areas or topics that I simply haven’t gotten around to writing about. For that Claude can also be helpful though. I asked it to create thematic clusters of abstracts from the FAccT conference this year and it synthesized 8 topics that each connected at least 3 papers from the conference. None of the 8 were surprising to me, but they did draw my attention to a couple of trends at the conference that prompt me to want to write them up.

References

Wei Z, Ngongoma L, Cols J, et al. (2026) Artificial Intelligence (AI) Readiness to Support Evidence Synthesis by Workflow: Findings From a Review of Reviews. Campbell Systematic Reviews 22(2): 18911803261454704.

The OECD defines AI agents as “systems that perceive and act upon their environment with a degree of autonomy, using tools as needed to achieve specific goals and adapt to changing inputs and contexts.” Essentially they are AI systems that tend towards the higher end of autonomy and adaptiveness in their environment. But with increased autonomy (and the lack of direct human control) comes increased risk for these systems to take actions in the world (e.g. via tool calls, APIs, or other outputs) that might be harmful. How do we ensure accountability for the harms they will invariably cause through their actions?

Over the course of decades research has developed ideas for the governance of human agents in terms of principal-agent models (Ross, 1973). The basic idea is that some actor (the principal) delegates a task to another actor (the agent) where there might be differences in goals or risk profiles between the two and it is difficult for the principal to verify what the agent is doing. Principal-agent theory encompasses a flexible array of models that explore this relationship, and have been applied across fields such as economics, management, and public administration (Ross, 1973; Eisenhardt, 1989; Gailmard, 2012).

Principal-agent models set up an information regime where the principal can specify the outcome or behavior they want from the agent, provide incentives to do so, and get information back to evaluate whether it was done to their liking. But information asymmetry and goal divergence can lead to issues of “moral hazard” (i.e. the agent does something at odds with the principal’s goals or risk profile) or “adverse selection” (i.e. the principal lacks information about the capability of agent before delegation) (Eisenhardt, 1989). A certain degree of agency loss is pragmatically inevitable for the principal (Gailmard, 2012) since eliminating such loss would require a fully specified contract for all expected behavior across all contexts of action and with perfect surveillance and verification of agent behavior (Hadfield-Menell and Hadfield, 2019). In economic terms the principal invariably has to cede some control over the outcome if it wants to save time or cost through delegation. Sure you could have a principal-in-the-loop check every output, but at scale the principal loses the efficiency benefit gained through delegation.

In writing about the governance of AI agents through the principal-agent lens, Kolt (2024) elaborates issues of authority and discretion ceded to agents, loyalty of an agent to a principal, and the challenge of multiple co-principals (sometimes referred to as “common agency”). He raises an important critique which is that shaping agent behavior toward what the principal wants though incentive or sanction mechanisms doesn’t really work for AI agents — they don’t respond to financial motivation or potential negative social sanction (e.g. psychological or reputation effects) the same way people do. Though I would add that even as we accept that AI agents themselves are not susceptible to psychological or reputational effects we shouldn’t forget that their human developers still are.

Several ideas have surfaced which could strengthen the accountability of the principal for actions taken by their agent. One of the most well-developed is to increase the visibility (i.e. transparency) of the agent vis-a-vis the principal. This might include the use of agent identifiers (flagging when an AI agent is involved in an activity), real-time monitoring (continuously tracking and analyzing agent behavior), and logging (recording and documenting what agents do) (Chan et al, 2024). Additional aspects to track and disclose include system documentation (e.g. parameters, versions) and tool use documentation (Ezell et al, 2025) as well as reasoning traces, confidence maps, counterfactuals, and guardrail events (Prause, 2026). Logging and monitoring approaches incur costs and are challenged by the speed and scale of AI agents, but they can help narrow information asymmetries, agency loss, and ultimately empower the principal (Kolt, 2024). A principal may well delegate the monitoring function to another entity—including to an AI system—that can better keep up with the scale, pace, and detail of the agent’s logs.

Beyond monitoring, Prause (2026) adds that screening mechanisms can help principals understand the capabilities of agents before anything is delegated. This could take the form of AI agent benchmarks that help close information asymmetries and support principals taking responsibility for ensuring an AI agent is capable before delegation. Hacker and Holweg (2026) propose elaborating public policy on AI agents by specifying the frequency and scope of human oversight together with a documentation mandate. Some actions may require additional structural safeguards (such as strict human review requirements), or should be outright banned for agents to take (e.g. financial transactions above some threshold) (Hacker and Holweg 2026; Prause, 2026). Nama (2026) further suggests that principals delegating to AI agents meet agentic AI literacy standards. This could include understanding the nature and scope of authority delegated, how to effectively monitor and intervene on the agent, and understanding available recourse or reversibility options. Such an approach would support the knowledge criterion central to establishing principal responsibility for the actions their AI agent takes on their behalf.

Empirical research is just beginning to examine how principals delegate tasks to AI agents differently than human agents, reflecting principals’ beliefs about AI agents’ obedience but also the recognition that they need to overspecify tasks to avoid mis-alignment and provide necessary knowledge and expertise they believe deficient in the agent (Petridis et al, 2026). People appear to find it somewhat freeing to not have to deal with the “social overhead” of managing another human being. Interaction methods and paradigms for principals to better control agents might include providing rough drafts and test runs for a principal to evaluate as well as establishing check-in criteria for the agent to communicate with the principal, which all align well with proposals for improved monitoring and screening.

A looming gap in the AI agent literature appears to be an in-depth and extended treatment of the co-principal issue. AI Agents almost always have co-principals (or perhaps hierarchical principals): their developers and their users. There is ample room for moral hazard when the co-principals disagree about what the agent should do, and apportioning responsibility between the two seems to be the crux of the issue. Proposals for agentic AI literacy and better benchmarks for screening agent capability strengthen the role and responsibility of the user, while monitoring and logging demand infrastructural and access considerations that realistically can only fall on developers. While transparency is perhaps the most viable option for ensuring principals maintain control, policy should also be mindful that increased monitoring and logging (including potentially of principals’ oversight) also tends towards surveillance.

References

Chan A, Salganik R, Markelius A, et al. (2023) Harms from Increasingly Agentic Algorithmic Systems. 2023 ACM Conference on Fairness, Accountability, and Transparency: 651–666.

Chan A, Ezell C, Kaufmann M, et al. (2024) Visibility into AI Agents. The 2024 ACM Conference on Fairness, Accountability, and Transparency: 958–973.

Eisenhardt KM (1989) Agency Theory: An Assessment and Review. The Academy of Management Review 14(1): 57.

Ezell C, Roberts-Gaal X and Chan A (2025) Incident Analysis for AI Agents. Proc. AI, Ethics, and Society (AIES) DOI: 10.48550/arxiv.2508.14231.

Gailmard S (2012) Accountability and Principal-Agent Models. In: Oxford Handbook of Public Accountability. Oxford University Press.

Hacker P and Holweg M (2026) A pragmatic approach to regulating AI agents. arXiv. https://arxiv.org/abs/2604.22819

Hadfield-Menell D and Hadfield GK (2019) Incomplete Contracting and AI Alignment. Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society: 417–422.

Kolt N (2024) Governing AI Agents. Notre Dame Law Review 101.

Nama R (2026) From evaluator to principal: the agentic AI literacy framework (AALF) for delegated autonomy. AI and Ethics 6(3): 299.

Petridis S, Liu MX, Fiannaca AJ, et al. (2026) Compass vs Railway Tracks: Unpacking User Mental Models for Communicating Long-Horizon Work to Humans vs. AI. Proceedings of the 2026 Designing Interactive Systems Conference: 1188–1204.

Prause M (2026) No skin in the game: why agentic AI requires principal-agent governance. AI and Ethics 6(2): 199.

Ross SA (1973) The Principal’s Problem. The American Economic Review 63(2).

In a recent paper published in Digital Journalism we report on an interview study where we talked to news audiences in the US about some of the problems AI labeling might address for them. Participants described how labels can enhance their understanding of responsibility, clarify attribution, cue credibility, maintain trust, and enhance their autonomy. But it wasn’t just an “AI” label that they desired. They also wanted more clarity on how people were involved in the content creation process, such as oversight or specific tasks they were assisted with. The signalling of human oversight was seen as a way to re-assure audiences about information integrity. Labeling AI output was seen as a way to support reader autonomy so they could effectively opt in or out of consuming a piece of content depending on their context.

Audiences see some potential benefit from AI labeling, and apparently policymakers do too. It’s the main idea behind recently passed (but not yet signed by the governor) legislation in the state of New York called the “fundamental artificial intelligence requirements in (FAIR) news act”. That proposal calls for news media published or disseminated in New York to “conspicuously imprint” at the top of the page that the content was “substantially created by generative artificial intelligence.”

The recently published EU Code of Practice on AI-Generated Content also specifies how to fulfill the EU AI Act’s requirement for labeling by deployers (i.e. users) of AI used for creating content. Whereas the NY law doesn’t specify what labels should look like in practice, the EU code offers a set of reference icons that deployers can use for labeling fully AI generated or AI modified content:

While the core idea is similar, the scope of the two laws is different and reflects differing value trade-offs. The NY law is scoped around news media which is broadly defined to include “news, weather, traffic, sports, or entertainment” published across a range of formats, whereas the EU law applies to all AI-generated images, audio, and video as well as text “with the purpose of informing the public on matters of public interest.” The EU law has an exception for content with human oversight, where the obligation to label doesn’t apply when “the AI-generated content has undergone a process of human review or editorial control and where a natural or legal person holds editorial responsibility for the publication of the content.” Whereas New York is explicitly targeting news media, the EU has what is essentially a carve out for news media as long as they ensure there’s editorial oversight and a person who takes responsibility and is publicly identified. That exception trades-off reader autonomy in choosing to opt out of AI-generated content in exchange for naming an explicit entity being responsible/accountable for the content. In that sense the New York law is a stronger statement for empowering the reader with the knowledge that something was produced by AI, regardless of any editorial oversight. The EU rule seems to be a slippery slope: Who defines the level or degree of editorial oversight that warrants this exception?

Another point to consider is that neither policy proposal is particularly clear on what is probably the most typical case: AI-assisted content creation. Especially given the audience demand for understanding human-AI co-involvement in creation that we saw in our study, how can we label content when both AI and people play different roles? This is a real struggle highlighted by a recent case where a McClatchy newsroom unionized in order for reporters to assert that they didn’t want to be named in a byline used to label content that had been automatically generated based on their original reporting and writing. Whereas the company thought that by naming the reporter this would provide the responsibility and accountability for the content, the reporters themselves felt that their loss of autonomy in the deployment and use of the technology undermined any sense of responsibility they felt for the AI-generated output. In this case, the reporters felt that a byline indicating “AI assistance” was unwarranted given the role they played in producing the output.

I don’t blame policymakers for not wanting to wade into the sticky territory of determining how to label jointly-produced content. It’s easier to stick to a clear “AI” (or not) label. If AI was used to produce the content, then label it, and let end-users opt out accordingly. If there was enough human effort to not have to label it “AI”, either because there is editorial oversight in the EU, or because it meets the standard for copyrightability in New York, then maybe it doesn’t need that AI label at all. In that case it’s perhaps better to have a human byline to make responsibility for the content clear. But that person also has to be willing to assume that responsibility and not have a corporation foist it on them.

Subscribe now

Apologies are statements that typically express regret and sometimes include an attribution that elaborates why an undesired outcome came about. Their social utility is in acknowledging blame and communicating a willingness to improve in the future, or in explaining why the apologizer couldn’t control the outcome. There are three broad apology types studied in the literature: (1) the basic “we’re sorry” which doesn’t identify the source of the problem, (2) internal blame that acknowledges the role of the apologizer, (3) external blame that attributes factors outside of the apologizer’s control for the cause of the error.

Through a series of experiments with human respondents Turel and Cui find that there are some tasks and types of apologies that do work to repair reliance in the AI system they study. Simple apologies didn’t work at all. But apologies from the system that blamed external factors (i.e. that the data fed into the system was insufficient) repaired reliance better than an internal attribution to the system’s own limitations (i.e. the AI tool lacked capability to do the task). This was true for what the authors call “objective” tasks — in this case estimating a person’s weight from an image.

What I find troubling about the findings is that deflecting the blame worked better than acknowledging it. The risk for accountability is that individuals subject to AI errors might inappropriately brush aside an error rather than contesting it and asking for a deeper explanation. Perhaps AI system apologies should instead locate and identify the human actors in the system who are offering the apology. The findings further remind us of the importance of faithfulness in AI explanations. Given the issues with sycophancy in LLMs it’s not hard to imagine a situation where models might learn to deflect blame in their “apologies” by offering not-so-faithful renditions of why an error occurred due to some external factor beyond the system. While there are certainly legitimate situations where an explanation due to external factors is warranted, policy desperately needs to grapple with the validity of explanations rendered directly by AI systems.

References

Turel O and Cui T (2026) Apologizing artificial intelligence: designing and evaluating effective AI apologies after errors. AI & SOCIETY: 1–18.

Like accountability, responsibility is a relational concept: people are responsible to someone for something, creating “an expectation of an action or its result” (Lenk, 2006). Depending on the who and what in that relationship can lead to different flavors of responsibility, like moral responsibility, or legal responsibility. Vincent (2011) counts at least six different concepts referred to by the word “responsibility”, including virtue responsibility (i.e. character and commitment to doing what’s right), role responsibility (i.e. the duties an actor should or shouldn’t do), outcome responsibility (i.e. backward looking attribution for a state of affairs), causal responsibility (i.e. the cause of the state of affairs), capacity responsibility (i.e. whether the actor has the requisite cognitive or material ability to be responsible), and liability responsibility (i.e. the financial or other action needed to make things right).

Building on decades of research on the psychology and cognition of how humans assign responsibility (e.g. Attribution Theory — Kelley and Michela, 1980), Franklin and colleagues (2022) published a model of AI responsibility that implicates nine different factors and their relationships in attributing responsibility:

There are overlaps and analogues to some of the philosophical conceptions of responsibility Vincent outlines, including causality, role, character (i.e. virtue), and capability. Additional factors include intent—classically associated with assignments of moral responsibility (Nissenbaum, 1994), desire/aim (i.e. an orientation that precedes commitment to an intended action), objective foreseeability (i.e. how likely an outcome really is, regardless of the agent’s expectations), and autonomy (i.e. ability for agent to freely make decisions). Failure to clarify these factors can lead to what has been termed a “responsibility gap”, such as when a person lacks sufficient knowledge over an autonomous AI system whose interactions with the environment make its behavior difficult to foresee (Matthias, 2004).

Some of these factors can be elaborated or nuanced further. As one example, the causality factor can be broken down into an internal (i.e. factors about the automation itself like its programming) vs. external (i.e. situational factors or data quality) cause, which also shapes how people perceive responsibility (Pareek et al, 2025). Another dimension that is relevant to how people view responsibility is whether the situation is low or high-stakes (Tsumura and Yamada, 2025; Pareek et al, 2025). Moreover, a cross-cultural study showed that while intent is implicated in moral judgements in many large-scale industrialized societies, there are some small-scale societies where it is not (Barrett et al, 2016), which has implications for global AI responsibility standards and policy.

These various factors are helpful for analyzing responsibility in an AI system. For instance, an autonomous vehicle (AV) may have physically caused the damage to another car that it rear-ends during a snow storm. But if the driver was supposed to be vigilant (i.e. that was their role responsibility) we might think they share some of the outcome responsibility. If the driver was distracted by a TV show they were watching on their phone, this might weigh the responsibility more toward them (i.e. due to a character failure). In contrast, if the deploying company had knowledge that the AV lacked the capability to detect and stop before hitting another vehicle in a common situation like a snow storm, they could have objectively foreseen such an incident, weighing the responsibility toward them.

What does this all mean practically for AI accountability? These factors enumerated above should play a more explicit role in what we expect from AI system explanations. In order to make nuanced decisions about assigning responsibility and calling for accountability, explanations of AI systems need to include information that helps accountability forums assess things like capability, knowledge, causality, character, role, intent, and autonomy of actors in the system. These factors suggest bits of information needed to inform responsibility judgements and should be the basis for explanation requirements and transparency regimes so that the needed information is available. Accountability forums such as the media or administrative agencies can use the factors to inform lines of questioning, AI incident cards could be designed to include descriptions of each factor, and a growing list of AI regulations around the world could be analyzed through this framework to assess whether they’re really compelling the right information needed for responsibility assignment.

References

Barrett HC, Bolyanatz A, Crittenden AN, et al. (2016) Small-scale societies exhibit fundamental variation in the role of intentions in moral judgment. Proceedings of the National Academy of Sciences 113(17): 4688–4693.

Franklin M, Ashton H, Awad E, et al. (2022) Causal Framework of Artificial Autonomous Agent Responsibility. Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society: 276–284.

Kelley HH and Michela JL (1980) Attribution theory and research. Annual review of psychology 31(1): 457–501.

Lenk H (2006) What is Responsibility? Philosophy Now (56). https://philosophynow.org/issues/56/What_is_Responsibility

Matthias A (2004) The responsibility gap: Ascribing responsibility for the actions of learning automata. Ethics and Information Technology 6(3): 175–183.

Nissenbaum H (1994) Computing and accountability. Communications of the ACM 37(1): 72–80.

Pareek S, Schömbs S, Velloso E, et al. (2025) “It’s Not the AI’s Fault Because It Relies Purely on Data”: How Causal Attributions of AI Decisions Shape Trust in AI Systems. Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems: 1–18.

Tsumura T and Yamada S (2025) Effects of knowledge and importance on responsibility in human-AI decision making. Scientific Reports 16(1): 2670.

Vincent NA (2011) A Structured Taxonomy of Responsibility Concepts. In: Moral Responsibility. Library of Ethics and AppliedPhilosophy, pp. 15–35.

From the 30 cases analyzed the authors report that in about 87% of cases the whistleblowers were motivated, at least in part, by moral considerations, with 27% indicating some kind of financial motivation. At least 90% were insiders at the offending organization and roughly 80% were mid-level employees or executives. But stepping forward was costly: 57–67% faced retaliation (e.g. harassment or unjust termination), 43–57% suffered negative career consequences, and 13% received death threats. Only 13% sought anonymity—although the authors caution that this likely reflects sampling bias toward famous cases. In short, whistleblowers in the dataset tended to be morally motivated insiders who paid a steep personal price.

Based on these patterns and their own observations the authors develop a design sketch for an AI whistleblower office. They claim it should: (1) financially reward tipsters with a percentage of sanctions, in the spirit of the SEC and CFTC programs, given that this can be a motivator ; (2) prohibit retaliation and offer witness protection (plus S visas for international tipsters); (3) enable anonymous tipping via lawyers or a secure online platform; (4) be adequately staffed and funded for effective “tip-sifting”; and (5) invest in messaging to raise awareness for the office and an advisory body to help would-be whistleblowers determine whether they have reasonable cause.

For AI accountability, this work adds a new dimension to transparency. Mandated disclosures and external audits will always leave gaps and insider reporting is one of the few channels likely to surface willful concealment. The recommendations align with a prospective accountability frame: supporting protection, anonymity, and an advice body for potential whistleblowers are forward-looking responsibilities that might make insider reporting a more viable option before harms have occurred. A sample of 30 is small, and the cases skew U.S., famous, and successfully-tipped—but as a starting point for thinking about policy the empirical grounding is valuable.

Note: This post was drafted by Claude Opus 4.7 under the prompting, supervision, and further editing by the author.

In the event of an incident where an AI system causes harm, the people responsible for that system may be expected to render an explanation to various accountability forums, such as the media, administrative bodies, or in a courtroom. This relationship between an actor and a forum where the actor is obliged to explain their conduct to the forum is a key aspect of accountability.

There is an array of research on the topic of explainable AI (i.e. XAI) extending back for nearly a decade (Lipton, 2018; Mittelstadt et al, 2018; Wachter et al, 2017), with a focus on making the technical components of AI systems more understandable. Explanation in this context has been defined as “the ability to articulate why a model produced a given output in a way that is accessible to human users” (Dhar et al, 2025). Explanation can be distinguished from justification, which focuses on the appropriateness or adequacy of the norms and reasons governing an output (e.g. decision) (Henin and Métayer, 2022; Schmude et al, 2026).

Research has raised the critique that the literature on XAI—which mirrors critiques of AI transparency more broadly— hasn’t always been clear about the goals of AI explanations (Dhar et al, 2025; Alpsancar et al, 2025). Who and what are they for, really? If explanations are meant to support accountability, there is still somewhat of a gap in the literature showing exactly how, particularly if we parse out the different goals of retrospective vs. prospective accountability.

Dhar et al present a framework for thinking about the goals of AI explanation in terms of who explanations are designed for, what information is conveyed, and how an explanation presents that information (2025). Different stakeholders such as AI system developers, operators, validators, and subjects may have different needs for explanations, including different modalities of presentation (e.g. visual, textual, interactive) that match information needs. Each stakeholder here might need something a bit different from an AI system explanation in order to contribute to accountability. For instance, a system developer might benefit from a highly technical interactive explanation that helps them debug an issue with the model and so help prevent future bias or fairness issues in decisions. A decision-subject might need something more accessible to help them understand why they got the outcome they did and potentially contest it if they think it’s wrong. A validator (e.g. an auditor) may need to verify input features used by the model to ensure they are accurate and appropriate. And an operator needs to be informed about how their actions lead to probable consequences with the system in order to be a responsible human-in-the-loop (Baum et al, 2022).

Besides who explanations are for, there are important dimensions about what should be explained (Dhar et al, 2025). Local explanations focus on individual outputs and are well-aligned to the goals of retrospective accountability, which focus on identifying and assigning blame for a specific individual decision. On the other hand global explanations, which orient towards overall patterns of output from a model across a range of inputs, are better suited to supporting goals of prospective accountability where a birds-eye view is needed to inform how to prevent anticipated harms at the system level. Post-hoc explanations of system behavior which track how inputs influence outputs are the key for both retrospective and prospective accountability, while mechanistic explanations that trace functional model internals are more narrowly useful for informing developers towards preventing unintended outcomes. In other words, while the classic view of accountability as retrospective doesn’t hinge on explaining model internals, a prospective view could additionally benefit from explanations of those internals to debug model failures and ensure better outcomes in the future.

An early paper to make a connection between AI explanation and the goals of accountability comes from Doshi-Velez et al (2017). The authors appropriately point out the potential for explanations to prevent or rectify errors in AI systems, helping to discern the appropriate or inappropriate use of criteria by a system. They note two key types of explanation that can play an important role in supporting accountability: feature importance and counterfactuals.

Feature importance/relevance explanations provide information about the weighting and priority of inputs to specific outputs, or to the overall distribution of outputs. Should some features be unacceptably correlated to outputs (e.g. race) overall, this can inform prospective accountability so that the model or its training data can be rectified. If features are correlated to outputs in ways that contradict a scientific causal account of what should be predictive of outcomes, this could be grounds for prospective accountability to align the model with scientific expectations. If the scientific account is well-established such a contradiction could also contribute to retrospective accountability for negligence.

Counterfactual explanations include a “statement of how the world would have to be different for a desirable outcome to occur” and “describe a dependency on the external facts that led to that decision” (Wachter et al, 2017) and have also been framed as a form of feature relevance explanation (Speith, 2022). If a decision-subject had their mortgage application denied and the counterfactual explanation indicated that they would have been approved if their race had been different, that would be clear grounds for that individual to contest the output.

While Doshi-Velez got it mostly right when it comes to supporting retrospective accountability, another explanation type elaborated in the literature (Speith, 2022)—model surrogates (e.g. linear approximations)—may also be narrowly useful for prospective accountability. What a model surrogate explanation can offer is a clear and interpretable feature importance explanation of a more complex model (e.g. a neural net, or other black-box model). If that feature importance explanation indicates an inappropriate bias this could be grounds for a developer to be prospectively responsible for addressing the apparent behavioral bias of their model. Even if the model itself doesn’t use inappropriate data, if its behavior appears to be inappropriate that might be grounds to call for it to change. Where model surrogates are not so useful is for retrospective accountability as they don’t reflect the actual decision-logic impacting a specific individual.

A recent paper from Alpsancar et al (2025) makes an explicit connection between AI explanation and the needs of assigning responsibility to support AI governance. The authors recount the classical model of moral responsibility which hinges on fulfilling three criteria to hold someone responsible for their actions: (1) causality (i.e. the person influenced the outcome), (2) freedom (i.e. the person was not coerced in their action), and (3) epistemic (i.e. the person is aware of the consequences of their actions). The authors also review what they term the trans-classical model of responsibility which is a systemic view of responsibility that helps cope with unintended and unforeseen consequences. In this view, the epistemic condition instead relates to knowledge of the potential for and probability of various outcomes in the system (i.e. risk) and responsibility is assigned for managing that risk.

In the classical view, the goal of explanation for accountability is clear: to help fulfill the three conditions so that responsibility can be assessed. AI system explanations should indicate causality, including who (or what) took what actions that were critical to the outcome. They should indicate the autonomy of entities and their actions, including how individuals may be influenced by AI systems in their judgements. And they should show whether individuals in the system were appropriately informed about the consequences of their actions. On the other hand, in the trans-classical view the goal of explanation should be to support the understanding of the risk (i.e. severity and prevalence) of outcomes. But it could also be important for explanations to show that there is not a direct causal actor responsible in the system, since otherwise we might revert to the classical model. Regardless of view there is a need for a sociotechnical approach to AI explanation. Explanations of technical models as discussed above are important for supporting the knowledge needs for either view.

There are several policy-relevant implications that can be derived here. First, explanation requirements for AI systems should specify the audience for the explanation. A disclosure rule that works for a decision-subject contesting a denied loan looks very different from one aimed at auditors verifying model inputs or developers debugging bias. Second, any explanation requirements should tie back to the accountability purpose being served. Retrospective accountability calls for local, post-hoc explanations including counterfactuals and feature importance explanations, while prospective accountability calls for global explanations about patterns across outputs. Third, policymakers should consider both the classical and trans-classical view of responsibility and how and whether they may want to blend or distinguish the two in assigning responsibility. Finally, standards bodies should resist technical definitions of explainability and consider sociotechnical elements related to the human use of AI systems and their explanations.

References

Alpsancar S, Buhl HM, Matzner T, et al. (2025) Explanation needs and ethical demands: unpacking the instrumental value of XAI. AI and Ethics 5(3): 3015–3033.

Baum, K., Mantel, S., Schmidt, E. & Speith, T. From Responsibility to Reason-Giving Explainable Artificial Intelligence. Philos. Technol. 35, 12 (2022).

Dhar R, Brandl S, Oldenburg N, et al. (2025) Beyond Technocratic XAI: The Who, What & How in Explanation Design. Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society 8(1): 745–759.

Doshi-Velez F, Kortz M, Budish R, et al. (2017) Accountability of AI Under the Law: The Role of Explanation. arXiv. DOI: 10.48550/arxiv.1711.01134.

Lipton, Z. C. 2018. The mythos of model interpretability:In machine learning, the concept of interpretability is both important and slippery. Queue, 16(3): 31–57

Mittelstadt B, Russell C and Wachter S (2019) Explaining Explanations in AI. Proceedings of the Conference on Fairness, Accountability, and Transparency: 279–288.

Speith T (2022) A Review of Taxonomies of Explainable Artificial Intelligence (XAI) Methods. 2022 ACM Conference on Fairness Accountability and Transparency: 2239–2250.

Wachter, S.; Mittelstadt, B.; and Russell, C. 2017. Counterfactual explanations without opening the black box: Automated decisions and the GDPR. Harv. JL & Tech., 31: 841

Explanations rendered by and about AI systems need to be as faithful as epistemically possible in order to support accountability. Buijsman describes the role of explanation in supporting accountability: “when a mistake has been made, the challenge is to find a reason why that mistake happened and the people responsible for fixing it.” (2026). A faithful explanation might help understand whether there may be an issue with faulty data, missing information, or incorrect reasoning, and ultimately help improve the system over time. Explanations that are not faithful could misdirect decision-making about how to assign blame or prevent future harms, frustrate attempts to contest a decision or diagnose mistakes and logical errors so they can be corrected, and ultimately to appropriately sanction actors if the explanation is unacceptable.

Faithfulness is especially relevant to questions of process accountability, where the goal is to hold an actor in the AI system accountable for how an outcome was computed. Explanations are a diagnostic tool for accountability, describing how inputs lead to the outcome and helping to trace instances of potential negligence or faulty logic in the system. If an unfaithful explanation of a mortgage decision says that you were rejected because your income is too low but the model decision was actually influenced by your race or zip code this undermines your ability to challenge the decision as unacceptably including protected characteristics.

LLMs are not able to provide faithful explanations, such as self-explanations generated by the model to render the “reasoning” behind their output in human-understandable language (Madsen et al, 2024; Mayne et al, 2025; Mutton et al, 2025). Madsen and colleagues (2024) show that larger models with more parameters generally produce more faithful explanations but that there is high variance across tasks. Mayne and colleagues (2025) focus on self-generated counterfactual explanations (SCEs) and indicate that their findings “suggest that SCEs are, at best, an ineffective explainability tool and, at worst, can provide misleading insights into model behaviour.” While models may be able to provide counterfactual explanations (e.g. if you change variables X and Y it will flip the decision outcome), these may be trivially true rather than articulating minimal changes to the input that would actually shed light on the decision.

The main implication here is that when accountability matters, such as for high-stakes situations where there is potential for severe impacts, faithful explanations are critical, but LLMs cannot provide such explanations. Policymakers may consider when AI providers need to demonstrate faithfulness of model explanations and establish thresholds around when models can be used in high-stakes contexts. Administrative bodies will also need to develop standardized benchmarks and measurements for faithfulness to support such policies.

References

Agarwal C, Tanneru SH and Lakkaraju H (2024) Faithfulness vs. Plausibility: On the (Un)Reliability of Explanations from Large Language Models. arXiv. DOI: 10.48550/arxiv.2402.04614.

Buijsman S (2026) Accuracy is not all you need! The Reasons to Require AI Explainability. Minds and Machines 36(1): 14.

Jacovi, A. & Goldberg, Y. Towards Faithfully Interpretable NLP Systems: How Should We Define and Evaluate Faithfulness? Proc. 58th Annu. Meet. Assoc. Comput. Linguistics 4198–4205 (2020) doi:10.18653/v1/2020.acl-main.386.

Lyu, Q., Apidianaki, M. & Callison-Burch, C. Towards Faithful Model Explanation in NLP: A Survey. Computational Linguistics 50, 657–723 (2024).

Madsen A, Chandar S and Reddy S (2024) Are self-explanations from Large Language Models faithful? In: Findings of the Association for Computational Linguistics: ACL, 2024.

Mayne H, Kearns RO, Yang Y, et al. (2025) LLMs Don’t Know Their Own Decision Boundaries: The Unreliability of Self-Generated Counterfactual Explanations. In: EMNLP, 2025.

Matton K, Ness RO, Guttag J, et al. (2025) Walk the Talk? Measuring the Faithfulness of Large Language Model Explanations. In: ICLR, 2025.

Rudin, C. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nat. Mach. Intell. 1, 206–215 (2019).

I recently came across a paper by Fok et al (2025) that’s been useful in helping me organize the various AI experiments I’ve been trying. Based on interviews with researchers who have written literature reviews the paper helps to understand their overall process and some of the ways they conceptualize the use of AI in that process.

The findings articulate a set of four phases to the literature review process that participants engaged in: search, appraisal, synthesis, and interpretation. The paper also identifies some of the ways AI can support updating of reviews, namely through automation and in providing a second opinion. My own use of AI for AIAR has been most useful for automating (with oversight) some of the appraisal aspects of the process, and in providing second opinions on appraisal and synthesis. I have also dabbled in some use cases that more directly do synthesis and interpretation, but these have been less successful. And I haven’t really tried any AI use cases for search, because I think that setting the search scope for the review is something that needs to be closely managed by me. Let’s walk through some of the different things I’ve tried.

Scraping, Formatting, and Promotion

Probably the most time-saving use case I’ve found is to use OpenAI’s Agent mode to help collect conference proceedings papers that I want to review. Some conferences have non-standard presentations of information, but Agent mode is pretty adept at navigating websites to collect papers and format them as RSS feeds. I plug those feeds into my triage workflow on InoReader, which streamlines the appraisal process of papers. It can help to be explicit in the prompt and identify a structured data (e.g. JSON) version of the proceedings. And while this process is mostly automated I find that I do still need to double-check the outputs to make sure it was a comprehensive scrape.

I have also experimented with using Agent mode to gather email addresses for each of the primary authors of papers cited by one of my posts, and to then draft a short personalized note notifying the person about the post. I wasn’t intrepid enough to automate the actual emailing, but I did manually copy and send some of the emails (after light editing) and even got a response from one. Promoting AIAR on social media could be a full time job, but having the AI do some of the grunt work of getting email addresses and drafting emails lowers the barrier a bit.

Article Appraisal

One of the nice built-in features of InoReader is that for any item in a feed I am tracking, I can trigger a custom prompt to an LLM. Using this feature I can get a quick second opinion from the LLM on whether the item might be relevant to my audience. Admittedly I don’t use this all that often, but I do occasionally engage it. One of the issues is that not all the RSS feeds I follow have full abstract text and so this limits the applicability. I do think there’s real potential in having AI help think through what items have implications for your intended audience, and there’s probably a lot more sophistication that could be applied in how to do this computationally beyond the integrated prompting in InoReader, such as by simulating ideal audience members and what they would want to know about an item.

Some articles on AIAR reflect the synthesis of a cluster of literature. As a living literature review the goal is to update these over time with other literature relevant to the cluster. I’ve been experimenting with LLMs to support this process. Using a Google Colab notebook I input the URL of the base article to be updated and scrape the full text. Then I prompt an LLM to evaluate a stream of literature for relevance to that article. The prompt is critical here. What I’m looking for are new papers that might directly update, change, or provide new context to any of the claims in the original article, to find new papers that might actually make a difference.

Each paper is rated for relevance, and that rating is paired with a table listing claims from the original article and ideas from the new paper that might bear on those claims. The table facilitates my appraisal of the new paper. The output looks like this:

So far this is promising, but there’s still work to do to evaluate it and set it up as an ongoing monitoring process that fully integrates with my InoReader appraisal workflow. In principle I’d set this up for each of the base articles in AIAR, and then monitor literature from something like OpenAlex to create a continuously updated feed of potentially relevant papers.

Grounded Synthesis

Google’s NotebookLM has turned into an increasingly powerful tool that can be used to interactively synthesize curations of articles. For my article on AI Ethics Principles and Accountability, I even published a notebook with all of the sources I had used to write the article. While the original goal with creating the notebook here was to allow readers to interactively explore the literature, I also realized that I could also use this to provide a second opinion on my own synthesis. Using Gemini, you can refer to a notebook of curated sources in NotebookLM and so I prompted it to create a table listing the supporting evidence for every claim in the post. In the absence of an editor, this can be a useful double-check to make sure you’re staying honest to the underlying literature in your synthesis. I think this kind of approach could potentially also be useful in an article update process to assess whether claims in new papers support or refute the existing claims you’ve written.

Still, I am a bit cautious about relying on LLMs, even closely grounded ones, in helping to synthesize literature for AIAR. In an early experiment, I loaded up NotebookLM with the entirety of the Fairness, Accountability, and Transparency Conference proceedings from 2025. I asked Gemini (with access to the Notebook) to look for clusters of papers that were thematically related to each other and to the topic of the blog. While some of these clusters seemed relevant and overlapped with my own perception of themes, others seemed more tenuous in the solidity of the theme and its relevance to AIAR. Synthesis is to a large degree about framing and finding a consistent thread, and I don’t think even the best LLMs are able to do this in a way that is satisfying.

As a Writing Aid

I have attempted to use LLMs (primarily Gemini, sometimes directly in NotebookLM) to help draft five of the posts for AIAR, three of which were based on translating a single paper, and two of which were based on clusters of papers.

I found that for the articles based on clusters the LLM was wholly unsuited to the task of synthesis: I ended up using none of the generated text. Even including all of the paper texts and my notes on those papers in the prompt, I was left feeling that the synthesized text didn’t capture what was interesting or important about the cluster. This again goes back to the idea of framing, structuring, and finding the aspects of relevance that I think are important within the field and to my intended audience. But this also relates to the interpretation phase and the “identification of key challenges, future trends, and open research opportunities” (Fok et al, 2025). All of this is consistent with what some editors at Science found when they tried to use ChatGPT to translate research papers.

For the three articles that were more direct translations of individual research papers I had slightly more success with incorporating AI generated text. In this post, I used almost 50% of the generated text in the final piece, which warranted a disclosure at the bottom of the post: “Some text in this post was adapted based on suggestions from AI.” I think this was somewhat successful because I prompted the model with details on the aspects of the paper I wanted the post to focus on, and that the post itself was more descriptive than synthetic or interpretive. The parts of the post that I wrote were the more interpretive aspects, putting the research into a broader context and considering its relevance to the audience. In another post (excepted below), I was also able to use some chunks of descriptive text that were generated by the LLM.

In Closing

Much like everything else on AIAR, this post will be a work-in-progress and is subject to update. The most compelling use-case I’ve found so far for AI is in automating the collection and formatting of references into my RSS workflow as this lets me do something that I might not otherwise make time for. I also find the article appraisal workflow compelling and plan to keep pushing on that to integrate it more into my regular workflow for keeping AIAR posts updated. I may also revisit use cases related to grounded synthesis and writing though I’m generally less optimistic about AI providing a real lift there. The work of framing and making connections in the literature, contextualizing findings, and thinking about what matters to an audience seem like they really need an expert eye, though perhaps LLMs can assist by offering a second opinion.

References

Fok R, Siu A and Weld DS (2025) Toward Living Narrative Reviews: An Empirical Study of the Processes and Challenges in Updating Survey Articles in Computing Research. Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems: 1–10.

Probably the biggest issue I see with the law is in the definitions. The RAISE act is geared towards regulating “frontier models” which it defines as: “an artificial intelligence model trained using greater than 10º26 computational operations (e.g., integer or floating-point operations), the compute cost of which exceeds one hundred million dollars” (or a different definition that applies to models produced through knowledge distillation). This is a bad definition because it merges two criteria that are arbitrary, shifting over time, and, most importantly, which model developers are not required to disclose. They are arbitrary because there’s no reason to think that 10^26 computing operations is a magical threshold at which danger suddenly materializes. Based on estimates from Epoch, none of the current breed of frontier models surpasses this threshold, so it’s not clear that the law applies to anything in the real world. And, again, even if OpenAI, Google, or Anthropic has exceeded this threshold in its training there’s no way for us to know because the law doesn’t make them tell anyone. It’s a sort of scouts honor, opt-in system. Also, the definition of compute is in conjunction with a cost greater than $100 million. Because the definition has to match both criteria, a model could use more than the compute threshold but be done for less than $100 million and then the law wouldn’t apply. But compute costs are always getting cheaper, and some model developers like Google control the market price of their computing and so can game this, not to mention that the value of the dollar could change.

The main mechanism for specifying transparency in the law is that large developers of models create a “safety and security protocol” —a form of transparency report—before deployment of the model. There are also provisions to require the reporting of “safety incidents” that might be a case of or increased risk of critical harm. The protocol report is shared with administrative accountability forums such as the attorney general and division of homeland security, as well as being made publicly available in redacted form for media or social forums. The unredacted protocol plus additional information about the tests and test results that inform the protocol need to be maintained for however long the model is deployed plus five years, presumably so that those records are potentially available for discovery by legal forums in the event they are needed. In this sense, the law does pretty well in providing for accessibility of the safety and security protocol to various accountability forums.

The overall aim of the law towards “frontier models” and “critical harm” scopes and sets limits on the relevance of the information in the protocol. Critical harm is defined as causing $1 billion or more in damage or loss of 100+ human lives. But with that scope in mind the definition of the protocol is reasonable as it specifies what should be included, including organizational procedures and sociotechnical measures meant to mitigate the potential for critical harm, as well as the testing procedures used to “evaluate if the frontier model poses an unreasonable risk of critical harm”. The protocol must also designate a person that is responsible for compliance — this is a critical component that ensures accountability for overseeing the protocol. The timeliness of the report is also referenced and calls for the developer to update the protocol on an annual basis as per any changes.

An area where the protocol falls short is in either specifying or auditing the accuracy of the information in the protocol. An earlier version of the law had provisions requiring 3rd party auditing, but those were removed from the final version signed into law. That would have strengthened the law considerably by having an independent entity checking the validity of procedures and the accuracy of provided information in the protocol. What’s left is the comparatively weaker request that large developers not lie, i.e. “shall not knowingly make false or materially misleading statements or omissions.” We can’t really assess whether the information in protocols would be understandable and fit for the purpose of accountability. A stronger law would have created a standard for the protocol that would be considered adequate.

The law provides reasonable carve outs to address typical criticisms and stakeholder pushback about transparency, including that disclosures might undermine privacy, confidentiality, trade secrets, or be used to game the system. Redactions to public safety and security protocols can be undertaken to protect these other interests. The law also protects fundamental innovation by not applying to academic research done at accredited colleges and universities. In addressing the tensions between transparency and other interests at stake, the law probably does about as well as it could, especially because administrative forums like the attorney general can gain access to copies of the protocol that are less redacted, i.e. where redactions only need to respect federal law, and fully unredacted reports must be maintained for possible discovery in legal forums.

Overall, much like its Californian counterpart, New York’s RAISE act is geared towards prospective accountability — trying to prevent future harm. Its scope is narrow around “critical harms”. While it does well to specify the accessibility of the transparency information it calls for, and align that information so it is relevant and timely to its scope, it lacks provisions for ensuring the accuracy of the information, and leaves the understandability of that information up to the large developers who’ll be creating the reports. But it’s not a powerful law because it doesn’t apply to anything in the real world (yet), and it’s unclear whether model developers will ever raise their hand and say that the law actually applies to them. It does provide an example of AI governance through transparency that can inform future legislation. The next version of the law, proposed by the governor’s office and under consideration by the state legislature, is already drastically different in many ways.

Before anyone can be held accountable for an AI system’s behavior we’re going to need some information about that system. What was the system’s behavior and was its performance unexpected? What are the underlying values and goals of its designers? Did the developers take appropriate steps to test for and prevent harmful outcomes? How are organizational policies designed and implemented for the ongoing operation of the system? Transparency is the umbrella idea of closing these kinds of knowledge gaps, and should be differentiated from explanation which is a more specific approach (Corbett and Denton, 2025; Hayes et al, 2023). More formally, transparency can be defined as “the availability of information about an actor allowing other actors to monitor the workings or performance of this actor” (Meijer et al, 2014). And while transparency in itself cannot ensure accountability, it often plays a critical supporting role, providing the informational substrate for understanding AI system behavior that can then filter into various forums that might seek to hold actors in an AI system accountable.

Transparency sets up a relationship between two entities—here an AI system and a forum—where information about the AI system becomes available to the forum. Because AI systems are sociotechnical this includes information about both the data and technical model in the system, as well as the human components such as organizational policies, procedures or practices, and user behaviors (Diakopoulos, 2020). For the sake of accountability, provided information should help the forum determine congruence with relevant values, goals, and normative or legal expectations of behavior (Hayes et al, 2023; Fleischmann and Wallace, 2005). Transparency information can be voluntary (e.g. a blog post), obligatory (e.g. legally mandated disclosure to an administrator), or involuntary (e.g. external audits, or leaks), though recent research has underscored the inadequacy of volunteered “first-party” transparency information compared to external “third-party” evaluations of social impacts (Reuel et al, 2025).

To be useful for accountability, transparency information needs to reflect high information quality. At a minimum it needs to be accessible, understandable, relevant, and accurate (Hayes et al, 2023; Diakopoulos, 2020; Turilli and Floridi, 2009). Beyond just availability, information needs to be accessible so that it can be easily used by audiences such as various accountability forums. Access can be facilitated through mechanisms such as negotiated data access agreements, researcher APIs, scraping, tailored interfaces, or data donations. Transparency information also needs to be understandable or usable by those audiences and aligned to their information processing capabilities and capacities. It needs to be relevant to diagnosing some behavior of interest whether that be in shedding light on some negative outcome for retrospective accountability, or providing critical context to inform prospective accountability. And it needs to be accurate such that it is valid, reliable, and free of error (Turilli and Floridi, 2009), since otherwise it can suffer from strategic activities that shape or distort information, leading to uninformative or boilerplate disclosures (Marin et al, 2025). Other pertinent aspects of information quality include the currency or timeliness of the information, and its comprehensiveness. For instance, research API access from TikTok and Meta was found to be incomplete with respect to the data that was available via the user interface, undermining the capacity for monitoring consequential systemic risks (Bekavac and Mayor, 2026). AI transparency will typically fall short when the above aspects of information quality aren’t adequately addressed.

A reoccurring pattern we see in the literature is a failure to clearly articulate the intended audience or forum for transparency information, with implications for how the information would be maximally accessible, understandable, and relevant for that audience. For instance, in the 2025 Foundation Model Transparency Index (Wan et al, 2025), the authors establish a set of 100 indicators that they apply to various models to evaluate how transparent they are in terms of data, training, compute usage, modeling, and downstream impacts and use policies. But the audience for all of this information—and its utility for accountability—is anything but clear. What transparency initiatives like this one need to do is clearly articulate the public interest and accountability purpose of each indicator, helping to connect over to the audience or forum that would then use that information for accountability. Similarly, a recent proposal for AI agent transparency (Ezell et al, 2025) appears oriented somewhat towards technical developers “debugging” agent incidents. If the information in that framework could be made available to administrative or judicial forums, it’s likely they would benefit from at least some of the information. But the ideal would be a more parsimonious framework that more closely tracks the needs of those forums for specific issues they may need to assess for accountability.

Although transparency is a necessary pre-condition for accountability, critics point out that it is not an unalloyed positive force. It shouldn’t be assumed to always enable accountability (Corbett and Denton, 2023), though policies that shape adherence to the attributes of quality transparency information described above should increase the likelihood of its utility. Transparency can also come into tension with other values, such as privacy, freedom of expression, security, or intellectual property (Ananny and Crawford, 2018; Diakopoulos, 2020; Turilli and Floridi, 2009) leading to situations where tradeoffs need to be made in highly context-specific ways, or addressed through specific implementation strategies such as by using secure enclaves and differential privacy protections (Burnat and Davidson, 2026) or structured access levels to frontier models to help manage security risks (Charnock et al, 2026). One of the most frequent counter arguments to more transparency is that it could enable gaming or manipulation of the system (van Bekkum and Borgesius, 2021), though careful context-specific engineering, threat modeling, and consideration to forum-specific access provisions should alleviate this issue (Diakopoulos, 2020). We might also consider the idea that social or public forums may use manipulation as a way to sanction a system—in other words manipulating a system may in some contexts and situations be considered a component of holding a system accountable for unwanted behavior. Ultimately, the choices around what, when, and how AI systems are made transparent are political (Corbett and Denton, 2023).

The role of policy here is to thread the needle through these criticisms to scope transparency and shape it towards positive outcomes for society. Policy must create obligations for actors within AI systems to produce the information needed by any given forum (e.g. administrative, legal, etc.) to make the relevant assessment of system performance. This information needs to meet accessibility, understandability, relevance, accuracy, currency, and comprehensiveness quality criteria. One way to do this is to be more specific about standards for AI system transparency information production: what standard processes and practices should be evidenced by actors making transparency information available? Public sector policy makers cannot leave this unspecified, otherwise there is too much room for strategic and performative behavior. Just requiring that researchers be provided an API to access platform data doesn’t mean that data is complete or responsive to accountability needs (Bekavac and Mayer, 2026). Another role for policy makers is to engage in the politics of where and how to make tradeoffs with other values such as privacy; looking to public attitudes should probably inform this. Transparency policies need to be user-centered (e.g. towards whatever forum the information is intended for) and context-specific, and would benefit from human-centered engineering and evaluation to refine their scope, meet user needs, and maximize their utility for accountability.

References

Ananny M and Crawford K (2018) Seeing without knowing: Limitations of the transparency ideal and its application to algorithmic accountability. New Media & Society 20(3): 973–989.

Bekavac L and Mayer S (2026) Platforms’ Research API Data Access: What Users See vs. What Researchers can Retrieve. Proceedings of the 2026 ACM Conference on Fairness, Accountability, and Transparency: 8276–8299.

Burnat FAD and Davidson BI (2026) The Accountability Paradox: How Platform API Restrictions Undermine AI Transparency Mandates. Proceedings of the 2026 ACM Conference on Fairness, Accountability, and Transparency: 2416–2433.

Charnock J, Tlaie A, O’Brien K, et al. (2026) Expanding External Access to Frontier AI Models for Dangerous Capability Evaluations. Proceedings of the 2026 ACM Conference on Fairness, Accountability, and Transparency: 5095–5113.

Corbett E and Denton R (2023) Interrogating the T in FAccT. Conference on Fairness, Accountability, and Transparency: 1624–1634.

Diakopoulos N. (2020) Transparency. Oxford Handbook of Ethics and AI. Eds. Markus Dubber, Frank Pasquale, Sunit Das.

Ezell C, Roberts-Gaal X and Chan A (2025) Incident Analysis for AI Agents. Proc. AI, Ethics, and Society (AIES) DOI: 10.48550/arxiv.2508.14231.

Fleischmann KR and Wallace WA (2005) A covenant with transparency. Communications of the ACM 48(5): 93–97.

Hayes P, Poel I van de and Steen M (2023) Moral transparency of and concerning algorithmic tools. AI and Ethics 3(2): 585–600

Marin, L. G. U.-B., Rijsbosch, B., Spanakis, G. & Kollnig, K. Are Companies Taking AI Risks Seriously? A Systematic Analysis of Companies’ AI Risk Disclosures in SEC 10-K forms. arXiv (2025). https://arxiv.org/abs/2508.19313

Meijer A, Bovens M and Schillemans T (2014) Transparency. The Oxford Handbook of Public Accountability. Oxford University Press.

Reuel A, Ghosh A, Chim J, et al. (2025) Who Evaluates AI’s Social Impacts? Mapping Coverage and Gaps in First and Third Party Evaluations. arXiv. https://arxiv.org/abs/2511.05613

Turilli, M., Floridi, L.: The ethics of information transparency. Ethics Inform. Technol. 11, 105–112 (2009)

Bekkum M van and Borgesius FZ (2021) Digital welfare fraud detection and the Dutch SyRI judgment. European Journal of Social Security 23(4): 323–340.

Wan A, Klyman K, Kapoor S, et al. (2025) The 2025 Foundation Model Transparency Index. arXiv. DOI: 10.48550/arxiv.2512.10169.

The crux of the analysis is in comparing 186 first-party reports that were part of model releases by model developers to 183 post-release evaluations that were run by various third parties. These reports were assessed based on the level of detail provided in evaluations of any of seven social impact dimensions as identified by Solaiman et al (2023). The seven dimensions assessed were Bias and Harm, Sensitive Content (e.g. outputting hate speech), Performance Disparity (e.g. unequal results across subpopulations), Environmental Costs and Emissions, Privacy and Data, Financial Costs, and Moderation Labor (e.g. working conditions of data annotators). The rating scale ranged from a 0 (no evaluation present), 1 (vague mention), 2 (concrete results but limited clarity on methods and context), and 3 (sufficient detail to understand and contextualize the evaluation). All the ratings are available here.

The main take-away is that third party evaluations were considerably more detailed, on average, than first-party evaluations (2.62 vs. 0.72 on the 0-3 scale). The implication is that the tech companies and other organizations training models are not releasing as much detail about their evaluations of social impacts in comparison to third parties who run evaluations. The authors note that the most popular models from the US (and to a lesser extent China) tend to attract the most third party evaluations, exposing a gap in evaluation of less-popular models. They also note that certain impact types such as data and content moderation impacts (as well as some others like environmental impacts) are not prevalent at all and are almost entirely absent from third-party evaluations, exposing the reality that third-parties just do not have access to the information they would need to properly evaluate certain issues.

The take-aways for policy here seem pretty clear. First-party evaluations of models by model providers are insufficient when it comes to evaluations of social impacts. There is a fair bit of variance in what level of attention different models receive and what dimensions of social impact are evaluated at all. Transparency standards are needed to provide more consistency and expectations for what evaluations need to be run and how, or which data needs to be disclosed so that third parties can cover more terrain with their evaluations. In addition, there need to be standards around which models demand a full evaluation. And there needs to be sufficient capacity in the evaluation landscape of third parties to be comprehensive. Advancing consistent transparency standards for AI models would support AI accountability by providing the information needed by different accountability forums.

References

Reuel A, Ghosh A, Chim J, et al. (2025) Who Evaluates AI’s Social Impacts? Mapping Coverage and Gaps in First and Third Party Evaluations. arXiv. https://arxiv.org/abs/2511.05613

Solaiman I, Talat Z, Agnew W, et al. (2023) Evaluating the Social Impact of Generative AI Systems in Systems and Society. arXiv. https://arxiv.org/abs/2306.05949v2

Subscribe now

Early analyses of the numerous published AI guidelines have identified a few core principles. These include privacy, fairness/justice, accountability/responsibility/explicability, transparency, beneficence, non-malfeasance/safety, and human autonomy (Jobin et al, 2019; Hagendorff 2020; Floridi et al, 2018). Despite mostly stable underlying ideas the exact terminology can vary and leads to a lack of clarity (Morley et al, 2021). A longer tail of principles includes ideas like trust, sustainability, dignity, and solidarity (Jobin et al, 2019).

Principles can come from different sources and so be biased in different ways, such as towards ideas in dominant geographies or from power holders such as experts or companies (Hickok, 2021). They can come from researchers and experts in the field (Floridi, 2018), from professional codes of conduct in domains of practice (Diakopoulos et al, 2024), from broad consensus documents like the UN declaration of human rights (Latonero, 2018), and be further informed from public evaluations (Kieslich et al, 2024). What’s the most legitimate source of principles for AI accountability? While a treaty like the Framework Convention on Artificial Intelligence has reached broad consensus, large swaths of the world still haven’t signed on. Achieving truly global principles will require ongoing political work.

Besides their potential to reflect biases, AI principles are also hard to actually implement in practice. Big abstractions need to be translated into concrete operationalizations (Hagendorff, 2020) if they are going to be used to measure AI system failures or guide AI system design to support prevention. Moreover, abstractions like fairness can hide contested ideas with conflicting perspectives (Mittelstadt, 2019) underlining the need to consider context-specific tradeoffs. Max (2026) suggests that ethics principles risk functioning more as political tools unless criteria for meeting or failing to meet them are provided in enough detail to permit criticism or contestation. Operationalizing principles also entails the challenge of adjudicating their prioritization, tensions, and trade-offs in specific contexts. Principles that are implausible to implement function more as rhetoric—”symbolic assurances”—than as viable governance instruments.

Prem (2023) analyzed more than 100 approaches from the literature for bridging the gap between principles and implementation. These include things like AI ethics criteria/checklists, metrics, process models, codes of practice, etc. He distinguishes approaches used during the design of a system (ex-ante), and those that are applied to an AI system after development or perhaps iteratively during development (ex-post). Ex-ante methods are relevant to prospective accountability, whereas ex-post methods are geared towards retrospective accountability (and also prospective if used iteratively during development). He notes that “Generally, there is a strong focus on those aspects for which technical solutions can be built,” exposing a further bias in the research on this topic.

Whereas designers and developers can adopt approaches to help prevent negative outcomes, AI system behavior itself should also be measured to assess adherence to principles. The idea of Ethics Based Auditing (EBA) applies the logic of auditing to the challenge of assessing system behavior “for consistency with relevant principles or norms.” (Mökander et al, 2021). This starts to get at a core issue of operationalizing principles into metrics that can evaluate (mis)alignment with a value. Principles just set the direction; effective accountability requires quantifiable performance metrics. This in turn requires supporting data access to inform those measurements.

Rismani and colleagues (2025) reviewed hundreds of these measures in the literature as they relate to different system components, hazards, harms, and principles. 90% of the measures they found were related to just four principles: fairness, transparency, privacy, and trust. To be useful for accountability metrics need to define some threshold of the metric which indicates the principle has been violated, that the system may create a hazard, and therefore warrants a call for accountability. Thresholds may be context-dependent, vary based on domain, and are subject to the risk tolerance of different stakeholders, but are rarely discussed in the literature (Rismani et al, 2025). This returns us to the normative question: How do you define an acceptable vs. unacceptable level of a measure of a principle? At what level might reasonable people agree there should be accountability? Public perceptions of acceptability may play a role here.

Principles serve as orienting ideas for what is valued. They can be used to determine what constitutes inappropriate behavior, necessitating accountability either retrospectively (blame for failure) or prospectively (prevention of harm). Bringing them into formal accountability forums (e.g. administrative, legal) hinges on mitigating biases in their enumeration and reaching a high degree of consensus. But implementing them in practice remains a challenge. They need to be translated into practices that designers and developers can use to mitigate the hazards created by an AI system, or to metrics with clear thresholds that can measure AI system behavior for signs of deviation. Policy should support the development of context- and domain-specific operationalizations of metrics and thresholds that are indicative of violations of principles by AI systems, as well as the data access provisions that would enable those measurements by the relevant accountability forums.

References

​​Diakopoulos N, Trattner C, Jannach D, et al. (2024) Leveraging Professional Ethics for Responsible AI. Communications of the ACM.​​

Floridi L, Cowls J, Beltrametti M, et al. (2018) AI4People—An Ethical Framework for a Good AI Society: Opportunities, Risks, Principles, and Recommendations. Minds and Machines 28(4): 689–707.

Hagendorff T (2020) The Ethics of AI Ethics: An Evaluation of Guidelines. Minds and Machines 30(1): 99–120.

Hickok M (2021) Lessons learned from AI ethics principles for future actions. AI and Ethics 1(1): 41–47.

Jobin A, Ienca M and Vayena E (2019) The global landscape of AI ethics guidelines. Nature Machine Intelligence 1(9): 389–399

Kieslich K, Helberger N and Diakopoulos N (2024) My Future with My Chatbot: A Scenario-Driven, User-Centric Approach to Anticipating AI Impacts. Conference on Fairness, Accountability, and Transparency: 2071–2085.

Latonero M (2018) Governing Artificial Intelligence: Upholding Human Rights & Dignity. Data & Society. https://datasociety.net/library/governing-artificial-intelligence/

Max R (2026) Guidelines without guidance: underdetermination in AI ethics. AI and Ethics 6(3): 323.

Mittelstadt B (2019) Principles alone cannot guarantee ethical AI. Nature Machine Intelligence 1(11): 501–507.

Morley J, Kinsey L, Elhalal A, et al. (2021) Operationalising AI ethics: barriers, enablers and next steps. AI & SOCIETY 38(1): 411–423.

Mökander J, Morley J, Taddeo M, et al. (2021) Ethics-Based Auditing of Automated Decision-Making Systems: Nature, Scope, and Limitations. Science and Engineering Ethics 27(4): 44.

Poel I van de (2020) Embedding Values in Artificial Intelligence (AI) Systems. Minds and Machines 30(3): 385–409.

Prem E (2023) From ethical AI frameworks to tools: a review of approaches. AI and Ethics 3(3): 699–716.

Rismani S, Shelby R, Davis L, et al. (2025) Measuring What Matters: Connecting AI Ethics Evaluations to System Attributes, Hazards, and Harms. Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society 8(3): 2199–2213.