The OECD defines AI agents as “systems that perceive and act upon their environment with a degree of autonomy, using tools as needed to achieve specific goals and adapt to changing inputs and contexts.” Essentially they are AI systems that tend towards the higher end of autonomy and adaptiveness in their environment. But with increased autonomy (and the lack of direct human control) comes increased risk for these systems to take actions in the world (e.g. via tool calls, APIs, or other outputs) that might be harmful. How do we ensure accountability for the harms they will invariably cause through their actions?

Over the course of decades research has developed ideas for the governance of human agents in terms of principal-agent models (Ross, 1973). The basic idea is that some actor (the principal) delegates a task to another actor (the agent) where there might be differences in goals or risk profiles between the two and it is difficult for the principal to verify what the agent is doing. Principal-agent theory encompasses a flexible array of models that explore this relationship, and have been applied across fields such as economics, management, and public administration (Ross, 1973; Eisenhardt, 1989; Gailmard, 2012).

Principal-agent models set up an information regime where the principal can specify the outcome or behavior they want from the agent, provide incentives to do so, and get information back to evaluate whether it was done to their liking. But information asymmetry and goal divergence can lead to issues of “moral hazard” (i.e. the agent does something at odds with the principal’s goals or risk profile) or “adverse selection” (i.e. the principal lacks information about the capability of agent before delegation) (Eisenhardt, 1989). A certain degree of agency loss is pragmatically inevitable for the principal (Gailmard, 2012) since eliminating such loss would require a fully specified contract for all expected behavior across all contexts of action and with perfect surveillance and verification of agent behavior (Hadfield-Menell and Hadfield, 2019). In economic terms the principal invariably has to cede some control over the outcome if it wants to save time or cost through delegation. Sure you could have a principal-in-the-loop check every output, but at scale the principal loses the efficiency benefit gained through delegation.

In writing about the governance of AI agents through the principal-agent lens, Kolt (2024) elaborates issues of authority and discretion ceded to agents, loyalty of an agent to a principal, and the challenge of multiple co-principals (sometimes referred to as “common agency”). He raises an important critique which is that shaping agent behavior toward what the principal wants though incentive or sanction mechanisms doesn’t really work for AI agents — they don’t respond to financial motivation or potential negative social sanction (e.g. psychological or reputation effects) the same way people do. Though I would add that even as we accept that AI agents themselves are not susceptible to psychological or reputational effects we shouldn’t forget that their human developers still are.

Several ideas have surfaced which could strengthen the accountability of the principal for actions taken by their agent. One of the most well-developed is to increase the visibility (i.e. transparency) of the agent vis-a-vis the principal. This might include the use of agent identifiers (flagging when an AI agent is involved in an activity), real-time monitoring (continuously tracking and analyzing agent behavior), and logging (recording and documenting what agents do) (Chan et al, 2024). Additional aspects to track and disclose include system documentation (e.g. parameters, versions) and tool use documentation (Ezell et al, 2025) as well as reasoning traces, confidence maps, counterfactuals, and guardrail events (Prause, 2026). Logging and monitoring approaches incur costs and are challenged by the speed and scale of AI agents, but they can help narrow information asymmetries, agency loss, and ultimately empower the principal (Kolt, 2024). A principal may well delegate the monitoring function to another entity—including to an AI system—that can better keep up with the scale, pace, and detail of the agent’s logs.

Beyond monitoring, Prause (2026) adds that screening mechanisms can help principals understand the capabilities of agents before anything is delegated. This could take the form of AI agent benchmarks that help close information asymmetries and support principals taking responsibility for ensuring an AI agent is capable before delegation. Hacker and Holweg (2026) propose elaborating public policy on AI agents by specifying the frequency and scope of human oversight together with a documentation mandate. Some actions may require additional structural safeguards (such as strict human review requirements), or should be outright banned for agents to take (e.g. financial transactions above some threshold) (Hacker and Holweg 2026; Prause, 2026). Nama (2026) further suggests that principals delegating to AI agents meet agentic AI literacy standards. This could include understanding the nature and scope of authority delegated, how to effectively monitor and intervene on the agent, and understanding available recourse or reversibility options. Such an approach would support the knowledge criterion central to establishing principal responsibility for the actions their AI agent takes on their behalf.

Empirical research is just beginning to examine how principals delegate tasks to AI agents differently than human agents, reflecting principals’ beliefs about AI agents’ obedience but also the recognition that they need to overspecify tasks to avoid mis-alignment and provide necessary knowledge and expertise they believe deficient in the agent (Petridis et al, 2026). People appear to find it somewhat freeing to not have to deal with the “social overhead” of managing another human being. Interaction methods and paradigms for principals to better control agents might include providing rough drafts and test runs for a principal to evaluate as well as establishing check-in criteria for the agent to communicate with the principal, which all align well with proposals for improved monitoring and screening.

A looming gap in the AI agent literature appears to be an in-depth and extended treatment of the co-principal issue. AI Agents almost always have co-principals (or perhaps hierarchical principals): their developers and their users. There is ample room for moral hazard when the co-principals disagree about what the agent should do, and apportioning responsibility between the two seems to be the crux of the issue. Proposals for agentic AI literacy and better benchmarks for screening agent capability strengthen the role and responsibility of the user, while monitoring and logging demand infrastructural and access considerations that realistically can only fall on developers. While transparency is perhaps the most viable option for ensuring principals maintain control, policy should also be mindful that increased monitoring and logging (including potentially of principals’ oversight) also tends towards surveillance.

References

Chan A, Salganik R, Markelius A, et al. (2023) Harms from Increasingly Agentic Algorithmic Systems. 2023 ACM Conference on Fairness, Accountability, and Transparency: 651–666.

Chan A, Ezell C, Kaufmann M, et al. (2024) Visibility into AI Agents. The 2024 ACM Conference on Fairness, Accountability, and Transparency: 958–973.

Eisenhardt KM (1989) Agency Theory: An Assessment and Review. The Academy of Management Review 14(1): 57.

Ezell C, Roberts-Gaal X and Chan A (2025) Incident Analysis for AI Agents. Proc. AI, Ethics, and Society (AIES) DOI: 10.48550/arxiv.2508.14231.

Gailmard S (2012) Accountability and Principal-Agent Models. In: Oxford Handbook of Public Accountability. Oxford University Press.

Hacker P and Holweg M (2026) A pragmatic approach to regulating AI agents. arXiv. https://arxiv.org/abs/2604.22819

Hadfield-Menell D and Hadfield GK (2019) Incomplete Contracting and AI Alignment. Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society: 417–422.

Kolt N (2024) Governing AI Agents. Notre Dame Law Review 101.

Nama R (2026) From evaluator to principal: the agentic AI literacy framework (AALF) for delegated autonomy. AI and Ethics 6(3): 299.

Petridis S, Liu MX, Fiannaca AJ, et al. (2026) Compass vs Railway Tracks: Unpacking User Mental Models for Communicating Long-Horizon Work to Humans vs. AI. Proceedings of the 2026 Designing Interactive Systems Conference: 1188–1204.

Prause M (2026) No skin in the game: why agentic AI requires principal-agent governance. AI and Ethics 6(2): 199.

Ross SA (1973) The Principal’s Problem. The American Economic Review 63(2).

In a recent paper published in Digital Journalism we report on an interview study where we talked to news audiences in the US about some of the problems AI labeling might address for them. Participants described how labels can enhance their understanding of responsibility, clarify attribution, cue credibility, maintain trust, and enhance their autonomy. But it wasn’t just an “AI” label that they desired. They also wanted more clarity on how people were involved in the content creation process, such as oversight or specific tasks they were assisted with. The signalling of human oversight was seen as a way to re-assure audiences about information integrity. Labeling AI output was seen as a way to support reader autonomy so they could effectively opt in or out of consuming a piece of content depending on their context.

Audiences see some potential benefit from AI labeling, and apparently policymakers do too. It’s the main idea behind recently passed (but not yet signed by the governor) legislation in the state of New York called the “fundamental artificial intelligence requirements in (FAIR) news act”. That proposal calls for news media published or disseminated in New York to “conspicuously imprint” at the top of the page that the content was “substantially created by generative artificial intelligence.”

The recently published EU Code of Practice on AI-Generated Content also specifies how to fulfill the EU AI Act’s requirement for labeling by deployers (i.e. users) of AI used for creating content. Whereas the NY law doesn’t specify what labels should look like in practice, the EU code offers a set of reference icons that deployers can use for labeling fully AI generated or AI modified content:

While the core idea is similar, the scope of the two laws is different and reflects differing value trade-offs. The NY law is scoped around news media which is broadly defined to include “news, weather, traffic, sports, or entertainment” published across a range of formats, whereas the EU law applies to all AI-generated images, audio, and video as well as text “with the purpose of informing the public on matters of public interest.” The EU law has an exception for content with human oversight, where the obligation to label doesn’t apply when “the AI-generated content has undergone a process of human review or editorial control and where a natural or legal person holds editorial responsibility for the publication of the content.” Whereas New York is explicitly targeting news media, the EU has what is essentially a carve out for news media as long as they ensure there’s editorial oversight and a person who takes responsibility and is publicly identified. That exception trades-off reader autonomy in choosing to opt out of AI-generated content in exchange for naming an explicit entity being responsible/accountable for the content. In that sense the New York law is a stronger statement for empowering the reader with the knowledge that something was produced by AI, regardless of any editorial oversight. The EU rule seems to be a slippery slope: Who defines the level or degree of editorial oversight that warrants this exception?

Another point to consider is that neither policy proposal is particularly clear on what is probably the most typical case: AI-assisted content creation. Especially given the audience demand for understanding human-AI co-involvement in creation that we saw in our study, how can we label content when both AI and people play different roles? This is a real struggle highlighted by a recent case where a McClatchy newsroom unionized in order for reporters to assert that they didn’t want to be named in a byline used to label content that had been automatically generated based on their original reporting and writing. Whereas the company thought that by naming the reporter this would provide the responsibility and accountability for the content, the reporters themselves felt that their loss of autonomy in the deployment and use of the technology undermined any sense of responsibility they felt for the AI-generated output. In this case, the reporters felt that a byline indicating “AI assistance” was unwarranted given the role they played in producing the output.

I don’t blame policymakers for not wanting to wade into the sticky territory of determining how to label jointly-produced content. It’s easier to stick to a clear “AI” (or not) label. If AI was used to produce the content, then label it, and let end-users opt out accordingly. If there was enough human effort to not have to label it “AI”, either because there is editorial oversight in the EU, or because it meets the standard for copyrightability in New York, then maybe it doesn’t need that AI label at all. In that case it’s perhaps better to have a human byline to make responsibility for the content clear. But that person also has to be willing to assume that responsibility and not have a corporation foist it on them.

Subscribe now

Apologies are statements that typically express regret and sometimes include an attribution that elaborates why an undesired outcome came about. Their social utility is in acknowledging blame and communicating a willingness to improve in the future, or in explaining why the apologizer couldn’t control the outcome. There are three broad apology types studied in the literature: (1) the basic “we’re sorry” which doesn’t identify the source of the problem, (2) internal blame that acknowledges the role of the apologizer, (3) external blame that attributes factors outside of the apologizer’s control for the cause of the error.

Through a series of experiments with human respondents Turel and Cui find that there are some tasks and types of apologies that do work to repair reliance in the AI system they study. Simple apologies didn’t work at all. But apologies from the system that blamed external factors (i.e. that the data fed into the system was insufficient) repaired reliance better than an internal attribution to the system’s own limitations (i.e. the AI tool lacked capability to do the task). This was true for what the authors call “objective” tasks — in this case estimating a person’s weight from an image.

What I find troubling about the findings is that deflecting the blame worked better than acknowledging it. The risk for accountability is that individuals subject to AI errors might inappropriately brush aside an error rather than contesting it and asking for a deeper explanation. Perhaps AI system apologies should instead locate and identify the human actors in the system who are offering the apology. The findings further remind us of the importance of faithfulness in AI explanations. Given the issues with sycophancy in LLMs it’s not hard to imagine a situation where models might learn to deflect blame in their “apologies” by offering not-so-faithful renditions of why an error occurred due to some external factor beyond the system. While there are certainly legitimate situations where an explanation due to external factors is warranted, policy desperately needs to grapple with the validity of explanations rendered directly by AI systems.

References

Turel O and Cui T (2026) Apologizing artificial intelligence: designing and evaluating effective AI apologies after errors. AI & SOCIETY: 1–18.

Like accountability, responsibility is a relational concept: people are responsible to someone for something, creating “an expectation of an action or its result” (Lenk, 2006). Depending on the who and what in that relationship can lead to different flavors of responsibility, like moral responsibility, or legal responsibility. Vincent (2011) counts at least six different concepts referred to by the word “responsibility”, including virtue responsibility (i.e. character and commitment to doing what’s right), role responsibility (i.e. the duties an actor should or shouldn’t do), outcome responsibility (i.e. backward looking attribution for a state of affairs), causal responsibility (i.e. the cause of the state of affairs), capacity responsibility (i.e. whether the actor has the requisite cognitive or material ability to be responsible), and liability responsibility (i.e. the financial or other action needed to make things right).

Building on decades of research on the psychology and cognition of how humans assign responsibility (e.g. Attribution Theory — Kelley and Michela, 1980), Franklin and colleagues (2022) published a model of AI responsibility that implicates nine different factors and their relationships in attributing responsibility:

There are overlaps and analogues to some of the philosophical conceptions of responsibility Vincent outlines, including causality, role, character (i.e. virtue), and capability. Additional factors include intent—classically associated with assignments of moral responsibility (Nissenbaum, 1994), desire/aim (i.e. an orientation that precedes commitment to an intended action), objective foreseeability (i.e. how likely an outcome really is, regardless of the agent’s expectations), and autonomy (i.e. ability for agent to freely make decisions). Failure to clarify these factors can lead to what has been termed a “responsibility gap”, such as when a person lacks sufficient knowledge over an autonomous AI system whose interactions with the environment make its behavior difficult to foresee (Matthias, 2004).

Some of these factors can be elaborated or nuanced further. As one example, the causality factor can be broken down into an internal (i.e. factors about the automation itself like its programming) vs. external (i.e. situational factors or data quality) cause, which also shapes how people perceive responsibility (Pareek et al, 2025). Another dimension that is relevant to how people view responsibility is whether the situation is low or high-stakes (Tsumura and Yamada, 2025; Pareek et al, 2025). Moreover, a cross-cultural study showed that while intent is implicated in moral judgements in many large-scale industrialized societies, there are some small-scale societies where it is not (Barrett et al, 2016), which has implications for global AI responsibility standards and policy.

These various factors are helpful for analyzing responsibility in an AI system. For instance, an autonomous vehicle (AV) may have physically caused the damage to another car that it rear-ends during a snow storm. But if the driver was supposed to be vigilant (i.e. that was their role responsibility) we might think they share some of the outcome responsibility. If the driver was distracted by a TV show they were watching on their phone, this might weigh the responsibility more toward them (i.e. due to a character failure). In contrast, if the deploying company had knowledge that the AV lacked the capability to detect and stop before hitting another vehicle in a common situation like a snow storm, they could have objectively foreseen such an incident, weighing the responsibility toward them.

What does this all mean practically for AI accountability? These factors enumerated above should play a more explicit role in what we expect from AI system explanations. In order to make nuanced decisions about assigning responsibility and calling for accountability, explanations of AI systems need to include information that helps accountability forums assess things like capability, knowledge, causality, character, role, intent, and autonomy of actors in the system. These factors suggest bits of information needed to inform responsibility judgements and should be the basis for explanation requirements and transparency regimes so that the needed information is available. Accountability forums such as the media or administrative agencies can use the factors to inform lines of questioning, AI incident cards could be designed to include descriptions of each factor, and a growing list of AI regulations around the world could be analyzed through this framework to assess whether they’re really compelling the right information needed for responsibility assignment.

References

Barrett HC, Bolyanatz A, Crittenden AN, et al. (2016) Small-scale societies exhibit fundamental variation in the role of intentions in moral judgment. Proceedings of the National Academy of Sciences 113(17): 4688–4693.

Franklin M, Ashton H, Awad E, et al. (2022) Causal Framework of Artificial Autonomous Agent Responsibility. Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society: 276–284.

Kelley HH and Michela JL (1980) Attribution theory and research. Annual review of psychology 31(1): 457–501.

Lenk H (2006) What is Responsibility? Philosophy Now (56). https://philosophynow.org/issues/56/What_is_Responsibility

Matthias A (2004) The responsibility gap: Ascribing responsibility for the actions of learning automata. Ethics and Information Technology 6(3): 175–183.

Nissenbaum H (1994) Computing and accountability. Communications of the ACM 37(1): 72–80.

Pareek S, Schömbs S, Velloso E, et al. (2025) “It’s Not the AI’s Fault Because It Relies Purely on Data”: How Causal Attributions of AI Decisions Shape Trust in AI Systems. Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems: 1–18.

Tsumura T and Yamada S (2025) Effects of knowledge and importance on responsibility in human-AI decision making. Scientific Reports 16(1): 2670.

Vincent NA (2011) A Structured Taxonomy of Responsibility Concepts. In: Moral Responsibility. Library of Ethics and AppliedPhilosophy, pp. 15–35.

From the 30 cases analyzed the authors report that in about 87% of cases the whistleblowers were motivated, at least in part, by moral considerations, with 27% indicating some kind of financial motivation. At least 90% were insiders at the offending organization and roughly 80% were mid-level employees or executives. But stepping forward was costly: 57–67% faced retaliation (e.g. harassment or unjust termination), 43–57% suffered negative career consequences, and 13% received death threats. Only 13% sought anonymity—although the authors caution that this likely reflects sampling bias toward famous cases. In short, whistleblowers in the dataset tended to be morally motivated insiders who paid a steep personal price.

Based on these patterns and their own observations the authors develop a design sketch for an AI whistleblower office. They claim it should: (1) financially reward tipsters with a percentage of sanctions, in the spirit of the SEC and CFTC programs, given that this can be a motivator ; (2) prohibit retaliation and offer witness protection (plus S visas for international tipsters); (3) enable anonymous tipping via lawyers or a secure online platform; (4) be adequately staffed and funded for effective “tip-sifting”; and (5) invest in messaging to raise awareness for the office and an advisory body to help would-be whistleblowers determine whether they have reasonable cause.

For AI accountability, this work adds a new dimension to transparency. Mandated disclosures and external audits will always leave gaps and insider reporting is one of the few channels likely to surface willful concealment. The recommendations align with a prospective accountability frame: supporting protection, anonymity, and an advice body for potential whistleblowers are forward-looking responsibilities that might make insider reporting a more viable option before harms have occurred. A sample of 30 is small, and the cases skew U.S., famous, and successfully-tipped—but as a starting point for thinking about policy the empirical grounding is valuable.

Note: This post was drafted by Claude Opus 4.7 under the prompting, supervision, and further editing by the author.

There is an array of research on the topic of explainable AI (i.e. XAI) extending back for nearly a decade (Lipton, 2018; Mittelstadt et al, 2018; Wachter et al, 2017), with a focus on making the technical components of AI systems more understandable. Explanation in this context has been defined as “the ability to articulate why a model produced a given output in a way that is accessible to human users” (Dhar et al, 2025). But recent research (Dhar et al, 2025; Alpsancar et al, 2025) has raised the critique that the literature on XAI hasn’t always been clear about the goals of AI explanations. Who and what are they for, really? If explanations are meant to support accountability, there is still somewhat of a gap in the literature showing exactly how, particularly if we parse out the different goals of retrospective vs. prospective accountability.

Dhar et al present a framework for thinking about the goals of AI explanation in terms of who explanations are designed for, what information is conveyed, and how an explanation presents that information (2025). Different stakeholders such as AI system developers, operators, validators, and subjects may have different needs for explanations, including different modalities of presentation (e.g. visual, textual, interactive) that match information needs. Each stakeholder here might need something a bit different from an AI system explanation in order to contribute to accountability. For instance, a system developer might benefit from a highly technical interactive explanation that helps them debug an issue with the model and so help prevent future bias or fairness issues in decisions. A decision-subject might need something more accessible to help them understand why they got the outcome they did and potentially contest it if they think it’s wrong. A validator (e.g. an auditor) may need to verify input features used by the model to ensure they are accurate and appropriate. And an operator needs to be informed about how their actions lead to probable consequences with the system in order to be a responsible human-in-the-loop (Baum et al, 2022).

Besides who explanations are for, there are important dimensions about what should be explained (Dhar et al, 2025). Local explanations focus on individual outputs and are well-aligned to the goals of retrospective accountability, which focus on identifying and assigning blame for a specific individual decision. On the other hand global explanations, which orient towards overall patterns of output from a model across a range of inputs, are better suited to supporting goals of prospective accountability where a birds-eye view is needed to inform how to prevent anticipated harms at the system level. Post-hoc explanations of system behavior which track how inputs influence outputs are the key for both retrospective and prospective accountability, while mechanistic explanations that trace functional model internals are more narrowly useful for informing developers towards preventing unintended outcomes. In other words, while the classic view of accountability as retrospective doesn’t hinge on explaining model internals, a prospective view could additionally benefit from explanations of those internals to debug model failures and ensure better outcomes in the future.

An early paper to make a connection between AI explanation and the goals of accountability comes from Doshi-Velez et al (2017). The authors appropriately point out the potential for explanations to prevent or rectify errors in AI systems, helping to discern the appropriate or inappropriate use of criteria by a system. They note two key types of explanation that can play an important role in supporting accountability: feature importance and counterfactuals.

Feature importance/relevance explanations provide information about the weighting and priority of inputs to specific outputs, or to the overall distribution of outputs. Should some features be unacceptably correlated to outputs (e.g. race) overall, this can inform prospective accountability so that the model or its training data can be rectified. If features are correlated to outputs in ways that contradict a scientific causal account of what should be predictive of outcomes, this could be grounds for prospective accountability to align the model with scientific expectations. If the scientific account is well-established such a contradiction could also contribute to retrospective accountability for negligence.

Counterfactual explanations include a “statement of how the world would have to be different for a desirable outcome to occur” and “describe a dependency on the external facts that led to that decision” (Wachter et al, 2017) and have also been framed as a form of feature relevance explanation (Speith, 2022). If a decision-subject had their mortgage application denied and the counterfactual explanation indicated that they would have been approved if their race had been different, that would be clear grounds for that individual to contest the output.

While Doshi-Velez got it mostly right when it comes to supporting retrospective accountability, another explanation type elaborated in the literature (Speith, 2022)—model surrogates (e.g. linear approximations)—may also be narrowly useful for prospective accountability. What a model surrogate explanation can offer is a clear and interpretable feature importance explanation of a more complex model (e.g. a neural net, or other black-box model). If that feature importance explanation indicates an inappropriate bias this could be grounds for a developer to be prospectively responsible for addressing the apparent behavioral bias of their model. Even if the model itself doesn’t use inappropriate data, if its behavior appears to be inappropriate that might be grounds to call for it to change. Where model surrogates are not so useful is for retrospective accountability as they don’t reflect the actual decision-logic impacting a specific individual.

A recent paper from Alpsancar et al (2025) makes an explicit connection between AI explanation and the needs of assigning responsibility to support AI governance. The authors recount the classical model of moral responsibility which hinges on fulfilling three criteria to hold someone responsible for their actions: (1) causality (i.e. the person influenced the outcome), (2) freedom (i.e. the person was not coerced in their action), and (3) epistemic (i.e. the person is aware of the consequences of their actions). The authors also review what they term the trans-classical model of responsibility which is a systemic view of responsibility that helps cope with unintended and unforeseen consequences. In this view, the epistemic condition instead relates to knowledge of the potential for and probability of various outcomes in the system (i.e. risk) and responsibility is assigned for managing that risk.

In the classical view, the goal of explanation for accountability is clear: to help fulfill the three conditions so that responsibility can be assessed. AI system explanations should indicate causality, including who (or what) took what actions that were critical to the outcome. They should indicate the autonomy of entities and their actions, including how individuals may be influenced by AI systems in their judgements. And they should show whether individuals in the system were appropriately informed about the consequences of their actions. On the other hand, in the trans-classical view the goal of explanation should be to support the understanding of the risk (i.e. severity and prevalence) of outcomes. But it could also be important for explanations to show that there is not a direct causal actor responsible in the system, since otherwise we might revert to the classical model. Regardless of view there is a need for a sociotechnical approach to AI explanation. Explanations of technical models as discussed above are important for supporting the knowledge needs for either view.

There are several policy-relevant implications that can be derived here. First, explanation requirements for AI systems should specify the audience for the explanation. A disclosure rule that works for a decision-subject contesting a denied loan looks very different from one aimed at auditors verifying model inputs or developers debugging bias. Second, any explanation requirements should tie back to the accountability purpose being served. Retrospective accountability calls for local, post-hoc explanations including counterfactuals and feature importance explanations, while prospective accountability calls for global explanations about patterns across outputs. Third, policymakers should consider both the classical and trans-classical view of responsibility and how and whether they may want to blend or distinguish the two in assigning responsibility. Finally, standards bodies should resist technical definitions of explainability and consider sociotechnical elements related to the human use of AI systems and their explanations.

References

Alpsancar S, Buhl HM, Matzner T, et al. (2025) Explanation needs and ethical demands: unpacking the instrumental value of XAI. AI and Ethics 5(3): 3015–3033.

Baum, K., Mantel, S., Schmidt, E. & Speith, T. From Responsibility to Reason-Giving Explainable Artificial Intelligence. Philos. Technol. 35, 12 (2022).

Dhar R, Brandl S, Oldenburg N, et al. (2025) Beyond Technocratic XAI: The Who, What & How in Explanation Design. Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society 8(1): 745–759.

Doshi-Velez F, Kortz M, Budish R, et al. (2017) Accountability of AI Under the Law: The Role of Explanation. arXiv. DOI: 10.48550/arxiv.1711.01134.

Lipton, Z. C. 2018. The mythos of model interpretability:In machine learning, the concept of interpretability is both important and slippery. Queue, 16(3): 31–57

Mittelstadt B, Russell C and Wachter S (2019) Explaining Explanations in AI. Proceedings of the Conference on Fairness, Accountability, and Transparency: 279–288.

Speith T (2022) A Review of Taxonomies of Explainable Artificial Intelligence (XAI) Methods. 2022 ACM Conference on Fairness Accountability and Transparency: 2239–2250.

Wachter, S.; Mittelstadt, B.; and Russell, C. 2017. Counterfactual explanations without opening the black box: Automated decisions and the GDPR. Harv. JL & Tech., 31: 841

Explanations rendered by and about AI systems need to be as faithful as epistemically possible in order to support accountability. Buijsman describes the role of explanation in supporting accountability: “when a mistake has been made, the challenge is to find a reason why that mistake happened and the people responsible for fixing it.” (2026). A faithful explanation might help understand whether there may be an issue with faulty data, missing information, or incorrect reasoning, and ultimately help improve the system over time. Explanations that are not faithful could misdirect decision-making about how to assign blame or prevent future harms, frustrate attempts to contest a decision or diagnose mistakes and logical errors so they can be corrected, and ultimately to appropriately sanction actors if the explanation is unacceptable.

Faithfulness is especially relevant to questions of process accountability, where the goal is to hold an actor in the AI system accountable for how an outcome was computed. Explanations are a diagnostic tool for accountability, describing how inputs lead to the outcome and helping to trace instances of potential negligence or faulty logic in the system. If an unfaithful explanation of a mortgage decision says that you were rejected because your income is too low but the model decision was actually influenced by your race or zip code this undermines your ability to challenge the decision as unacceptably including protected characteristics.

LLMs are not able to provide faithful explanations, such as self-explanations generated by the model to render the “reasoning” behind their output in human-understandable language (Madsen et al, 2024; Mayne et al, 2025; Mutton et al, 2025). Madsen and colleagues (2024) show that larger models with more parameters generally produce more faithful explanations but that there is high variance across tasks. Mayne and colleagues (2025) focus on self-generated counterfactual explanations (SCEs) and indicate that their findings “suggest that SCEs are, at best, an ineffective explainability tool and, at worst, can provide misleading insights into model behaviour.” While models may be able to provide counterfactual explanations (e.g. if you change variables X and Y it will flip the decision outcome), these may be trivially true rather than articulating minimal changes to the input that would actually shed light on the decision.

The main implication here is that when accountability matters, such as for high-stakes situations where there is potential for severe impacts, faithful explanations are critical, but LLMs cannot provide such explanations. Policymakers may consider when AI providers need to demonstrate faithfulness of model explanations and establish thresholds around when models can be used in high-stakes contexts. Administrative bodies will also need to develop standardized benchmarks and measurements for faithfulness to support such policies.

References

Agarwal C, Tanneru SH and Lakkaraju H (2024) Faithfulness vs. Plausibility: On the (Un)Reliability of Explanations from Large Language Models. arXiv. DOI: 10.48550/arxiv.2402.04614.

Buijsman S (2026) Accuracy is not all you need! The Reasons to Require AI Explainability. Minds and Machines 36(1): 14.

Jacovi, A. & Goldberg, Y. Towards Faithfully Interpretable NLP Systems: How Should We Define and Evaluate Faithfulness? Proc. 58th Annu. Meet. Assoc. Comput. Linguistics 4198–4205 (2020) doi:10.18653/v1/2020.acl-main.386.

Lyu, Q., Apidianaki, M. & Callison-Burch, C. Towards Faithful Model Explanation in NLP: A Survey. Computational Linguistics 50, 657–723 (2024).

Madsen A, Chandar S and Reddy S (2024) Are self-explanations from Large Language Models faithful? In: Findings of the Association for Computational Linguistics: ACL, 2024.

Mayne H, Kearns RO, Yang Y, et al. (2025) LLMs Don’t Know Their Own Decision Boundaries: The Unreliability of Self-Generated Counterfactual Explanations. In: EMNLP, 2025.

Matton K, Ness RO, Guttag J, et al. (2025) Walk the Talk? Measuring the Faithfulness of Large Language Model Explanations. In: ICLR, 2025.

Rudin, C. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nat. Mach. Intell. 1, 206–215 (2019).

I recently came across a paper by Fok et al (2025) that’s been useful in helping me organize the various AI experiments I’ve been trying. Based on interviews with researchers who have written literature reviews the paper helps to understand their overall process and some of the ways they conceptualize the use of AI in that process.

The findings articulate a set of four phases to the literature review process that participants engaged in: search, appraisal, synthesis, and interpretation. The paper also identifies some of the ways AI can support updating of reviews, namely through automation and in providing a second opinion. My own use of AI for AIAR has been most useful for automating (with oversight) some of the appraisal aspects of the process, and in providing second opinions on appraisal and synthesis. I have also dabbled in some use cases that more directly do synthesis and interpretation, but these have been less successful. And I haven’t really tried any AI use cases for search, because I think that setting the search scope for the review is something that needs to be closely managed by me. Let’s walk through some of the different things I’ve tried.

Scraping, Formatting, and Promotion

Probably the most time-saving use case I’ve found is to use OpenAI’s Agent mode to help collect conference proceedings papers that I want to review. Some conferences have non-standard presentations of information, but Agent mode is pretty adept at navigating websites to collect papers and format them as RSS feeds. I plug those feeds into my triage workflow on InoReader, which streamlines the appraisal process of papers. It can help to be explicit in the prompt and identify a structured data (e.g. JSON) version of the proceedings. And while this process is mostly automated I find that I do still need to double-check the outputs to make sure it was a comprehensive scrape.

I have also experimented with using Agent mode to gather email addresses for each of the primary authors of papers cited by one of my posts, and to then draft a short personalized note notifying the person about the post. I wasn’t intrepid enough to automate the actual emailing, but I did manually copy and send some of the emails (after light editing) and even got a response from one. Promoting AIAR on social media could be a full time job, but having the AI do some of the grunt work of getting email addresses and drafting emails lowers the barrier a bit.

Article Appraisal

One of the nice built-in features of InoReader is that for any item in a feed I am tracking, I can trigger a custom prompt to an LLM. Using this feature I can get a quick second opinion from the LLM on whether the item might be relevant to my audience. Admittedly I don’t use this all that often, but I do occasionally engage it. One of the issues is that not all the RSS feeds I follow have full abstract text and so this limits the applicability. I do think there’s real potential in having AI help think through what items have implications for your intended audience, and there’s probably a lot more sophistication that could be applied in how to do this computationally beyond the integrated prompting in InoReader, such as by simulating ideal audience members and what they would want to know about an item.

Some articles on AIAR reflect the synthesis of a cluster of literature. As a living literature review the goal is to update these over time with other literature relevant to the cluster. I’ve been experimenting with LLMs to support this process. Using a Google Colab notebook I input the URL of the base article to be updated and scrape the full text. Then I prompt an LLM to evaluate a stream of literature for relevance to that article. The prompt is critical here. What I’m looking for are new papers that might directly update, change, or provide new context to any of the claims in the original article, to find new papers that might actually make a difference.

Each paper is rated for relevance, and that rating is paired with a table listing claims from the original article and ideas from the new paper that might bear on those claims. The table facilitates my appraisal of the new paper. The output looks like this:

So far this is promising, but there’s still work to do to evaluate it and set it up as an ongoing monitoring process that fully integrates with my InoReader appraisal workflow. In principle I’d set this up for each of the base articles in AIAR, and then monitor literature from something like OpenAlex to create a continuously updated feed of potentially relevant papers.

Grounded Synthesis

Google’s NotebookLM has turned into an increasingly powerful tool that can be used to interactively synthesize curations of articles. For my article on AI Ethics Principles and Accountability, I even published a notebook with all of the sources I had used to write the article. While the original goal with creating the notebook here was to allow readers to interactively explore the literature, I also realized that I could also use this to provide a second opinion on my own synthesis. Using Gemini, you can refer to a notebook of curated sources in NotebookLM and so I prompted it to create a table listing the supporting evidence for every claim in the post. In the absence of an editor, this can be a useful double-check to make sure you’re staying honest to the underlying literature in your synthesis. I think this kind of approach could potentially also be useful in an article update process to assess whether claims in new papers support or refute the existing claims you’ve written.

Still, I am a bit cautious about relying on LLMs, even closely grounded ones, in helping to synthesize literature for AIAR. In an early experiment, I loaded up NotebookLM with the entirety of the Fairness, Accountability, and Transparency Conference proceedings from 2025. I asked Gemini (with access to the Notebook) to look for clusters of papers that were thematically related to each other and to the topic of the blog. While some of these clusters seemed relevant and overlapped with my own perception of themes, others seemed more tenuous in the solidity of the theme and its relevance to AIAR. Synthesis is to a large degree about framing and finding a consistent thread, and I don’t think even the best LLMs are able to do this in a way that is satisfying.

As a Writing Aid

I have attempted to use LLMs (primarily Gemini, sometimes directly in NotebookLM) to help draft five of the posts for AIAR, three of which were based on translating a single paper, and two of which were based on clusters of papers.

I found that for the articles based on clusters the LLM was wholly unsuited to the task of synthesis: I ended up using none of the generated text. Even including all of the paper texts and my notes on those papers in the prompt, I was left feeling that the synthesized text didn’t capture what was interesting or important about the cluster. This again goes back to the idea of framing, structuring, and finding the aspects of relevance that I think are important within the field and to my intended audience. But this also relates to the interpretation phase and the “identification of key challenges, future trends, and open research opportunities” (Fok et al, 2025). All of this is consistent with what some editors at Science found when they tried to use ChatGPT to translate research papers.

For the three articles that were more direct translations of individual research papers I had slightly more success with incorporating AI generated text. In this post, I used almost 50% of the generated text in the final piece, which warranted a disclosure at the bottom of the post: “Some text in this post was adapted based on suggestions from AI.” I think this was somewhat successful because I prompted the model with details on the aspects of the paper I wanted the post to focus on, and that the post itself was more descriptive than synthetic or interpretive. The parts of the post that I wrote were the more interpretive aspects, putting the research into a broader context and considering its relevance to the audience. In another post (excepted below), I was also able to use some chunks of descriptive text that were generated by the LLM.

In Closing

Much like everything else on AIAR, this post will be a work-in-progress and is subject to update. The most compelling use-case I’ve found so far for AI is in automating the collection and formatting of references into my RSS workflow as this lets me do something that I might not otherwise make time for. I also find the article appraisal workflow compelling and plan to keep pushing on that to integrate it more into my regular workflow for keeping AIAR posts updated. I may also revisit use cases related to grounded synthesis and writing though I’m generally less optimistic about AI providing a real lift there. The work of framing and making connections in the literature, contextualizing findings, and thinking about what matters to an audience seem like they really need an expert eye, though perhaps LLMs can assist by offering a second opinion.

References

Fok R, Siu A and Weld DS (2025) Toward Living Narrative Reviews: An Empirical Study of the Processes and Challenges in Updating Survey Articles in Computing Research. Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems: 1–10.

Probably the biggest issue I see with the law is in the definitions. The RAISE act is geared towards regulating “frontier models” which it defines as: “an artificial intelligence model trained using greater than 10º26 computational operations (e.g., integer or floating-point operations), the compute cost of which exceeds one hundred million dollars” (or a different definition that applies to models produced through knowledge distillation). This is a bad definition because it merges two criteria that are arbitrary, shifting over time, and, most importantly, which model developers are not required to disclose. They are arbitrary because there’s no reason to think that 10^26 computing operations is a magical threshold at which danger suddenly materializes. Based on estimates from Epoch, none of the current breed of frontier models surpasses this threshold, so it’s not clear that the law applies to anything in the real world. And, again, even if OpenAI, Google, or Anthropic has exceeded this threshold in its training there’s no way for us to know because the law doesn’t make them tell anyone. It’s a sort of scouts honor, opt-in system. Also, the definition of compute is in conjunction with a cost greater than $100 million. Because the definition has to match both criteria, a model could use more than the compute threshold but be done for less than $100 million and then the law wouldn’t apply. But compute costs are always getting cheaper, and some model developers like Google control the market price of their computing and so can game this, not to mention that the value of the dollar could change.

The main mechanism for specifying transparency in the law is that large developers of models create a “safety and security protocol” —a form of transparency report—before deployment of the model. There are also provisions to require the reporting of “safety incidents” that might be a case of or increased risk of critical harm. The protocol report is shared with administrative accountability forums such as the attorney general and division of homeland security, as well as being made publicly available in redacted form for media or social forums. The unredacted protocol plus additional information about the tests and test results that inform the protocol need to be maintained for however long the model is deployed plus five years, presumably so that those records are potentially available for discovery by legal forums in the event they are needed. In this sense, the law does pretty well in providing for accessibility of the safety and security protocol to various accountability forums.

The overall aim of the law towards “frontier models” and “critical harm” scopes and sets limits on the relevance of the information in the protocol. Critical harm is defined as causing $1 billion or more in damage or loss of 100+ human lives. But with that scope in mind the definition of the protocol is reasonable as it specifies what should be included, including organizational procedures and sociotechnical measures meant to mitigate the potential for critical harm, as well as the testing procedures used to “evaluate if the frontier model poses an unreasonable risk of critical harm”. The protocol must also designate a person that is responsible for compliance — this is a critical component that ensures accountability for overseeing the protocol. The timeliness of the report is also referenced and calls for the developer to update the protocol on an annual basis as per any changes.

An area where the protocol falls short is in either specifying or auditing the accuracy of the information in the protocol. An earlier version of the law had provisions requiring 3rd party auditing, but those were removed from the final version signed into law. That would have strengthened the law considerably by having an independent entity checking the validity of procedures and the accuracy of provided information in the protocol. What’s left is the comparatively weaker request that large developers not lie, i.e. “shall not knowingly make false or materially misleading statements or omissions.” We can’t really assess whether the information in protocols would be understandable and fit for the purpose of accountability. A stronger law would have created a standard for the protocol that would be considered adequate.

The law provides reasonable carve outs to address typical criticisms and stakeholder pushback about transparency, including that disclosures might undermine privacy, confidentiality, trade secrets, or be used to game the system. Redactions to public safety and security protocols can be undertaken to protect these other interests. The law also protects fundamental innovation by not applying to academic research done at accredited colleges and universities. In addressing the tensions between transparency and other interests at stake, the law probably does about as well as it could, especially because administrative forums like the attorney general can gain access to copies of the protocol that are less redacted, i.e. where redactions only need to respect federal law, and fully unredacted reports must be maintained for possible discovery in legal forums.

Overall, much like its Californian counterpart, New York’s RAISE act is geared towards prospective accountability — trying to prevent future harm. Its scope is narrow around “critical harms”. While it does well to specify the accessibility of the transparency information it calls for, and align that information so it is relevant and timely to its scope, it lacks provisions for ensuring the accuracy of the information, and leaves the understandability of that information up to the large developers who’ll be creating the reports. But it’s not a powerful law because it doesn’t apply to anything in the real world (yet), and it’s unclear whether model developers will ever raise their hand and say that the law actually applies to them. It does provide an example of AI governance through transparency that can inform future legislation. The next version of the law, proposed by the governor’s office and under consideration by the state legislature, is already drastically different in many ways.

Transparency sets up a relationship between two entities—here an AI system and a forum—where information about the AI system becomes available to the forum. Because AI systems are sociotechnical this includes information about both the data and technical model in the system, as well as the human components such as organizational policies, procedures or practices, and user behaviors (Diakopoulos, 2020). For the sake of accountability, provided information should help the forum determine congruence with relevant values, goals, and normative or legal expectations of behavior (Hayes et al, 2023; Fleischmann and Wallace, 2005). Transparency information can be voluntary (e.g. a blog post), obligatory (e.g. legally mandated disclosure to an administrator), or involuntary (e.g. external audits, or leaks), though recent research has underscored the inadequacy of volunteered “first-party” transparency information compared to external “third-party” evaluations of social impacts (Reuel et al, 2025).

To be useful for accountability, transparency information needs to reflect high information quality. At a minimum it needs to be accessible, understandable, relevant, and accurate (Hayes et al, 2023; Diakopoulos, 2020; Turilli and Floridi, 2009). Beyond just availability, information needs to be accessible so that it can be easily found by audiences such as various accountability forums. It also needs to be understandable or usable by those audiences and aligned to their information processing capabilities and capacities. It needs to be relevant to diagnosing some behavior of interest whether that be in shedding light on some negative outcome for retrospective accountability, or providing critical context to inform prospective accountability. Information also needs to be accurate such that it is valid, reliable, and free of error (Turilli and Floridi, 2009), since otherwise it can suffer from strategic activities that shape or distort information, leading to uninformative or boilerplate disclosures (Marin et al, 2025). Other aspects of information quality that are pertinent include the currency or timeliness of the information, and its comprehensiveness. AI transparency will typically fall short when the above factors and attributes aren’t adequately addressed.

A reoccurring pattern we see in the literature is a failure to clearly articulate the intended audience or forum for transparency information, with implications for how the information would be maximally accessible, understandable, and relevant for that audience. For instance, in the 2025 Foundation Model Transparency Index (Wan et al, 2025), the authors establish a set of 100 indicators that they apply to various models to evaluate how transparent they are in terms of data, training, compute usage, modeling, and downstream impacts and use policies. But the audience for all of this information—and its utility for accountability—is anything but clear. What transparency initiatives like this one need to do is clearly articulate the public interest and accountability purpose of each indicator, helping to connect over to the audience or forum that would then use that information for accountability. Similarly, a recent proposal for AI agent transparency (Ezell et al, 2025) appears oriented somewhat towards technical developers “debugging” agent incidents. If the information in that framework could be made available to administrative or judicial forums, it’s likely they would benefit from at least some of the information. But the ideal would be a more parsimonious framework that more closely tracks the needs of those forums for specific issues they may need to assess for accountability.

While I would argue that transparency is a necessary pre-condition for accountability, critics point out that transparency is not an unalloyed positive force. It shouldn’t be assumed to always enable accountability (Corbett and Denton, 2023), though policies that shape adherence to the attributes of quality transparency information described above should increase the likelihood of its utility. Transparency can also come into tension with other values, such as privacy, freedom of expression, or intellectual property (Ananny and Crawford, 2018; Diakopoulos, 2020; Turilli and Floridi, 2009) leading to situations where tradeoffs need to be made in highly context-specific ways. One of the most frequent counter arguments to more transparency is that it could enable gaming or manipulation of the system (van Bekkum and Borgesius, 2021), though careful context-specific engineering, threat modeling, and consideration to forum-specific access provisions should alleviate this issue (Diakopoulos, 2020). We might also consider the idea that social forums may use manipulation as a way to sanction a system—in other words manipulating a system may in some contexts and situations be considered a component of holding a system accountable for unwanted behavior. Ultimately, the choices around what, when, and how AI systems are made transparent are political (Corbett and Denton, 2023).

The role of policy here is to thread the needle through these criticisms to scope transparency and shape it towards positive outcomes for society. Policy must create obligations for actors within AI systems to produce the information needed by any given forum (e.g. administrative, legal, etc.) to make the relevant assessment of system performance. This information needs to meet accessibility, understandability, relevance, accuracy, currency, and comprehensiveness quality criteria. One way to do this is to be more specific about standards for AI system transparency information production: what standard processes and practices should be evidenced by actors making transparency information available? Public sector policy makers cannot leave this unspecified, otherwise there is too much room for strategic and performative behavior. Another role for policy makers is to engage in the politics of where and how to make tradeoffs with other values such as privacy; looking to public attitudes should probably inform this. Transparency policies need to be user-centered (e.g. towards whatever forum the information is intended for) and context-specific, and would benefit from human-centered engineering and evaluation to refine their scope, meet user needs, and maximize their utility for accountability.

References

Ananny M and Crawford K (2018) Seeing without knowing: Limitations of the transparency ideal and its application to algorithmic accountability. New Media & Society 20(3): 973–989.

Corbett E and Denton R (2023) Interrogating the T in FAccT. Conference on Fairness, Accountability, and Transparency: 1624–1634.

Diakopoulos N. (2020) Transparency. Oxford Handbook of Ethics and AI. Eds. Markus Dubber, Frank Pasquale, Sunit Das.

Ezell C, Roberts-Gaal X and Chan A (2025) Incident Analysis for AI Agents. Proc. AI, Ethics, and Society (AIES) DOI: 10.48550/arxiv.2508.14231.

Fleischmann KR and Wallace WA (2005) A covenant with transparency. Communications of the ACM 48(5): 93–97.

Hayes P, Poel I van de and Steen M (2023) Moral transparency of and concerning algorithmic tools. AI and Ethics 3(2): 585–600

Marin, L. G. U.-B., Rijsbosch, B., Spanakis, G. & Kollnig, K. Are Companies Taking AI Risks Seriously? A Systematic Analysis of Companies’ AI Risk Disclosures in SEC 10-K forms. arXiv (2025). https://arxiv.org/abs/2508.19313

Meijer A, Bovens M and Schillemans T (2014) Transparency. The Oxford Handbook of Public Accountability. Oxford University Press.

Reuel A, Ghosh A, Chim J, et al. (2025) Who Evaluates AI’s Social Impacts? Mapping Coverage and Gaps in First and Third Party Evaluations. arXiv. https://arxiv.org/abs/2511.05613

Turilli, M., Floridi, L.: The ethics of information transparency. Ethics Inform. Technol. 11, 105–112 (2009)

Bekkum M van and Borgesius FZ (2021) Digital welfare fraud detection and the Dutch SyRI judgment. European Journal of Social Security 23(4): 323–340.

Wan A, Klyman K, Kapoor S, et al. (2025) The 2025 Foundation Model Transparency Index. arXiv. DOI: 10.48550/arxiv.2512.10169.

The crux of the analysis is in comparing 186 first-party reports that were part of model releases by model developers to 183 post-release evaluations that were run by various third parties. These reports were assessed based on the level of detail provided in evaluations of any of seven social impact dimensions as identified by Solaiman et al (2023). The seven dimensions assessed were Bias and Harm, Sensitive Content (e.g. outputting hate speech), Performance Disparity (e.g. unequal results across subpopulations), Environmental Costs and Emissions, Privacy and Data, Financial Costs, and Moderation Labor (e.g. working conditions of data annotators). The rating scale ranged from a 0 (no evaluation present), 1 (vague mention), 2 (concrete results but limited clarity on methods and context), and 3 (sufficient detail to understand and contextualize the evaluation). All the ratings are available here.

The main take-away is that third party evaluations were considerably more detailed, on average, than first-party evaluations (2.62 vs. 0.72 on the 0-3 scale). The implication is that the tech companies and other organizations training models are not releasing as much detail about their evaluations of social impacts in comparison to third parties who run evaluations. The authors note that the most popular models from the US (and to a lesser extent China) tend to attract the most third party evaluations, exposing a gap in evaluation of less-popular models. They also note that certain impact types such as data and content moderation impacts (as well as some others like environmental impacts) are not prevalent at all and are almost entirely absent from third-party evaluations, exposing the reality that third-parties just do not have access to the information they would need to properly evaluate certain issues.

The take-aways for policy here seem pretty clear. First-party evaluations of models by model providers are insufficient when it comes to evaluations of social impacts. There is a fair bit of variance in what level of attention different models receive and what dimensions of social impact are evaluated at all. Transparency standards are needed to provide more consistency and expectations for what evaluations need to be run and how, or which data needs to be disclosed so that third parties can cover more terrain with their evaluations. In addition, there need to be standards around which models demand a full evaluation. And there needs to be sufficient capacity in the evaluation landscape of third parties to be comprehensive. Advancing consistent transparency standards for AI models would support AI accountability by providing the information needed by different accountability forums.

References

Reuel A, Ghosh A, Chim J, et al. (2025) Who Evaluates AI’s Social Impacts? Mapping Coverage and Gaps in First and Third Party Evaluations. arXiv. https://arxiv.org/abs/2511.05613

Solaiman I, Talat Z, Agnew W, et al. (2023) Evaluating the Social Impact of Generative AI Systems in Systems and Society. arXiv. https://arxiv.org/abs/2306.05949v2

Subscribe now

Early analyses of the numerous published AI guidelines have identified a few core principles. These include privacy, fairness/justice, accountability/responsibility/explicability, transparency, beneficence, non-malfeasance/safety, and human autonomy (Jobin et al, 2019; Hagendorff 2020; Floridi et al, 2018). Despite mostly stable underlying ideas the exact terminology can vary and leads to a lack of clarity (Morley et al, 2021). A longer tail of principles includes ideas like trust, sustainability, dignity, and solidarity (Jobin et al, 2019).

Principles can come from different sources and so be biased in different ways, such as towards ideas in dominant geographies or from power holders such as experts or companies (Hickok, 2021). They can come from researchers and experts in the field (Floridi, 2018), from professional codes of conduct in domains of practice (Diakopoulos et al, 2024), from broad consensus documents like the UN declaration of human rights (Latonero, 2018), and be further informed from public evaluations (Kieslich et al, 2024). What’s the most legitimate source of principles for AI accountability? While a treaty like the Framework Convention on Artificial Intelligence has reached broad consensus, large swaths of the world still haven’t signed on. Achieving truly global principles will require ongoing political work.

Besides their potential to reflect biases, AI principles are also hard to actually implement in practice. Big abstractions need to be translated into concrete operationalizations (Hagendorff, 2020) if they are going to be used to measure AI system failures or guide AI system design to support prevention. Moreover, abstractions like fairness can hide contested ideas with conflicting perspectives (Mittelstadt, 2019) underlining the need to consider context-specific tradeoffs. Max (2026) suggests that ethics principles risk functioning more as political tools unless criteria for meeting or failing to meet them are provided in enough detail to permit criticism or contestation. Operationalizing principles also entails the challenge of adjudicating their prioritization, tensions, and trade-offs in specific contexts. Principles that are implausible to implement function more as rhetoric—”symbolic assurances”—than as viable governance instruments.

Prem (2023) analyzed more than 100 approaches from the literature for bridging the gap between principles and implementation. These include things like AI ethics criteria/checklists, metrics, process models, codes of practice, etc. He distinguishes approaches used during the design of a system (ex-ante), and those that are applied to an AI system after development or perhaps iteratively during development (ex-post). Ex-ante methods are relevant to prospective accountability, whereas ex-post methods are geared towards retrospective accountability (and also prospective if used iteratively during development). He notes that “Generally, there is a strong focus on those aspects for which technical solutions can be built,” exposing a further bias in the research on this topic.

Whereas designers and developers can adopt approaches to help prevent negative outcomes, AI system behavior itself should also be measured to assess adherence to principles. The idea of Ethics Based Auditing (EBA) applies the logic of auditing to the challenge of assessing system behavior “for consistency with relevant principles or norms.” (Mökander et al, 2021). This starts to get at a core issue of operationalizing principles into metrics that can evaluate (mis)alignment with a value. Principles just set the direction; effective accountability requires quantifiable performance metrics. This in turn requires supporting data access to inform those measurements.

Rismani and colleagues (2025) reviewed hundreds of these measures in the literature as they relate to different system components, hazards, harms, and principles. 90% of the measures they found were related to just four principles: fairness, transparency, privacy, and trust. To be useful for accountability metrics need to define some threshold of the metric which indicates the principle has been violated, that the system may create a hazard, and therefore warrants a call for accountability. Thresholds may be context-dependent, vary based on domain, and are subject to the risk tolerance of different stakeholders, but are rarely discussed in the literature (Rismani et al, 2025). This returns us to the normative question: How do you define an acceptable vs. unacceptable level of a measure of a principle? At what level might reasonable people agree there should be accountability? Public perceptions of acceptability may play a role here.

Principles serve as orienting ideas for what is valued. They can be used to determine what constitutes inappropriate behavior, necessitating accountability either retrospectively (blame for failure) or prospectively (prevention of harm). Bringing them into formal accountability forums (e.g. administrative, legal) hinges on mitigating biases in their enumeration and reaching a high degree of consensus. But implementing them in practice remains a challenge. They need to be translated into practices that designers and developers can use to mitigate the hazards created by an AI system, or to metrics with clear thresholds that can measure AI system behavior for signs of deviation. Policy should support the development of context- and domain-specific operationalizations of metrics and thresholds that are indicative of violations of principles by AI systems, as well as the data access provisions that would enable those measurements by the relevant accountability forums.

References

​​Diakopoulos N, Trattner C, Jannach D, et al. (2024) Leveraging Professional Ethics for Responsible AI. Communications of the ACM.​​

Floridi L, Cowls J, Beltrametti M, et al. (2018) AI4People—An Ethical Framework for a Good AI Society: Opportunities, Risks, Principles, and Recommendations. Minds and Machines 28(4): 689–707.

Hagendorff T (2020) The Ethics of AI Ethics: An Evaluation of Guidelines. Minds and Machines 30(1): 99–120.

Hickok M (2021) Lessons learned from AI ethics principles for future actions. AI and Ethics 1(1): 41–47.

Jobin A, Ienca M and Vayena E (2019) The global landscape of AI ethics guidelines. Nature Machine Intelligence 1(9): 389–399

Kieslich K, Helberger N and Diakopoulos N (2024) My Future with My Chatbot: A Scenario-Driven, User-Centric Approach to Anticipating AI Impacts. Conference on Fairness, Accountability, and Transparency: 2071–2085.

Latonero M (2018) Governing Artificial Intelligence: Upholding Human Rights & Dignity. Data & Society. https://datasociety.net/library/governing-artificial-intelligence/

Max R (2026) Guidelines without guidance: underdetermination in AI ethics. AI and Ethics 6(3): 323.

Mittelstadt B (2019) Principles alone cannot guarantee ethical AI. Nature Machine Intelligence 1(11): 501–507.

Morley J, Kinsey L, Elhalal A, et al. (2021) Operationalising AI ethics: barriers, enablers and next steps. AI & SOCIETY 38(1): 411–423.

Mökander J, Morley J, Taddeo M, et al. (2021) Ethics-Based Auditing of Automated Decision-Making Systems: Nature, Scope, and Limitations. Science and Engineering Ethics 27(4): 44.

Poel I van de (2020) Embedding Values in Artificial Intelligence (AI) Systems. Minds and Machines 30(3): 385–409.

Prem E (2023) From ethical AI frameworks to tools: a review of approaches. AI and Ethics 3(3): 699–716.

Rismani S, Shelby R, Davis L, et al. (2025) Measuring What Matters: Connecting AI Ethics Evaluations to System Attributes, Hazards, and Harms. Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society 8(3): 2199–2213.

A growing number of surveys consider the public perception and acceptance of AI across different use cases, such as health care, surveillance, and automation (Eom et al, 2024), personal health and labor replacement (Mun et al, 2025), AI in tax fraud detection (Kieslich et al, 2022), media, health, and justice domains (Araujo et al, 2020) and others. One study showed that overall judgments of the value of AI across a wide range of use cases is strongly shaped by perceived benefits, with perceived risks also playing a significant role (Brauner et al, 2025).

A recurring result in many of these survey studies is that there is variance in user acceptance of AI across people of different backgrounds. Factors such as the knowledge, literacy, education, or even political orientation of respondents, as well as their age and gender can play a role in the perception of risk, benefit, and acceptance of AI. For instance, younger respondents often view AI as less risky and more beneficial than older respondents (Brauner at al, 2025). A critical factor in individual perception is the level of AI knowledge the person has (and their confidence in that knowledge), where higher knowledge can lead to lower risk assessment, i.e. “risk blindness” (Said et al, 2023). Because of these differences, policy should ideally be informed by representative population samples, or perhaps population samples weighted by those who might bear the greater risk.

Kieslich et al (2021) take the perspective that we also need to understand public perception of the principles underlying AI systems. This in effect is a measure of whether the system is “aligned” with the perspectives and values of the person evaluating it. They measure perceptions of principles like explainability, fairness, security, accountability, accuracy, privacy, and limited machine autonomy for a scenario related to use of AI in tax fraud detection. For their representative sample of respondents from Germany they find that accountability was perceived as the most important principle. This underscores the idea that accountability is a critical property of AI systems that the public cares about.

Mun et al (2025) pairs a quantitative survey of various AI use cases together with open-ended follow-up questions where respondents elaborate on why they think a use case should or shouldn’t be developed, and what would need to change for them to switch their opinion of the use case. As with Brauner et al (2025) they find that cost-benefit reasoning dominates, but that in some cases virtue-based reasoning is somewhat more prevalent, such as for the Elementary School Teacher or Digital Medical Advice scenarios. They further analyze these rationale through the lens of Moral Foundations Theory and find that Care (i.e. dislike of pain of others or feelings of empathy and compassion) was the most prevalent reason mentioned overall, but fairness also dominated some use cases (e.g. Lawyer). This finding about how a moral foundation or value towards something like care aligns with one of the surveys reported on by Eom et al (2024) where 64% of respondents thought it was a bad idea for “robotic nurses for bedridden patients that can diagnose situations and decide when to administer medicine.” In other words, use cases where care is an underlying moral proposition seem to make people less accepting of the use of AI. In terms of accountability, then, we need to consider not only perceived risk, but also whether there is some kind of underlying value in society that is being violated.

One of the gaps identified by Araujo et al (2020) is that public perception of AI acceptance in a use case doesn’t necessarily tell us if people would personally accept a specific AI decision, or reject it and instead call for accountability. Important work remains to be done to understand this ego-centric retrospective case. On the other hand, for prospective accountability, research has begun to explore public perceptions around which stakeholders should be responsible for taking action to prevent negative outcomes (Barnett et al, 2025). This research uses written scenarios depicting harm from AI in the media ecosystem as a basis for a survey to gather public input about which stakeholders are in a position to take action to prevent the harm. Participants assigned responsibility to any of 12 different stakeholders that emerged from the data, including government, tech companies, news publishers, schools, social media platforms, independent third parties, local communities, public health officials, media companies, NGOs, employers, and unions. Specific actions that these stakeholders could take were then rated in terms of whether they should be taken, and also whether the action should be prioritized, resulting in rich data that could inform policy on how to assign responsibility for prevention, though ideally this process would be re-run with a representative sample.

Public opinion plays a critical role in shaping legitimate norms and standards for AI behavior. Policymakers should recognize that expectations of AI systems — including what is considered “acceptable” — are rooted in social perceptions. Surveys show that these perceptions vary based on demographic or other individual factors such as knowledge, and that there is variance across use case contexts. Policy should therefore be grounded in representative and inclusive data that is tailored to the specific use case contexts to be governed. Although cost-benefit reasoning dominates rationale for AI acceptance, value-based reasoning also needs to be considered. Finally, there is still much open research to do by drilling further into perceptions of who is responsible for what across a variety of situations.

Share

References

Araujo T, Helberger N, Kruikemeier S, et al. (2020) In AI we trust? Perceptions about automated decision-making by artificial intelligence. AI & SOCIETY 35(3): 611–623.

Barnett J, Kieslich K, Helberger N, et al. (2025) Envisioning Stakeholder-Action Pairs to Mitigate Negative Impacts of AI: A Participatory Approach to Inform Policy Making. Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency: 1424–1449.

Brauner P, Glawe F, Liehner GL, et al. (2025) Mapping public perception of artificial intelligence: Expectations, risk–benefit tradeoffs, and value as determinants for societal acceptance. Technological Forecasting and Social Change 220: 124304.

Eom D, Newman T, Brossard D, et al. (2024) Societal guardrails for AI? Perspectives on what we know about public opinion on artificial intelligence. Science and Public Policy 51(5): 1004–1013.

Kieslich K, Keller B and Starke C (2022) Artificial intelligence ethics by design. Evaluating public perception on the importance of ethical design principles of artificial intelligence. Big Data & Society 9(1): 20539517221092956.

Mun J, Yeong WBA, Deng WH, et al. (2025) Why (Not) Use AI? Analyzing People’s Reasoning and Conditions for AI Acceptability. Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society 8(2): 1771–1784.

Said N, Potinteu AE, Brich I, et al. (2023) An artificial intelligence perspective: How knowledge and confidence shape risk and benefit perception. Computers in Human Behavior 149: 107855.

The paper outlines three factors that contribute to AI agent incidents: (1) system factors, (2) contextual factors, and (3) cognitive errors. System factors are things like training and feedback data, learning methods, the system prompt, and scaffolding software around the agent. Contextual factors include aspects of the task definition, tools that the agent uses, and information the agent uses or needs to perform tasks. Finally, “cognitive” errors are basically flaws in how the AI agent functions leading to failure, which result from faulty observation of the environment, understanding of inputs, decision-making, and action execution to achieve a goal.

Based on these classes of factors the authors go on to outline a range of information that would be helpful to disclose as part of an AI agent incident. They organize this information into three categories: (1) activity logs, (2) system documentation and access, and (3) tool-related information. Activity logs would include a record of all inputs and outputs to the agent including system and user prompts, external information included in inputs, model reasoning traces, model outputs and actions taken, and necessary metadata like timestamps to contextualize all of this. System documentation and access refers to information about the AI model such as any model or system cards, version information (and change logs), and other parameters (e.g. temperature, random seeds) that might inform an incident reconstruction. Tool information is there to document any tools that agents use including identifying them, their version, the actions the tool enables, and any information about how the tool might adapt to the user.

This paper goes a long way toward outlining the necessary information that should be included in an incident report. But from a policy perspective there are some open questions about incident reporting. For one, how long should a developer maintain an activity log? This might depend on the risk profile of the use case, as well as whether there are any privacy considerations and how those might be handled. Another key question is who gets access to an incident report including any activity logs as well as system and tool-related information? The severity of the incident may create different tiers of access. Administrative and judicial forums might need access to the detailed information outlined in this paper for root-cause analysis and for assessing accountability, but it’s unclear that it should be made fully public due to privacy or trade secrecy issues. Still, secure infrastructure and access control will be needed and policy should consider how to create a shared and standardized infrastructure that AI developers can report into.

There are a few issues that the authors don’t address but which I also think will be important to policy. Related to the access control dimension, a common critique of providing transparency information is that it can enable gaming and manipulation (Diakopoulos, 2020). The many information factors that the authors outline need to be stress tested against how an adversary might be able to manipulate the agent if they were made public. This can also inform which pieces of information need to be withheld for specific closed-door forums, like administrative agencies or judicial cleanrooms. Another open question relates to AI agents using tools that use other tools. If tool use is implicated in an incident, then presumably we would want to recursively evaluate all the tools it may have in turn relied on. This then creates additional monitoring and activity logging demands on tools that are made available to agents. Finally, from a sociotechnical standpoint I think there could be aspects of AI agent transparency that disclose more about the human context around an incident, such as the roles and activities of supervisors, users, or other humans in the loop that may have had access or authority over intermediate results for the agent.

References

Diakopoulos N (2020) Transparency. Oxford Handbook of Ethics and AI. Eds. Markus Dubber, Frank Pasquale, Sunit Das.

Ezell C, Roberts-Gaal X and Chan A (2025) Incident Analysis for AI Agents. Proc. AI, Ethics, and Society (AIES) DOI: 10.48550/arxiv.2508.14231.

Different forums operate in different ways, have different capacities for obtaining information or explanation, and may have different standards of expected behavior or ways to sanction the actor. There are also differences in how their authority is constituted, with legal or administrative authority formally flowing from the state, while professional, social, and media forums gain their authority through other informal social processes. These distinctions correspond to vertical accountability, where a forum formally holds power over the actor often due to a hierarchical relationship between them, and horizontal accountability which is essentially voluntary and where there is no formal obligation to provide an account. Forums can also be public as is the case for political, legal, professional, social, and media forums, while others like administrative forums may be partially public or non-public.

Because of their different capacities to know, act, and sanction, forums often work in concert to hold an actor accountable. In a networked view of accountability the interplay between forums is a necessary feature of how accountability is ultimately rendered (Wieringa, 2020). For instance a forum with informal power and a horizontal relationship to the actor in question (e.g. media) may contribute knowledge that is publicized and which informs a forum with formal power and a vertical relationship (e.g. a relevant governing agency) that can further pursue accountability, if needed in a non-public space that accommodates issues such as trade secrecy or privacy. Different forums respond and react to one another.

Wieringa (2023) provides a detailed description of how networked accountability works, illustrating it with the case of the Dutch welfare fraud system, SyRI (System Risk Indication). Briefly, SyRI was a system implemented by the Dutch government and used by municipalities from 2015-2019 to try to detect potential fraud based on welfare beneficiary data. In 2020 a Dutch court ruled that the law authorizing the creation of SyRI was unlawful because it conflicted with the right to privacy ensured by the European Convention on Human Rights (van Bekkum and Borgesius, 2021). While there had been some administrative forums early in the development leading up to the law which tried to pump the brakes, those forums were ultimately not successful in shaping what became the law before parliament passed it.

How was accountability achieved here? The following figure illustrates many of the various relationships described by Wieringa in the case.

It was ultimately the legal forum that provided the formal accountability and authority to overrule the law authorizing the creation of SyRI. In essence the case was about holding accountable the legislators who delegated authority to create the AI system to risk rate people using private personal information. There were clear limits here though as the legal forum was unable to compel disclosure of detailed information about how the SyRI algorithm actually works, with the government arguing that disclosure of that information could enable fraudsters to game or evade the system (van Bekkum and Borgesius, 2021). As Wieringa (2023) writes, the court indicated that the State “needed to explain how the algorithmic system was designed, tested, applied, and how it operates” but failed to do so. As the court opinion wrote, “[w]ithout insight into the risk indicators and the risk model, or at least without further legal safeguards to compensate for this lack of insight, the SyRI legislations provides insufficient points of reference for the conclusion that by using SyRI the interference with the right to respect for private life is always proportional and therefore necessary…” (Meuwese, 2020). This highlights that even a formal forum such as a courtroom may not be able to bridge knowledge gaps about an AI system, and that insufficient transparency about such systems is a core impediment to accountability.

While the legal forum was able to provide the formal accountability to stop the use of SyRI, both social and media forums also played critical roles in achieving that outcome, and the political forum was further activated in the process as well. Indeed the impetus for the court case originally came from a collection of civil society actors, “The Privacy Coalition”, which in 2016 filed a public records request to find out more about the system (Wieringa, 2023). The critical issue in the public records response was that “crucial information, such as audit reports and PIAs [Privacy Impact Assessments], needed to evaluate the proportionality of the system was withheld”. There simply wasn’t enough information to assess whether the privacy violations at stake might be warranted. In short, the legitimacy of the system couldn’t be established on the basis of the information provided: the state hadn’t provided a sufficient account to the social actor. Unsatisfied with the level of detail provided, The Privacy Coalition then sued the state in 2018, moving into the legal forum.

The lawsuit also stimulated some activity in the political forum, with two ministers of parliament (MPs) filing to make the SyRI system transparent, which was denied by the state. Around this time The Privacy Coalition activated the media forum through a campaign to educate the public about SyRI and shape public attention, opinion, and awareness of the system and the issues it exposed. This had the apparent effect of also stimulating more social actors in the form of citizen demonstrations, which were then covered and amplified by the media further. The media forum also participated by scrutinizing SyRI and developing arguments against it through published editorials and commentaries, and by asking members of parliament or of municipal councils to account for the system.

Accountability is not a clean process. It involves lots of relationships, connections, and back and forth as different forums gain information and trigger or reinforce each other. Forums with informal, horizontal accountability relationships are needed to mobilize information, however at the end of the day there needs to be formal accountability from a forum with the power to change the situation and sanction actors, in this case by overturning a law. That means we need laws that define what AI behavior is permissible (or as in this case, what values like privacy need to be preserved in AI system behavior), and that other forums need to have capabilities to gain knowledge of AI behavior such that they can potentially activate formal accountability in a legal (judicial) forum. To the extent that the state would want to defend or reimplement a system akin to SyRI that system would need to offer more algorithmic transparency to clearly demonstrate how the government interest in efficiency of fraud detection is balanced against relevant fundamental rights.

References

Bekkum M van and Borgesius FZ (2021) Digital welfare fraud detection and the Dutch SyRI judgment. European Journal of Social Security 23(4): 323–340.

Bovens M (2007) Analysing and Assessing Accountability: A Conceptual Framework. European Law Journal 13(4): 447–468.

Meuwese A (2020) Regulating algorithmic decision-making one case at the time: A note on the Dutch “SyRI” judgment. European Review of Digital Administration & Law 1(1).

Wieringa M (2020) What to account for when accounting for algorithms: a systematic literature review on algorithmic accountability. FAT* ’20: Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency: 1–18.

Wieringa M (2023) “Hey SyRI, tell me about algorithmic accountability”: Lessons from a landmark case. Data & Policy 5.

The scope of the law is quite narrow, however, as it only applies to “catastrophic risk” and “critical safety incidents” related to “frontier foundation models”. Unlike the wider scope of something like the EU’s AI Act, SB53 is really targeted. A “frontier” model is defined as one that is trained with more than a threshold number of numerical operations. What makes something “catastrophic” according to the law is that more than 50 people are seriously harmed, or damages amount to at least $1B from a single incident. The risks in focus here are hypothetical including AIs assisting with making or releasing chemical, biological, radiological, or nuclear weapons; unsupervised AI’s that engage in conduct that you might recognize as murder, assault, extortion, or theft; or evading control of the developer or user.

Because perhaps none of these risks have ever actually materialized, it’s appropriate to see this law as an implementation of the precautionary principle, the idea that action should be taken to prevent potential harm, even when scientific proof of the risk is incomplete or uncertain. Basically, better safe than sorry. The law is a nice example of creating prospective accountability — assigning responsibilities for preventing outcomes that are in this case still quite uncertain but which we want to minimize the chance of coming about. In this case this puts additional onus on the processes implemented that should prevent such risks from materializing, and for monitoring the system for indicators of such risks.

Transparency can be defined as “the availability of information about an actor allowing other actors to monitor the workings or performance of this actor” [1] and is recognized as a enabler for accountability [2]. We can’t hold accountable what we don’t know about. The law addresses the knowledge dimension of the AI Accountability problem using three mechanisms to increase the supply of information about catastrophic risks from frontier foundation models: (1) transparency reports with a wide range of information on risk prevention processess and assessments, (2) incident reporting of critical safety incidents or risks found from internal uses, and (3) reinforcing whistleblower protections so that insiders with direct experience can raise an alarm.

Reflecting a prospective accountability perspective the law requires frontier model developers to publish on their website a “frontier AI framework” that includes a lot of details meant to create processess that prevent catastrophic risks. The framework needs to include information about standards incorporated, thresholds for assessing catastrophic risk, mitigations taken for those risks, reviews of the adequecy of those mitigations, use of 3rd party assessments, updating the framework, security of model weights, the identification and response to critical safety incidents, internal governance practices to ensure these processes are implemented, and assessment of catastrophic risks from internal use. All of this is meant to establish a sound process for preventing these risks, and creates the conditions for accountability if the developer doesn’t have an adequate process.

Beyond the many bits of information that need to be disclosed in the frontier AI framework, developers also have to post a transparency report that includes a range of information about the model including, importantly, the intended uses and restrictions or conditions on uses of the model. These bits of information are important for accountability because they help define the appropriate behavior for users of the models. This transparency report also has to include additional information about the implementation of the frontier AI framework including specific assessments of catastrophic risk, the extent to which 3rd party evaluators (e.g. red teamers) were involved, and any other steps taken to implement the framework. In other words, the developer has to say how they’re fulfilling the process they outline in their framework.

Developers also have to disclose the results of assessments of catastrophic risks resulting from internal use of its models as well as any critical safety incidents to an administrative office of the state. So not only is the developer accountable to the public by way of posting a lot of information about its framwork and assessments to its website, but it is also accountable to an administrative forum where they report additional assessments and incidents. Interestingly, the incident reporting mechanism will also be open to the public, which is important insofar as one of the risks of concern is loss of control of the model by users.

The last bit here is that the law strengthens whistleblower protections. It basically clarifies that employees at frontier AI model developers who are responsible for “assessing, managing, or addressing risk of critical safety incidents” should be allowed to and not retaliated against if they disclose information to certain actors about whether activities of the frontier developer might pose danger resulting from a catastrophic risk.

The bill attaches clear consequences for failing to meet the obligations it lays out. The Attorney General of California can impose a civil penalty of up to $1,000,000 per violation for failing to publish required documents, making false statements, failing to report incidents, or not complying with its own frontier AI framework. This provides the sanctions that make the accountability relationship meaningful, though this only applies to large developers with more than $500M in annual gross revenue.

SB53 is generally a solid example of how you might go about legislating prospective AI accountability to prevent risks that are uncertain. It has provisions for updating over time that will be important for keeping it relevant as technology advances. Perhaps one of the biggest weaknesses I see is that the threshold of training compute used to define a “frontier model” (10^26 numerical calculations) is not required to be disclosed. At the end of the day it’s up to the model developer to raise their hand and say that this law applies to them and to identify which models it applies to. And by one estimate GPT-5 wouldn’t even fall in the remit of the law. We shall see how and whether the frontier model developers engage.

References

[1] Albert Meijer, “Transparency,” in The Oxford Handbook of Public Accountability, ed. Mark Bovens, Robert E. Goodin, and Thomas Schillemans (Oxford: Oxford University Press, 2014)

[2] Nicholas Diakopoulos, “Transparency,” in The Oxford Handbook of Ethics and AI. Eds. Markus Dubber, Frank Pasquale, Sunit Das. (Oxford: Oxford University Press, 2020)

A new article in the Computer Law & Security Review [1], argues that the robots.txt standard can operate as more than a polite suggestion. The authors propose that common law principles, specifically in contract and tort law, offer a viable path to hold AI developers accountable for how their bots access content on websites.

In case you’re not familiar, the robots.txt file is a public file that a website owner places on their server to set bounds on web crawlers and scraper bots. It specifies what parts of a website are off limits to different bots, helping to manage server load and control access to private or sensitive parts of the site. Major search engines generally respect it and an increasing number of sites online use it to control access by different AI bots [2], but its effectiveness relies on good faith.

The first argument in the article is that robots.txt actually functions as a contract. A webmaster makes an “offer” for the contract by having the robots.txt file on their site. In essence it conveys, "You may access my site under these specific conditions." An AI operator accepts this offer not with words, but with action. When it sends its bot to access the website's content, that action signifies acceptance of the terms laid out in the robots.txt file. The bot’s continued operation on the site demonstrates a deliberate engagement with the website's conditions. While this contract can be implied, it can be further strengthened by referring to robots.txt in the site's Terms of Use. This argument sets up contract law as a path for accountability of AI bot behavior in accessing websites.

In cases where the website blocks all bot access there can’t technically be a contract because no “offer” was made. In these cases the authors argue that the tort of negligence could be used to create legal accountability of AI bot behavior. The authors propose that AI operators owe a duty of care to website owners. Ignoring a robots.txt file is a breach of that duty because respecting the file is a well-established community norm. And when this breach causes harm—such as reputational damage from an AI model misrepresenting a site's content or consequential economic loss—the AI developer could be found liable for negligence.

For policymakers, this research offers a clear message: robots.txt can be treated as more than an informal guideline for AI behavior. But it still needs to be tested in court. Clarifying its legal standing could be the next step towards accountability in a legal forum. More generally, it’s worth considering whether contracts or civil claims of negligence should be a preferred route for governing and holding accountable AI system behavior.

References

[1] Chang, C.-Y. & He, X. The liabilities of robots.txt. Computer Law Security Review. 58, 106176 (2025). https://arxiv.org/abs/2503.06035

[2] Longpre, S. et al. Consent in Crisis: The Rapid Decline of the AI Data Commons. NeurIPS (2024) doi:10.48550/arxiv.2407.14933.

Disclosure: Some text in this post was adapted based on suggestions from AI.

A 10-K filing is documentation that public companies need to submit annually to the SEC. It provides a comprehensive overview of the business including operations, financial performance, and any significant risks. In recent years the SEC has become concerned with “AI Washing” around the risks of AI, essentially that businesses might be making false claims by over-hyping the technology or underindexing the risks. This interest has even continued under the new administration. Filings are legally binding, and insufficient disclosures can lead to litigation or other enforcement actions.

These disclosures can act as a set of expectations around corporate perceptions of AI risks. If the public knows the company knows there is a risk then we might expect them to do something to try to mitigate it. It also provides a little ray of light that might help accountability forums, such as the media, ask the company about what it’s doing about the risk.

So, what exactly are companies disclosing about AI risks in these filings? A recent paper on arXiv presented an analysis of more than 30,000 10-K filings from more than 7,000 companies made between 2020 and 2024 [2]. Analysis shows that just about half the companies by 2024 mentioned AI somewhere in their disclosure, which was up from only about 1 in 8 in 2020.

The researchers qualitatively analyzed a sample of 50 companies, including 10 of the top tech companies. In that sample they found a wide range of societal risks from AI being cited, including discrimination, privacy, misinformation, malicious use, interactional harms, and so on. The risks were also framed in particular ways to dodge responsibility: “The top-tech firms often seem to externalise societal AI risks, attributing them to third-party misuse (e.g., faulty datasets or misuse of their models) while rarely acknowledging their own role in developing and deploying systems that may contribute to these risks…”

Oftentimes companies rely on vague or broad boilerplate language when they talk about risks, though there are at times more specific statements. In the paper the researchers quote the disclosure from Cognizant Technology Solutions: “The uncertainty around the safety and security of new and emerging AI applications requires significant investment to test for security, accuracy, bias, and other variables - efforts that can be complex, costly, and potentially impact our profit margins.” That’s the kind of statement that might be useful for accountability purposes.

Perhaps just as interesting are the risks the researchers didn’t observe in the sub-sample, which included environmental harms of AI, socioeconomic displacements, dangerous AI capabilities, multi-agent risks, and information ecosystem pollution. These are the risks that it seems companies haven’t yet recognized are anything they need to worry about. That may also limit accountability proceedings if companies don’t think these are issues they need to address.

There are clear limitations for informing AI accountability from 10-K filings both due to vague language and responsibility shirking. At the same time, this study does show that there can sometimes be bits of useful transparency included in these disclosures. Still, a more effective policy might more clearly indicate the types and specificity of AI risk information that are expected in these kinds of filings.

References

[1] Albert Meijer, “Transparency,” in The Oxford Handbook of Public Accountability, ed. Mark Bovens, Robert E. Goodin, and Thomas Schillemans (Oxford: Oxford University Press, 2014)

[2] Marin, L. G. U.-B., Rijsbosch, B., Spanakis, G. & Kollnig, K. Are Companies Taking AI Risks Seriously? A Systematic Analysis of Companies’ AI Risk Disclosures in SEC 10-K forms. arXiv (2025). https://arxiv.org/abs/2508.19313

Jacobs and Schillemans present a typology for how the media contribute to accountability of public institutions, outlining four distinct functions: spark, forum, amplifier, and trigger (Jacobs and Schillemans, 2019). As a spark, the ordinary activity of news reporting (“just asking questions”) may cause organizations to reconsider their behavior or role in a process. As a forum, the media act as a space where investigations uncover unwanted behaviors leading to critical questions that are posed to the actor for explanation. The media can also amplify the impact of other accountability forums, for example, by bringing more attention to congressional hearings. The last role, trigger, is where the media contributes to enabling other accountability forums by producing relevant information that spurs formal accountability in other forums.

Unlike legal or administrative forums the media is an informal forum and has no real authority to enforce infractions from the actors they address. Media forums wield power by drawing public attention to issues, with the consequences being largely reputational in nature. An actor who fails to provide a satisfactory account of an outcome may appear negligent in the public eye or draw the disapproval of the public for its conduct, negatively impacting its reputation.

While its teeth may not be as sharp as some other forums’ the media still has important contributions to make towards closing information and knowledge gaps around AI systems. Using techniques such as interviews with various stakeholders, examination of leaked documents, public information requests (Fink, 2017), external data-driven audits of system behavior (Diakopoulos, 2015), or large-scale investigation of AI systems (Veerbeek, 2025), media can inject valuable observations about the behavior of AI systems that trigger a call for accountability. Media can also surface information that informs and triggers other forums that do have teeth. For instance, Reuters’ reporting on an internal Meta document detailing chatbot policies led to Senate committee investigations. Other journalistic investigations, such as ProPublica’s look at algorithmic rent-setting in Texas, have eventually led to legal settlements.

Media also play a critical role in establishing or maintaining norms around acceptable behaviors for AI systems in society as well as who may be answerable for explaining violations of behavior. This includes propagating both descriptive norms (i.e. what actors do) and injunctive norms (i.e. what actors ought to do) (Lapinski and Rimal, 2005). Journalists apply a range of values around what kinds of outcomes or behaviors of actors may be normatively detrimental and therefore warrant scrutiny. In their daily decisions around what is newsworthy they have to assess what impacts in society are worthy of broader attention. This is the agenda-setting power of the media. By selecting and framing impacts of AI to report on, media can help establish beliefs or reinforce attitudes, which can eventually develop into social norms or expectations for the behavior of AI systems (Shehata et al, 2021). And of course the media is not homogenous. News outlets on the left vs. the right of the political spectrum prioritize different risks and impacts of AI in society in their coverage (Allaham et al, 2025).

In the course of their reporting journalists may seek accounts to help explain some observed behavior — why did the AI system produce some bad outcome? This activity helps to establish accountability relationships between actors in the system and the media as a forum. To do this journalists parse the complex sociotechnical system and consider which actors might take responsibility. By asking certain actors for explanations (e.g. a tech developer or data annotation provider), journalists audition expectations that the actor may need to answer for some outcome or (in)action. Some actors may not respond to requests for explanations, though by including these gaps in their article (“i.e. XYZ did not respond to requests for comment”), journalists subtly signal an injunctive norm — perhaps the actor should have provided an account. Journalists can also query other stakeholders in the system such as experts who study the system to ask them who they think ought to be responsible for some outcome, thus further contributing to the development of injunctive norms.

Policy Implications

The media’s power to shape the public and political agenda around AI, to investigate and expose problems, and to contribute to the development of social norms makes it a critical forum for enabling AI accountability. Policymakers should consider how to support the media’s role to foster a more accountable AI ecosystem.

For one, policies that support the media’s capacity for producing information about AI system behavior can be augmented. This could include everything from strengthening public records requests laws and whistleblower protections to increased data access provisions for auditing. Investing in more journalists working on the AI accountability beat would also serve to increase the stock of information, which is why it’s encouraging to see programs from the Pulitzer Center and the Tarbell Center focused on exactly that.

But also, policymakers need to be cognizant of how different media and perspectives in society are representing the norms and standards of behavior for AI systems. The agenda setting power of media (including new AI-driven media) influences what the public and, consequently, policymakers consider important. Policy should invest resources in large scale tracking surveys of public attitudes towards a range of AI behaviors. Moreover, a media monitor should be set up to track discourse and assess valuations of AI behavior in news, editorials, and other social media. Survey and tracking results can then inform standards for AI system behavior.

References

Allaham, M., Kieslich, K., Diakopoulos, N. Informing AI Risk Assessment with News Media: Analyzing National and Political Variation in the Coverage of AI Risks. Proceedings of the Conference on AI, Ethics, and Society (AIES). 2025. https://arxiv.org/abs/2507.23718

Diakopoulos, N. Algorithmic Accountability: Journalistic investigation of computational power structures. Digital Journalism 3, 398–415 (2015). https://doi.org/10.1080/21670811.2014.976411

Fink, K. Opening the government’s black boxes: freedom of information and algorithmic accountability. 17, 1–19 (2017). https://doi.org/10.1080/1369118X.2017.1330418

Jacobs, S. & Schillemans, T. Media and public accountability: typology and research agenda. In Media and Governance, Eds. T. Schillmans and J. Pierre. (Polity Press, 2019).

Lapinski, M. K. & Rimal, R. N. An Explication of Social Norms. Communication Theory 15, 127–147 (2005). https://doi.org/10.1111/j.1468-2885.2005.tb00329.x

Shehata, A. et al. Conceptualizing long-term media effects on societal beliefs. Annals of the International Communication Association 45, 1–19 (2021). https://doi.org/10.1080/23808985.2021.1921610

Veerbeek, J. Fighting Fire with Fire: Journalistic Investigations of Artificial Intelligence Using Artificial Intelligence Techniques. Journalism Practice, 1–19 (2025). https://doi.org/10.1080/17512786.2025.2479499

Wieringa, M. What to account for when accounting for algorithms: a systematic literature review on algorithmic accountability. FAT* ’20: Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency 1–18 (2020) doi:10.1145/3351095.3372833.

The underlying problem addressed by the paper is that LLMs can “memorize” content that they’ve been trained on, reproducing portions of their training data verbatim. LLM developers are currently facing dozens of legal cases alleging copyright violations. The paper argues that one of the challenges facing courts is that assessing output copyright violations hinges on showing substantial similarity between the original and output. But substantial similarity is a subjective legal concept that resists algorithmic implementation, meaning that we can’t necessarily expect LLMs to be able to reliably monitor and detect whether their output meets any kind of substantial similarity legal standard. Moreover, because users might prompt a model in adversarial ways to nudge a model towards outputting a response that is a copyright violation, this muddies the water around responsibility for the violation. How much responsibility should the user have?

This paper proposes an alternative focus: instead of debating whether an output looks “too similar,” legal forums might scrutinize whether training decisions substantially increased (or decreased) the risk of memorization. The paper refers to this as a “fair learning doctrine” and the authors argue that “By setting an appropriate standard, the doctrine can incentivize design choices that align with ethical and legal norms.” In essence, this reframing would allow developers to be held accountable if they didn’t implement the standard.

The paper works through a couple of analyses using Pythia, an open-source LLM trained on The Pile [2] to offer a proof-of-concept of such training standards. In one experiment the authors show that upweighting the number of times a document appears in a training dataset doesn’t substantially affect the memorization of that document. This analysis demonstrates a method that developers might use to analyze whether their model is sensitive to this kind of upweighting. In another analysis, the authors simulate what would happen if an entire dataset (like FreeLaw or PubMed Central) were excluded from training. Here they find that overlaps in data density can affect memorization risks—suggesting the relevance of dataset curation choices.

In general, these analyses are indicative, but there needs to be additional research to really flesh out what a development standard for minimizing memorization in LLMs should look like. After sufficient research, a technical standards body such as ISO or IEEE might then formalize it and socialize it. At that stage it could be used as a benchmark for any model developer. The main contribution of the paper is that it starts building a bridge between law and model training, suggesting legally informed development standards that might one day be operationalized and used for the purposes of accountability.

References

[1] Wei, J. T.-Z., Wang, M., Godbole, A., Choi, J. & Jia, R. Interrogating LLM design under copyright law. Proc. 2025 ACM Conf. Fairness, Accountability, Transparency. 3030–3045 (2025) doi:10.1145/3715275.3732193.

[2] Gao, L. et al. The Pile: An 800GB Dataset of Diverse Text for Language Modeling. arXiv (2020) doi:10.48550/arxiv.2101.00027.