Claude Cowork for Research Screening
You don’t need Claude Science to support literature reviews
The volume of articles that might be relevant to the AI Accountability Review is huge. I track about a dozen highly relevant journals and pre-print servers and screen upwards of 100 articles every week just from those. But there are also a number of conferences that publish on the topic, including conferences like FAccT, AIES, ICLR, NeurIPS, and CHI. When a big conference publishes its proceedings there could be hundreds or even thousands of new articles to screen all in one batch. Recent research has demonstrated that AI tools show significant promise for accelerating relevance screening for literature reviews (Wei et al, 2026). So I wanted to see whether a generalist agentic AI system like Claude Cowork could help with screening conference papers.
Cowork is a desktop tool that lets Anthropic models act agentically on your computer, working with local files and using the Chrome browser to access information on the web. The basic workflow is that I first create a new project in Cowork for the conference. I then prompt the system to scrape the papers for the conference by directing it to the proceedings or digital library page. For some mega-conferences like ICLR I point it at a subsection like “Social Aspects” as that already cuts down the volume. The agent is usually pretty good at figuring out how to download the papers into a structured file including titles, authors, abstracts, and DOIs or links. Sometimes there’s some back and forth to help it navigate specific issues. For instance, in one case the agent got blocked when accessing the ACM Digital Library because it was working too quickly and triggered an IP block. A university VPN is helpful for getting around that.
Once you’ve got a list of papers and their abstracts in a structured JSON file, the next step is to screen them according to a relevance filter. Because AIAR is a living literature review, the premise is that I have a set of articles I’ve written on sub-topics, like responsibility, transparency, or explanation, and then want to find literature to advance, update, or elaborate those articles. In my workflow, I use the text of my articles to define the relevance of new papers. Each of my articles is included in the Cowork project as a markdown file. Then I use something like the following prompt to rate each candidate paper against each of my articles:
“Assess the relevance of each of the abstracts in the iclr2026 social aspects papers json file on a three point scale: low=1, medium=2, and high=3 in terms of how relevant the information is to the ideas reflected in each of the blog posts contained in the .md files (except for About_AIAR.md). So there should be 10 relevance scores for each abstract in all. If ANY of the 10 relevance scores equals 3 (i.e. high) then include that abstract as well as other metadata about the paper (title, authors, DOI, link) into a new rated.rss output file. If a paper makes it into rated.rss, its description should also include a list indicating which of the 10 blog posts it’s highly relevant to. If rated.rss contains more than 100 items, then split it into subsets of no more than 100 items.”
Claude will typically spin up a number of sub-agents (e.g. it used 9 in the above case) and tear through the ratings process in a matter of minutes. I output to RSS feeds because it integrates easily with Inoreader for manually reviewing and deciding whether to read it and include it in the AIAR.
The potential risks from using a tool like Claude to help with this screening work is that I potentially miss papers that are relevant to AIAR (false negative), or that some papers are classified as relevant but I decide after manual review that they’re really not-so-relevant (false positive). False positive errors add a minimal amount of work to my manual screening pass, so are not a big deal. False negatives are potentially an issue though, since there could be papers being published that would be highly relevant to AIAR but that I miss. Still, this type of error is not catastrophic for a narrative review like mine (which makes no claim to have found every paper on a topic). And if a paper was really resonating on social media or via other network channels I might still become aware of it and include it in the review.
For systematic literature reviews it’s typically important to evaluate the false positive and false negative rates for screening support tools to show both what might be missed, and also indicate how much of an efficiency gain is created. Cowork can also help with this by integrating Artifacts which are live interactive apps that can pull in the data from the project and be used to assess relevance ratings against my own judgement. When I was screening abstracts from FAccT 2026, I prompted Claude to create an artifact interface that showed a sample of 10 papers marked as high relevance and 10 marked as low or medium relevance. Then I spent about 5-10 minutes manually (and blindly) rating those 20, and was able to measure the false positive and false negative rates (for the record, 1 false positive and 1 false negative). The interface looks like this:
The workflow above has been hugely valuable for refining large sets of articles into more rarified sets that save me time. Instead of wading through more than 1500 papers from the CHI conference this year, I looked at about 150. One area of unease I have, though, is that because relevance is defined in terms of what I’ve already written, the screening might miss newly emerging areas or topics that I simply haven’t gotten around to writing about. For that Claude can also be helpful though. I asked it to create thematic clusters of abstracts from the FAccT conference this year and it synthesized 8 topics that each connected at least 3 papers from the conference. None of the 8 were surprising to me, but they did draw my attention to a couple of trends at the conference that prompt me to want to write them up.
References
Wei Z, Ngongoma L, Cols J, et al. (2026) Artificial Intelligence (AI) Readiness to Support Evidence Synthesis by Workflow: Findings From a Review of Reviews. Campbell Systematic Reviews 22(2): 18911803261454704.