Agentic Programming in practice: building a harness for BioMysteryBench

In November of 2025, Anthropic Engineering published an article that had a state-change effect on how I used AI for agentic coding — and, judging by how quickly the wider industry converged on the same scaffold-first thinking, I doubt I was alone [1]. The article's central argument is that letting an agent make progress across multiple context windows depends far more on the surrounding scaffolding — a features list, a progress file, git commits, end-to-end verification — than on the raw capability of the model. In this article, I want to describe my current practice and demonstrate its use in tackling a problem that is interesting in its own right: creating a harness for an AI agent that autonomously answers questions in Anthropic's bioinformatics benchmark test for agents, BioMysteryBench [2].

Along the way, I'll explain agentic programming, using agents to answer benchmark questions, and techniques to improve the performance of both tasks. It is my hope that this will be useful to anyone interested in this area; I have also made the GitHub repository [18] available for those who code and might want to dive into the details.

Agentic programming is the practice of using a Large Language Model (LLM) to help write code. There is no standardised nomenclature for how people use AI to code, but it is common to describe the practice as having a set of ascending levels [4][5]. The published taxonomies I've seen tend to have five levels, and they all turn on a single variable — how much work the AI does before it hands back to you. Ive added a level 0 for completeness.

You will come across multiple accountings of what constitutes agentic programming on YouTube, LinkedIn, and your-favourite-ai-tech-site. The distinguishing factor between the different levels is the degree of freedom that the developer affords the AI, and the shift in attention that the developer makes — from writing and reading code toward defining and examining the behaviour of the final software product that the code generates.

In April of 2026, Anthropic published an article called Evaluating Claude's bioinformatics research capabilities with BioMysteryBench [2]. BioMysteryBench is a set of 99 questions in the domain of bioinformatics, written by domain experts, meant to serve as a test set for assessing an AI system's ability to answer bioinformatics questions.

It is not the first such benchmark — LAB-Bench [6], BixBench [7], Biomni [8], SciGym [10] and ProteinGym [9] all precede it — but it is distinctive in an important way. Rather than grading against a scientist's derived conclusion, BioMysteryBench grades against objective, ground-truth properties of the data — questions whose answers come from controlled, objective properties of the data or from orthogonally validated metadata — while still allowing full freedom in analytical strategy. Anthropic highlights three properties: the benchmark is method-agnostic (research freedom and creativity are allowed), it has objective ground-truth answers, and — because it does not require a question to be human-solvable — it permits "human-difficult" question inclusion [2].

That last property matters. Of the 99 questions, 76 are human-solvable and 23 are "human-difficult" — questions that Anthropic's "panel of experts" could not solve. Claude's own models made real progress even on the hard subset: Claude Opus 4.6 scored 77.4% on the human-solvable problems and 23.5% on the human-difficult ones, while Claude Mythos Preview reached roughly 30% on the human-difficult set [2].

BioMysteryBench presented a unique opportunity. Although the benchmark questions are freely available through Hugging Face [3] (the preview set of five is public; the full set requires access approval), there is no readily available default application that can iterate through these questions and attempt to answer them. So this was a non-trivial problem in my domain that I could use to demonstrate the power of agentic coding to create such an application. But most importantly, it represented what ought to be the heart of every agentic programming project — a set of test questions.

There are many ways to carry out agentic coding. Even if you stick with one LLM provider, each can be used in multiple ways. Anthropic itself provides three different interfaces for coding — the web interface at claude.ai, the Claude desktop application, and the Claude CLI (command-line interface, "Claude Code"). All the major players (Google, Microsoft, OpenAI) offer analogous products vying for your attention, and the internet is rife with reviews claiming one to be better than another. All of them change constantly. My advice is to pick one and stick with it while you learn, then explore others. (For a getting-started orientation, see the official docs for Claude Code [12], OpenAI Codex [13] and GitHub Copilot [14], and a recent side-by-side comparison [11] — the pattern most teams settled on in 2026 is Copilot as a broad autocomplete baseline plus Claude Code for high-leverage agentic work.)

Here's my full setup:

Getting started was as simple as opening a text file and describing the system I wanted to build. The whole project began with a single message pasted into a fresh session — a link to the BioMysteryBench article and this request:

That prompt already encodes most of what I care about: a clear deliverable, a request for references rather than guesses, an explicit safety constraint, and an instruction to write its findings down where I can review them. This first step — describing the system and getting a plan you agree with — is the single most important part of the whole process. Everything after it is iteration.

Mechanically, here is how the first session went. I created a directory on my laptop called bio-mystery-bench and, inside it, another directory called documents. I then opened Claude Desktop, clicked Code (to select a coding session as opposed to a Chat or Cowork session), and then + New Session. At the bottom of the window, just above the input dialog bar, I selected the bio-mystery-bench directory to tell Claude where I wanted to work, and selected Plan mode to tell Claude I wanted a plan based on my input before any code was written.

Claude then asked me a few questions — about the dataset to start with (the first 5 questions of BioMysteryBench or all 99; it suggested sticking with the first 5) and about the default model to answer the questions (it suggested claude-sonnet-4-6). I accepted the suggestions and it generated a plan, which it presented in a side panel. I reviewed the plan and accepted it, allowing Claude to create the first version of the code base. Finally, Claude gave me a summary of what it had made.

I then asked Claude to push these changes to a GitHub repository where I could review and accept them. This constitutes the basic cycle I would continue in the next 60 hours of session time and dozens of separate conversations over the course of two and a half weeks.

The initial creation of the first untested repo took less than two hours.

What follows is less a diary than a catalogue of behaviours — the practices that made the loop productive, the division of labour between me and Claude Code, and the gotchas that cost time. If you take one thing from this section, let it be that the interesting work was almost never writing code; it was deciding what to build next and how to keep the system honest.

A useful way to see the project is as two roles that rarely overlapped:

The shift the November article describes is exactly this: my attention moved from reading and writing code to defining and inspecting behaviour.

A number of habits accreted over the first week and then hardened into rules (many of them living in the project's CLAUDE.md, the file Claude Code reads automatically at the start of every session):

Two of these — code-learnings and session-close — are worth dwelling on, because they are the mechanism by which a stateless model accumulates something like institutional memory. The model does not remember previous sessions; the harness remembers for it.

The failures were more instructive than the successes:

Before I optimised anything, I wanted to know the solution was sound. The earliest questions I put to Claude were deliberately about properties of the whole system, not features. I summarised them to myself as SERF — Scalable, Extensible, Robust, Feasible — and probed each with a concrete question and a follow-up demand:

Those questions map onto the four SERF properties directly: reasoning logs and a passing test suite make the system robust (testable, transparent, observable); the container sandbox and the dataset format make it feasible and extensible; and the deliberate refusal to depend on my laptop's hardware keeps it scalable. Establishing all four before chasing benchmark scores is what let the later performance work in Part 2 proceed without constantly re-litigating whether the foundation was trustworthy.

Continue to Part 2 — Improving the performance of the base model