Exploring the Design Space of Mathematical Agents

Chao Xu

The task

A mathematical statement passes through a mathematical agent to a correct proof. The proof is the desired output.

Reasoning model

a single call context reasoning … pages of it reply internal reasoning a summary may be shown you see this

Each call uses only the context supplied to it.

token — a chunk of text, such as a word or part of a word

context — the tokens supplied as input to the call

effort — a setting that guides how much reasoning to do before the reply

What is a Harness

Context management
A memoryless model receives context kept by the harness and returns new output. Two objects linked by arrows.
Environment interaction
The model acts on the environment through tools and receives results. Two objects linked by arrows.
AGENT = MODEL + HARNESS

More compute, more solved problems

OpenAI's chart data: pass rate rises from 8.4% to 17.0% for GPT-6 Astra and from 26.2% to 47.4% for its internal model as test-time compute increases. Logarithmic compute axis and standard-error bars.
Likely: OpenAI has a mathreasoning harness

Why one-shot is not enough

Two issues with a one-shot model call and how a mathematical harness handles them: in one call, context, reasoning, and reply share a finite context window with no cross-call memory and suffer context rot as the window fills, so the harness arranges exploration across calls with persistent note state and fresh windows. Second, self-verification inside the same call is weak, so the harness uses a separate model call in a fresh window to verify.

Goal: Minimize average cost per correct solve

A∈HA\in\mathcal H: harness design. x∼Dx\sim D: problem.

pp: probability of a correct proof.

cc: expected cost of a run.

min⁡A∈H  Ex∼D[cp].\min_{A\in\mathcal H}\;\mathbb E_{x\sim D}\left[\frac{c}{p}\right].

Secondary goal: save the user's time.

Harnesses change cost on the same model

FrontierHarness Eval scatter plot of 12 harness configurations running the same model (Kimi K3) on identical software engineering tasks: pass rate spans 50.0% to 66.7%, while median cost per task spans from $1.05 (Exo Harness), $2.43 (Pi), and $3.47 (Codex) along the Pareto frontier to $18.34 (Claude Code).
Holding Kimi K3 fixed, harness choice changes median cost per task by 17×.

Cost and time on a single example

1 h 2 h 4 h 8 h 16 h $10 $100 $1,000 time cost Codex-goal$19, 1.4 h ProofCouncil$29, 2.1 h Rethlas$61, 1.8 h QED$71, 7.0 h Danus*$1,979, 12.1 h Xean$9, 1.7 h
Baselines: 25–26 August 2026. Xean: 18 September 2026. *Danus at 12 h.

Why study mathematical harnesses?

A mathematical harness operates in a self-contained environment with statement and notes only, no external world state or flaky tools, and multi-hour search with clear verification. Those context-management, dependency-tracking, and cost-aware verification designs transfer to general long-horizon harnesses.
Mathematics isolates long-horizon search and verification to guide general harness design.

Tokens in a model call

GPT-6 Astra, per 1M tokens

Input Output cached input fresh input reasoning visible output $1 $10 $50

Early edits break prefix reuse

The same 80,000-token problem-and-notes prefix and 20,000-token route as the previous slide. The first call has zero cached input, route B reuses 80,000 tokens, and editing the beginning makes all 100,000 displayed tokens fresh.

Replaying history accumulates input

Schematic model: ℓ new tokens per turn, all prior turns retained
turn 1 turn 2 turn 3 turn 4 turn 5 message reply re-sent re-sent re-sent re-sent N turns → ℓN(N−1)/2 old tokens re-sent
Caching and retention determine the monetary cost.

When the chat outgrows the window

conversation history m₁ r₁ m₂ r₂ m₃ r₃ m₄ r₄ m₅ one possible input after trimming m₁ r₁ m₂ r₂ m₃ r₃ m₄ r₄ m₅ room to reply context window omitted from this call Earlier definitions may be missing from the next model call. The application can trim or summarize to make space.

Design philosophy

1. Use the most reasoning.
2. Trust the model.
3. Keep the harness simple, and let it eat the environment.

Inference spending

reasoning visible output uncached input cached input Codex-goal 24% 10% 20% 45% ProofCouncil 31% 12% 16% 41% Rethlas 23% 34% 35% QED 19% 17% 25% 38% Danus* 14% 34% 45% Xean 68% 21%
Reasoning is 14 to 31% of cost in five systems, 68% in Xean.
Baselines: 25–26 August 2026. Xean: 18 September 2026. *Danus at 12 h.

Why trust the model

judgmentyour code (cost: thousands) vs a reasoning model (cost: billions) informationthe reasoning is hidden
the model sees reasoning reply the harness sees hidden reply

They build for the next model

Boris Cherny
“If you wait for the next model, and if you project out maybe three months, six months, most ideas will probably be good.”
Noam Brown
“The approach that we wanted to take was to just go toward the extreme end of baking in as little structure as we could and give the agents very primitive tools to use, and they figure out for themselves how to use them effectively.”
today harness model environment tomorrow harness model environment
The model eats the harness, the harness eats the environment.

Existing mathematical harnesses

ProofCouncil
Original ProofCouncil architecture figure showing author, critic, model council, and computation support.
Rethlas
Original Rethlas architecture figure showing theorem retrieval, decomposition, recursive subagents, and verification.
Pharos
Original Pharos architecture diagram showing main agent, proof workers, helper agents, verifier, and shared mathematical records.

Solve a mathematical conjecture with AI

Verification
A single checkmark representing verification.
Exploration
An unlabeled branching tree representing exploration.

Verification is an engineering problem

Lean 100% correctness High formalization cost Natural-language verifier Almost always right Guides proof search Trains models
We verify in natural language with separate model calls.

Xean

Explorer and Verifier on the note DAG

Explorer Reads note index Chooses full notes Appends lemmas and proofs Records failed routes append read Note DAG select verdicts Verifier Selects unverified notes Checks sources and steps Checks task completion Reconstructs dependencies
The shared note DAG is the only coordination state between exploration and verification.

A DAG of notes with dependencies and checks

A directed acyclic graph of circular note nodes connected by dependency arrows: green circles with check marks for verified notes, red circles with crosses for rejected notes, and empty circles for unverified or open notes.

Available knowledge

Three information settings: a model with the task alone, a model with a fixed library, and a model with live search over external sources.

What Xean verifies

Four illustrated cards showing Xean's verification checks: 1. Sources (Does the cited theorem apply?), 2. Correctness (Does the argument hold?), 3. Task requirements (Is the exact task solved?), and 4. Reconstruction (Can a fresh call rebuild the proof with the original proof hidden?).
Separate fresh model calls check sources, steps, task completion, and blind reconstruction.

Built on Pi

Two-layer architecture diagram: Xean implements the mathematical harness (Explorer and Verifier agents, note DAG, and context and acceptance policy) on top of Pi, which provides durability (SQLite checkpoints, crash recovery, atomic outcomes), task scheduling and conversation trees, and unified model streaming with retries and cost accounting.

Strange failures: connections break after 15 minutes

Comparison of a standard streaming call versus durable stream recovery when reasoning exceeds 15 minutes: without recovery, a high-probability connection break after 15 minutes loses the entire call and restarts from zero, whereas Pi and Xean checkpoint completed reasoning items during streaming and replay them to finish the call.

Xean on 460 problems

190 model-audited passes, one GPT-6 Astra campaign per problem.

$9.96
mean per passing problem
$124 if every campaign is charged
65.6% reasoning spend
Median: $3.78 (quartiles $2.30, $7.95)
Verdicts: 5 failed, 1 inconclusive
0 50 100 150 150 <$10 16 $10–20 11 $20–40 11 $40–80 2 $80–160 cost
Launch: 14 September 2026. Accounting: 18 September 2026.

How I use Xean

Search whole collections of problems

Open problems Independent searches Results for review Candidate proof Counterexample Partial progress Unresolved
Each running campaign uses ≈200 MB of local memory, making large parallel batches practical.

How I use Xean

Why I rarely run an agent past 24 hours

solved 0 h 24 h 72 h+ time Next model (months later) Current model Flat solve rate, rising token cost
If 24 hours is not enough, waiting for the next stronger model is cheaper than running longer.

Open research problems for any harness

Time, cost, capability, simplification, evaluation, and meta-harnesses.

Time

Parallel swarms

“If you have four agents working on the problem, it is done twice as fast … you’re paying 2× more to get an answer twice as quickly.”
“I wouldn’t even attribute 10% of the credit to multi-agent.”
■ reasoning■ shared prefix most branches stop quickly one path leads somewhere
Swarms buy time with money; how to use parallel agents effectively remains unclear.

Cost

A cost-aware harness

An exploration DAG leading toward a Goal node. A bottleneck node that supports ten downstream exploration nodes is marked Verify (Many branches depend on it), whereas an isolated node with no dependents is marked Do not verify (Nothing depends on it).
Rational metareasoning
Choose the next computation
Bayesian optimization
Choose the next expensive evaluation

Capability

Code execution during proof exploration

Targeted computation checks small cases or counterexamples, while an unconstrained coding loop during proof exploration can cycle through writing, debugging, and testing scripts instead of proving the general claim.
Use code for small cases and counterexamples without derailing deductive search.

Simplification

Simplify the mathematics that accumulates

Accumulated mathematical arguments are reorganized into a reusable theorem with its assumptions preserved, supporting the next problem. A question mark marks simplification as an open challenge.
Mathematical library learning: no current model does this well and cheaply.

Evaluation

How do we know a harness change helps—cheaply?

A harness change is tested by comparing baseline and candidate runs on hard problems. Because each attempt costs $10 to $100+ and outcomes have high variance across runs, telling whether a change actually helps or is just noise takes thousands of dollars.
Each run is expensive, and high variance means we need many runs to tell if a change helps.

Meta-harness

Let the model write its own harness

Gödel machine Enumerate candidate programs Prove rewrite improves utility Requires 100% formal proof LLM meta-harness LLM writes harness code & prompts Judge “probably better” under c / p Specializes harness to your goal
A cost-aware harness adapts policy; a meta-harness rewrites the harness itself.

Recent improvements in Codex and ChatGPT push custom harnesses to work harder to catch up

Screenshot of ChatGPT Pro running for 214 minutes with three completed subagents—Construct algorithm, Oracle obstruction, and Tight lattice—to produce a deterministic quadratic-query algorithm.

Opening, 30 seconds. The subject is the design space of mathematical-agent harnesses. State the objective: greater mathematical capability at a given token cost, with reliability as a requirement and simplicity as a design principle. This is a technical design talk with a short survey and a sustained Xean case study. The cover deliberately has no venue or date. Event and timing live in the view metadata. This draft uses the current main Xean checkout for the core workflow and presents historical experiments as historical experiments.

One minute. Success means a correct proof of the supplied statement. This defines the outcome before introducing capability and cost. The agent can fail to find a proof. In the experiments, a separate reviewer assesses the saved argument against the task. Next, introduce a single reasoning model call and the harness that surrounds it.

One minute. A reasoning model call takes a sequence of tokens called the context, generates internal reasoning tokens before producing the visible reply, and has no memory outside the context supplied to that call. A token is a chunk of text such as a word or part of a word. An effort setting guides how much internal reasoning the model performs. Next, introduce the harness that manages context and environment interaction across calls.

The model is memoryless. Each call sees the supplied context. The harness manages that context and connects model actions to an environment.

Report the compute budget alongside the success rate. The chart uses OpenAI's September 8, 2026 curated open-mathematics problem set, with standard-error bars. Noam Brown's comparison to students taking different amounts of time: https://www.youtube.com/watch?v=Kx9qjX-SbUA . The second reveal corrects the initial math-harness framing to reasoning harness.

Two minutes. Now that we have seen test-time compute scaling, this slide explains why scaling beyond a single one-shot model call requires a mathematical harness. First, it is hard to explore further inside one call: the supplied context, internal reasoning, and visible reply must all fit inside one finite context window with no memory across calls, and as that window fills up, retrieval and reasoning quality degrade before the hard limit is reached, a phenomenon known as context rot. A harness therefore arranges exploration across calls with persistent note state and fresh context windows. Second, asking the same model call to verify its own work is weak because it shares the same context and blind spots. Using a separate model call in a fresh window to verify works much better than verifying within the generating call itself. Next, define the optimization objective for comparing mathematical harnesses.

Fix a mathematical problem and an agent A. An attempt includes that agent's entire exploration, internal verification, retries, and stopping policy. The agent can use memory and adaptive decisions within this attempt. The quantities c and p average over repeated runs of this complete attempt on the fixed problem. Correctness means the proof selected and returned by the agent satisfies the exact task. Independent review evaluates that returned proof and remains fallible. A reviewer finding a correct candidate elsewhere in the saved notebook measures a different outcome. Include the cost of failed attempts. Keep the allowed information and tools fixed. We first combine cost and capability for this problem, then average across the problems we care about. No underlying sample-space notation is needed. Both c and p depend on the chosen design A and problem x. The notation suppresses these arguments to keep the illustrative objective readable. A run is the complete problem-solving attempt. A model call is one invocation within it. H is the collection of available harness designs. Each design A includes its model choices and attempt policy. For each problem, divide the expected cost of a complete attempt, including failed work, by the probability that it returns a correct proof. Under independent, identically distributed fresh attempts and correctly recognized success, this ratio is the expected spend until success. With fallible verification, it is an evaluation index rather than a guaranteed stopping rule. Then average these per-problem ratios under D. Uniform sampling from a fixed task set gives its arithmetic mean. This is the primary objective for the talk. It differs from dividing pooled spend by pooled successes, which can conceal tasks that the agent never solves. If the true success probability on a problem is zero, assign infinite cost per solve. A finite sample with no success leaves that probability and cost unresolved rather than proving them zero and infinite. The expected objective can also be infinite for a difficult tail. These are limits of the ideal objective, and experiments report success counts, cost, and estimation uncertainty together. Minimum notation states the design objective, with an infimum if the best value is not attained. Saving the user's time is a secondary goal: reduce the attention needed to direct, supervise, and review the work. User effort and elapsed runtime are distinct measures. We assume the intended users are usually constrained by spending rather than a strict response deadline. This is a modeling assumption. We track elapsed runtime but keep it secondary to cost. Simplicity is a design preference.

One minute. Before looking at mathematical harnesses, note that harness choice already has a large effect on cost in general coding benchmarks. This chart from FrontierHarness Eval compares 12 harness configurations running the exact same model—Kimi K3—on identical software engineering tasks from identical cold-start VM snapshots (360 trials total). Pass rate stays within a narrow band from 50.0% to 66.7%, while median cost per task spans more than 17 times: from $1.05 in Exo Harness, $2.43 in Pi (60.0%), and $3.47 in Codex (66.7%) along the Pareto frontier all the way to $18.34 in Claude Code (63.3%). The UC Berkeley and Arena HarnessTax study finds a similar 1.5 to 5 times cost spread across seven models on SWE-bench Lite and Terminal-Bench 2.0. Next, see how much wider that cost range becomes on a long-horizon mathematical problem.

The five baseline points come from the August 26 workshop snapshot. The Xean point is a separate September 18 GPT-5.6 Sol max run: $9.113073 recorded priced usage over 1.655083 hours. Nine native Codex source calls have no usage or price. The chart therefore does not measure the complete price-equivalent cost of those calls. Unequal settings and stopping points remain. Data: assets/mathai2026-run-data.json and assets/xean-cyclic-couples-solmax-20260918.json .

One minute. Why focus on mathematical harnesses? Mathematics is an isolated, self-contained domain: the input is a short statement, the working state consists of definitions, lemmas, and proofs, and there is no noisy external world state or flaky third-party APIs. At the same time, proving a hard theorem requires hours of multi-call exploration, dependency tracking, and strict verification. That isolation makes mathematics a clean testbed for long-horizon agent work, and the harness mechanisms developed here—context assembly, note DAGs, and cost-aware verification—can later inform much more general long-horizon harnesses. Next, examine the token cost model across calls.

Use a continuing mathematical call as the example. The previously supplied problem and notes form an eligible 80,000-token cached prefix. The new route adds 20,000 fresh input tokens. Fresh means ordinary uncached input. The model generates 4,000 internal reasoning tokens and 1,000 visible output tokens. Reasoning is counted within the 5,000 total output tokens and charged once. It is not an exposed transcript of the model's private reasoning. The labels group cached and fresh tokens under input, and reasoning and visible tokens under output. The single-row diagram reuses the SVG from the canonical token-pricing slide. Its widths identify the categories rather than encode token counts. The displayed prices are rates per million tokens. The token counts and total below are a spoken example. The rates are USD per million tokens: cached input $1, fresh input $10, and output $50. Both reasoning and visible output use the output rate. The charges are $0.08 + $0.20 + $0.20 + $0.05 = $0.53. This call assumes an existing cache hit and ordinary billing for the fresh suffix, with no new cache writes or additional charges. Earlier spending to create the cache is outside this call. An explicit-only cache breakpoint at the end of the existing prefix is one way to obtain this billing scope. Cache storage and eligibility must still satisfy the provider's rules. These are teaching token counts, not measured Xean usage. The rates were verified against official OpenAI documentation on October 25, 2026, for the standard short-context tier. Other models, tiers, and context bands can differ. API usage prices are distinct from subscriptions. The October 21 tariff record remains historical. This example and its sources are recorded separately in assets/ustc-xean/token-types-example.json.

One minute. Repeat the first two calls from the preceding slide with exactly the same blocks, token counts, and cached regions. The alternative third call edits the beginning of the 80,000-token problem-and-notes block and uses route A. Its displayed input has zero cached tokens. Previously, returning to unchanged route A reused all 100,000 tokens. Here the early edit prevents both that full hit and the shared 80,000-token hit. The cache matches the beginning of the rendered input in order. Later text can remain identical while its preceding context changes, so that later state cannot be reused from the old entry. The illustration assumes no other matching entry for the edited version. Earlier unchanged system or tool content outside the illustrated block may still be cached. Avoid routinely rewriting a stable opening or shuffling old notes if they can be retained unchanged. Put new branch-specific work after shared material when the prompt and provider's cache rules permit it. This is exact prefix reuse, not a semantic cache that recognizes equivalent mathematics.

This is a schematic input-volume calculation, not a universal monetary-cost law. With ℓ tokens added by each completed turn and all earlier turns supplied again, repeated old input across N turns is ℓ(0+1+...+N-1). An initial fixed prefix adds a linear term. Variable turn lengths, compaction, partial retention, and caching change volume or price. Provider reasoning retention differs, so hidden reasoning is not assumed to be replayed. A tool response may enter a later model call, and multiple tool calls can share a response.

The stored conversation can exceed the model's working context. Each call must leave space for its input and generated output, including reasoning where applicable. The diagram illustrates trimming older messages to make room. Applications can also select relevant material or summarize older context. It does not assert a universal oldest-first policy, permanent storage, or a particular ChatGPT warning interface. An omitted definition can affect later reasoning unless the needed information is supplied again. Summaries can also lose hypotheses and proof details. Saved research notes preserve mathematical work beyond a context window. Source: the OpenAI conversation-state guide defines the window and token budget. The OpenAI Docs glossary defines compaction as summarizing older context so work can continue: https://learn.chatgpt.com/docs/glossary. Allow about 40 seconds.

Thirty seconds. State the core design philosophy upfront before unpacking each point: spend the budget on mathematical reasoning rather than context replay, trust the model to choose mathematical strategy, and keep the harness simple while letting it absorb environment details such as durable state, context assembly, and verification checks.

Five baseline rows use the August workshop data. Xean uses the September 18 GPT-5.6 Sol max rerun. Its roughly 68% reasoning share is calculated from priced usage, excluding nine unpriced native Codex source calls. This matches the earlier cost-and-time comparison. Reasoning is part of total output. Visible output is the remainder, so these cost categories do not double-count it. Input and visible output are not automatically wasted work. A higher reasoning share alone does not establish better capability or lower cost per correct solve. This is a separate run from the August Xean result of 75.3%. Data: assets/mathai2026-run-data.json and assets/xean-cyclic-couples-solmax-20260918.json .

Two minutes. Both quotes express the design principle behind our harness: bake in minimal structure, give the model simple primitives, and let the model choose the mathematical strategy. Hand-built tactic trees compete with model improvements, whereas a small harness that records notes and enforces acceptance rules stays understandable as models improve. This is a design philosophy rather than a claim of proved optimality. The next slide pictures how that boundary shifts over time.

One minute. As models improve, they absorb strategy and tactical planning that earlier systems hard-coded into the harness, while the harness expands outward into the environment: managing campaign state, knowledge libraries, and verification rules across calls. Next, compare existing mathematical harnesses with Xean.

Two minutes. Existing mathematical harnesses wire together elaborate multi-stage pipelines: ProofCouncil combines author-critic loops with model councils and computation helpers, Rethlas combines example generation, theorem retrieval, decomposition, and recursive proof subagents, and Pharos coordinates route planners, worker pools, literature helpers, and verifiers. Because we trust the model to choose the mathematical strategy, we deliberately avoid designing many specialized stages. Instead, we split the harness's job into two primitive responsibilities: verification and exploration.

Verification checks the claims, assumptions, and proof steps. Exploration carries useful lemmas, failed routes, and open directions into further attempts. The following slides develop these two tasks.

One minute. We treat verification as an engineering problem rather than a formal-methods problem. Lean gives 100% certainty, at the cost of formalizing every definition and lemma. For most things we care about, we do not need 100% correctness: a natural-language verifier such as GPT-6 Astra that is almost always right already suffices to guide proof search and even to train stronger models, as in DeepSeekMath-V2. We therefore verify in natural language with separate model calls and focus the rest of the talk on exploration.

Thirty seconds. Reset from the survey to the system case study. Explain how an ordinary candidate becomes a retained note, enters a verification task, and affects the next decision. The following design is grounded in the clean Xean 4.1.0 release. Joint reconstruction appears in this section. Detailed source matching remains in the library.

One minute. In the latest Xean design, we removed the separate Coordinator role altogether. Only two kinds of agents run on the shared note DAG: Explorer and Verifier. Explorer reads the note index, chooses which full notes to read, and appends new notes with summaries—lemmas, candidate proofs, or failed routes. Verifier selects unverified notes from the DAG and runs fresh model calls to check sources, step correctness, task requirements, and blind reconstruction, writing verdicts back to the DAG. Next, look closer at the note DAG itself.

Two minutes. Instead of carrying a long chat transcript, Xean stores mathematical state as a directed acyclic graph of notes. Each node is a mathematical note, edges record which supporting notes an argument depends on, and each node carries its verification status: green check marks for verified claims, red crosses for rejected claims, and empty nodes for unverified drafts or open exploration. When a note fails verification, any branch that depends on it cannot be accepted until that dependency is replaced. Although the underlying runtime handles recovery and batching details, the core abstraction for exploration is simply a DAG of notes with dependencies and verification statuses.

Two minutes. Relevant results may be absent from the model, especially recent developments. The agent can use its learned knowledge, a fixed library, or live search, depending on the resources supplied. These access policies constrain what evidence it can obtain. I deliberately designed toward a closed setting because many problems I work on need little outside information. This is a choice for that problem distribution, not a claim that all mathematical research can work without external resources. The harness chooses what information the agent can obtain. A model-only run receives the task and uses its learned knowledge. A fixed-library run can retrieve from an approved collection of papers, books, or notes. These documents are outside the model's learned parameters, even when we control every document in the collection. A live-search run can discover documents beyond that collection. Search results and source contents may change, and the harness does not control their completeness or correctness. Closed here means restricting access to prescribed material, not assuming that an absent statement is false. The useful distinction has two parts: whether knowledge comes from outside the model, and whether the accessible sources are fixed under our control. A fixed library is already external knowledge. Freeze source versions for a controlled comparison, and record retrieved passages and provenance for live work. Source access changes both capability and token cost, so comparisons should state which setting they use.

Sources checks the cited results, their hypotheses, applicability, and declared dependencies. Lookup is optional: the checker may use supplied sources or its mathematical knowledge. Correctness checks the claims and argument of each note, so a correct partial result can pass. Task requirements separately checks whether a proposed solution meets every completion criterion of the exact task. Reconstruction asks a fresh call to prove the candidate together with its generated dependencies, with their original proofs withheld. Admitted external premises remain available. Extraction and comparison are separate steps, so these four checks do not mean exactly four model calls. These are natural-language model checks. Fresh context does not guarantee statistically independent errors or formal certification. The DAG on the preceding slide explains why the checks also concern supporting claims. Next, show how Pi preserves the work through execution failures. Implementation reference: Xean release 9e0c2f4dc70cde2a4dd18c3cf790f51e379b4cac, src/policy/prompts.ts:215–240,328–413, src/verification.ts:52–103, src/policy/support.ts:35–61, and src/state.ts:90–107. This source identifies the verification design rather than the historical benchmark runtime.

One minute. Xean is built on Pi, a durable agent runtime. Keeping the general runtime in Pi lets Xean stay small and focus on mathematical policy: Explorer and Verifier agents, the note DAG, and verification checks. Underneath, Pi owns durable SQLite checkpoints, crash recovery, atomic task outcomes, conversation trees, and provider transport with retries and cost accounting. Every conversation entry, tool call, and note is committed to storage so multi-hour mathematical campaigns can be interrupted and resumed without losing committed work. Next, look at a strange failure mode that makes this durability essential in long reasoning runs.

One minute. When running deep reasoning models on hard mathematical problems, a single call can reason for fifteen minutes or longer. A strange practical issue we encountered is that once a single reasoning call runs past roughly fifteen minutes, there is a high probability that the network or provider connection breaks before the response finishes. Without stream durability, a dropped connection discards the entire fifteen-minute call and forces a restart from zero, which often drops again. On top of Pi, Xean checkpoints completed reasoning items as they stream and replays those committed reasoning blocks when reconnecting so the call can finish. Next, examine the historical 460-problem campaign results and their token costs.

Two minutes. Historical evaluation: campaigns beginning September 14, result archive snapshot September 15, accounting checked September 18, 2026. Preserve the original recorded outcomes and accounting. The launch selected 447 catalog open problems plus 13 golden benchmarks. These 190 are model-audited passes, including affirmative results, counterexamples, and conditional or hardness barriers, rather than 190 certified new proofs of open conjectures. The five failed and one inconclusive verdicts refer to audited outputs, not a partition of all 460 problems. The $9.96 mean uses passing-campaign spend only (median $3.78, quartiles $2.30 and $7.95, with 150 of the 190 passing campaigns costing under $10, 16 in $10–20, 11 in $20–40, 11 in $40–80, and 2 in $80–160). Charging all problem-solving campaigns, including unsuccessful work, gives about $124 per pass, excluding independent-review costs. Campaign spend is $23,536.49, and independent reviews add $118.80. Recorded total spend is $23,655.29, including reviews. The reasoning bar divides $15,528.64 of reasoning charges by that full known priced total, giving 65.6455%, displayed as 65.6%. There are 9,409 rows without priced usage, so missing spend remains unknown. The source reports full-codex-audit-v1, not expert certification. Next, explain how I use Xean in practice across problem collections and why I stop runs after 24 hours.

One minute. In my own day-to-day usage, I use Xean to explore whole collections of open problems at the same time. Collect precise problem statements and completion criteria, launch independent campaigns within a shared resource budget, and review their outputs. Each campaign retains its own task and mathematical record, and the separate rows represent different problems rather than subagents decomposing the same problem. Because the model is hosted remotely, a running Xean campaign used about 200 MB of local process memory at an observed live snapshot (176.39 MiB RSS on Jupiter on November 4, 2026, xean 4.5.0; synthetic probes reached 147.8 MiB for a small campaign and 339–361 MiB across five larger campaigns), making parallel batches practical on a single server. Hosted-model spending and result evaluation still determine how much search is affordable. Next, explain why I rarely run an individual campaign past 24 hours.

One minute. Second, in my own usage, I rarely run a campaign for more than 24 hours. Running an agent for days costs a lot of money, and if a problem is still unresolved after 24 hours, it is usually beyond the current model's capability. Past that point, extra runtime mostly burns tokens on a flat plateau, so it is cheaper to stop the run and wait a few months for the next stronger model. Next, turn to open research problems for agent harnesses.

Thirty seconds. Each of the six future directions ahead is an open research problem for any long-horizon agent harness, with mathematics serving as our clean testbed: trading money for time with swarms, making cost-aware decisions inside the harness, expanding capability with bounded code execution, simplifying accumulated knowledge, evaluating harness changes cheaply, and building a meta-harness that develops the harness itself.

Two minutes. First, on the time axis: running parallel agent swarms trades money for wall-clock speed rather than lowering the cost per solved problem. As Noam Brown noted, four parallel agents may finish twice as fast at twice the cost. Moreover, a single long reasoning call already tries and abandons many approaches internally; when we fan out across ten subagents today, all ten pay for the shared context prefix while nine stop after a brief attempt and only one path leads somewhere. How to use parallel agents effectively—breaking a hard problem into genuinely useful subproblems—remains unclear. Next, look at the cost axis: a cost-aware harness.

One minute. Second, on the cost axis: a cost-aware harness is the first level of cost optimization. Here the harness code itself is fixed, and based on the input and current search state, its policy decides what computation to run next by weighing expected contribution against cost. Checking a bottleneck lemma can prevent wasted work across many dependent arguments, whereas an isolated side note is a lower priority unless later work needs it. The same reasoning applies to exploring a new approach, loading a full paper, or stopping. The diagram illustrates proposed priorities, not an implemented optimal policy. Rational metareasoning is the broader framework for choosing computations and stopping decisions using their expected value and cost. Hay, Russell, Tolpin, and Shimony, Selecting Computations: Theory and Applications, UAI 2012, formalize such decisions and study practical approximations. Estimating those values cheaply enough remains part of the problem. Bayesian optimization provides a related approach for expensive evaluations: update a probabilistic model of an objective from observations, then use an acquisition rule to choose the next evaluation. Snoek, Larochelle, and Adams, Practical Bayesian Optimization of Machine Learning Algorithms, NeurIPS 2012, include evaluation duration in their selection criterion. Tuning harness configurations is a natural application. Within a mathematical run, heterogeneous actions such as checking, exploring, and retrieving require an appropriate model of their effects. This connection does not imply that the illustrated dependency heuristic or current xean implements Bayesian optimization. Next, turn to capability and code execution during proof exploration.

Two minutes. Third, on the capability axis: giving the prover Python, computer algebra, or an optimization solver looks natural for checking small cases and searching for counterexamples. In our early exploratory runs, unconstrained code execution inside the proof explorer often diverted token budget into writing and debugging scripts or testing finite inputs while leaving the general theorem unproved. It also adds sandbox and tool-state complexity. Current Xean therefore keeps Explorer in pure mathematical reasoning without code or shell tools. Systems such as ProofCouncil and Rethlas invoke computation or example generation as separate helper steps. Integrating bounded computational checks without derailing deductive search remains an open design question. Both panels are conceptual illustrations rather than a measured ablation. Next, look at simplifying the mathematics that accumulates across notes.

One minute. Fourth, as longer runs accumulate notes and lemmas, we want to periodically reorganize the accumulated mathematical work so exploration can continue from a clearer foundation: consolidate overlapping statements, extract reusable theorems, and find simpler arguments while preserving exact claims and necessary assumptions. This is the mathematical analogue of library learning, and currently no model can do this well and cheaply. Shorter proofs, better lemmas, and fewer unnecessary dependencies require genuine mathematical compression rather than surface text summarization. The question mark marks that open step, and H1 and H2 stand for the retained assumptions of a conceptual theorem. Next, ask how we know whether any of these harness improvements actually helps—cheaply.

One minute. Fifth, across all of these harness ideas—swarms, cost-aware decisions, code execution, and mathematical simplification—a central open problem is how to evaluate whether a harness change actually helps—cheaply. Modifying a context rule, verifier policy, or prompt is easy, while testing whether that change improves solve rate or cost per solve is expensive: each attempt on a hard problem costs ten to over a hundred dollars, and outcomes have high variance from run to run. Because of that variance, a few test runs cannot separate a genuine improvement from luck, and running enough trials to get a reliable signal costs thousands of dollars. Once we have an evaluation signal, that raises one more question: why write the harness ourselves at all?

One minute. Sixth, connect cost-aware search to self-improving harnesses. A cost-aware harness is the first level: the harness code is fixed, and the input decides what action the policy takes next. At the next level, we still minimize expected cost per solve, and even the harness code itself can be modified. That resembles Jürgen Schmidhuber's Gödel machine, with two changes that make it practical for open-ended problems: instead of enumerating candidate programs, an LLM writes targeted edits to the harness code; and instead of requiring a 100% formal proof that the rewrite improves utility—just as we replaced Lean with a natural-language verifier earlier—an LLM and empirical evaluation judge whether the new harness is probably better under uncertainty and cost c / p. In recent literature this is called a Darwin Gödel Machine, a Gödel Agent, or a Meta-Harness—recursive self-improvement at the harness level—where the model writes and specializes its own harness for your goal. Next, close by looking at how fast general products are moving.

One minute. As this screenshot shows, ChatGPT Pro and Codex already run multi-hour campaigns and spawn subagents of their own. Rapid progress in those general tools raises the baseline and pushes custom mathematical harnesses to work harder—through better cost-aware exploration, cleaner dependency management, and stronger verification—to stay ahead at a given budget. Compare any custom harness against the current general tools at the same cost.