Opening, 30 seconds. The subject is the design space of mathematical-agent harnesses. State the objective: greater mathematical capability at a given token cost, with reliability as a requirement and simplicity as a design principle. This is a technical design talk with a short survey and a sustained Xean case study. The cover deliberately has no venue or date. Event and timing live in the view metadata. This draft uses the current main Xean checkout for the core workflow and presents historical experiments as historical experiments.
One minute. Success means a correct proof of the supplied statement. This defines the outcome before introducing capability and cost. The agent can fail to find a proof. In the experiments, a separate reviewer assesses the saved argument against the task. Next, introduce a single reasoning model call and the harness that surrounds it.
One minute. A reasoning model call takes a sequence of tokens called the context, generates internal reasoning tokens before producing the visible reply, and has no memory outside the context supplied to that call. A token is a chunk of text such as a word or part of a word. An effort setting guides how much internal reasoning the model performs. Next, introduce the harness that manages context and environment interaction across calls.
The model is memoryless. Each call sees the supplied context. The harness manages that context and connects model actions to an environment.
Report the compute budget alongside the success rate. The chart uses OpenAI's September 8, 2026 curated open-mathematics problem set, with standard-error bars. Noam Brown's comparison to students taking different amounts of time: https://www.youtube.com/watch?v=Kx9qjX-SbUA . The second reveal corrects the initial math-harness framing to reasoning harness.
Two minutes. Now that we have seen test-time compute scaling, this slide explains why scaling beyond a single one-shot model call requires a mathematical harness. First, it is hard to explore further inside one call: the supplied context, internal reasoning, and visible reply must all fit inside one finite context window with no memory across calls, and as that window fills up, retrieval and reasoning quality degrade before the hard limit is reached, a phenomenon known as context rot. A harness therefore arranges exploration across calls with persistent note state and fresh context windows. Second, asking the same model call to verify its own work is weak because it shares the same context and blind spots. Using a separate model call in a fresh window to verify works much better than verifying within the generating call itself. Next, define the optimization objective for comparing mathematical harnesses.
Fix a mathematical problem and an agent A. An attempt includes that agent's entire exploration, internal verification, retries, and stopping policy. The agent can use memory and adaptive decisions within this attempt. The quantities c and p average over repeated runs of this complete attempt on the fixed problem. Correctness means the proof selected and returned by the agent satisfies the exact task. Independent review evaluates that returned proof and remains fallible. A reviewer finding a correct candidate elsewhere in the saved notebook measures a different outcome. Include the cost of failed attempts. Keep the allowed information and tools fixed. We first combine cost and capability for this problem, then average across the problems we care about. No underlying sample-space notation is needed. Both c and p depend on the chosen design A and problem x. The notation suppresses these arguments to keep the illustrative objective readable. A run is the complete problem-solving attempt. A model call is one invocation within it. H is the collection of available harness designs. Each design A includes its model choices and attempt policy. For each problem, divide the expected cost of a complete attempt, including failed work, by the probability that it returns a correct proof. Under independent, identically distributed fresh attempts and correctly recognized success, this ratio is the expected spend until success. With fallible verification, it is an evaluation index rather than a guaranteed stopping rule. Then average these per-problem ratios under D. Uniform sampling from a fixed task set gives its arithmetic mean. This is the primary objective for the talk. It differs from dividing pooled spend by pooled successes, which can conceal tasks that the agent never solves. If the true success probability on a problem is zero, assign infinite cost per solve. A finite sample with no success leaves that probability and cost unresolved rather than proving them zero and infinite. The expected objective can also be infinite for a difficult tail. These are limits of the ideal objective, and experiments report success counts, cost, and estimation uncertainty together. Minimum notation states the design objective, with an infimum if the best value is not attained. Saving the user's time is a secondary goal: reduce the attention needed to direct, supervise, and review the work. User effort and elapsed runtime are distinct measures. We assume the intended users are usually constrained by spending rather than a strict response deadline. This is a modeling assumption. We track elapsed runtime but keep it secondary to cost. Simplicity is a design preference.
One minute. Before looking at mathematical harnesses, note that harness choice already has a large effect on cost in general coding benchmarks. This chart from FrontierHarness Eval compares 12 harness configurations running the exact same model—Kimi K3—on identical software engineering tasks from identical cold-start VM snapshots (360 trials total). Pass rate stays within a narrow band from 50.0% to 66.7%, while median cost per task spans more than 17 times: from $1.05 in Exo Harness, $2.43 in Pi (60.0%), and $3.47 in Codex (66.7%) along the Pareto frontier all the way to $18.34 in Claude Code (63.3%). The UC Berkeley and Arena HarnessTax study finds a similar 1.5 to 5 times cost spread across seven models on SWE-bench Lite and Terminal-Bench 2.0. Next, see how much wider that cost range becomes on a long-horizon mathematical problem.
The five baseline points come from the August 26 workshop snapshot. The Xean point is a separate September 18 GPT-5.6 Sol max run: $9.113073 recorded priced usage over 1.655083 hours. Nine native Codex source calls have no usage or price. The chart therefore does not measure the complete price-equivalent cost of those calls. Unequal settings and stopping points remain. Data: assets/mathai2026-run-data.json and assets/xean-cyclic-couples-solmax-20260918.json .
One minute. Why focus on mathematical harnesses? Mathematics is an isolated, self-contained domain: the input is a short statement, the working state consists of definitions, lemmas, and proofs, and there is no noisy external world state or flaky third-party APIs. At the same time, proving a hard theorem requires hours of multi-call exploration, dependency tracking, and strict verification. That isolation makes mathematics a clean testbed for long-horizon agent work, and the harness mechanisms developed here—context assembly, note DAGs, and cost-aware verification—can later inform much more general long-horizon harnesses. Next, examine the token cost model across calls.
Use a continuing mathematical call as the example. The previously supplied problem and notes form an eligible 80,000-token cached prefix. The new route adds 20,000 fresh input tokens. Fresh means ordinary uncached input. The model generates 4,000 internal reasoning tokens and 1,000 visible output tokens. Reasoning is counted within the 5,000 total output tokens and charged once. It is not an exposed transcript of the model's private reasoning. The labels group cached and fresh tokens under input, and reasoning and visible tokens under output. The single-row diagram reuses the SVG from the canonical token-pricing slide. Its widths identify the categories rather than encode token counts. The displayed prices are rates per million tokens. The token counts and total below are a spoken example. The rates are USD per million tokens: cached input $1, fresh input $10, and output $50. Both reasoning and visible output use the output rate. The charges are $0.08 + $0.20 + $0.20 + $0.05 = $0.53. This call assumes an existing cache hit and ordinary billing for the fresh suffix, with no new cache writes or additional charges. Earlier spending to create the cache is outside this call. An explicit-only cache breakpoint at the end of the existing prefix is one way to obtain this billing scope. Cache storage and eligibility must still satisfy the provider's rules. These are teaching token counts, not measured Xean usage. The rates were verified against official OpenAI documentation on October 25, 2026, for the standard short-context tier. Other models, tiers, and context bands can differ. API usage prices are distinct from subscriptions. The October 21 tariff record remains historical. This example and its sources are recorded separately in assets/ustc-xean/token-types-example.json.
One minute. Repeat the first two calls from the preceding slide with exactly the same blocks, token counts, and cached regions. The alternative third call edits the beginning of the 80,000-token problem-and-notes block and uses route A. Its displayed input has zero cached tokens. Previously, returning to unchanged route A reused all 100,000 tokens. Here the early edit prevents both that full hit and the shared 80,000-token hit. The cache matches the beginning of the rendered input in order. Later text can remain identical while its preceding context changes, so that later state cannot be reused from the old entry. The illustration assumes no other matching entry for the edited version. Earlier unchanged system or tool content outside the illustrated block may still be cached. Avoid routinely rewriting a stable opening or shuffling old notes if they can be retained unchanged. Put new branch-specific work after shared material when the prompt and provider's cache rules permit it. This is exact prefix reuse, not a semantic cache that recognizes equivalent mathematics.
This is a schematic input-volume calculation, not a universal monetary-cost law. With ℓ tokens added by each completed turn and all earlier turns supplied again, repeated old input across N turns is ℓ(0+1+...+N-1). An initial fixed prefix adds a linear term. Variable turn lengths, compaction, partial retention, and caching change volume or price. Provider reasoning retention differs, so hidden reasoning is not assumed to be replayed. A tool response may enter a later model call, and multiple tool calls can share a response.
The stored conversation can exceed the model's working context. Each call must leave space for its input and generated output, including reasoning where applicable. The diagram illustrates trimming older messages to make room. Applications can also select relevant material or summarize older context. It does not assert a universal oldest-first policy, permanent storage, or a particular ChatGPT warning interface. An omitted definition can affect later reasoning unless the needed information is supplied again. Summaries can also lose hypotheses and proof details. Saved research notes preserve mathematical work beyond a context window. Source: the OpenAI conversation-state guide defines the window and token budget. The OpenAI Docs glossary defines compaction as summarizing older context so work can continue: https://learn.chatgpt.com/docs/glossary. Allow about 40 seconds.
Thirty seconds. State the core design philosophy upfront before unpacking each point: spend the budget on mathematical reasoning rather than context replay, trust the model to choose mathematical strategy, and keep the harness simple while letting it absorb environment details such as durable state, context assembly, and verification checks.
Five baseline rows use the August workshop data. Xean uses the September 18 GPT-5.6 Sol max rerun. Its roughly 68% reasoning share is calculated from priced usage, excluding nine unpriced native Codex source calls. This matches the earlier cost-and-time comparison. Reasoning is part of total output. Visible output is the remainder, so these cost categories do not double-count it. Input and visible output are not automatically wasted work. A higher reasoning share alone does not establish better capability or lower cost per correct solve. This is a separate run from the August Xean result of 75.3%. Data: assets/mathai2026-run-data.json and assets/xean-cyclic-couples-solmax-20260918.json .
Two minutes. Both quotes express the design principle behind our harness: bake in minimal structure, give the model simple primitives, and let the model choose the mathematical strategy. Hand-built tactic trees compete with model improvements, whereas a small harness that records notes and enforces acceptance rules stays understandable as models improve. This is a design philosophy rather than a claim of proved optimality. The next slide pictures how that boundary shifts over time.
One minute. As models improve, they absorb strategy and tactical planning that earlier systems hard-coded into the harness, while the harness expands outward into the environment: managing campaign state, knowledge libraries, and verification rules across calls. Next, compare existing mathematical harnesses with Xean.
Two minutes. Existing mathematical harnesses wire together elaborate multi-stage pipelines: ProofCouncil combines author-critic loops with model councils and computation helpers, Rethlas combines example generation, theorem retrieval, decomposition, and recursive proof subagents, and Pharos coordinates route planners, worker pools, literature helpers, and verifiers. Because we trust the model to choose the mathematical strategy, we deliberately avoid designing many specialized stages. Instead, we split the harness's job into two primitive responsibilities: verification and exploration.
Verification checks the claims, assumptions, and proof steps. Exploration carries useful lemmas, failed routes, and open directions into further attempts. The following slides develop these two tasks.
One minute. We treat verification as an engineering problem rather than a formal-methods problem. Lean gives 100% certainty, at the cost of formalizing every definition and lemma. For most things we care about, we do not need 100% correctness: a natural-language verifier such as GPT-6 Astra that is almost always right already suffices to guide proof search and even to train stronger models, as in DeepSeekMath-V2. We therefore verify in natural language with separate model calls and focus the rest of the talk on exploration.
Thirty seconds. Reset from the survey to the system case study. Explain how an ordinary candidate becomes a retained note, enters a verification task, and affects the next decision. The following design is grounded in the clean Xean 4.1.0 release. Joint reconstruction appears in this section. Detailed source matching remains in the library.
One minute. In the latest Xean design, we removed the separate Coordinator role altogether. Only two kinds of agents run on the shared note DAG: Explorer and Verifier. Explorer reads the note index, chooses which full notes to read, and appends new notes with summaries—lemmas, candidate proofs, or failed routes. Verifier selects unverified notes from the DAG and runs fresh model calls to check sources, step correctness, task requirements, and blind reconstruction, writing verdicts back to the DAG. Next, look closer at the note DAG itself.
Two minutes. Instead of carrying a long chat transcript, Xean stores mathematical state as a directed acyclic graph of notes. Each node is a mathematical note, edges record which supporting notes an argument depends on, and each node carries its verification status: green check marks for verified claims, red crosses for rejected claims, and empty nodes for unverified drafts or open exploration. When a note fails verification, any branch that depends on it cannot be accepted until that dependency is replaced. Although the underlying runtime handles recovery and batching details, the core abstraction for exploration is simply a DAG of notes with dependencies and verification statuses.
Two minutes. Relevant results may be absent from the model, especially recent developments. The agent can use its learned knowledge, a fixed library, or live search, depending on the resources supplied. These access policies constrain what evidence it can obtain. I deliberately designed toward a closed setting because many problems I work on need little outside information. This is a choice for that problem distribution, not a claim that all mathematical research can work without external resources. The harness chooses what information the agent can obtain. A model-only run receives the task and uses its learned knowledge. A fixed-library run can retrieve from an approved collection of papers, books, or notes. These documents are outside the model's learned parameters, even when we control every document in the collection. A live-search run can discover documents beyond that collection. Search results and source contents may change, and the harness does not control their completeness or correctness. Closed here means restricting access to prescribed material, not assuming that an absent statement is false. The useful distinction has two parts: whether knowledge comes from outside the model, and whether the accessible sources are fixed under our control. A fixed library is already external knowledge. Freeze source versions for a controlled comparison, and record retrieved passages and provenance for live work. Source access changes both capability and token cost, so comparisons should state which setting they use.
Sources checks the cited results, their hypotheses, applicability, and declared dependencies. Lookup is optional: the checker may use supplied sources or its mathematical knowledge. Correctness checks the claims and argument of each note, so a correct partial result can pass. Task requirements separately checks whether a proposed solution meets every completion criterion of the exact task. Reconstruction asks a fresh call to prove the candidate together with its generated dependencies, with their original proofs withheld. Admitted external premises remain available. Extraction and comparison are separate steps, so these four checks do not mean exactly four model calls. These are natural-language model checks. Fresh context does not guarantee statistically independent errors or formal certification. The DAG on the preceding slide explains why the checks also concern supporting claims. Next, show how Pi preserves the work through execution failures. Implementation reference: Xean release 9e0c2f4dc70cde2a4dd18c3cf790f51e379b4cac, src/policy/prompts.ts:215–240,328–413, src/verification.ts:52–103, src/policy/support.ts:35–61, and src/state.ts:90–107. This source identifies the verification design rather than the historical benchmark runtime.
One minute. Xean is built on Pi, a durable agent runtime. Keeping the general runtime in Pi lets Xean stay small and focus on mathematical policy: Explorer and Verifier agents, the note DAG, and verification checks. Underneath, Pi owns durable SQLite checkpoints, crash recovery, atomic task outcomes, conversation trees, and provider transport with retries and cost accounting. Every conversation entry, tool call, and note is committed to storage so multi-hour mathematical campaigns can be interrupted and resumed without losing committed work. Next, look at a strange failure mode that makes this durability essential in long reasoning runs.
One minute. When running deep reasoning models on hard mathematical problems, a single call can reason for fifteen minutes or longer. A strange practical issue we encountered is that once a single reasoning call runs past roughly fifteen minutes, there is a high probability that the network or provider connection breaks before the response finishes. Without stream durability, a dropped connection discards the entire fifteen-minute call and forces a restart from zero, which often drops again. On top of Pi, Xean checkpoints completed reasoning items as they stream and replays those committed reasoning blocks when reconnecting so the call can finish. Next, examine the historical 460-problem campaign results and their token costs.
Two minutes. Historical evaluation: campaigns beginning September 14, result archive snapshot September 15, accounting checked September 18, 2026. Preserve the original recorded outcomes and accounting. The launch selected 447 catalog open problems plus 13 golden benchmarks. These 190 are model-audited passes, including affirmative results, counterexamples, and conditional or hardness barriers, rather than 190 certified new proofs of open conjectures. The five failed and one inconclusive verdicts refer to audited outputs, not a partition of all 460 problems. The $9.96 mean uses passing-campaign spend only (median $3.78, quartiles $2.30 and $7.95, with 150 of the 190 passing campaigns costing under $10, 16 in $10–20, 11 in $20–40, 11 in $40–80, and 2 in $80–160). Charging all problem-solving campaigns, including unsuccessful work, gives about $124 per pass, excluding independent-review costs. Campaign spend is $23,536.49, and independent reviews add $118.80. Recorded total spend is $23,655.29, including reviews. The reasoning bar divides $15,528.64 of reasoning charges by that full known priced total, giving 65.6455%, displayed as 65.6%. There are 9,409 rows without priced usage, so missing spend remains unknown. The source reports full-codex-audit-v1, not expert certification. Next, explain how I use Xean in practice across problem collections and why I stop runs after 24 hours.
One minute. In my own day-to-day usage, I use Xean to explore whole collections of open problems at the same time. Collect precise problem statements and completion criteria, launch independent campaigns within a shared resource budget, and review their outputs. Each campaign retains its own task and mathematical record, and the separate rows represent different problems rather than subagents decomposing the same problem. Because the model is hosted remotely, a running Xean campaign used about 200 MB of local process memory at an observed live snapshot (176.39 MiB RSS on Jupiter on November 4, 2026, xean 4.5.0; synthetic probes reached 147.8 MiB for a small campaign and 339–361 MiB across five larger campaigns), making parallel batches practical on a single server. Hosted-model spending and result evaluation still determine how much search is affordable. Next, explain why I rarely run an individual campaign past 24 hours.
One minute. Second, in my own usage, I rarely run a campaign for more than 24 hours. Running an agent for days costs a lot of money, and if a problem is still unresolved after 24 hours, it is usually beyond the current model's capability. Past that point, extra runtime mostly burns tokens on a flat plateau, so it is cheaper to stop the run and wait a few months for the next stronger model. Next, turn to open research problems for agent harnesses.
Thirty seconds. Each of the six future directions ahead is an open research problem for any long-horizon agent harness, with mathematics serving as our clean testbed: trading money for time with swarms, making cost-aware decisions inside the harness, expanding capability with bounded code execution, simplifying accumulated knowledge, evaluating harness changes cheaply, and building a meta-harness that develops the harness itself.
Two minutes. First, on the time axis: running parallel agent swarms trades money for wall-clock speed rather than lowering the cost per solved problem. As Noam Brown noted, four parallel agents may finish twice as fast at twice the cost. Moreover, a single long reasoning call already tries and abandons many approaches internally; when we fan out across ten subagents today, all ten pay for the shared context prefix while nine stop after a brief attempt and only one path leads somewhere. How to use parallel agents effectively—breaking a hard problem into genuinely useful subproblems—remains unclear. Next, look at the cost axis: a cost-aware harness.
One minute. Second, on the cost axis: a cost-aware harness is the first level of cost optimization. Here the harness code itself is fixed, and based on the input and current search state, its policy decides what computation to run next by weighing expected contribution against cost. Checking a bottleneck lemma can prevent wasted work across many dependent arguments, whereas an isolated side note is a lower priority unless later work needs it. The same reasoning applies to exploring a new approach, loading a full paper, or stopping. The diagram illustrates proposed priorities, not an implemented optimal policy. Rational metareasoning is the broader framework for choosing computations and stopping decisions using their expected value and cost. Hay, Russell, Tolpin, and Shimony, Selecting Computations: Theory and Applications, UAI 2012, formalize such decisions and study practical approximations. Estimating those values cheaply enough remains part of the problem. Bayesian optimization provides a related approach for expensive evaluations: update a probabilistic model of an objective from observations, then use an acquisition rule to choose the next evaluation. Snoek, Larochelle, and Adams, Practical Bayesian Optimization of Machine Learning Algorithms, NeurIPS 2012, include evaluation duration in their selection criterion. Tuning harness configurations is a natural application. Within a mathematical run, heterogeneous actions such as checking, exploring, and retrieving require an appropriate model of their effects. This connection does not imply that the illustrated dependency heuristic or current xean implements Bayesian optimization. Next, turn to capability and code execution during proof exploration.
Two minutes. Third, on the capability axis: giving the prover Python, computer algebra, or an optimization solver looks natural for checking small cases and searching for counterexamples. In our early exploratory runs, unconstrained code execution inside the proof explorer often diverted token budget into writing and debugging scripts or testing finite inputs while leaving the general theorem unproved. It also adds sandbox and tool-state complexity. Current Xean therefore keeps Explorer in pure mathematical reasoning without code or shell tools. Systems such as ProofCouncil and Rethlas invoke computation or example generation as separate helper steps. Integrating bounded computational checks without derailing deductive search remains an open design question. Both panels are conceptual illustrations rather than a measured ablation. Next, look at simplifying the mathematics that accumulates across notes.
One minute. Fourth, as longer runs accumulate notes and lemmas, we want to periodically reorganize the accumulated mathematical work so exploration can continue from a clearer foundation: consolidate overlapping statements, extract reusable theorems, and find simpler arguments while preserving exact claims and necessary assumptions. This is the mathematical analogue of library learning, and currently no model can do this well and cheaply. Shorter proofs, better lemmas, and fewer unnecessary dependencies require genuine mathematical compression rather than surface text summarization. The question mark marks that open step, and H1 and H2 stand for the retained assumptions of a conceptual theorem. Next, ask how we know whether any of these harness improvements actually helps—cheaply.
One minute. Fifth, across all of these harness ideas—swarms, cost-aware decisions, code execution, and mathematical simplification—a central open problem is how to evaluate whether a harness change actually helps—cheaply. Modifying a context rule, verifier policy, or prompt is easy, while testing whether that change improves solve rate or cost per solve is expensive: each attempt on a hard problem costs ten to over a hundred dollars, and outcomes have high variance from run to run. Because of that variance, a few test runs cannot separate a genuine improvement from luck, and running enough trials to get a reliable signal costs thousands of dollars. Once we have an evaluation signal, that raises one more question: why write the harness ourselves at all?
One minute. Sixth, connect cost-aware search to self-improving harnesses. A cost-aware harness is the first level: the harness code is fixed, and the input decides what action the policy takes next. At the next level, we still minimize expected cost per solve, and even the harness code itself can be modified. That resembles Jürgen Schmidhuber's Gödel machine, with two changes that make it practical for open-ended problems: instead of enumerating candidate programs, an LLM writes targeted edits to the harness code; and instead of requiring a 100% formal proof that the rewrite improves utility—just as we replaced Lean with a natural-language verifier earlier—an LLM and empirical evaluation judge whether the new harness is probably better under uncertainty and cost c / p. In recent literature this is called a Darwin Gödel Machine, a Gödel Agent, or a Meta-Harness—recursive self-improvement at the harness level—where the model writes and specializes its own harness for your goal. Next, close by looking at how fast general products are moving.
One minute. As this screenshot shows, ChatGPT Pro and Codex already run multi-hour campaigns and spawn subagents of their own. Rapid progress in those general tools raises the baseline and pushes custom mathematical harnesses to work harder—through better cost-aware exploration, cleaner dependency management, and stronger verification—to stay ahead at a given budget. Compare any custom harness against the current general tools at the same cost.