Skip to content
Reason Commons
All research
Development findings · 14 Aug 2026

When persistent reasoning did not help

A three-world development run: neither persistent treatment improved safe continuation. What failed, and what that changes.

An open research record. Read the methodology and limitations alongside the claims.

This is a three-world development run, not a preregistered result. It used openai/gpt-4o-mini on RCB v1 seed 701. All worlds share one hidden topology.

The raw run is in reports/rm-paired/instrument-2026-08-14/.

Result

Condition Safe continuation Successor exact Propagation F1 Stale reliance Action accuracy Prompt tokens Violation incidents
Direct full history 0.0% 0.0% 66.7% 0.0% 100.0% 27,898 0
Persistent neutral 0.0% 0.0% 0.0% 0.0% 100.0% 124,049 8
Persistent LTP 0.0% 0.0% 0.0% 33.3% 83.3% 126,762 16

Neither persistent treatment improved the primary outcome. Neutral persistence preserved action safety but cost 4.45 times as many prompt tokens and lost all propagation credit. LTP guidance cost 4.54 times as many prompt tokens, increased stale reliance by 33.3 percentage points, and failed the safety gate.

The comparison is too small and structurally narrow to estimate a general effect. It is sufficient to reject the apparatus-level expectation that adding LTP instructions to the current graph updater reliably fixes its capture errors.

Trace diagnosis

The neutral condition produced:

  • one self-dependency;
  • four snapshots containing a hard cycle; and
  • three cases where a unit was both challenged and confirmed.

The LTP condition produced:

  • one self-dependency;
  • one duplicate dependency;
  • nine snapshots containing a hard cycle; and
  • five challenged-and-confirmed state conflicts.

Both conditions also populated confirmedUnitIds with accepted or merely mentioned units despite instructions limiting confirmation to admissible observations. This reproduces the exact category error the protocol is intended to prevent: acceptance and occurrence collapse during capture.

Decision

Do not scale this model-only updater to more generated seeds and do not treat LTP prompting as the full Reasoning Memory treatment. More repetitions of the same topology would estimate the current failure more precisely without fixing the mechanism.

The next stage gate should add a governed capture condition in which extracted warrants and dependency scopes are checked before they can control execution. That condition should use typed ratification, execution, observation, and assessment records rather than a single ambiguous confirmation set. Only if it passes the safety gate should the project invest in private multi-topology holdouts or the human H7 study.

Back to the top ↑