A ratification and falsification protocol for shared reasoning systems
Status: Protocol specification, version 1.0
[!IMPORTANT] Normative semantics remain current. This document governs authority, evidence, validity, and revision. It does not define the current screen architecture or implementation order; use
INDEX.md,CURRENT_STATE.md, and the product redesign source set for those.
Formerly: the Grounded Reasoning Protocol (GRP) — renamed 2026-08-05 with no semantic change. Applies to: Any system in which reasoning is constructed collaboratively — by humans, AI agents, or both — ratified into a shared canonical model, and used to direct action in the world. Relationship to prior art: Extends ratification-based reasoning systems (e.g. Logical Thinking Process tree governance) with a falsification layer. Compatible with, but not limited to, Dettmer's six-tree LTP model.
0. The problem this protocol solves
A ratification system governs how reasoning becomes accepted. Left alone, it has a failure mode: the shared model can grow forever, in perfect internal order, without ever being tested. Acceptance of a prediction quietly becomes the reality for the next step. Disappointing results are absorbed by patching the plan, blaming execution, or adding assumptions — never by revising the causal model.
The decisive moment for any reasoning system is not when the model explains reality. It is when reality produces an observation the model did not predict. This protocol makes that moment a first-class, formally governed event.
RCP therefore adds four things to a ratification system:
- Scoped dependencies — separating what may be agreed, what may be executed, and what may be relied upon.
- An evidence channel — a second pathway into the system, distinct from ratification, through which reality speaks.
- A two-level outcome assessment — separating "was this a valid test?" from "what did the test say?"
- A revision protocol — an enumerated, enforced set of consequences for contradiction, designed so that rationalization is structurally more expensive than revision.
1. Normative language
MUST, MUST NOT, SHOULD, SHOULD NOT, and MAY are normative.
Throughout, participant means a human or an AI agent acting within the system. Reviewer means a participant holding ratification authority. The graph means the accepted shared reasoning.
2. Foundational principles
P1 — Two channels, never confused
Reasoning changes only through ratification: staged proposal, validation, authorised acceptance, provenance, supersession-not-deletion.
Validity changes only through evidence: immutable observation records entering through an append-only channel that does not pass through the ratification gate.
Neither channel may impersonate the other. Evidence MUST NOT rewrite accepted reasoning; it may only change validity annotations on it and generate revision proposals. Ratification MUST NOT create, edit, or suppress observation records; reviewers decide what evidence means, never whether it exists.
Why evidence bypasses ratification. If observations queue behind reviewers, inconvenient evidence can be politely never-ratified. That is the original rationalization failure mode rebuilt with extra steps. The record of what was observed is not a claim the team accepts or rejects; it is an event the team must reckon with. What remains ratifiable is every interpretation of the record.
P2 — Acceptance is not occurrence
An accepted prediction is a reasoning artifact. An observed result is an evidence artifact. The system MUST maintain distinct objects and states for: the accepted prediction, the execution of the action, the observed result, and the assessment of prediction against result. No rule may treat the first as a substitute for the third.
P3 — Predictions precede outcomes
Any accepted causal claim that directs action SHOULD carry at least one prediction specification, and every prediction specification MUST be registered — with its confirmation, contradiction, and inconclusive rules — before its observation window opens. A prediction registered after outcomes are knowable is invalid as a test and MUST be flagged as retrodiction.
Pre-registration is what removes discretion at the moment of disappointment. It is the single highest-leverage rule in this protocol.
P4 — Tests are judged before claims
An outcome assessment is two verdicts, strictly ordered. First: was this a valid test? Only if yes: what did the test say about the claim? Execution failures, unmet preconditions, unreliable measurement, and incomplete windows invalidate the test, and say nothing — yet — about the claim (see §8 for what repeated test invalidity says).
P5 — Contradiction must resolve
A contradicted prediction MUST NOT be left dangling. It MUST resolve into exactly one of an enumerated set of outcomes (§9), each with its own obligations. In particular, saving a claim by narrowing its scope MUST generate a new registered prediction. A revision that only explains away the anomaly is flagged; a revision that risks something new is not.
P6 — Correlated agreement is not evidence
Multiple AI agents agreeing on a claim carries approximately zero marginal evidential weight beyond one agent, because their errors are correlated through shared training, prompts, and context. Human agreement is weakly independent at best. Evidence records MUST be typed by source class (§6.4), and validity transitions MUST weight source classes, not source counts.
P7 — Mechanical where decidable, judgment where scarce
Wherever a pre-registered rule decides an outcome, the assessment MUST be computed, not deliberated. Wherever judgment is genuinely required, the protocol MUST name who exercises it and schedule it as the scarce resource it is. Two roles follow (§10): the protocol referee (automatable; enforces well-formedness) and the judgment referee (scarce; decides substance).
P8 — The protocol instruments itself
The system MUST record enough about its own operation to answer: where is the constraint on our rate of validated learning? (§11.) A reasoning system that cannot locate its own bottleneck will optimize whatever is most pleasant to optimize.
3. The dependency model: strength × scope
Every dependency between units of reasoning carries two independent fields:
dependency:
dependent_id: string
prerequisite:
kind: accepted_unit | proposal_item | evidence_record
id: string
strength: hard | soft
scope: ratification | execution | validity
rule_code: string
explanation: string
source: system | explicit_reference | agent | reviewer
Strength answers how binding: hard blocks; soft advises.
Scope answers binding for what purpose:
| Scope | Question it governs | Hard form | Soft form |
|---|---|---|---|
ratification |
May this be accepted into the graph? | Blocks acceptance until the prerequisite is accepted (or earlier in the same atomic batch) | Advisory on review order; never disables accept |
execution |
May this be acted on in the world? | Blocks executability until the prerequisite is observed (an evidence record satisfying it exists) | Recommended-but-overridable precondition; override MUST be recorded with who and why |
validity |
May this continue to be relied upon? | Supersession, material revision, or evidence-driven state change of the prerequisite propagates to the dependent (§7.4) | Weak evidential link; produces an advisory on the dependent when the prerequisite degrades |
Rules:
- One pair of units MAY be linked by dependencies of different scopes simultaneously. The canonical example: transition step N+1 has a hard ratification dependency on the accepted expected effect of step N (the plan can be agreed in advance) and a hard execution dependency on the observed result of step N (the world must have complied before anyone acts on it).
- Cycle detection MUST run per scope. A ratification-scope cycle is invalid. An execution-scope cycle is invalid. A validity-scope cycle MAY be accepted only when explicitly designated and reviewed as a reinforcing loop.
- Only an evidence record — never a ratification act — MAY satisfy a hard execution-scope dependency whose prerequisite is an observation.
- Legacy systems with a bare
hard | softfield MUST migrate by defaulting existing dependencies toscope: ratification, then re-deriving execution and validity edges from their rules. No legacy dependency may silently acquire execution semantics.
3.1 The two frontiers
Both frontiers are derived views over the scoped dependency graph, not stored state:
- The Ratification Frontier is the set of pending items whose hard ratification-scope prerequisites are accepted or included earlier in a valid selected batch. (Identical to the classical ratification frontier.)
- The Execution Frontier is the set of accepted, executable-class items whose hard execution-scope prerequisites are satisfied by evidence and whose own validity state permits execution (§7.3).
A user interface MUST present these frontiers separately. "Ready to agree" and "ready to do" are different lists, and conflating them is the acceptance-becomes-reality failure mode in visual form.
4. The evidence domain
The evidence domain adds the following object classes. Objects marked ⊘ are immutable once written: they may be superseded by a corrective record that references them, but never edited or deleted.
4.1 Execution record ⊘
Testimony that an accepted action was attempted.
execution_record:
id: string
action_id: string # the accepted action executed
executor: participant_ref
started_at: timestamp
ended_at: timestamp | null
status: completed | aborted | partial
conformance_notes: string # deviations from the action as ratified
context: string # conditions at execution time
An execution record asserts attempt, not success. Fidelity to the ratified action is judged later (§5.1), not self-declared here.
4.2 Observation record ⊘
Testimony that something was measured or witnessed.
observation_record:
id: string
statement: string # what was observed, as a complete proposition
observer: participant_ref # or instrument identifier
source_class: instrument | human_report | operational_data |
agent_inference | external_document # see §6.4
source_version: string | null # model/instrument version where applicable
observed_at: timestamp
recorded_at: timestamp
method: string # how it was measured
scope: string # population, system, or context covered
execution_record_id: string | null # the execution it pertains to, if any
uncertainty: string | null # known limitations
observed_at and recorded_at are distinct on purpose: a long gap between them is itself a data-quality signal.
4.3 Prediction specification
The pre-registered, testable commitment attached to an accepted causal claim, action, or expected effect. A prediction specification is a reasoning artifact — it is proposed and ratified like any other unit, because deciding what would count as confirmation is a judgment the team must own. But once its window opens it becomes ⊘ immutable; changing a live prediction is superseding it, and the supersession is visible.
prediction_specification:
id: string
subject_id: string # the causal edge, action, or expected effect under test
indicator: string # what will be measured
expected: direction | range | event # the committed expectation
window:
opens_at: timestamp | trigger
closes_at: timestamp | trigger
necessary_conditions: [string] # conditions under which the test is meaningful
confirmation_rule: string # decidable where possible
contradiction_rule: string
inconclusive_rule: string
registered_at: timestamp # MUST precede window.opens_at
A claim MAY carry multiple predictions. A prediction whose rules are decidable (a threshold, a direction, a countable event) enables mechanical assessment (P7); a prediction whose rules require judgment MUST say so explicitly, so the judgment is scheduled rather than discovered.
4.4 Benchmark provenance
Any benchmark that makes an effect desirable or undesirable MUST declare its source type: goal_condition | regulation | contract | historical_baseline | stakeholder_judgment | local_target. "The condition exists" and "the condition is undesirable" are separate claims with separate evidence requirements, and the second is only as strong as its benchmark's provenance.
5. The two-level outcome assessment
When a prediction's window closes (or its contradiction rule fires early), the system MUST produce assessments in this order.
5.1 Level one: test validity assessment
Was this a valid test of the claim?
conforming the test is valid; proceed to level two
execution_nonconforming the action executed materially differed from the action ratified
preconditions_not_met the prediction's necessary conditions did not hold
measurement_unreliable the observation method cannot support the rule's resolution
observation_window_incomplete the window did not complete or coverage was partial
A non-conforming verdict terminates the assessment of this test. It MUST NOT be recorded as evidence for or against the claim — but it MUST be recorded against the test-validity ledger of the claim (§8).
5.2 Level two: prediction result assessment
What did the valid test say?
confirmed
partially_confirmed
contradicted
inconclusive
Where the pre-registered rules are decidable, this verdict MUST be computed by the protocol referee and presented as fact. Where the rules require judgment, the verdict is a ratifiable assessment produced by a judgment referee — but the referee decides against the pre-registered rule, not against their preferences at the moment of disappointment.
5.3 Discrepancy classification
Every contradicted and partially_confirmed verdict MUST carry a discrepancy classification linking it to the causal structure: which edge or assumption does this observation most directly challenge? This classification is a ratifiable assessment; it is the entry point of the revision protocol (§9).
6. Evidence weighting
6.4 Source classes and their treatment
| Source class | Independence | Treatment |
|---|---|---|
instrument |
High (given calibration) | Full weight; calibration metadata required |
operational_data |
High | Full weight; extraction method recorded |
human_report |
Moderate | Weight by observer proximity and incentive; multiple independent reporters compound |
external_document |
Varies | Weight by provenance |
agent_inference |
Correlated | One agent's inference is a hypothesis, not an observation. Additional agreeing agents add ≈ zero weight (P6). Agent inference MUST NOT, alone, drive a validity transition beyond contested |
6.5 The load-bearing composition rule
For every assessment the team relies on for action (sufficiency conclusions, executability conclusions), the system MUST be able to report the evidence composition of its transitive support: what fraction rests solely on agent_inference? This number is a standing meta-metric (§11). No critical causal edge may be accepted, or remain current under challenge, solely because multiple agents assert it.
7. Validity states and their transitions
7.1 States
Accepted units carry one of:
current relied upon; no unresolved credible challenge
contested credible contrary evidence is attached and unresolved
needs_revalidation a structured review is required (from upstream change or contest escalation)
suspended MUST NOT be relied upon for execution or as a hard prerequisite
superseded retained historically; excluded from the canonical projection
7.2 Transition table
| From → To | Trigger | Authority |
|---|---|---|
current → contested |
A level-two contradicted or qualifying partially_confirmed verdict attaches, or credible contrary evidence of sufficient source class arrives |
Mechanical. No one decides whether the graph admits a challenge; the evidence's existence is the transition |
contested → current |
The challenge resolves: the discrepancy is attributed elsewhere with evidence, or a superseding corrective observation lands | Judgment referee, recorded with rationale |
contested → needs_revalidation |
Severity, repetition, or risk crosses the policy threshold (e.g. a second independent contradiction; any contradiction on a claim supporting an irreversible action) | Policy-mechanical where thresholds are set; otherwise judgment referee |
contested / needs_revalidation → suspended |
Continued reliance is unsafe pending review | Judgment referee; MAY be policy-mechanical for high-risk classes |
any → superseded |
A ratified revision replaces the unit | Ratification transaction only |
needs_revalidation → current |
Revalidation review confirms the unit unchanged | Judgment referee via the revalidation process |
Evidence severity, repetition, and the reversibility of dependent actions — not the identity or seniority of whoever dislikes the evidence — are the only admissible inputs to escalation thresholds.
7.3 Effect on execution
An item on the Execution Frontier MUST leave it immediately when any unit in its hard execution- or validity-scope support becomes suspended, and MUST be flagged when any such unit becomes contested. Suspension is precisely the "do not rely on this" bit; an execution system that ignores it has no falsification layer, only falsification theater.
7.4 Propagation
When a unit enters contested, needs_revalidation, or suspended:
- Traverse hard validity-scope dependents transitively.
suspendedpropagates asneeds_revalidationto hard dependents (and revokes executability per §7.3).contestedpropagates as an advisory, not a state change — a challenge to a root cause is not yet a challenge to everything downstream, but everyone downstream should see it.- Soft validity dependents receive advisories only.
- Every propagation MUST be traceable to the originating evidence record.
8. The test-validity ledger
Every claim under test accumulates a ledger of level-one verdicts. The ledger is where the "blame the implementation" escape hatch is closed:
- Rule of repetition. When a claim's tests return non-
conformingverdictsNtimes consecutively (defaultN = 3), the system MUST raise a mandatory meta-assessment with exactly three admissible findings: the claim is untestable as formulated (revise it), the execution capability is deficient (a real constraint, now named, elsewhere in the system), or the classification itself is motivated (a governance finding). "We never tested it properly," repeated, is evidence — about the claim, the executor, or the classifier. - The ledger and its rates are inputs to the meta layer (§11), not just local bookkeeping.
9. The revision protocol
The revision protocol is the formal answer to what happens when reality disagrees. It exists because, absent enumeration, teams and agents will patch plans, blame execution, or add epicycles — anything but touch the causal tree.
9.1 Mandated resolutions
A contradicted verdict on a valid test MUST resolve, within a policy-defined period, into exactly one of:
R1 supersede_causal_claim the challenged edge/assumption is superseded by a
ratified revision of the causal model
R2 scope_limitation a ratified assumption is added that narrows the
claim's domain to exclude the contradicting case
— obligations in §9.2
R3 evidence_invalidation a ratified assessment, itself supported by evidence,
finds the contradicting observation unreliable
(supersedes it with a corrective record; never deletes it)
R4 benchmark_revision the prediction tested the wrong commitment; the
prediction specification is superseded — obligations in §9.2
Until resolution, the claim remains at least contested, and the contradiction is visible on every dependent the team touches.
9.2 The progressive-revision rule
Resolutions R2 and R4 — the two that save the claim by changing the target — carry a mandatory obligation: the revision MUST register at least one new prediction specification that the revised claim exposes to failure, with a window that will actually close.
The validator MUST flag as ad hoc any R2/R4 resolution lacking this, and an ad hoc resolution MUST NOT return the claim to current; the best it can reach is contested (ad hoc revision outstanding).
This is Lakatos, operationalized: a revision that merely explains away the anomaly is degenerate; a revision that risks a new prediction is progressive. The protocol does not forbid saving a claim — it prices the save at one new falsifiable commitment.
9.3 The rationalization asymmetry
The protocol is deliberately arranged so that the lazy path and the honest path point the same way:
- Ignoring evidence is impossible (mechanical
contested). - Blaming execution is bounded (the test-validity ledger, §8).
- Patching by assumption costs a new prediction (§9.2).
- Superseding the causal claim (R1) is the cheapest fully-clearing resolution.
When revising the model is administratively cheaper than defending it, the model gets revised.
10. Roles and authority
10.1 The protocol referee (automatable)
Enforces well-formedness. MUST gate every entry to either channel. Checks, among others: propositions complete; predictions registered before windows; observation metadata complete; decidable rules computed; frontiers derived correctly; propagation executed; ad hoc revisions flagged; ledgers maintained. The protocol referee holds no authority over substance and MUST NOT be able to block an observation for anything other than structural incompleteness.
10.2 The judgment referee (scarce)
Decides substance: escalations, non-decidable verdicts, contest resolutions, revalidations, R1–R4 selection. Judgment referees are the constrained resource of the whole system; the protocol referee's job is to ensure nothing reaches them that a rule could have decided, and the meta layer's job (§11) is to notice when they are the bottleneck.
10.3 Human and agent parity
The protocol is identical for human and agent participants, with three default asymmetries (revisable per deployment, in this order only — loosening requires evidence from the meta layer):
- Agents MUST NOT hold judgment-referee authority initially.
- Agent-originated observations are
agent_inferenceclass regardless of confidence expressed (P6). - Agents MAY draft prediction specifications, discrepancy classifications, and revision proposals freely — drafting is cheap and useful; deciding is neither.
11. The meta layer (TOC²)
The system's true throughput is not accepted units per week; it is the rate of validated learning — predictions resolved against reality per unit time, weighted by the load they bear. Inventory is everything accepted-but-untested: unvalidated hypotheses, unobserved expected effects, open windows. Operating expense is participant attention.
Terminology note (2026-08). This section predates the Closed-Loop protocol's three-way split of measures. In current vocabulary the rate described here is Learning Flow; unqualified throughput is reserved for the Goal Throughput of the modeled system. See
THROUGHPUT.md.
11.1 Required standing metrics
prediction coverage fraction of load-bearing causal edges carrying ≥1 registered prediction
resolution latency time from window close to level-two verdict
contradiction yield fraction of resolved predictions contradicted or partial
(≈0 means predictions are too safe; the model is not being risked)
revision integrity R1 : R2/R4 ratio, and fraction of R2/R4 flagged ad hoc
test validity rate conforming fraction of level-one verdicts, per claim and overall
evidence composition fraction of load-bearing support resting solely on agent_inference
suspension responsiveness time from suspension to execution-frontier revocation (SHOULD be ~0)
judgment queue depth open items awaiting a judgment referee, and their age
11.2 Constraint review
At a policy-defined cadence, the system MUST ask its own question of itself: which single stage currently limits the rate of validated learning? — generation, ratification, execution, observation, assessment, or revision — and record the answer with the metric that supports it. Elevating a non-constraint stage is waste; the metrics exist so the constraint is located by measurement rather than by which stage is most enjoyable to improve.
Two migration patterns to expect: in agent-heavy deployments the constraint arrives at observation and assessment (generation is nearly free); in human deployments it arrives at observation entry friction first and judgment capacity second. Design accordingly before the meta layer confirms it.
12. Conformance levels
Implementations MAY adopt RCP incrementally. Each level is independently valuable and each is a prerequisite for the next.
RCP-0 Scoped dependencies.
strength × scope on all dependencies; ratification and execution frontiers
derived and displayed separately; per-scope cycle detection.
RCP-1 The closed loop.
Execution records, observation records, prediction specifications;
pre-registration enforced; two-level assessment with mechanical verdicts
for decidable rules. One claim, one prediction, one observation, one
verdict — end to end — before any breadth.
RCP-2 Validity and revision.
Five validity states with the transition table; propagation; execution
revocation on suspension; the revision protocol with the
progressive-revision rule; the test-validity ledger.
RCP-3 The meta layer.
Standing metrics; constraint review; evidence-composition reporting;
agent participation in drafting evidence-domain artifacts.
A system at RCP-0 has stopped confusing agreement with occurrence. A system at RCP-1 is falsifiable. A system at RCP-2 is self-correcting. A system at RCP-3 is self-improving — it can locate the constraint on its own learning and elevate it deliberately.
13. Final rule
Reasoning is accepted by people; validity is decided by reality. The protocol's whole purpose is to keep those two authorities distinct, connected, and impossible to quietly swap.