diff --git a/harness/README.md b/harness/README.md index e5a81cb..d55cfe3 100644 --- a/harness/README.md +++ b/harness/README.md @@ -46,9 +46,11 @@ page *Harness-parity* in LLeMbas CLI's wiki. - **Requires** (spec 3.0.0): what must hold for a tool to be offered at all — `page`: a browser page of the chat is open and attached (not a schedule, not a background job); `display`: the session is shown somewhere a file can be opened (the web UI, Desktop; not a bare terminal); - `forge`: the session's view holds a repository mounted from a forge. A requirement that goes away - between the offer and the call makes the call answer the tool's `unavailable` text, word for - word, as its result. + `forge`: the session's view holds a repository mounted from a forge; `plan`: the session is in + plan mode. A requirement that goes away between the offer and the call makes the call answer the + tool's `unavailable` text, word for word, as its result. **From 3.1.0 a requirement that can come + and go during a session does not take the tool out of the list** (`context.md` §10, rule 4): the + tool is offered from the start and answers `unavailable` while the requirement is not met. - **Confirm** (spec 3.0.0): `always` — the call is put to the person before it runs, in every mode, auto included, and runs on a yes only; nothing remembers the answer (`permission/modes.json`). - A tool's other names, per project, are listed in its `aliases` and work as the tool. A former @@ -57,11 +59,25 @@ page *Harness-parity* in LLeMbas CLI's wiki. `in_progress` → `doing`, `completed` → `done`, `cancelled` → `dropped`; `conformance/plan.json`). - **The plan** (`plan_write`, `plan_update`, `plan_submit`) is one structure both keep per session — title, summary, findings, objectives, phases of tasks with a status each, ids given by the - implementation and never by the model — shown as a card in every UI and put into the prompt - (`{{plan_context}}`) on every step while it exists. `plan_write` and `plan_update` are offered in - every mode; `plan_submit` only in plan mode, where it is the approval. Their permission key is + implementation and never by the model — shown as a card in every UI. From 3.1.0 it is in the + system prompt (`{{plan_context}}`) only as it was when the prompt was made; the model sees it change + in its own `plan_write` / `plan_update` results, and a change by the person comes as a note + (`context.md` §10). All three are offered in every mode; `plan_submit` (`requires: plan`) is the + approval in plan mode and answers its `unavailable` text outside it. Their permission key is `plan` (`permission/defaults.json`), which a rule written for the former `todo` key still reaches. +## 3.1.0 + +Written ahead of both implementations as `3.1.0-draft`. **The prefix stays still** (`context.md` §10). Found on 2026-11-05 by measuring a local model's +prompt cache: a harness that rebuilds its system prompt for every request re-reads the whole session +whenever any part of it changes — the date at midnight, a permission mode, a memory, the plan. So: +the system prompt is made once per session and resent byte for byte; no clock is in it; what +changes during a session comes as a `` on the next user message (`loop.json` +`context.note`); the tool list is fixed for the session (`requires` that can come and go no longer +remove a tool — `plan_submit` gains `requires: plan` and an `unavailable` text); compaction asks +for its summary as the next turn of the same conversation instead of a separate request with a +pasted transcript (§7); every request records its cached input tokens. + ## 3.0.0 Written ahead of both implementations as `3.0.0-draft`, and `3.0.0` once both carried it (LLeMbas diff --git a/harness/VERSION b/harness/VERSION index 4a36342..8c9072d 100644 --- a/harness/VERSION +++ b/harness/VERSION @@ -1 +1 @@ -3.0.0 +3.1.0-draft diff --git a/harness/context.md b/harness/context.md index cc91df1..9818009 100644 --- a/harness/context.md +++ b/harness/context.md @@ -1,6 +1,7 @@ # Context hygiene -Spec 3.0.0. What a session does to keep tool results from filling the model's context — the same in +Spec 3.1.0. What a session does to keep tool results from filling the model's context, and (§10, from +3.1.0) what it keeps still so the model's server can reuse what it has already read — the same in a chat and in an agent session, in both projects. The texts and numbers are `loop.json` `context`; the cases are `conformance/context.json`; what the model is told is `prompts/blocks/context.md`. @@ -125,8 +126,15 @@ two results. ## 7. Compaction -Compaction is the one deliberate break of the cache, and it already exists. In 3.0.0 two things -come with it: +Compaction is the one deliberate break of the cache, and it already exists. **From 3.1.0 the +summary is asked for as the next turn of the same conversation**: the same system prompt, the same +tools, every message as it was, and then one more user message — `prompts/tasks/compact.md`, whose +`{{transcript}}` is empty in this form, since the model has the conversation itself. So the request +that makes the summary costs only that message; a separate request with a pasted transcript was a +full re-read of everything, and a transcript capped to fit it lost the session's middle. A tool +call in the summary's reply is not run; its text before the call is the summary, and a reply with +none fails the compaction as an empty one does. The summary request is never shown as a turn of the +session, and it is not stored as one. In 3.0.0 two things came with compaction, and still do: - After the summary the harness adds — itself, not the model — a table of the **reads the summary covers**: the full reads made since the previous compaction (or since the session began) in turns @@ -151,6 +159,50 @@ come with it: - **The dedupe table is emptied**, so no stub ever points at content that is no longer in the context. A read after compaction is a full read; a `recall` of any earlier id still works. +## 10. The prefix: what the model has already read + +A model's server keeps what it has read of a request and reuses it for the next one only as far as +the two are the same from the first byte. The system prompt and the tool list come first, so **a +change to either is a full re-read** of everything after them — on a long session minutes of work +on a local server, and the full input price again on a cloud API. Measured (2026-11-05, a local +model): a date that changed at midnight sent a 16,912-token session from 99 % cached to 0 %, +139 seconds. So, from 3.1.0: + +1. **The system prompt is made once per session and sent unchanged.** It is built when the session + starts, kept with the session, and every request sends those exact bytes. It is built again only + where everything is read again anyway: after a compaction (§7), or when the session moves to + another model. A session that is resumed sends what it kept. +2. **Nothing in it is a clock.** No date or time. The date goes in a note (rule 3): the first user + message of a session carries it, and so does the next one after the date has changed. +3. **What changes during a session goes at the end, as a note** on the next user message, never + into earlier text: the permission mode (`modes/.md` as the note's body when it changes, + and its first line is in the prompt only as it was at the start), the plan when the person changed + it (the model's own `plan_write` and `plan_update` already answer with it), a memory or + instruction file the person edited, a personality chosen, an MCP server's instructions that + arrived later, the date. The note is a block after the person's words: + + ``` + + The permission mode is now plan: … + + ``` + + (`loop.json` `context.note`.) A note is part of that message from then on, like anything sent. + A request with no user message (a tool step) carries no note; it waits for the next one. +4. **The tool list is the same for the whole session.** A tool whose `requires` can come and go + during a session — `page`, `display`, `forge`, and plan mode for `plan_submit` — is offered from + the start whenever the session could ever meet it, and a call made while it is not met answers + the tool's `unavailable` text. Only what is fixed for the session may leave a tool out: the + session kind (`sessions`), the model (`vision`), a subagent's narrowing, what the project's + configuration or the person switched off before the session began. +5. **Nothing already sent is rewritten** — the one rule above, for results; old tool output is + never pruned in place. +6. **Every request's cache use is recorded** where the provider reports it (cached input tokens), + beside the request's other usage, so a slow prompt has a cause on record. + +What a server does with a matching prefix is its own business; a harness that keeps these rules has +done what it can. + ## 8. Earlier reasoning Left to the model's own chat template: the harness sends the history as it does, and neither strips @@ -160,7 +212,7 @@ reasoning a template keeps nor adds what it drops. `prompts/blocks/context.md`, in every session that has `recall`. -## Not in 3.0.0 +## Not in 3.1.0 Eliding old content in batches behind a threshold, and measuring quality and tokens with and without these measures, are left for a later spec. Every measure above ships with its unit and conformance diff --git a/harness/loop.json b/harness/loop.json index 9ceae80..740ef96 100644 --- a/harness/loop.json +++ b/harness/loop.json @@ -62,7 +62,7 @@ "lifecycle": "A subagent's own background commands stop when it ends, and its report says so. A session's jobs stop when the session is deleted. A front end asked to exit while jobs run asks the person first; one with nobody to ask waits for the jobs the prompt started, then stops the rest." }, "context": { - "about": "Context hygiene (spec 3.0.0, context.md): every measure acts when a result is inserted, and nothing already sent to the model is ever rewritten — changing earlier context throws away the prompt cache from that point. conformance/context.json gives the texts.", + "about": "Context hygiene (spec 3.0.0, the prefix from 3.1.0; context.md): every measure acts when a result is inserted, and nothing already sent to the model is ever rewritten — changing earlier context throws away the prompt cache from that point. conformance/context.json gives the texts.", "ids": { "about": "Every tool result gets an id: `#` and its number in the session, counting from 1, in lowercase hex, at least 4 digits (`#0001`, `#000a`, `#10000`) — unique in the session and never reused (not after a compaction either), so `recall` finds any of them. It is the first thing in the result, in brackets.", "digits": 4 @@ -105,6 +105,18 @@ "line": "Read a file again for its content; recall shows a result by its id.", "rows": 50 }, - "reasoning": "Earlier turns' reasoning is left to the model's chat template: the harness sends the history as it does, neither stripping reasoning the template keeps nor adding what it drops." + "reasoning": "Earlier turns' reasoning is left to the model's chat template: the harness sends the history as it does, neither stripping reasoning the template keeps nor adding what it drops.", + "note": { + "about": "A note on the next user message (context.md §10, rule 3): what changed since the system prompt was made, after the person's words. Lines in this order, each only when it applies.", + "open": "", + "close": "", + "date": "Today is {{date}}.", + "mode": "The permission mode is now {{mode}}:\n{{mode_text}}", + "plan": "The person changed the plan; it is now:\n{{plan}}", + "memory": "The person changed what is remembered; it is now:\n{{memory}}", + "instructions": "{{path}} changed; it now reads:\n{{text}}", + "personality": "The person chose a different personality:\n{{personality}}", + "mcp": "Instructions from an MCP server that connected since the session began:\n{{instructions}}" + } } } diff --git a/harness/tools/plan_submit.json b/harness/tools/plan_submit.json index 7693ca3..c364c6f 100644 --- a/harness/tools/plan_submit.json +++ b/harness/tools/plan_submit.json @@ -6,6 +6,10 @@ "exclusive": true, "block": "tool", "purpose": false, + "requires": [ + "plan" + ], + "unavailable": "Not in plan mode: plan_submit is how a plan made in plan mode is put to the person for approval. Carry on with the work; keep the plan current with plan_update.", "aliases": { "cli": [ "plan_exit"