LLeMbas/LLeMbas-CLI
1
0 Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases 10 Wiki Activity

Harness spec 3.0.0-draft: context ids in the cases are hex; several files in one patch

Browse Source 
conformance/context.json named results #g001…#k001, which are not the
lowercase hex context.md requires (an implementation that finds a result's id
at the head of the result by that rule cannot see them); they are #1001…#5001
now, with the same answers. Its `about` names the CLI's module,
session/results.ts.

context.md §5 says what it left open: a deleted file's bracket, and an
apply_patch over several files — the id alone, then each file in its own
bracket with its change, in the patch's order; a tool's note after the diff.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
HomerandClaude Opus 5.5 committed 2026-10-27 20:09:22 +00:00
1 parent 4bbee18cee
commit e7300b26d5
2 files changed
+49 -36
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

harness/conformance/context.json
+35 -35
View File
@@ -1,5 +1,5 @@
{ {
"about": "Context hygiene (context.md, loop.json `context`): what is inserted into the conversation for each step of a session, in order. Steps: `read` (`path`, the file's whole `content`, `offset`, `limit` — the read tool's body: `<n>: ` lines, the note when lines remain), `edit` / `write` (`path`, `before` — absent for a new file — and `after`), `output` (any other tool's result: `tool`, `args`, `text`), `recall` (`target`, `offset`, `limit`) and `compact` (the table added after a summary; the dedupe table is emptied). Each step but `compact` carries the `id` the session gives its result. `limits` overrides loop.json's `cap` for the case. The answer is the text each step inserts: a read headed by path, lines, length and the first 8 hex digits of the content's SHA-256, a stub for an unchanged re-read inside an earlier full read, a diff for an edit, a capped output with its elided line, a stub for a repeated listing or search with the same result. LLeMbas: services/context.py (to be written); CLI: session/context.ts (to be written).", "about": "Context hygiene (context.md, loop.json `context`): what is inserted into the conversation for each step of a session, in order. Steps: `read` (`path`, the file's whole `content`, `offset`, `limit` — the read tool's body: `<n>: ` lines, the note when lines remain), `edit` / `write` (`path`, `before` — absent for a new file — and `after`), `output` (any other tool's result: `tool`, `args`, `text`), `recall` (`target`, `offset`, `limit`) and `compact` (the table added after a summary; the dedupe table is emptied). Each step but `compact` carries the `id` the session gives its result. `limits` overrides loop.json's `cap` for the case. The answer is the text each step inserts: a read headed by path, lines, length and the first 8 hex digits of the content's SHA-256, a stub for an unchanged re-read inside an earlier full read, a diff for an edit, a capped output with its elided line, a stub for a repeated listing or search with the same result. LLeMbas: services/context.py (to be written); CLI: session/results.ts.",
"cases": [ "cases": [
{ {
"steps": [ "steps": [
@@ -223,7 +223,7 @@
"steps": [ "steps": [
{ {
"op": "output", "op": "output",
"id": "#g001", "id": "#1001",
"tool": "bash", "tool": "bash",
"args": { "args": {
"command": "git status" "command": "git status"
@@ -232,7 +232,7 @@
}, },
{ {
"op": "output", "op": "output",
"id": "#g002", "id": "#1002",
"tool": "bash", "tool": "bash",
"args": { "args": {
"command": "git status" "command": "git status"
@@ -241,7 +241,7 @@
}, },
{ {
"op": "output", "op": "output",
"id": "#g003", "id": "#1003",
"tool": "grep", "tool": "grep",
"args": { "args": {
"pattern": "TODO" "pattern": "TODO"
@@ -250,7 +250,7 @@
}, },
{ {
"op": "output", "op": "output",
"id": "#g004", "id": "#1004",
"tool": "grep", "tool": "grep",
"args": { "args": {
"pattern": "TODO" "pattern": "TODO"
@@ -259,35 +259,35 @@
} }
], ],
"expect": [ "expect": [
"[#g001]\nnothing to commit", "[#1001]\nnothing to commit",
"[#g002]\nnothing to commit", "[#1002]\nnothing to commit",
"[#g003]\na.ts:3: // TODO", "[#1003]\na.ts:3: // TODO",
"[#g004] Same result as #g003." "[#1004] Same result as #1003."
] ]
}, },
{ {
"steps": [ "steps": [
{ {
"op": "read", "op": "read",
"id": "#h001", "id": "#2001",
"path": "a.md", "path": "a.md",
"content": "alpha\n" "content": "alpha\n"
}, },
{ {
"op": "read", "op": "read",
"id": "#h002", "id": "#2002",
"path": "b.md", "path": "b.md",
"content": "beta\n" "content": "beta\n"
}, },
{ {
"op": "read", "op": "read",
"id": "#h003", "id": "#2003",
"path": "a.md", "path": "a.md",
"content": "alpha\n" "content": "alpha\n"
}, },
{ {
"op": "output", "op": "output",
"id": "#h004", "id": "#2004",
"tool": "glob", "tool": "glob",
"args": { "args": {
"pattern": "*.md" "pattern": "*.md"
@@ -299,13 +299,13 @@
}, },
{ {
"op": "read", "op": "read",
"id": "#h005", "id": "#2005",
"path": "a.md", "path": "a.md",
"content": "alpha\n" "content": "alpha\n"
}, },
{ {
"op": "output", "op": "output",
"id": "#h006", "id": "#2006",
"tool": "glob", "tool": "glob",
"args": { "args": {
"pattern": "*.md" "pattern": "*.md"
@@ -314,19 +314,19 @@
}, },
{ {
"op": "recall", "op": "recall",
"id": "#h007", "id": "#2007",
"target": "#h002" "target": "#2002"
} }
], ],
"expect": [ "expect": [
"[#h001 a.md lines 1-1 of 1 sha b6a98d9c]\n1: alpha", "[#2001 a.md lines 1-1 of 1 sha b6a98d9c]\n1: alpha",
"[#h002 b.md lines 1-1 of 1 sha f2c82dec]\n1: beta", "[#2002 b.md lines 1-1 of 1 sha f2c82dec]\n1: beta",
"[#h003 a.md lines 1-1 of 1 sha b6a98d9c] Unchanged since #h001, which already shows lines 1-1.", "[#2003 a.md lines 1-1 of 1 sha b6a98d9c] Unchanged since #2001, which already shows lines 1-1.",
"[#h004]\na.md\nb.md", "[#2004]\na.md\nb.md",
"## Files read before this summary\n\nRead a file again for its content; recall shows a result by its id.\n\n| Path | Lines | sha | Result |\n|---|---|---|---|\n| a.md | 1-1 of 1 | b6a98d9c | #h001 |\n| b.md | 1-1 of 1 | f2c82dec | #h002 |", "## Files read before this summary\n\nRead a file again for its content; recall shows a result by its id.\n\n| Path | Lines | sha | Result |\n|---|---|---|---|\n| a.md | 1-1 of 1 | b6a98d9c | #2001 |\n| b.md | 1-1 of 1 | f2c82dec | #2002 |",
"[#h005 a.md lines 1-1 of 1 sha b6a98d9c]\n1: alpha", "[#2005 a.md lines 1-1 of 1 sha b6a98d9c]\n1: alpha",
"[#h006]\na.md\nb.md", "[#2006]\na.md\nb.md",
"[#h007 recall #h002 lines 1-1 of 1]\n1: beta" "[#2007 recall #2002 lines 1-1 of 1]\n1: beta"
] ]
}, },
{ {
@@ -336,33 +336,33 @@
}, },
{ {
"op": "recall", "op": "recall",
"id": "#i001", "id": "#3001",
"target": "#ffff" "target": "#ffff"
} }
], ],
"expect": [ "expect": [
"", "",
"[#i001] No stored result #ffff in this session." "[#3001] No stored result #ffff in this session."
] ]
}, },
{ {
"steps": [ "steps": [
{ {
"op": "write", "op": "write",
"id": "#j001", "id": "#4001",
"path": "new.md", "path": "new.md",
"after": "# Title\n\nText.\n" "after": "# Title\n\nText.\n"
}, },
{ {
"op": "read", "op": "read",
"id": "#j002", "id": "#4002",
"path": "empty.txt", "path": "empty.txt",
"content": "" "content": ""
} }
], ],
"expect": [ "expect": [
"[#j001 new.md sha fd9dba9b] Created, 3 lines.", "[#4001 new.md sha fd9dba9b] Created, 3 lines.",
"[#j002 empty.txt empty sha e3b0c442]\n(empty file)" "[#4002 empty.txt empty sha e3b0c442]\n(empty file)"
] ]
}, },
{ {
@@ -377,7 +377,7 @@
"steps": [ "steps": [
{ {
"op": "output", "op": "output",
"id": "#k001", "id": "#5001",
"tool": "bash", "tool": "bash",
"args": { "args": {
"command": "cat min.js" "command": "cat min.js"
@@ -386,7 +386,7 @@
}, },
{ {
"op": "output", "op": "output",
"id": "#k002", "id": "#5002",
"tool": "bash", "tool": "bash",
"args": { "args": {
"command": "cat log" "command": "cat log"
@@ -395,8 +395,8 @@
} }
], ],
"expect": [ "expect": [
"[#k001]\nxxxxxxxxxxxxxxxxxxxx […cut]\n[0 lines elided, full output #k001]", "[#5001]\nxxxxxxxxxxxxxxxxxxxx […cut]\n[0 lines elided, full output #5001]",
"[#k002]\nfirst line\n[1 lines elided, full output #k002]\nlast" "[#5002]\nfirst line\n[1 lines elided, full output #5002]\nlast"
] ]
} }
] ]
harness/context.md
+14 -1
View File
@@ -86,7 +86,20 @@ naming the path and the new content's hash:
``` ```
`write` of a new file answers `[#e202 docs/new.md sha 52b6e3a0] Created, 14 lines.` — the model `write` of a new file answers `[#e202 docs/new.md sha 52b6e3a0] Created, 14 lines.` — the model
wrote the content and has it. The diff is capped like any output. wrote the content and has it. A deleted file is `[#e203 old.md deleted]`. An `apply_patch` over
several files answers the id alone, then each file in its own bracket with its change, in the
patch's order:
```
[#e204]
[src/a.ts sha 7d0a91c3]
@@ -1,2 +1,2 @@
[docs/new.md sha 52b6e3a0] Created, 14 lines.
[old.md deleted]
```
A note the tool adds (a skill file written outside skill_manage) follows the diff. The diff is
capped like any output, the first line kept.
## 6. The same search or listing twice is a stub ## 6. The same search or listing twice is a stub