LLeMbas/LLeMbas-CLI
1
0 Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases 10 Wiki Activity

Harness spec 3.0.0-draft: context ids in the cases are hex; several files in one patch

Browse Source 
conformance/context.json named results #g001…#k001, which are not the
lowercase hex context.md requires (an implementation that finds a result's id
at the head of the result by that rule cannot see them); they are #1001…#5001
now, with the same answers. Its `about` names the CLI's module,
session/results.ts.

context.md §5 says what it left open: a deleted file's bracket, and an
apply_patch over several files — the id alone, then each file in its own
bracket with its change, in the patch's order; a tool's note after the diff.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
HomerandClaude Opus 5.5 committed 2026-10-27 20:09:22 +00:00
1 parent 4bbee18cee
commit e7300b26d5
2 files changed
+49 -36
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

harness/conformance/context.json
+35 -35
View File
@@ -1,5 +1,5 @@
{
"about": "Context hygiene (context.md, loop.json `context`): what is inserted into the conversation for each step of a session, in order. Steps: `read` (`path`, the file's whole `content`, `offset`, `limit` — the read tool's body: `<n>: ` lines, the note when lines remain), `edit` / `write` (`path`, `before` — absent for a new file — and `after`), `output` (any other tool's result: `tool`, `args`, `text`), `recall` (`target`, `offset`, `limit`) and `compact` (the table added after a summary; the dedupe table is emptied). Each step but `compact` carries the `id` the session gives its result. `limits` overrides loop.json's `cap` for the case. The answer is the text each step inserts: a read headed by path, lines, length and the first 8 hex digits of the content's SHA-256, a stub for an unchanged re-read inside an earlier full read, a diff for an edit, a capped output with its elided line, a stub for a repeated listing or search with the same result. LLeMbas: services/context.py (to be written); CLI: session/context.ts (to be written).",
"about": "Context hygiene (context.md, loop.json `context`): what is inserted into the conversation for each step of a session, in order. Steps: `read` (`path`, the file's whole `content`, `offset`, `limit` — the read tool's body: `<n>: ` lines, the note when lines remain), `edit` / `write` (`path`, `before` — absent for a new file — and `after`), `output` (any other tool's result: `tool`, `args`, `text`), `recall` (`target`, `offset`, `limit`) and `compact` (the table added after a summary; the dedupe table is emptied). Each step but `compact` carries the `id` the session gives its result. `limits` overrides loop.json's `cap` for the case. The answer is the text each step inserts: a read headed by path, lines, length and the first 8 hex digits of the content's SHA-256, a stub for an unchanged re-read inside an earlier full read, a diff for an edit, a capped output with its elided line, a stub for a repeated listing or search with the same result. LLeMbas: services/context.py (to be written); CLI: session/results.ts.",
"cases": [
{
"steps": [
@@ -223,7 +223,7 @@
"steps": [
{
"op": "output",
"id": "#g001",
"id": "#1001",
"tool": "bash",
"args": {
"command": "git status"
@@ -232,7 +232,7 @@
},
{
"op": "output",
"id": "#g002",
"id": "#1002",
"tool": "bash",
"args": {
"command": "git status"
@@ -241,7 +241,7 @@
},
{
"op": "output",
"id": "#g003",
"id": "#1003",
"tool": "grep",
"args": {
"pattern": "TODO"
@@ -250,7 +250,7 @@
},
{
"op": "output",
"id": "#g004",
"id": "#1004",
"tool": "grep",
"args": {
"pattern": "TODO"
@@ -259,35 +259,35 @@
}
],
"expect": [
"[#g001]\nnothing to commit",
"[#g002]\nnothing to commit",
"[#g003]\na.ts:3: // TODO",
"[#g004] Same result as #g003."
"[#1001]\nnothing to commit",
"[#1002]\nnothing to commit",
"[#1003]\na.ts:3: // TODO",
"[#1004] Same result as #1003."
]
},
{
"steps": [
{
"op": "read",
"id": "#h001",
"id": "#2001",
"path": "a.md",
"content": "alpha\n"
},
{
"op": "read",
"id": "#h002",
"id": "#2002",
"path": "b.md",
"content": "beta\n"
},
{
"op": "read",
"id": "#h003",
"id": "#2003",
"path": "a.md",
"content": "alpha\n"
},
{
"op": "output",
"id": "#h004",
"id": "#2004",
"tool": "glob",
"args": {
"pattern": "*.md"
@@ -299,13 +299,13 @@
},
{
"op": "read",
"id": "#h005",
"id": "#2005",
"path": "a.md",
"content": "alpha\n"
},
{
"op": "output",
"id": "#h006",
"id": "#2006",
"tool": "glob",
"args": {
"pattern": "*.md"
@@ -314,19 +314,19 @@
},
{
"op": "recall",
"id": "#h007",
"target": "#h002"
"id": "#2007",
"target": "#2002"
}
],
"expect": [
"[#h001 a.md lines 1-1 of 1 sha b6a98d9c]\n1: alpha",
"[#h002 b.md lines 1-1 of 1 sha f2c82dec]\n1: beta",
"[#h003 a.md lines 1-1 of 1 sha b6a98d9c] Unchanged since #h001, which already shows lines 1-1.",
"[#h004]\na.md\nb.md",
"## Files read before this summary\n\nRead a file again for its content; recall shows a result by its id.\n\n| Path | Lines | sha | Result |\n|---|---|---|---|\n| a.md | 1-1 of 1 | b6a98d9c | #h001 |\n| b.md | 1-1 of 1 | f2c82dec | #h002 |",
"[#h005 a.md lines 1-1 of 1 sha b6a98d9c]\n1: alpha",
"[#h006]\na.md\nb.md",
"[#h007 recall #h002 lines 1-1 of 1]\n1: beta"
"[#2001 a.md lines 1-1 of 1 sha b6a98d9c]\n1: alpha",
"[#2002 b.md lines 1-1 of 1 sha f2c82dec]\n1: beta",
"[#2003 a.md lines 1-1 of 1 sha b6a98d9c] Unchanged since #2001, which already shows lines 1-1.",
"[#2004]\na.md\nb.md",
"## Files read before this summary\n\nRead a file again for its content; recall shows a result by its id.\n\n| Path | Lines | sha | Result |\n|---|---|---|---|\n| a.md | 1-1 of 1 | b6a98d9c | #2001 |\n| b.md | 1-1 of 1 | f2c82dec | #2002 |",
"[#2005 a.md lines 1-1 of 1 sha b6a98d9c]\n1: alpha",
"[#2006]\na.md\nb.md",
"[#2007 recall #2002 lines 1-1 of 1]\n1: beta"
]
},
{
@@ -336,33 +336,33 @@
},
{
"op": "recall",
"id": "#i001",
"id": "#3001",
"target": "#ffff"
}
],
"expect": [
"",
"[#i001] No stored result #ffff in this session."
"[#3001] No stored result #ffff in this session."
]
},
{
"steps": [
{
"op": "write",
"id": "#j001",
"id": "#4001",
"path": "new.md",
"after": "# Title\n\nText.\n"
},
{
"op": "read",
"id": "#j002",
"id": "#4002",
"path": "empty.txt",
"content": ""
}
],
"expect": [
"[#j001 new.md sha fd9dba9b] Created, 3 lines.",
"[#j002 empty.txt empty sha e3b0c442]\n(empty file)"
"[#4001 new.md sha fd9dba9b] Created, 3 lines.",
"[#4002 empty.txt empty sha e3b0c442]\n(empty file)"
]
},
{
@@ -377,7 +377,7 @@
"steps": [
{
"op": "output",
"id": "#k001",
"id": "#5001",
"tool": "bash",
"args": {
"command": "cat min.js"
@@ -386,7 +386,7 @@
},
{
"op": "output",
"id": "#k002",
"id": "#5002",
"tool": "bash",
"args": {
"command": "cat log"
@@ -395,8 +395,8 @@
}
],
"expect": [
"[#k001]\nxxxxxxxxxxxxxxxxxxxx […cut]\n[0 lines elided, full output #k001]",
"[#k002]\nfirst line\n[1 lines elided, full output #k002]\nlast"
"[#5001]\nxxxxxxxxxxxxxxxxxxxx […cut]\n[0 lines elided, full output #5001]",
"[#5002]\nfirst line\n[1 lines elided, full output #5002]\nlast"
]
}
]