Files
LLeMbas-CLI/tests/conformance.test.ts
HomerandClaude Fable 5.1 837f38c744 Harness spec 1.1.0: one plan, prototypes and the library shared; the CLI implements it
The spec: plan_write and plan_update (new, shared, offered in every mode) and
plan_submit (now shared) keep one structured plan — findings, objectives,
phases of tasks — in place of the todo list, which lives on as a former name
of plan_write. The shared system text is one text for both projects, with
variables for what differs; the mode texts, a prototypes block and a library
block are shared. acp.md: the client hands the device what only it knows
(context, library, prototypes), the plan travels as a structure (plan_state),
library calls carry the chat's scope, and a client may rewind a session.
loop.json says its numbers as numbers. Four conformance areas: plan,
prototype, library_scope, toolcall_index.

The CLI: the plan is kept per session in the store and drawn above the prompt
and on the approval card; the prompt is assembled from the spec's texts with
their variables filled; the link reads the instance's instructions,
personalization, knowledge scope and prototype offer, sends the plan as a
structure and answers rewind; library calls to the instance carry the session
as scope; a prototype written in the terminal is saved as a file and named in
the transcript; a tool-call fragment without an index continues the last call;
the default theme is Moria; dead exports and Hermes leftovers removed.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-18 04:02:12 +02:00

27 lines
1.3 KiB
TypeScript

// Every conformance case of the harness spec (harness/conformance/), through LLeMbas CLI. LLeMbas
// runs the same files through its own implementation; a case that fails on one side only is a
// difference between the two harnesses, which the spec exists to prevent.
import { expect, test } from "bun:test"
import { readdirSync, readFileSync } from "node:fs"
import { join } from "node:path"
import { AREAS } from "./conformance.ts"
const dir = join(import.meta.dir, "..", "harness", "conformance")
for (const f of readdirSync(dir).filter((x) => x.endsWith(".json")).sort()) {
const area = f.replace(/\.json$/, "")
const doc = JSON.parse(readFileSync(join(dir, f), "utf8")) as { cases: Record<string, unknown>[] }
// An area nobody here answers yet is skipped and named, not failed: the harness spec can be a
// step ahead of this implementation inside one round of work.
if (!AREAS[area]) {
test.skip(`conformance: ${area} — no runner in tests/conformance.ts yet`, () => {})
continue
}
test(`conformance: ${area} (${doc.cases.length} cases)`, () => {
for (const c of doc.cases) {
const { expect: want, ...input } = c
expect(want, `${area}: ${JSON.stringify(input)} has no expect — bun run harness record`).toBeDefined()
expect({ input, got: AREAS[area]!(input) }).toEqual({ input, got: want })
}
})
}