From 02c7282e74401aa5a2fbe19116169acdb779ac36 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jaroslav=20Bene=C5=A1?= Date: Mon, 9 Nov 2026 16:56:31 +0000 Subject: [PATCH] =?UTF-8?q?Harness=20spec=203.3.0:=20subagents=20=E2=80=94?= =?UTF-8?q?=20the=20word,=20the=20model,=20capacity,=20the=20offer?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - "Subagent" in every text a person or a model reads; "helper" retired (the team area's step helper_done is now task_done). - A subagent runs on the session's model unless the person named another in this session's own messages; task's and team_add's `model` is refused otherwise. No setting chooses a subagent's model. - Capacity: another model on the session's model's one-model server is refused; the session's model as its own subagent needs parallel >= 2 or no limit; no fallback. - The agents {min, max} setting is shown as Subagents, offered when a model could serve; context.note.team_own when the session's model cannot. - subagents.md, loop.json subagents, conformance/subagents.json (26 cases). Co-Authored-By: Claude Opus 5.5 --- harness/README.md | 36 +- harness/VERSION | 2 +- harness/conformance/subagents.json | 1253 ++++++++++++++++++++++++++++ harness/conformance/team.json | 8 +- harness/loop.json | 29 +- harness/models/schema.json | 6 +- harness/prompts/blocks/subagent.md | 2 +- harness/subagents.md | 87 ++ harness/team.md | 44 +- harness/tools/task.json | 4 +- harness/tools/team_add.json | 2 +- tests/conformance.ts | 2 +- 12 files changed, 1436 insertions(+), 39 deletions(-) create mode 100644 harness/conformance/subagents.json create mode 100644 harness/subagents.md diff --git a/harness/README.md b/harness/README.md index faa6fa7..208e7e3 100644 --- a/harness/README.md +++ b/harness/README.md @@ -22,10 +22,11 @@ page *Harness-parity* in LLeMbas CLI's wiki. | `permission/defaults.json` | The rules every session starts from | | `permission/modes.json` | The four modes — manual, edit, auto, plan — as what each does with each risk class | | `models/schema.json` | Model metadata: context, output limit, efforts and the thinking switch, vision, tools, notes, capacity (`parallel`); `levels` — the effort levels a chat offers and what each sends | -| `loop.json` | How one prompt runs: the step backstop, budgets, wrap-up, nudges, parallel calls, capacity; `jobs` — background work, its marks and limits; `team` — a team's size, names, marks, refusals and texts; `context` — result ids, stubs, the output cap, the compaction table, the environment lines, the session notes | +| `loop.json` | How one prompt runs: the step backstop, budgets, wrap-up, nudges, parallel calls, capacity; `jobs` — background work, its marks and limits; `subagents` — the model refusals, what names a model, the setting's label; `team` — a team's size, names, marks, refusals and texts; `context` — result ids, stubs, the output cap, the compaction table, the environment lines, the session notes | | `filesystem.md` | The file system a session works in: paths, node kinds, the four access levels, errors, listings, search and indexing by tokens, stacked instruction files, skills, memory | | `context.md` | Context hygiene: what happens to a tool result when it is inserted, and never afterwards; the prefix that stays still; reasoning sent back during its turn; no absolute paths in the prompt | | `team.md` | Teams: a lead and the members it adds, the size setting, the room, who receives what, reports, a manager's refusals, capacity, stopping | +| `subagents.md` | Subagents: the word, which model one runs on (the session's, or one the person named), the capacity refusals, and when the Subagents setting is offered | | `quirks.json` | What real servers do, and how both harnesses cope | | `conformance/.json` | Cases with their expected results, run by both test suites | | `acp.md` | The link between them: ACP plus every `_lembas/*` method, field and capability | @@ -33,7 +34,7 @@ page *Harness-parity* in LLeMbas CLI's wiki. ## Vocabulary - **Risk classes:** `read`, `write`, `execute`, `interact`. **Reading and writing are never one - tool** among the shared ones, so a read-only helper is given exactly the readers. + tool** among the shared ones, so a read-only subagent is given exactly the readers. - **Modes:** `manual`, `edit`, `auto`, `plan` (`unrestricted` is read as `auto`). - **Scopes:** `shared` — both projects implement it; `cli` or `llembas` — only that one. The same two names key what is per project elsewhere: a tool's `aliases`, `loop.json`'s budgets, a @@ -48,8 +49,9 @@ page *Harness-parity* in LLeMbas CLI's wiki. page of the chat is open and attached (not a schedule, not a background job); `display`: the session is shown somewhere a file can be opened (the web UI, Desktop; not a bare terminal); `forge`: the session's view holds a repository mounted from a forge; `plan`: the session is in - plan mode; `team` (3.2.0): the session's team setting allows more than one agent, on a model that - serves two requests at once (`team.md` §2). A requirement that goes away between the offer and the call makes the call answer the + plan mode; `team` (3.2.0): the session's team setting allows more than one agent, and a model could + serve as a subagent (3.3.0: `subagents.md` §4; before, the chat's model serving two requests at + once). A requirement that goes away between the offer and the call makes the call answer the tool's `unavailable` text, word for word, as its result. **From 3.1.0 a requirement that can come and go during a session does not take the tool out of the list** (`context.md` §10, rule 4): the tool is offered from the start and answers `unavailable` while the requirement is not met. @@ -68,6 +70,32 @@ page *Harness-parity* in LLeMbas CLI's wiki. approval in plan mode and answers its `unavailable` text outside it. Their permission key is `plan` (`permission/defaults.json`), which a rule written for the former `todo` key still reaches. +## 3.3.0 + +Written ahead of both implementations; both carry it in the same round (`subagents.md`). + +- **Subagent** is the word everywhere a person or a model reads one — a `task` subagent and a + team's member alike; *helper* is retired. Tool names and aliases are unchanged. The `team` + conformance area's step `helper_done` is now **`task_done`**. +- **The model**: a subagent runs on the session's model, unless the person named another one in + this session's own messages (its served id, its id at its connection or its display name, a whole + word, without case — never the model's text or a tool's output). `task`'s and `team_add`'s + `model` is refused otherwise: "Use another model for subagents only when the person names it." + No setting chooses a subagent's model; an agent definition's own `model` is not used for one. +- **Capacity, the logical restrictions only**: another model on the session's model's one-model + server is refused ("That model is on the same one-model server as this chat's model; name a + model on another server."); the session's model as its own subagent needs `parallel ≥ 2` or no + limit, else "This model serves one request at a time: name another model, on another server, + for subagents." — and nothing else is picked. A member is refused for these too, and still only + waits for a full model. +- **The offer**: the `agents {min, max}` setting is shown as **Subagents** (*fewest*, *most*; + `loop.json` `subagents.setting`), and with it the team tools, when subagents are enabled and at + least one model could serve: the session's, with the capacity, or any usable model off its + one-model server. `context.note.team_own` tells the model, and `subagents.setting.own` the + person, when the session's model cannot serve subagents itself. +- `task`'s and `team_add`'s `model` descriptions, `task`'s description (a line on the model), + `models/schema.json` capacity texts, `team.md` §2, §4, §8. Conformance area **`subagents`**. + ## 3.2.0 Written ahead of both implementations; both carry it in the same round. diff --git a/harness/VERSION b/harness/VERSION index 944880f..15a2799 100644 --- a/harness/VERSION +++ b/harness/VERSION @@ -1 +1 @@ -3.2.0 +3.3.0 diff --git a/harness/conformance/subagents.json b/harness/conformance/subagents.json new file mode 100644 index 0000000..a9000e3 --- /dev/null +++ b/harness/conformance/subagents.json @@ -0,0 +1,1253 @@ +{ + "about": "Subagents (subagents.md, spec 3.3.0): which model a subagent runs on, and when subagents are offered. `models`: the models the person may use, by served id — `id` at its connection, display `name`, `capacity` (models/schema.json; {} — no limit); `session`: the session's model, one of them. `op` `model` — a task or team_add call with `model` (absent: none given) after the person typed `said` in this session (oldest first; `other`: texts that are not the person's — the model's, a tool's) — answers `{model}`: the served id the subagent runs on, or `{refused}`: the text, word for word (loop.json subagents.texts). `op` `offer` (`subagents`: enabled, default true) — `{offered, own, note}`: whether the Subagents setting and the team tools are offered, whether the session's model can serve its own subagents, and the note line that says it cannot (context.note.team_own) where subagents are offered, else null. LLeMbas: services/subagents.py; CLI: team/subagents.ts.", + "cases": [ + { + "about": "No model given: the session's own, on a model with no limit.", + "op": "model", + "session": "api/atlas-1", + "models": { + "lab/qwen-27b": { + "id": "qwen-27b", + "name": "Qwen 27B", + "capacity": { + "group": "lab", + "parallel": 1 + } + }, + "lab/gemma-12b": { + "id": "gemma-12b", + "name": "Gemma 12B", + "capacity": { + "group": "lab", + "parallel": 1 + } + }, + "box/coder-30b": { + "id": "coder-30b", + "name": "Coder 30B", + "capacity": { + "parallel": 1 + } + }, + "box/coder-30b-instruct": { + "id": "coder-30b-instruct", + "name": "Coder 30B Instruct", + "capacity": { + "parallel": 2 + } + }, + "lab2/fast-8b": { + "id": "fast-8b", + "name": "Fast 8B", + "capacity": { + "group": "lab2", + "parallel": 4 + } + }, + "api/atlas-1": { + "id": "atlas-1", + "name": "Atlas 1", + "capacity": {} + } + }, + "said": [ + "Look into the parser." + ], + "expect": { + "model": "api/atlas-1" + } + }, + { + "about": "No model given: the session's own, on a model that serves several at once.", + "op": "model", + "session": "lab2/fast-8b", + "models": { + "lab/qwen-27b": { + "id": "qwen-27b", + "name": "Qwen 27B", + "capacity": { + "group": "lab", + "parallel": 1 + } + }, + "lab/gemma-12b": { + "id": "gemma-12b", + "name": "Gemma 12B", + "capacity": { + "group": "lab", + "parallel": 1 + } + }, + "box/coder-30b": { + "id": "coder-30b", + "name": "Coder 30B", + "capacity": { + "parallel": 1 + } + }, + "box/coder-30b-instruct": { + "id": "coder-30b-instruct", + "name": "Coder 30B Instruct", + "capacity": { + "parallel": 2 + } + }, + "lab2/fast-8b": { + "id": "fast-8b", + "name": "Fast 8B", + "capacity": { + "group": "lab2", + "parallel": 4 + } + }, + "api/atlas-1": { + "id": "atlas-1", + "name": "Atlas 1", + "capacity": {} + } + }, + "expect": { + "model": "lab2/fast-8b" + } + }, + { + "about": "No model given and the session's model serves one request at a time: refused, and nothing else is picked.", + "op": "model", + "session": "lab/qwen-27b", + "models": { + "lab/qwen-27b": { + "id": "qwen-27b", + "name": "Qwen 27B", + "capacity": { + "group": "lab", + "parallel": 1 + } + }, + "lab/gemma-12b": { + "id": "gemma-12b", + "name": "Gemma 12B", + "capacity": { + "group": "lab", + "parallel": 1 + } + }, + "box/coder-30b": { + "id": "coder-30b", + "name": "Coder 30B", + "capacity": { + "parallel": 1 + } + }, + "box/coder-30b-instruct": { + "id": "coder-30b-instruct", + "name": "Coder 30B Instruct", + "capacity": { + "parallel": 2 + } + }, + "lab2/fast-8b": { + "id": "fast-8b", + "name": "Fast 8B", + "capacity": { + "group": "lab2", + "parallel": 4 + } + }, + "api/atlas-1": { + "id": "atlas-1", + "name": "Atlas 1", + "capacity": {} + } + }, + "said": [ + "Split this up between subagents." + ], + "expect": { + "refused": "This model serves one request at a time: name another model, on another server, for subagents." + } + }, + { + "about": "Another model the person never named: refused.", + "op": "model", + "session": "lab/qwen-27b", + "models": { + "lab/qwen-27b": { + "id": "qwen-27b", + "name": "Qwen 27B", + "capacity": { + "group": "lab", + "parallel": 1 + } + }, + "lab/gemma-12b": { + "id": "gemma-12b", + "name": "Gemma 12B", + "capacity": { + "group": "lab", + "parallel": 1 + } + }, + "box/coder-30b": { + "id": "coder-30b", + "name": "Coder 30B", + "capacity": { + "parallel": 1 + } + }, + "box/coder-30b-instruct": { + "id": "coder-30b-instruct", + "name": "Coder 30B Instruct", + "capacity": { + "parallel": 2 + } + }, + "lab2/fast-8b": { + "id": "fast-8b", + "name": "Fast 8B", + "capacity": { + "group": "lab2", + "parallel": 4 + } + }, + "api/atlas-1": { + "id": "atlas-1", + "name": "Atlas 1", + "capacity": {} + } + }, + "said": [ + "Hello." + ], + "model": "box/coder-30b", + "expect": { + "refused": "Use another model for subagents only when the person names it." + } + }, + { + "about": "Named by its served id: allowed, though it serves one request at a time — it is on another server.", + "op": "model", + "session": "lab/qwen-27b", + "models": { + "lab/qwen-27b": { + "id": "qwen-27b", + "name": "Qwen 27B", + "capacity": { + "group": "lab", + "parallel": 1 + } + }, + "lab/gemma-12b": { + "id": "gemma-12b", + "name": "Gemma 12B", + "capacity": { + "group": "lab", + "parallel": 1 + } + }, + "box/coder-30b": { + "id": "coder-30b", + "name": "Coder 30B", + "capacity": { + "parallel": 1 + } + }, + "box/coder-30b-instruct": { + "id": "coder-30b-instruct", + "name": "Coder 30B Instruct", + "capacity": { + "parallel": 2 + } + }, + "lab2/fast-8b": { + "id": "fast-8b", + "name": "Fast 8B", + "capacity": { + "group": "lab2", + "parallel": 4 + } + }, + "api/atlas-1": { + "id": "atlas-1", + "name": "Atlas 1", + "capacity": {} + } + }, + "said": [ + "Use box/coder-30b for the subagents." + ], + "model": "box/coder-30b", + "expect": { + "model": "box/coder-30b" + } + }, + { + "about": "Named by its id at its connection, in other letter case.", + "op": "model", + "session": "lab/qwen-27b", + "models": { + "lab/qwen-27b": { + "id": "qwen-27b", + "name": "Qwen 27B", + "capacity": { + "group": "lab", + "parallel": 1 + } + }, + "lab/gemma-12b": { + "id": "gemma-12b", + "name": "Gemma 12B", + "capacity": { + "group": "lab", + "parallel": 1 + } + }, + "box/coder-30b": { + "id": "coder-30b", + "name": "Coder 30B", + "capacity": { + "parallel": 1 + } + }, + "box/coder-30b-instruct": { + "id": "coder-30b-instruct", + "name": "Coder 30B Instruct", + "capacity": { + "parallel": 2 + } + }, + "lab2/fast-8b": { + "id": "fast-8b", + "name": "Fast 8B", + "capacity": { + "group": "lab2", + "parallel": 4 + } + }, + "api/atlas-1": { + "id": "atlas-1", + "name": "Atlas 1", + "capacity": {} + } + }, + "said": [ + "try CODER-30B" + ], + "model": "box/coder-30b", + "expect": { + "model": "box/coder-30b" + } + }, + { + "about": "Named by its display name, without case.", + "op": "model", + "session": "lab/qwen-27b", + "models": { + "lab/qwen-27b": { + "id": "qwen-27b", + "name": "Qwen 27B", + "capacity": { + "group": "lab", + "parallel": 1 + } + }, + "lab/gemma-12b": { + "id": "gemma-12b", + "name": "Gemma 12B", + "capacity": { + "group": "lab", + "parallel": 1 + } + }, + "box/coder-30b": { + "id": "coder-30b", + "name": "Coder 30B", + "capacity": { + "parallel": 1 + } + }, + "box/coder-30b-instruct": { + "id": "coder-30b-instruct", + "name": "Coder 30B Instruct", + "capacity": { + "parallel": 2 + } + }, + "lab2/fast-8b": { + "id": "fast-8b", + "name": "Fast 8B", + "capacity": { + "group": "lab2", + "parallel": 4 + } + }, + "api/atlas-1": { + "id": "atlas-1", + "name": "Atlas 1", + "capacity": {} + } + }, + "said": [ + "The subagents can use coder 30b" + ], + "model": "box/coder-30b", + "expect": { + "model": "box/coder-30b" + } + }, + { + "about": "White space in a display name matches any run of white space.", + "op": "model", + "session": "api/atlas-1", + "models": { + "lab/qwen-27b": { + "id": "qwen-27b", + "name": "Qwen 27B", + "capacity": { + "group": "lab", + "parallel": 1 + } + }, + "lab/gemma-12b": { + "id": "gemma-12b", + "name": "Gemma 12B", + "capacity": { + "group": "lab", + "parallel": 1 + } + }, + "box/coder-30b": { + "id": "coder-30b", + "name": "Coder 30B", + "capacity": { + "parallel": 1 + } + }, + "box/coder-30b-instruct": { + "id": "coder-30b-instruct", + "name": "Coder 30B Instruct", + "capacity": { + "parallel": 2 + } + }, + "lab2/fast-8b": { + "id": "fast-8b", + "name": "Fast 8B", + "capacity": { + "group": "lab2", + "parallel": 4 + } + }, + "api/atlas-1": { + "id": "atlas-1", + "name": "Atlas 1", + "capacity": {} + } + }, + "said": [ + "Give that to Gemma\n 12B, please." + ], + "model": "lab/gemma-12b", + "expect": { + "model": "lab/gemma-12b" + } + }, + { + "about": "A whole word only: a longer id that begins with the name does not name it.", + "op": "model", + "session": "lab/qwen-27b", + "models": { + "lab/qwen-27b": { + "id": "qwen-27b", + "name": "Qwen 27B", + "capacity": { + "group": "lab", + "parallel": 1 + } + }, + "lab/gemma-12b": { + "id": "gemma-12b", + "name": "Gemma 12B", + "capacity": { + "group": "lab", + "parallel": 1 + } + }, + "box/coder-30b": { + "id": "coder-30b", + "name": "Coder 30B", + "capacity": { + "parallel": 1 + } + }, + "box/coder-30b-instruct": { + "id": "coder-30b-instruct", + "name": "Coder 30B Instruct", + "capacity": { + "parallel": 2 + } + }, + "lab2/fast-8b": { + "id": "fast-8b", + "name": "Fast 8B", + "capacity": { + "group": "lab2", + "parallel": 4 + } + }, + "api/atlas-1": { + "id": "atlas-1", + "name": "Atlas 1", + "capacity": {} + } + }, + "said": [ + "coder-30b-instruct is good at this." + ], + "model": "box/coder-30b", + "expect": { + "refused": "Use another model for subagents only when the person names it." + } + }, + { + "about": "The longer id names its own model.", + "op": "model", + "session": "lab/qwen-27b", + "models": { + "lab/qwen-27b": { + "id": "qwen-27b", + "name": "Qwen 27B", + "capacity": { + "group": "lab", + "parallel": 1 + } + }, + "lab/gemma-12b": { + "id": "gemma-12b", + "name": "Gemma 12B", + "capacity": { + "group": "lab", + "parallel": 1 + } + }, + "box/coder-30b": { + "id": "coder-30b", + "name": "Coder 30B", + "capacity": { + "parallel": 1 + } + }, + "box/coder-30b-instruct": { + "id": "coder-30b-instruct", + "name": "Coder 30B Instruct", + "capacity": { + "parallel": 2 + } + }, + "lab2/fast-8b": { + "id": "fast-8b", + "name": "Fast 8B", + "capacity": { + "group": "lab2", + "parallel": 4 + } + }, + "api/atlas-1": { + "id": "atlas-1", + "name": "Atlas 1", + "capacity": {} + } + }, + "said": [ + "coder-30b-instruct is good at this." + ], + "model": "box/coder-30b-instruct", + "expect": { + "model": "box/coder-30b-instruct" + } + }, + { + "about": "A full stop or a colon that ends the sentence ends the word.", + "op": "model", + "session": "lab/qwen-27b", + "models": { + "lab/qwen-27b": { + "id": "qwen-27b", + "name": "Qwen 27B", + "capacity": { + "group": "lab", + "parallel": 1 + } + }, + "lab/gemma-12b": { + "id": "gemma-12b", + "name": "Gemma 12B", + "capacity": { + "group": "lab", + "parallel": 1 + } + }, + "box/coder-30b": { + "id": "coder-30b", + "name": "Coder 30B", + "capacity": { + "parallel": 1 + } + }, + "box/coder-30b-instruct": { + "id": "coder-30b-instruct", + "name": "Coder 30B Instruct", + "capacity": { + "parallel": 2 + } + }, + "lab2/fast-8b": { + "id": "fast-8b", + "name": "Fast 8B", + "capacity": { + "group": "lab2", + "parallel": 4 + } + }, + "api/atlas-1": { + "id": "atlas-1", + "name": "Atlas 1", + "capacity": {} + } + }, + "said": [ + "Run the subagents on coder-30b." + ], + "model": "box/coder-30b", + "expect": { + "model": "box/coder-30b" + } + }, + { + "about": "A colon followed by a letter continues the word: a tag is another name.", + "op": "model", + "session": "lab/qwen-27b", + "models": { + "lab/qwen-27b": { + "id": "qwen-27b", + "name": "Qwen 27B", + "capacity": { + "group": "lab", + "parallel": 1 + } + }, + "lab/gemma-12b": { + "id": "gemma-12b", + "name": "Gemma 12B", + "capacity": { + "group": "lab", + "parallel": 1 + } + }, + "box/coder-30b": { + "id": "coder-30b", + "name": "Coder 30B", + "capacity": { + "parallel": 1 + } + }, + "box/coder-30b-instruct": { + "id": "coder-30b-instruct", + "name": "Coder 30B Instruct", + "capacity": { + "parallel": 2 + } + }, + "lab2/fast-8b": { + "id": "fast-8b", + "name": "Fast 8B", + "capacity": { + "group": "lab2", + "parallel": 4 + } + }, + "api/atlas-1": { + "id": "atlas-1", + "name": "Atlas 1", + "capacity": {} + } + }, + "said": [ + "coder-30b:q4 would do." + ], + "model": "box/coder-30b", + "expect": { + "refused": "Use another model for subagents only when the person names it." + } + }, + { + "about": "Named in an earlier message of the session: still named.", + "op": "model", + "session": "lab/qwen-27b", + "models": { + "lab/qwen-27b": { + "id": "qwen-27b", + "name": "Qwen 27B", + "capacity": { + "group": "lab", + "parallel": 1 + } + }, + "lab/gemma-12b": { + "id": "gemma-12b", + "name": "Gemma 12B", + "capacity": { + "group": "lab", + "parallel": 1 + } + }, + "box/coder-30b": { + "id": "coder-30b", + "name": "Coder 30B", + "capacity": { + "parallel": 1 + } + }, + "box/coder-30b-instruct": { + "id": "coder-30b-instruct", + "name": "Coder 30B Instruct", + "capacity": { + "parallel": 2 + } + }, + "lab2/fast-8b": { + "id": "fast-8b", + "name": "Fast 8B", + "capacity": { + "group": "lab2", + "parallel": 4 + } + }, + "api/atlas-1": { + "id": "atlas-1", + "name": "Atlas 1", + "capacity": {} + } + }, + "said": [ + "Use coder-30b for subagents today.", + "Now the tests." + ], + "model": "box/coder-30b", + "expect": { + "model": "box/coder-30b" + } + }, + { + "about": "Only the person's words count: a model named in the model's text or a tool's output is not named.", + "op": "model", + "session": "lab/qwen-27b", + "models": { + "lab/qwen-27b": { + "id": "qwen-27b", + "name": "Qwen 27B", + "capacity": { + "group": "lab", + "parallel": 1 + } + }, + "lab/gemma-12b": { + "id": "gemma-12b", + "name": "Gemma 12B", + "capacity": { + "group": "lab", + "parallel": 1 + } + }, + "box/coder-30b": { + "id": "coder-30b", + "name": "Coder 30B", + "capacity": { + "parallel": 1 + } + }, + "box/coder-30b-instruct": { + "id": "coder-30b-instruct", + "name": "Coder 30B Instruct", + "capacity": { + "parallel": 2 + } + }, + "lab2/fast-8b": { + "id": "fast-8b", + "name": "Fast 8B", + "capacity": { + "group": "lab2", + "parallel": 4 + } + }, + "api/atlas-1": { + "id": "atlas-1", + "name": "Atlas 1", + "capacity": {} + } + }, + "said": [ + "Go on." + ], + "other": [ + "Maybe box/coder-30b would help.", + "models: box/coder-30b, lab/gemma-12b" + ], + "model": "box/coder-30b", + "expect": { + "refused": "Use another model for subagents only when the person names it." + } + }, + { + "about": "Named, but on the session's model's one-model server: refused — it would unload the lead's model.", + "op": "model", + "session": "lab/qwen-27b", + "models": { + "lab/qwen-27b": { + "id": "qwen-27b", + "name": "Qwen 27B", + "capacity": { + "group": "lab", + "parallel": 1 + } + }, + "lab/gemma-12b": { + "id": "gemma-12b", + "name": "Gemma 12B", + "capacity": { + "group": "lab", + "parallel": 1 + } + }, + "box/coder-30b": { + "id": "coder-30b", + "name": "Coder 30B", + "capacity": { + "parallel": 1 + } + }, + "box/coder-30b-instruct": { + "id": "coder-30b-instruct", + "name": "Coder 30B Instruct", + "capacity": { + "parallel": 2 + } + }, + "lab2/fast-8b": { + "id": "fast-8b", + "name": "Fast 8B", + "capacity": { + "group": "lab2", + "parallel": 4 + } + }, + "api/atlas-1": { + "id": "atlas-1", + "name": "Atlas 1", + "capacity": {} + } + }, + "said": [ + "use gemma-12b" + ], + "model": "lab/gemma-12b", + "expect": { + "refused": "That model is on the same one-model server as this chat's model; name a model on another server." + } + }, + { + "about": "Not named and on the same server: not_named is said first.", + "op": "model", + "session": "lab/qwen-27b", + "models": { + "lab/qwen-27b": { + "id": "qwen-27b", + "name": "Qwen 27B", + "capacity": { + "group": "lab", + "parallel": 1 + } + }, + "lab/gemma-12b": { + "id": "gemma-12b", + "name": "Gemma 12B", + "capacity": { + "group": "lab", + "parallel": 1 + } + }, + "box/coder-30b": { + "id": "coder-30b", + "name": "Coder 30B", + "capacity": { + "parallel": 1 + } + }, + "box/coder-30b-instruct": { + "id": "coder-30b-instruct", + "name": "Coder 30B Instruct", + "capacity": { + "parallel": 2 + } + }, + "lab2/fast-8b": { + "id": "fast-8b", + "name": "Fast 8B", + "capacity": { + "group": "lab2", + "parallel": 4 + } + }, + "api/atlas-1": { + "id": "atlas-1", + "name": "Atlas 1", + "capacity": {} + } + }, + "model": "lab/gemma-12b", + "expect": { + "refused": "Use another model for subagents only when the person names it." + } + }, + { + "about": "A one-model server's model is fine when the session's model is not on it.", + "op": "model", + "session": "api/atlas-1", + "models": { + "lab/qwen-27b": { + "id": "qwen-27b", + "name": "Qwen 27B", + "capacity": { + "group": "lab", + "parallel": 1 + } + }, + "lab/gemma-12b": { + "id": "gemma-12b", + "name": "Gemma 12B", + "capacity": { + "group": "lab", + "parallel": 1 + } + }, + "box/coder-30b": { + "id": "coder-30b", + "name": "Coder 30B", + "capacity": { + "parallel": 1 + } + }, + "box/coder-30b-instruct": { + "id": "coder-30b-instruct", + "name": "Coder 30B Instruct", + "capacity": { + "parallel": 2 + } + }, + "lab2/fast-8b": { + "id": "fast-8b", + "name": "Fast 8B", + "capacity": { + "group": "lab2", + "parallel": 4 + } + }, + "api/atlas-1": { + "id": "atlas-1", + "name": "Atlas 1", + "capacity": {} + } + }, + "said": [ + "Use Gemma 12B for that." + ], + "model": "lab/gemma-12b", + "expect": { + "model": "lab/gemma-12b" + } + }, + { + "about": "The session's own model given as `model` needs no naming.", + "op": "model", + "session": "lab2/fast-8b", + "models": { + "lab/qwen-27b": { + "id": "qwen-27b", + "name": "Qwen 27B", + "capacity": { + "group": "lab", + "parallel": 1 + } + }, + "lab/gemma-12b": { + "id": "gemma-12b", + "name": "Gemma 12B", + "capacity": { + "group": "lab", + "parallel": 1 + } + }, + "box/coder-30b": { + "id": "coder-30b", + "name": "Coder 30B", + "capacity": { + "parallel": 1 + } + }, + "box/coder-30b-instruct": { + "id": "coder-30b-instruct", + "name": "Coder 30B Instruct", + "capacity": { + "parallel": 2 + } + }, + "lab2/fast-8b": { + "id": "fast-8b", + "name": "Fast 8B", + "capacity": { + "group": "lab2", + "parallel": 4 + } + }, + "api/atlas-1": { + "id": "atlas-1", + "name": "Atlas 1", + "capacity": {} + } + }, + "model": "lab2/fast-8b", + "expect": { + "model": "lab2/fast-8b" + } + }, + { + "about": "The session's own model, named, still serves one request at a time: refused.", + "op": "model", + "session": "lab/qwen-27b", + "models": { + "lab/qwen-27b": { + "id": "qwen-27b", + "name": "Qwen 27B", + "capacity": { + "group": "lab", + "parallel": 1 + } + }, + "lab/gemma-12b": { + "id": "gemma-12b", + "name": "Gemma 12B", + "capacity": { + "group": "lab", + "parallel": 1 + } + }, + "box/coder-30b": { + "id": "coder-30b", + "name": "Coder 30B", + "capacity": { + "parallel": 1 + } + }, + "box/coder-30b-instruct": { + "id": "coder-30b-instruct", + "name": "Coder 30B Instruct", + "capacity": { + "parallel": 2 + } + }, + "lab2/fast-8b": { + "id": "fast-8b", + "name": "Fast 8B", + "capacity": { + "group": "lab2", + "parallel": 4 + } + }, + "api/atlas-1": { + "id": "atlas-1", + "name": "Atlas 1", + "capacity": {} + } + }, + "said": [ + "use qwen-27b for them" + ], + "model": "lab/qwen-27b", + "expect": { + "refused": "This model serves one request at a time: name another model, on another server, for subagents." + } + }, + { + "about": "A model with no limit serves its own subagents.", + "op": "offer", + "session": "api/atlas-1", + "models": { + "api/atlas-1": { + "id": "atlas-1", + "name": "Atlas 1", + "capacity": {} + } + }, + "expect": { + "offered": true, + "own": true, + "note": null + } + }, + { + "about": "A model that serves several at once, alone: offered.", + "op": "offer", + "session": "lab2/fast-8b", + "models": { + "lab2/fast-8b": { + "id": "fast-8b", + "name": "Fast 8B", + "capacity": { + "group": "lab2", + "parallel": 4 + } + } + }, + "expect": { + "offered": true, + "own": true, + "note": null + } + }, + { + "about": "A one-request model whose only other models are on its own one-model server: nothing is offered.", + "op": "offer", + "session": "lab/qwen-27b", + "models": { + "lab/qwen-27b": { + "id": "qwen-27b", + "name": "Qwen 27B", + "capacity": { + "group": "lab", + "parallel": 1 + } + }, + "lab/gemma-12b": { + "id": "gemma-12b", + "name": "Gemma 12B", + "capacity": { + "group": "lab", + "parallel": 1 + } + } + }, + "expect": { + "offered": false, + "own": false, + "note": null + } + }, + { + "about": "A one-request model with a model on another server: offered, and the note says the session's model cannot serve subagents itself.", + "op": "offer", + "session": "lab/qwen-27b", + "models": { + "lab/qwen-27b": { + "id": "qwen-27b", + "name": "Qwen 27B", + "capacity": { + "group": "lab", + "parallel": 1 + } + }, + "lab/gemma-12b": { + "id": "gemma-12b", + "name": "Gemma 12B", + "capacity": { + "group": "lab", + "parallel": 1 + } + }, + "box/coder-30b": { + "id": "coder-30b", + "name": "Coder 30B", + "capacity": { + "parallel": 1 + } + } + }, + "expect": { + "offered": true, + "own": false, + "note": "This model serves one request at a time, so subagents cannot run on it: they need another model, on another server, that the person names in this chat." + } + }, + { + "about": "Another server's model counts whatever its own capacity: one request at a time is enough.", + "op": "offer", + "session": "box/coder-30b", + "models": { + "box/coder-30b": { + "id": "coder-30b", + "name": "Coder 30B", + "capacity": { + "parallel": 1 + } + }, + "lab/qwen-27b": { + "id": "qwen-27b", + "name": "Qwen 27B", + "capacity": { + "group": "lab", + "parallel": 1 + } + } + }, + "expect": { + "offered": true, + "own": false, + "note": "This model serves one request at a time, so subagents cannot run on it: they need another model, on another server, that the person names in this chat." + } + }, + { + "about": "Subagents switched off: nothing is offered, whatever the models.", + "op": "offer", + "session": "api/atlas-1", + "models": { + "api/atlas-1": { + "id": "atlas-1", + "name": "Atlas 1", + "capacity": {} + }, + "lab2/fast-8b": { + "id": "fast-8b", + "name": "Fast 8B", + "capacity": { + "group": "lab2", + "parallel": 4 + } + } + }, + "subagents": false, + "expect": { + "offered": false, + "own": true, + "note": null + } + }, + { + "about": "A one-request model alone: nothing is offered.", + "op": "offer", + "session": "box/coder-30b", + "models": { + "box/coder-30b": { + "id": "coder-30b", + "name": "Coder 30B", + "capacity": { + "parallel": 1 + } + } + }, + "expect": { + "offered": false, + "own": false, + "note": null + } + } + ] +} diff --git a/harness/conformance/team.json b/harness/conformance/team.json index 052b666..dbd42ca 100644 --- a/harness/conformance/team.json +++ b/harness/conformance/team.json @@ -1,5 +1,5 @@ { - "about": "Teams (team.md, spec 3.2.0): one lead session's team, step by step, written from a reference model of the rules. `start`: the size setting `min` and `max`, `subagents` (enabled for the session; default true) and the chat model's `capacity` (models/schema.json: `parallel`, `single_session`; {} — no limit). The lead is `lead`; its first prompt is running as the case begins, and every agent is on the chat's model. Steps and what each answers: `offered` (`by` an agent, default lead) — `{tools, setting}`: the team tools listed for that agent, sorted, and whether the setting is shown; `call` (`by`, `tool`, `args`) — `{result, delivered}`: the call's answer word for word (null where an ordinary tool would run: the case looks only at whether it is refused), and what was delivered to whom by this step, `{name: [message, …]}`; `team_read` answers `{room, delivered}` instead, `room` being the messages it shows as `{n, from, to}` (the tree and the text format are loop.json team.read's, checked by each project's own tests); `task` runs a helper that counts toward the size (and holds a slot) until `helper_done` (`by`) — answers null; `finish` (`by`, `reply`, `reason`: stop by default, or steps, budget, `error — `) — the agent's turn ends: a member's reply is its report — `{delivered}`; `set` (`min`, `max`, `capacity`: the person's setting or the model's metadata changed) — null; `person_stop` (`name`) — the person stopped that member in the UI — null; `user` — the person's message to the lead — `{note}`: the note on it (context.md §10), null when there is none; `state` — the team in tree order as `{name, manager, state}`: working, waiting (for a slot), finished, idle (the lead between turns). Delivering to a finished member or the idle lead wakes it (it is then working, or waiting when its model is full). LLeMbas: services/team.py; CLI: team/.", + "about": "Teams (team.md, spec 3.2.0): one lead session's team, step by step, written from a reference model of the rules. `start`: the size setting `min` and `max`, `subagents` (enabled for the session; default true) and the chat model's `capacity` (models/schema.json: `parallel`, `single_session`; {} — no limit). The lead is `lead`; its first prompt is running as the case begins, and every agent is on the chat's model, and it is the only model there is (spec 3.3.0: other models are subagents.json's). Steps and what each answers: `offered` (`by` an agent, default lead) — `{tools, setting}`: the team tools listed for that agent, sorted, and whether the setting is shown; `call` (`by`, `tool`, `args`) — `{result, delivered}`: the call's answer word for word (null where an ordinary tool would run: the case looks only at whether it is refused), and what was delivered to whom by this step, `{name: [message, …]}`; `team_read` answers `{room, delivered}` instead, `room` being the messages it shows as `{n, from, to}` (the tree and the text format are loop.json team.read's, checked by each project's own tests); `task` runs a subagent that counts toward the size (and holds a slot) until `task_done` (`by`) — answers null; `finish` (`by`, `reply`, `reason`: stop by default, or steps, budget, `error — `) — the agent's turn ends: a member's reply is its report — `{delivered}`; `set` (`min`, `max`, `capacity`: the person's setting or the model's metadata changed) — null; `person_stop` (`name`) — the person stopped that member in the UI — null; `user` — the person's message to the lead — `{note}`: the note on it (context.md §10), null when there is none; `state` — the team in tree order as `{name, manager, state}`: working, waiting (for a slot), finished, idle (the lead between turns). Delivering to a finished member or the idle lead wakes it (it is then working, or waiting when its model is full). LLeMbas: services/team.py; CLI: team/.", "cases": [ { "about": "Offered on a model that serves two requests at once: the lead has all four tools and the setting is shown.", @@ -52,7 +52,7 @@ ] }, { - "about": "A model that serves one request at a time offers no team: no tools, no setting.", + "about": "A model that serves one request at a time, with no other model, offers no team: no tools, no setting.", "start": { "min": 1, "max": 4, @@ -727,7 +727,7 @@ ] }, { - "about": "A task helper counts toward max while it runs, inside a team.", + "about": "A task subagent counts toward max while it runs, inside a team.", "start": { "min": 1, "max": 3, @@ -772,7 +772,7 @@ } }, { - "op": "helper_done", + "op": "task_done", "by": "lead" }, { diff --git a/harness/loop.json b/harness/loop.json index a39721f..0acd674 100644 --- a/harness/loop.json +++ b/harness/loop.json @@ -34,7 +34,7 @@ "about": "Calls in one step run together, at most 4; a call that waits on the user (ask_user, plan_submit) runs alone; two calls on one file run in order. A tool's error is its result, never the end of the prompt.", "max": 4 }, - "capacity": "A subagent (task) is not started on a model whose server cannot hold it beside the prompt's own model — or beside the model of a subagent already running: another model of a connection that holds one model at a time, or the same model when it already serves as many of this session's requests as it can at once (models/schema.json capacity: `parallel`, `single_session` being parallel 1). A team's members are not refused there: they wait for a slot (team.md, Capacity).", + "capacity": "A subagent (task) is not started on a model whose server cannot hold it beside the prompt's own model — or beside the model of a subagent already running: another model of a connection that holds one model at a time, or the same model when it already serves as many of this session's requests as it can at once (models/schema.json capacity: `parallel`, `single_session` being parallel 1). Spec 3.3.0: which model a subagent may run on at all, and the refusals for the session's own one-model server and a one-request model, are subagents.md and `subagents` below, checked first. A team's members are not refused for a full model: they wait for a slot (team.md, Capacity).", "jobs": { "about": "Background work owned by a session: a command (bash `background: true`, or one still running when its timeout passed) or a subagent (task `background: true`). One id sequence per session (job1, job2, …), one list (job_list), one way to read (job_output), to stop (job_stop) and, for a subagent, to message (job_send); a subagent's own tools see only the jobs it started. A job that ends wakes the session that owns it: while a prompt runs, as a user message at the next step boundary; otherwise as a new turn of its own, marked `machine` so a UI shows it as the harness's, not the person's. A session that is not open keeps the note until it is resumed. Every such message begins with its mark, said rather than hidden: a model refuses an unmarked one as an injection (conformance/jobs.json gives the texts).", "marks": { @@ -61,8 +61,28 @@ }, "lifecycle": "A subagent's own background commands stop when it ends, and its report says so. A session's jobs stop when the session is deleted. A front end asked to exit while jobs run asks the person first; one with nobody to ask waits for the jobs the prompt started, then stops the rest." }, + "subagents": { + "about": "Spec 3.3.0 (subagents.md): which model a subagent — a task subagent or a team member — runs on, and when subagents are offered. The session's own model unless the person named another in this session; no setting chooses one. The texts are said word for word, checked in this order: not_named, same_group, one_request. conformance/subagents.json.", + "texts": { + "not_named": "Use another model for subagents only when the person names it.", + "same_group": "That model is on the same one-model server as this chat's model; name a model on another server.", + "one_request": "This model serves one request at a time: name another model, on another server, for subagents." + }, + "named": { + "about": "A model is named when its served id, its id at its connection or its display name occurs as a whole word, without case, in a message the person typed in this session (never model text, tool output, files, notes, marked messages or reports; a compacted message still counts). Before the name: not a letter, digit, _, -, /, . or :. After it: not a letter, digit, _, - or /, nor a . or : followed by a letter or digit. White space in a display name matches any run of white space.", + "before": "[\\p{L}\\p{N}_\\-/.:]", + "after": "[\\p{L}\\p{N}_\\-/]|[.:][\\p{L}\\p{N}]" + }, + "setting": { + "about": "How the agents {min, max} setting (team.md §2) is shown to a person: label, the names of min and max, and `own` beside it when the session's model cannot serve subagents itself (subagents.md §4).", + "label": "Subagents", + "min": "fewest", + "max": "most", + "own": "This model serves one request at a time: subagents need another model, on another server, named in the chat." + } + }, "team": { - "about": "Teams (spec 3.2.0, team.md): a lead and the members it adds, who talk in the team's room. The texts are said word for word; {{size}} is the team's size after the call (team.md §3), {{names}} names joined with \", \" in tree order. conformance/team.json.", + "about": "Teams (spec 3.2.0, team.md; which model a member runs on and when a team is offered: spec 3.3.0, subagents.md): a lead and the members it adds, who talk in the team's room. The texts are said word for word; {{size}} is the team's size after the call (team.md §3), {{names}} names joined with \", \" in tree order. conformance/team.json.", "agents": { "about": "The chat setting agents {min, max}: the agents a team may have, the lead included. 1 ≤ min ≤ max ≤ cap. min and max are a new chat's defaults; cap is the instance's (LLeMbas) or the global configuration's (the CLI).", "min": 1, @@ -83,7 +103,7 @@ "report": "[Team report from {{from}} to {{to}}:]" }, "unavailable": { - "about": "What a team tool answers while its requirement `team` is not met (team.md §2). one_agent is also each team tool's own `unavailable`, and what task answers while a team is offered, max is 1 and the lead has no member.", + "about": "What a team tool answers while its requirement `team` is not met (team.md §2). one_agent is also each team tool's own `unavailable`, and what task answers while a team is offered, max is 1 and the lead has no member. one_request: no model can serve as a subagent any more (subagents.md §4) — the session's model's capacity changed after the prompt was made.", "one_agent": "The person allows one agent in this chat: there is no team. Do the work yourself.", "one_request": "This model serves one request at a time now, so there is no team. Do the work yourself." }, @@ -203,7 +223,7 @@ "started": "Started in: {{path}} (relative to the project root)" }, "note": { - "about": "A note on the next user message (context.md §10, rule 3): what changed since the system prompt was made, after the person's words. Lines in this order, each only when it applies. `plan_ended` is the mode block for a session that leaves plan mode for a mode with no text of its own (a chat). `plan_own` is the plan block for a session that does not resend earlier tool results (a chat): the model's own plan, as its tools left it. `personal` is the {{path}} of the instructions block for the person's own instructions. Spec 3.2.0, where a team is offered (team.md §10): `team` — the size setting, on the first user message and after it changes, followed in the same block by `team_lead` when min ≥ 2 ({{members}} = min − 1), or `team_one` in its place when max is 1; `team_stopped` — members the person stopped, a block of its own.", + "about": "A note on the next user message (context.md §10, rule 3): what changed since the system prompt was made, after the person's words. Lines in this order, each only when it applies. `plan_ended` is the mode block for a session that leaves plan mode for a mode with no text of its own (a chat). `plan_own` is the plan block for a session that does not resend earlier tool results (a chat): the model's own plan, as its tools left it. `personal` is the {{path}} of the instructions block for the person's own instructions. Spec 3.2.0, where a team is offered (team.md §10): `team` — the size setting, on the first user message and after it changes, followed in the same block by `team_lead` when min ≥ 2 ({{members}} = min − 1), or `team_one` in its place when max is 1; `team_stopped` — members the person stopped, a block of its own. Spec 3.3.0 (subagents.md §4): `team_own` after `team` (and after `team_lead`) in the same block when the session's model cannot serve subagents itself — it serves one request at a time — and another server's model could.", "open": "", "close": "", "date": "Today is {{date}}.", @@ -219,6 +239,7 @@ "team": "The team may have {{min}} to {{max}} agents, you included.", "team_lead": "Work through a team from the start: you lead, and members do the work — at least {{members}} of them.", "team_one": "The person allows one agent: work alone; the team tools and task are unavailable.", + "team_own": "This model serves one request at a time, so subagents cannot run on it: they need another model, on another server, that the person names in this chat.", "team_stopped": "The person stopped {{names}}; they have left the team.", "empty": "(none)" }, diff --git a/harness/models/schema.json b/harness/models/schema.json index 49747c7..d32b6f7 100644 --- a/harness/models/schema.json +++ b/harness/models/schema.json @@ -21,11 +21,11 @@ "notes": { "type": "string", "maxLength": 300, "description": "What the model is good at, in a line: other models are told about it in the roster, for handing work over." }, "capacity": { "type": "object", - "description": "How much the model's server can do at once, so a helper or a subagent on it is not started where it would evict or wait for the model already working.", + "description": "How much the model's server can do at once, so a subagent on it is not started where it would evict or wait for the model already working (subagents.md §3).", "properties": { - "group": { "type": "string", "description": "Models that share one server holding one model at a time (llama-swap): starting another model of the group unloads this one. The CLI: a model's `group`, or its connection when the connection sets one_model_at_a_time. LLeMbas: the connection's name when it sets one_model_at_a_time, which is what /v1/models says." }, + "group": { "type": "string", "description": "Models that share one server holding one model at a time (llama-swap): starting another model of the group unloads this one, so a subagent on another model of the session's model's group is refused (spec 3.3.0, subagents.md §3). The CLI: a model's `group`, or its connection when the connection sets one_model_at_a_time. LLeMbas: the connection's name when it sets one_model_at_a_time, which is what /v1/models says." }, "single_session": { "type": "boolean", "description": "The model serves one request at a time: a second one waits behind the first. The same as `parallel: 1`, which a server that knows `parallel` sends beside it for older clients." }, - "parallel": { "type": "integer", "minimum": 1, "description": "Spec 3.2.0: how many requests the model's server serves at once (llama.cpp's slots, a vLLM or SGLang batch); one more waits behind them. Read as: `parallel` when given, else 1 when single_session is true, else no limit (a hosted API). Where both are given and disagree, `parallel` wins. A team (team.md) is offered only on a model that serves at least 2." } + "parallel": { "type": "integer", "minimum": 1, "description": "Spec 3.2.0: how many requests the model's server serves at once (llama.cpp's slots, a vLLM or SGLang batch); one more waits behind them. Read as: `parallel` when given, else 1 when single_session is true, else no limit (a hosted API). Where both are given and disagree, `parallel` wins. The session's model is its own subagents' model only when it serves at least 2; a one-request session needs a named model on another server (spec 3.3.0, subagents.md §3, §4)." } }, "additionalProperties": false } diff --git a/harness/prompts/blocks/subagent.md b/harness/prompts/blocks/subagent.md index 5214ca0..ec228ea 100644 --- a/harness/prompts/blocks/subagent.md +++ b/harness/prompts/blocks/subagent.md @@ -1,5 +1,5 @@ + the mode text; LLeMbas's subagent says the same. The variables are what differs per run. --> You are the "{{name}}" subagent. You work for the main agent, not for the user directly, on one task it gave you; you cannot ask the user anything. {{instructions}} Your final reply is your report to the main agent — make it complete on its own: the result, then the evidence (file:line), then anything you could not settle.{{report}}{{read_only}} diff --git a/harness/subagents.md b/harness/subagents.md new file mode 100644 index 0000000..de00d80 --- /dev/null +++ b/harness/subagents.md @@ -0,0 +1,87 @@ +# Subagents + +Spec 3.3.0. **Which model a subagent runs on, and when subagents are offered at all.** A +**subagent** is any agent a session's agent starts to work for it: a `task` subagent (in the +foreground or as a background job) and a team's member (`team.md`) alike. The same rules in a chat +on the server and in an agent session on a device, in both projects. The texts are `loop.json` +`subagents`; the cases `conformance/subagents.json`. + +## 1. The word + +**Subagent** is the word in every text a person or a model reads: the UI (the web UI, Desktop, +Android, the CLI, the admin pages), the prompt texts and the tool texts. A team's members are +subagents too. The word *helper* is retired. Tool names, their former names (`aliases`) and +stored identifiers do not change. + +## 2. The model a subagent runs on + +**The session's own model**, always — unless **the person has named another model** in this +session. Nothing else chooses one: there is **no setting** for a subagent's model anywhere — not +in a chat, not in a person's or an instance's settings, not in the configuration, and an agent +definition's own `model` (the CLI's agent files) is not used for a subagent either. + +- **Named** means: in a message **the person typed** in this session, the model's served id, its + id at its connection, or its display name occurs as a whole word, letters compared without case. + Only the person's own words count — never the model's text, a tool's output, a file, a note, a + job's or a team's marked message, or a subagent's report. A message that was compacted away + still counts: what the person named stays named for the whole session. +- **A whole word**: the character before the name is not a letter, a digit, `_`, `-`, `/`, `.` or + `:`; the character after it is not a letter, a digit, `_`, `-` or `/`, nor a `.` or `:` followed + by a letter or a digit. So `coder-30b.` at the end of a sentence names `coder-30b`, and + `coder-30b-instruct` does not. Runs of white space in a display name match any run of white + space. +- The `model` parameter of `task` and of `team_add` takes a model the person may use, by the same + names. When it is **not the session's model and the person has not named it**, the call is + refused, word for word, with `subagents.texts.not_named`: + + > Use another model for subagents only when the person names it. + + The session's own model given as `model` is the default spelled out, and is never refused for + not being named. +- A team's members are of the lead's session: what the person named there counts for every + `team_add` in the team, a sub-manager's included. A member's default is the session's model, not + its manager's. + +## 3. Capacity: the logical restrictions, and no others + +What a model's server can do at once is `models/schema.json` `capacity`: `parallel` (`single_session` +being `parallel: 1`; neither said — no limit), and `group`, the server that holds one model at a +time. With **S** the session's model and **M** the subagent's, checked in this order, after +`not_named`: + +1. **M is another model on S's one-model server** (both have the same `group`): refused with + `subagents.texts.same_group` — starting it would unload the lead's model: + + > That model is on the same one-model server as this chat's model; name a model on another server. + +2. **M is S, and S serves one request at a time** (`parallel` 1): refused with + `subagents.texts.one_request` — the subagent would take the only slot from the lead: + + > This model serves one request at a time: name another model, on another server, for subagents. + + There is **no fallback**: nothing else is picked for the person, ever. A model the person named, + on another server, is the way out — and a model that serves one request at a time is fine there. + +These apply to `task` and to `team_add` alike, and before anything is started. Beyond them nothing +is refused for being the wrong model: a member past the free slots waits for one (`team.md` §8), +and `task` keeps its rule for a model that is already full (`loop.json` `capacity`). + +## 4. When subagents are offered + +The size setting `agents {min, max}` (`team.md` §2) is shown as **Subagents** — `subagents.setting`: +the label, and *fewest* and *most* for `min` and `max` (defaults 1 and 4, at most `cap` 32, kept +with the chat). It, the team tools and `blocks/team_lead.md` are **offered** when both hold: + +1. Subagents are enabled for the session (`team.md` §2, rule 1). +2. **At least one model could serve as a subagent**: S itself, when it serves two or more requests + at once (`parallel ≥ 2`, or no limit); or **any other model the person may use that is not on + S's one-model server** (no `group`, or another one) — whatever its own `parallel`. + +So a session on a one-request model is offered subagents when another server has a model, and +**its first note says so** (`context.note.team_own`, after `team`): the session's model cannot +serve subagents itself, and they need a model the person names. A UI showing the setting says the +same beside it (`subagents.setting.own`). Nothing is offered when neither holds — the same as +before 3.3.0 for a single model. + +What counts as **usable** is each project's: LLeMbas — the models the person may use in the chat's +picker; the CLI — the models of the server it is logged in to. diff --git a/harness/team.md b/harness/team.md index 7a4f5d8..bf313f1 100644 --- a/harness/team.md +++ b/harness/team.md @@ -1,6 +1,6 @@ # Teams -Spec 3.2.0. **Subagents that work together.** A session's agent — the **lead** — may add +Spec 3.2.0; the model a member runs on and when a team is offered, 3.3.0 (`subagents.md`). **Subagents that work together.** A session's agent — the **lead** — may add **members**, background agents that each own a part of the work, and lead them as a manager: it plans, splits the work, coordinates and reviews, and the members do it. Members talk to each other and to their managers in the team's **room**. A member may itself lead an area with members of its @@ -46,23 +46,27 @@ system prompt — when both hold, and only then: 1. Subagents are enabled for the session: LLeMbas — the instance allows them and the chat's model has its subagent tool; the CLI — its subagents are not switched off in the configuration. -2. **The chat's model serves at least two requests at once**: `capacity.parallel ≥ 2`, or neither - `parallel` nor `single_session` is set (a hosted API; `models/schema.json`). +2. **At least one model could serve as a subagent** (spec 3.3.0, `subagents.md` §4): the chat's + model, when it serves at least two requests at once (`capacity.parallel ≥ 2`, or neither + `parallel` nor `single_session` is set — a hosted API; `models/schema.json`); or any other model + the person may use that is not on the chat's model's one-model server (`capacity.group`). + +The setting is shown as **Subagents**, *fewest* to *most* (`loop.json` `subagents.setting`). Both are fixed for as long as the system prompt is (`context.md` §10, rule 4: a model switch makes a new prompt and a new list). Otherwise the setting is not shown and none of the four tools is listed, and the session's `task` works as before. -While the tools are listed they need **`team`** (`requires`): met when `max ≥ 2` and the model still -serves two at once — or, for the lead, when its team still has a member (so a lead whose person +While the tools are listed they need **`team`** (`requires`): met when `max ≥ 2` and a model could +still serve as a subagent (rule 2) — or, for the lead, when its team still has a member (so a lead whose person lowered `max` to 1 can still talk to, read and stop the members it has). A call while it is not -met answers, word for word, `team.unavailable.one_request` when the model's capacity no longer -allows a team (its metadata changed after the prompt was made), else `team.unavailable.one_agent` +met answers, word for word, `team.unavailable.one_request` when no model could serve as a subagent +any more (the model's metadata changed after the prompt was made), else `team.unavailable.one_agent` — which is also the tools' own `unavailable` text. The meaning of the setting: -- **`max = 1`**: one agent, no team, and no helper of any kind: `task` answers `one_agent` too +- **`max = 1`**: one agent, no team, and no subagent of any kind: `task` answers `one_agent` too (while the lead has no member; with members left from a larger setting, `full`). - **`min ≥ 2`**: the lead works through a team from the start — it is a manager before its first member (§6), and the note asks for at least `min − 1` members. @@ -72,14 +76,14 @@ The meaning of the setting: ## 3. Size, and what counts **The size** of a team is: the lead, every member (whatever its state — a finished member can be -woken by a message, so it keeps its place), and every `task` helper running in the team, started by +woken by a message, so it keeps its place), and every `task` subagent running in the team, started by any agent in it (the CLI's background `task` jobs and foreground ones alike, while they run). A stopped member does not count: stopping is how room is made. - **`team_add` is refused past `max`** — when the size would become more than `max`, with `team.texts.full`. So is `task`, while a team is offered: with `full`, or with `one_agent` when `max` is 1 and the lead has no member. A `job_send` that would start a finished background - `task` helper again is refused the same way. + `task` subagent again is refused the same way. - **Lowering `max` stops nobody.** Members already there stay; nothing new is added until the size is under the new `max`. - The size is across **the whole tree**: a member that leads adds within the same `max` as the lead. @@ -90,8 +94,9 @@ A stopped member does not count: stopping is how room is made. machinery (a hidden chat on LLeMbas with its `chat_jobs` row, a subagent job in the CLI), running unattended — no `ask_user`, the chat's own permission ceiling, its mode no looser than its manager's (`permission/modes.json`). -- **Its agent** is `general` (the default: it does the work) or `explore` (read-only); `model` is - as `task`'s — a model the person may use, the default being the caller's. +- **Its agent** is `general` (the default: it does the work) or `explore` (read-only). **Its model** + is the session's (the lead's) unless `model` gives one the person named in this session — as + `task`'s, refused with `subagents.texts` otherwise (`subagents.md` §2, §3). - **Its tools** are those its agent gives a subagent, plus `team_send` and `team_read`; one added with `lead: true` also has `team_add` and `team_stop`. They are fixed when it is made (§10 rule 4: a subagent's narrowing). `report` (`offered: subagent`) sends its manager a message before its @@ -108,7 +113,8 @@ A stopped member does not count: stopping is how room is made. `unavailable` (§2); `all_name` (the name `all`); `bad_name` (not `team.names.pattern`); `taken` (the name of an agent on the team, `lead` included); `no_goal`; `full`. Then the member is added: `added`, or `added_waiting` when it has to wait for a slot (§8). An unknown `agent` or `model` - answers as `task` does. + answers as `task` does; then `model`'s refusals, `not_named`, `same_group`, `one_request` + (`subagents.md`), in that order — before anything is added. - **Members are not jobs to the model.** `job_list` does not show them and the `job_*` tools do not take them; the team tools do. (An implementation may run them on its job machinery.) - **States**: `working` (a turn is running), `waiting` (it has work, and waits for a slot, §8), @@ -171,7 +177,7 @@ class. **The tool list does not change** when an agent becomes a manager (`context.md` §10, rule 4): the same tools are listed and the refused ones answer the refusal. An agent is a manager while it has a member — a finished one included; one whose members were all stopped is not, unless it is the -lead and `min ≥ 2` with the team offered. `task` stays: a one-off helper is delegation, and counts +lead and `min ≥ 2` with the team offered. `task` stays: a one-off subagent is delegation, and counts toward the size (§3). ## 7. Delivery, and waking @@ -180,7 +186,7 @@ A message or a report reaches its recipient **as a marked message** — the jobs (`loop.json` `jobs`): while the recipient's turn runs, as a user message at its next step boundary (which does not start a turn: `context.md` §8); otherwise it **wakes** the recipient — a finished member, or the idle lead — as a new turn of its own, marked `machine`, with its context intact, as -`job_send` restarts a finished helper. A woken agent needs a slot (§8). Several waiting at one step +`job_send` restarts a finished subagent. A woken agent needs a slot (§8). Several waiting at one step boundary go in one user message, in room order, separated by a blank line. Every one begins with its mark, on a line of its own, then the text: @@ -199,14 +205,16 @@ A session that is not open keeps them until it is resumed, as a job's note. ## 8. Capacity: a member waits for a slot A model serves `parallel` requests at once (`models/schema.json`). **An agent holds a slot of its -model while its turn runs** — the lead, a member, a `task` helper. An agent with work (added, woken, +model while its turn runs** — the lead, a member, a `task` subagent. An agent with work (added, woken, given a message while finished, the lead given the person's message) starts when its model has a free slot and none of its `capacity.group`'s other models is busy; otherwise it is `waiting`. -`team_add` is never refused for capacity (only for size): its answer says the member waits. +`team_add` is never refused for a full model (only for size, and for the model rules of +`subagents.md` §3: another model on the lead's one-model server, or a one-request session model): +its answer says the member waits. Waiting agents start **in order**: the lead first (the person waits on it), then the others in the order they began waiting. A slot frees when a turn ends or an agent is stopped. `task` keeps its own -rule (`loop.json` `capacity`: refused where its model is full), and its helper holds a slot. +rule (`loop.json` `capacity`: refused where its model is full), and its subagent holds a slot. ## 9. Stopping diff --git a/harness/tools/task.json b/harness/tools/task.json index a7e1549..2aec69a 100644 --- a/harness/tools/task.json +++ b/harness/tools/task.json @@ -12,7 +12,7 @@ "subagent_run" ] }, - "description": "Hand a self-contained piece of work to a subagent, which works on its own and returns only its final report — keeping the searching and reading out of your context.\n\n- Use it for broad research (\"where is X handled, and what calls it?\") or a separable job. For one file you already know, just read it.\n- `prompt` must stand on its own: the subagent sees nothing of this conversation. Say what to find or do, and what to report back.\n- `background: true` returns at once with a job id and you carry on; the report comes to you as a message when it ends. Meanwhile job_output shows its progress, job_send gives it a message, job_stop stops it. Without it, the call waits for the report.\n- Independent tasks can run in the same turn. Subagents that change files at the same time should each have `worktree: true`, so they work on separate branches.\n- Agents: {{agents}}", + "description": "Hand a self-contained piece of work to a subagent, which works on its own and returns only its final report — keeping the searching and reading out of your context.\n\n- Use it for broad research (\"where is X handled, and what calls it?\") or a separable job. For one file you already know, just read it.\n- `prompt` must stand on its own: the subagent sees nothing of this conversation. Say what to find or do, and what to report back.\n- `background: true` returns at once with a job id and you carry on; the report comes to you as a message when it ends. Meanwhile job_output shows its progress, job_send gives it a message, job_stop stops it. Without it, the call waits for the report.\n- Independent tasks can run in the same turn. Subagents that change files at the same time should each have `worktree: true`, so they work on separate branches.\n- It runs on your model. Another model only when the person named it in this conversation; nothing else chooses one.\n- Agents: {{agents}}", "parameters": { "type": "object", "properties": { @@ -38,7 +38,7 @@ "type": "boolean" }, "model": { - "description": "Run it on another configured model (connection/model, from the list in your instructions) — a second opinion, or a model better at this kind of work. Default: the agent's own model, else yours.", + "description": "Run it on another model — only one the person named in this conversation (its id or name, as in the list in your instructions). Default: your model.", "type": "string" } }, diff --git a/harness/tools/team_add.json b/harness/tools/team_add.json index c880d10..cd73524 100644 --- a/harness/tools/team_add.json +++ b/harness/tools/team_add.json @@ -47,7 +47,7 @@ }, "model": { "type": "string", - "description": "Run it on another model you may use (from the list in your instructions). Default: yours." + "description": "Run it on another model — only one the person named in this conversation (its id or name, as in the list in your instructions). Default: the session's model." } }, "required": [ diff --git a/tests/conformance.ts b/tests/conformance.ts index 067ad0d..d058ce6 100644 --- a/tests/conformance.ts +++ b/tests/conformance.ts @@ -272,7 +272,7 @@ export const AREAS: Record unknown> = { case "finish": team.finish(by, st.reply ?? "", st.reason ?? "stop") return { delivered: delivered() } - case "helper_done": + case "task_done": team.helperDoneBy(by) return null case "set":