Custom connections are gone: the CLI talks to the LLeMbas server it is logged in to

LLeMbas CLI 2.0 is LLeMbas-server-specific (CLI rule 2). Removed: the
anthropic, gemini, ollama and responses clients, context discovery and
llama-swap unloading (discover.ts), the Connection schema with its dialect,
auth, key_cmd, discover, unload_url and one_model_at_a_time, the model
`fallback` (it only ever moved to another server), effort_map and cache, and
the multi-connection refs. Kept: openai-chat, http, sse, think, effort,
tokens and all of src/lembas.

- connections.yaml holds the login (`type: webui`, as lembas login writes
  it). A custom entry from 1.x is read past with a warning that says how to
  log in, so an old file still starts; with several logins left, the latest
  is used and the others named.
- A model's ref is the id the server serves it under; the older forms
  (`<login>/<id>`, `<login>/<model>`) still resolve and config.yaml is
  rewritten once. Without a login there is no model, and the error (and the
  first-run screen, now the sign-in alone) says `lembas login <address>`.
- A login to another server signs this machine out of the previous one.
- The window is the server's word (no discovery); a capacity group comes
  from the server's `capacity.group` alone; embeddings use the server's
  /v1/embeddings.
- `lembas models` lists the server's models; install.sh's starter files, the
  README and the JSON schemas say the same.

Tests: every fixture is a login now (the fake server lists models, answers
other GETs with 404, and can hand one model to another script, as a server
with several backends); the dialect suites and the cross-connection tests are
gone with their code.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
HomerandClaude Opus 5.5 committed 2026-10-27 21:00:05 +00:00
1 parent f20e8c4307
commit 28ff205ed0
80 files changed
+388 -2376

No files matched your search

+9 -10
View File
@@ -4,12 +4,11 @@
# LLeMbas CLI # LLeMbas CLI
A terminal coding agent and project manager: the `lembas` command. It talks to any LLM API A terminal coding agent and project manager: the `lembas` command. It signs in to a
(OpenAI-compatible servers such as llama.cpp, vLLM or LM Studio, the OpenAI Responses API, [LLeMbas](https://github.com/LLeMbas/LLeMbas) server and uses its models, memory and search, works
Anthropic, Gemini, Ollama), works in your project with four permission modes and a git snapshot of in your project with four permission modes and a git snapshot of every turn, keeps its own notes,
every turn, remembers what it learns across sessions, and can sign in to a knowledge bases and skills on your machine, and lets the server's web UI work the same sessions.
[LLeMbas](https://github.com/LLeMbas/LLeMbas) server to use its models, library and search — and Linux, x64 and arm64.
let its web UI work the same sessions. Linux, x64 and arm64.
## Install ## Install
@@ -24,12 +23,12 @@ checksum before installing anything ([SECURITY.md](SECURITY.md) shows how to che
## First steps ## First steps
```sh ```sh
cd your-project && lembas # the TUI lembas login https://your-server # sign in to your LLeMbas server
lembas login https://your-server # sign in to a LLeMbas server and use its models cd your-project && lembas # the TUI, on the server's models
``` ```
Without a server, add a connection to `~/.config/lembas/connections.yaml` and set `model:` in From 2.0 the CLI talks only to the LLeMbas server it is logged in to; the custom connections of 1.x
`~/.config/lembas/config.yaml`. (`dialect:` entries in `connections.yaml`) are gone.
`lembas` updates itself (`lembas update` does it now); `lembas uninstall` removes it, and `lembas` updates itself (`lembas update` does it now); `lembas uninstall` removes it, and
`--purge` takes settings and sessions too. `--purge` takes settings and sessions too.
+7 -19
View File
@@ -221,8 +221,9 @@ if [ "$starter" -eq 1 ] && [ ! -e "$CONFIG/config.yaml" ]; then
# The full list: https://llembas.eu/docs/. An editor with a YAML language server # The full list: https://llembas.eu/docs/. An editor with a YAML language server
# completes and checks keys as you type (the first line points it at the schema). # completes and checks keys as you type (the first line points it at the schema).
# The model to start with, as connection/model from connections.yaml. # The model to start with, by its id on the LLeMbas server you log in to (lembas models lists
# model: local/qwen3 # them). Unset: the server's default for your account.
# model: deepseek/deepseek-flash
# manual (asks before changes) | edit (changes files in the project freely) | plan # manual (asks before changes) | edit (changes files in the project freely) | plan
mode: manual mode: manual
@@ -237,27 +238,14 @@ fi
if [ "$starter" -eq 1 ] && [ ! -e "$CONFIG/connections.yaml" ]; then if [ "$starter" -eq 1 ] && [ ! -e "$CONFIG/connections.yaml" ]; then
(umask 077 && cat >"$CONFIG/connections.yaml" <<'YAML' (umask 077 && cat >"$CONFIG/connections.yaml" <<'YAML'
# yaml-language-server: $schema=schema/connections.schema.json # yaml-language-server: $schema=schema/connections.schema.json
# Where the models are. Global only: a project can never point LLeMbas CLI elsewhere. # The LLeMbas server this machine is logged in to. Global only: a project can never point
# Keys are never written here in plain text: {env:VAR}, {file:~/path}, or key_cmd. # LLeMbas CLI elsewhere. Written by `lembas login <address>`, not by hand; the models come from
# Then set `model: <connection>/<model>` in config.yaml, and check with `lembas models`. # the server at every start (`lembas models` lists them).
connections: {} connections: {}
#
# connections:
# local: # llama.cpp / llama-swap / vLLM / LM Studio — OpenAI-compatible
# dialect: openai-chat
# base_url: http://127.0.0.1:8080/v1
# models:
# qwen3: { context: 131072, max_output: 65536 }
# anthropic:
# dialect: anthropic
# base_url: https://api.anthropic.com
# api_key: "{env:ANTHROPIC_API_KEY}"
# models:
# claude-sonnet-5: { context: 200000 }
YAML YAML
) )
chmod 600 "$CONFIG/connections.yaml" chmod 600 "$CONFIG/connections.yaml"
say " wrote $CONFIG/connections.yaml (0600) — add a connection there" say " wrote $CONFIG/connections.yaml (0600) — lembas login <address> fills it in"
fi fi
# The schemas for editors follow the binary, so they are rewritten on every install. # The schemas for editors follow the binary, so they are rewritten on every install.
"$TARGET" config schema >/dev/null && say " wrote $CONFIG/schema/ (JSON Schemas for editors)" "$TARGET" config schema >/dev/null && say " wrote $CONFIG/schema/ (JSON Schemas for editors)"
+1 -1
View File
@@ -1,7 +1,7 @@
{ {
"name": "lembas-cli", "name": "lembas-cli",
"version": "1.5.2", "version": "1.5.2",
"description": "LLeMbas CLI: a terminal coding agent and project manager — OpenTUI, any LLM API, memory, skills, MCP, git-native.", "description": "LLeMbas CLI: a terminal coding agent and project manager for a LLeMbas server — OpenTUI, memory, skills, MCP, git-native.",
"license": "MIT", "license": "MIT",
"private": true, "private": true,
"type": "module", "type": "module",
+3 -3
View File
@@ -3,11 +3,11 @@
"type": "object", "type": "object",
"properties": { "properties": {
"model": { "model": {
"description": "The model a session starts with: `connection/model`, a connection from connections.yaml and one of its models.", "description": "The model a session starts with: one the LLeMbas server you are logged in to serves, by its id there (`deepseek/deepseek-flash`). Default: the server's default for your account.",
"type": "string" "type": "string"
}, },
"small_model": { "small_model": {
"description": "`connection/model` for small jobs — session titles, commit messages. Default: the session's model. On a server that holds one model at a time (llama-swap), another model here swaps.", "description": "The model for small jobs — session titles, commit messages, by its id on the server. Default: the session's model.",
"type": "string" "type": "string"
}, },
"titles": { "titles": {
@@ -866,7 +866,7 @@
} }
}, },
"embedding": { "embedding": {
"description": "Global config only. connection/model of an embedding model (OpenAI-shaped /embeddings, or Ollama): knowledge bases are then searched by meaning as well as by words. lembas kb reindex after changing it.", "description": "Global config only. An embedding model the LLeMbas server serves, by its id there: knowledge bases are then searched by meaning as well as by words. lembas kb reindex after changing it.",
"type": "string" "type": "string"
}, },
"effort": { "effort": {
+5 -344
View File
@@ -4,14 +4,12 @@
"properties": { "properties": {
"connections": { "connections": {
"default": {}, "default": {},
"description": "Each LLM endpoint by a name of your choosing; a model is then `name/model-id`. `type: webui` is a LLeMbas instance, whose models come from it.", "description": "The LLeMbas server this machine is logged in to, as `lembas login` writes it: its address and this machine's token. Its models come from it at every start.",
"type": "object", "type": "object",
"propertyNames": { "propertyNames": {
"type": "string" "type": "string"
}, },
"additionalProperties": { "additionalProperties": {
"anyOf": [
{
"type": "object", "type": "object",
"properties": { "properties": {
"type": { "type": {
@@ -24,10 +22,7 @@
"description": "The instance's address, e.g. https://ai.example.org (no /v1)." "description": "The instance's address, e.g. https://ai.example.org (no /v1)."
}, },
"api_key": { "api_key": {
"description": "This machine's token, as login wrote it: {file:~/.config/lembas/lembas/<name>.key}. Or key_cmd.", "description": "This machine's token, as login wrote it: {file:~/.config/lembas/lembas/<name>.key}.",
"type": "string"
},
"key_cmd": {
"type": "string" "type": "string"
}, },
"tls": { "tls": {
@@ -167,7 +162,7 @@
] ]
}, },
"effort_style": { "effort_style": {
"description": "openai-chat only: where the effort goes. llama.cpp drops top-level, so `both` is the default.", "description": "Where the effort goes in a request: top-level, in chat_template_kwargs, or both (the default: llama.cpp drops top-level).",
"type": "string", "type": "string",
"enum": [ "enum": [
"top", "top",
@@ -175,26 +170,6 @@
"both" "both"
] ]
}, },
"effort_map": {
"description": "anthropic/gemini: effort → thinking budget tokens.",
"type": "object",
"propertyNames": {
"type": "string",
"enum": [
"minimal",
"low",
"medium",
"high",
"xhigh",
"max"
]
},
"additionalProperties": {
"type": "integer",
"exclusiveMinimum": 0,
"maximum": 9007199254740991
}
},
"vision": { "vision": {
"description": "The model reads images: @image files, pasted paths and view_image.", "description": "The model reads images: @image files, pasted paths and view_image.",
"type": "boolean" "type": "boolean"
@@ -203,10 +178,6 @@
"description": "false: a model without tool calls — it can only talk.", "description": "false: a model without tool calls — it can only talk.",
"type": "boolean" "type": "boolean"
}, },
"cache": {
"description": "anthropic: prompt caching breakpoints.",
"type": "boolean"
},
"headers": { "headers": {
"description": "Extra HTTP headers for this model's requests.", "description": "Extra HTTP headers for this model's requests.",
"type": "object", "type": "object",
@@ -230,19 +201,12 @@
"type": "string", "type": "string",
"maxLength": 300 "maxLength": 300
}, },
"fallback": {
"description": "Other models (connection/model), in order, to switch to when this one's server cannot be reached at all — never once a reply has begun.",
"type": "array",
"items": {
"type": "string"
}
},
"single_session": { "single_session": {
"description": "The server serves one request at a time: a subagent on this same model would take its only slot and push the session's cached prompt out, so it is refused.", "description": "The server serves one request at a time: a subagent on this same model would take its only slot and push the session's cached prompt out, so it is refused.",
"type": "boolean" "type": "boolean"
}, },
"group": { "group": {
"description": "Models sharing one server that holds one model at a time (llama-swap), named by any string: a subagent on another model of the same group would unload this one, so it is refused. One connection's one_model_at_a_time is the same rule for all its models; this is for models one connection serves from several such servers (a LLeMbas instance).", "description": "Models sharing one GPU server that holds one model at a time (llama-swap), as the LLeMbas server says (capacity.group): a subagent on another model of the same group would unload this one, so it is refused.",
"type": "string" "type": "string"
} }
}, },
@@ -255,313 +219,10 @@
"url" "url"
], ],
"additionalProperties": false "additionalProperties": false
},
{
"type": "object",
"properties": {
"dialect": {
"type": "string",
"enum": [
"openai-chat",
"responses",
"anthropic",
"gemini",
"ollama"
],
"description": "The API it speaks: openai-chat (llama.cpp, vLLM, LM Studio, OpenAI…), responses (OpenAI Responses), anthropic, gemini, ollama (its native /api/chat)."
},
"base_url": {
"type": "string",
"description": "Where the API is, e.g. https://api.openai.com/v1 or http://localhost:8080/v1."
},
"api_key": {
"description": "Never pasted: {env:NAME} or {file:~/path}. Or use key_cmd.",
"type": "string"
},
"key_cmd": {
"description": "A command whose output is the key, e.g. pass show openai.",
"type": "string"
},
"auth": {
"description": "How the key is sent. Default per dialect: anthropic x-api-key, gemini x-goog-api-key, others bearer.",
"type": "string",
"enum": [
"bearer",
"x-api-key",
"x-goog-api-key",
"none"
]
},
"headers": {
"description": "Extra HTTP headers for every request.",
"type": "object",
"propertyNames": {
"type": "string"
},
"additionalProperties": {
"type": "string"
}
},
"body": {
"description": "Merged into every request body.",
"type": "object",
"propertyNames": {
"type": "string"
},
"additionalProperties": {}
},
"tls": {
"description": "TLS for endpoints behind a private CA. `ca` is a PEM file added to the trusted roots; `insecure` skips verification entirely (last resort). The system store is always trusted.",
"type": "object",
"properties": {
"ca": {
"type": "string"
},
"insecure": {
"type": "boolean"
}
},
"additionalProperties": false
},
"timeout": {
"description": "Seconds of silence allowed: waiting for the response, then between two pieces of a streamed reply. Not a limit on how long a reply may take. Default 600.",
"type": "number",
"exclusiveMinimum": 0
},
"discover": {
"description": "Merge ids and context lengths from GET /models.",
"type": "boolean"
},
"unload_url": {
"description": "llama-swap style: frees VRAM when switching away.",
"type": "string"
},
"one_model_at_a_time": {
"description": "The server holds one model at a time (llama-swap in front of one GPU): a subagent on another of its models would unload the session's, so it is refused. The session's own model may still be its own subagent.",
"type": "boolean"
},
"unload_method": {
"description": "How unload_url is called (default POST).",
"type": "string",
"enum": [
"GET",
"POST"
]
},
"quirks": {
"description": "Server oddities; auto is right almost always.",
"type": "object",
"properties": {
"stream_usage": {
"description": "Ask for token usage in the stream (stream_options); auto learns when a server refuses it.",
"type": "string",
"enum": [
"auto",
"on",
"off"
]
},
"prompt_progress": {
"description": "Ask how far the server is through reading the prompt (llama.cpp's return_progress), shown while it reads; auto learns when a server refuses it, and one that ignores it shows nothing.",
"type": "string",
"enum": [
"auto",
"on",
"off"
]
},
"think_tags": {
"description": "<think>…</think> in a reply is taken as reasoning; off leaves it as text.",
"type": "string",
"enum": [
"auto",
"on",
"off"
]
},
"max_tokens_field": {
"description": "OpenAI reasoning models refuse `max_tokens`; most local servers only know it.",
"type": "string",
"enum": [
"max_tokens",
"max_completion_tokens"
]
}
},
"additionalProperties": false
},
"models": {
"default": {},
"description": "The models to offer, by the id the server knows them by.",
"type": "object",
"propertyNames": {
"type": "string"
},
"additionalProperties": {
"type": "object",
"properties": {
"name": {
"description": "A display name; the id is what is sent.",
"type": "string"
},
"family": {
"description": "Picks the prompt overlay: anthropic | gpt | gemini | local. Guessed from the id when absent.",
"type": "string"
},
"context": {
"description": "Total context window in tokens. 0 / absent = unknown: no percentage, no auto-compaction.",
"type": "integer",
"minimum": 0,
"maximum": 9007199254740991
},
"max_output": {
"description": "Max tokens per reply.",
"type": "integer",
"exclusiveMinimum": 0,
"maximum": 9007199254740991
},
"temperature": {
"description": "Sampling temperature; absent = the server's default.",
"type": "number",
"minimum": 0,
"maximum": 2
},
"top_p": {
"description": "Nucleus sampling; absent = the server's default.",
"type": "number",
"minimum": 0,
"maximum": 1
},
"efforts": {
"description": "The model's own effort vocabulary, in offering order.",
"type": "array",
"items": {
"type": "string",
"enum": [
"minimal",
"low",
"medium",
"high",
"xhigh",
"max"
]
}
},
"effort": {
"description": "Default effort; `off` sends none.",
"anyOf": [
{
"type": "string",
"enum": [
"minimal",
"low",
"medium",
"high",
"xhigh",
"max"
]
},
{
"type": "string",
"const": "off"
}
]
},
"effort_style": {
"description": "openai-chat only: where the effort goes. llama.cpp drops top-level, so `both` is the default.",
"type": "string",
"enum": [
"top",
"kwargs",
"both"
]
},
"effort_map": {
"description": "anthropic/gemini: effort → thinking budget tokens.",
"type": "object",
"propertyNames": {
"type": "string",
"enum": [
"minimal",
"low",
"medium",
"high",
"xhigh",
"max"
]
},
"additionalProperties": {
"type": "integer",
"exclusiveMinimum": 0,
"maximum": 9007199254740991
}
},
"vision": {
"description": "The model reads images: @image files, pasted paths and view_image.",
"type": "boolean"
},
"tools": {
"description": "false: a model without tool calls — it can only talk.",
"type": "boolean"
},
"cache": {
"description": "anthropic: prompt caching breakpoints.",
"type": "boolean"
},
"headers": {
"description": "Extra HTTP headers for this model's requests.",
"type": "object",
"propertyNames": {
"type": "string"
},
"additionalProperties": {
"type": "string"
}
},
"body": {
"description": "Merged into every request body for this model.",
"type": "object",
"propertyNames": {
"type": "string"
},
"additionalProperties": {}
},
"notes": {
"description": "What this model is good at, in a line: the model in use is told which others there are, for handing work over (task) or switching.",
"type": "string",
"maxLength": 300
},
"fallback": {
"description": "Other models (connection/model), in order, to switch to when this one's server cannot be reached at all — never once a reply has begun.",
"type": "array",
"items": {
"type": "string"
}
},
"single_session": {
"description": "The server serves one request at a time: a subagent on this same model would take its only slot and push the session's cached prompt out, so it is refused.",
"type": "boolean"
},
"group": {
"description": "Models sharing one server that holds one model at a time (llama-swap), named by any string: a subagent on another model of the same group would unload this one, so it is refused. One connection's one_model_at_a_time is the same rule for all its models; this is for models one connection serves from several such servers (a LLeMbas instance).",
"type": "string"
}
},
"additionalProperties": false
}
}
},
"required": [
"dialect",
"base_url"
],
"additionalProperties": false
}
]
} }
} }
}, },
"additionalProperties": false, "additionalProperties": false,
"title": "LLeMbas CLI connections.yaml", "title": "LLeMbas CLI connections.yaml",
"description": "~/.config/lembas/connections.yaml: the LLM endpoints. Global only. Keys as {env:NAME}, {file:path} or key_cmd — never pasted." "description": "~/.config/lembas/connections.yaml: the LLeMbas server this machine is logged in to, as lembas login writes it. Global only."
} }
+2 -2
View File
@@ -68,12 +68,12 @@ export async function drive(opts: { cols: number; rows: number; bin?: string; ho
} }
export function fakeHome(script: Parameters<typeof fakeProvider>[0]) { export function fakeHome(script: Parameters<typeof fakeProvider>[0]) {
const fake = fakeProvider(script) const fake = fakeProvider(script, { served: [{ id: "fake/coder" }, { id: "fake/other" }] })
const home = mkdtempSync(join(tmpdir(), "lembas-drive-")) const home = mkdtempSync(join(tmpdir(), "lembas-drive-"))
mkdirSync(join(home, ".config/lembas"), { recursive: true }) mkdirSync(join(home, ".config/lembas"), { recursive: true })
writeFileSync( writeFileSync(
join(home, ".config/lembas/connections.yaml"), join(home, ".config/lembas/connections.yaml"),
`connections:\n fake:\n dialect: openai-chat\n base_url: ${fake.url}\n models:\n coder: { context: 32768, efforts: [low, high], effort: low }\n other: { context: 8192 }\n`, `connections:\n fake:\n type: webui\n url: ${fake.url}\n api_key: smoke\n models:\n fake/coder: { context: 32768, efforts: [low, high], effort: low }\n fake/other: { context: 8192 }\n`,
{ mode: 0o600 }, { mode: 0o600 },
) )
writeFileSync(join(home, ".config/lembas/config.yaml"), "model: fake/coder\nupdate: { auto: off }\n") writeFileSync(join(home, ".config/lembas/config.yaml"), "model: fake/coder\nupdate: { auto: off }\n")
+5 -5
View File
@@ -26,11 +26,11 @@ const v = run(["--version"])
check("--version", v.code === 0 && /^\d+\.\d+\.\d+/.test(v.out), v.err) check("--version", v.code === 0 && /^\d+\.\d+\.\d+/.test(v.out), v.err)
// 2. a headless run against a fake endpoint, with the session store on (FTS5 inside the binary) // 2. a headless run against a fake endpoint, with the session store on (FTS5 inside the binary)
const fake = fakeProvider([{ chunks: [delta({ content: "smoke ok" }), usage(5, 2)] }]) const fake = fakeProvider([{ chunks: [delta({ content: "smoke ok" }), usage(5, 2)] }], { served: [{ id: "fake/m" }] })
mkdirSync(join(home, ".config/lembas"), { recursive: true }) mkdirSync(join(home, ".config/lembas"), { recursive: true })
writeFileSync( writeFileSync(
join(home, ".config/lembas/connections.yaml"), join(home, ".config/lembas/connections.yaml"),
`connections:\n fake:\n dialect: openai-chat\n base_url: ${fake.url}\n models:\n m: {}\n`, `connections:\n fake:\n type: webui\n url: ${fake.url}\n api_key: smoke\n models:\n fake/m: {}\n`,
{ mode: 0o600 }, { mode: 0o600 },
) )
writeFileSync(join(home, ".config/lembas/config.yaml"), "model: fake/m\nupdate: { auto: off }\n") writeFileSync(join(home, ".config/lembas/config.yaml"), "model: fake/m\nupdate: { auto: off }\n")
@@ -46,10 +46,10 @@ check("sessions → recorded (sqlite + FTS5 in the binary)", s.code === 0 && s.o
// 2b. serve --stdio: the ACP agent inside the binary answers initialize, opens a session and // 2b. serve --stdio: the ACP agent inside the binary answers initialize, opens a session and
// streams a reply — the code the LLeMbas link runs too. // streams a reply — the code the LLeMbas link runs too.
{ {
const f = fakeProvider([{ chunks: [delta({ content: "acp ok" })] }]) const f = fakeProvider([{ chunks: [delta({ content: "acp ok" })] }], { served: [{ id: "fake/m" }] })
writeFileSync( writeFileSync(
join(home, ".config/lembas/connections.yaml"), join(home, ".config/lembas/connections.yaml"),
`connections:\n fake:\n dialect: openai-chat\n base_url: ${f.url}\n models:\n m: {}\n`, `connections:\n fake:\n type: webui\n url: ${f.url}\n api_key: smoke\n models:\n fake/m: {}\n`,
{ mode: 0o600 }, { mode: 0o600 },
) )
const p = Bun.spawn([bin, "serve", "--stdio"], { cwd: home, env, stdin: "pipe", stdout: "pipe", stderr: "pipe" }) const p = Bun.spawn([bin, "serve", "--stdio"], { cwd: home, env, stdin: "pipe", stdout: "pipe", stderr: "pipe" })
@@ -181,7 +181,7 @@ for (const [cols, rows] of [
const d = await drive({ cols: 100, rows: 30, bin, home: h, cwd }) const d = await drive({ cols: 100, rows: 30, bin, home: h, cwd })
await d.waitFor("Trust this directory?") await d.waitFor("Trust this directory?")
await d.key("\r") await d.key("\r")
const setup = await d.waitFor("no model to talk to yet") const setup = await d.waitFor("not logged in to a LLeMbas server")
await Bun.write(join(cfgDir, "connections.yaml"), connections) await Bun.write(join(cfgDir, "connections.yaml"), connections)
await Bun.write(join(cfgDir, "config.yaml"), "model: fake/coder\nupdate: { auto: off }\n") await Bun.write(join(cfgDir, "config.yaml"), "model: fake/coder\nupdate: { auto: off }\n")
// The screen opens on the sign-in, its address focused: ctrl+r tries again from there. // The screen opens on the sign-in, its address focused: ctrl+r tries again from there.
+1 -20
View File
@@ -17,7 +17,6 @@ import { DEFAULT_RULES, toRules } from "./permission/evaluate.ts"
import { hardlineRules } from "./permission/hardline.ts" import { hardlineRules } from "./permission/hardline.ts"
import { defaultEffort } from "./provider/effort.ts" import { defaultEffort } from "./provider/effort.ts"
import { clientFor, familyOf, resolveModel } from "./provider/index.ts" import { clientFor, familyOf, resolveModel } from "./provider/index.ts"
import { discoverContext, unload } from "./provider/discover.ts"
import { assembleSystem, instructionFiles, knowledgeScope } from "./prompt/assemble.ts" import { assembleSystem, instructionFiles, knowledgeScope } from "./prompt/assemble.ts"
import { describeRepo, repoState } from "./git/repo.ts" import { describeRepo, repoState } from "./git/repo.ts"
import { findProject, persistProjectRule, trustOf, type Project } from "./project/root.ts" import { findProject, persistProjectRule, trustOf, type Project } from "./project/root.ts"
@@ -100,7 +99,7 @@ export interface App {
/** The subagents the task tool offers (built-in, global, and a trusted project's), as of start. */ /** The subagents the task tool offers (built-in, global, and a trusted project's), as of start. */
agents: import("./project/agents.ts").AgentSpec[] agents: import("./project/agents.ts").AgentSpec[]
store: Store | undefined store: Store | undefined
/** Every configured `connection/model`, in config order. */ /** Every model the server serves, by its id there, in its order. */
modelRefs(): string[] modelRefs(): string[]
/** A ref in any form it may be written in (the old `<login>/<model>` too) as the ref the /** A ref in any form it may be written in (the old `<login>/<model>` too) as the ref the
* model has now; the ref itself when it names nothing. */ * model has now; the ref itself when it names nothing. */
@@ -497,7 +496,6 @@ export function createApp(o: AppOptions): App {
...(at ? { root: at.root, cwd: at.cwd, projectDir: undefined } : {}), ...(at ? { root: at.root, cwd: at.cwd, projectDir: undefined } : {}),
jobs: { registry, owner: () => ({ sessionId: childSessionId, job }) }, jobs: { registry, owner: () => ({ sessionId: childSessionId, job }) },
}, },
fallback: engine.o.fallback,
steerMark: MARKS.to_subagent, steerMark: MARKS.to_subagent,
system: (mode, current, plan) => system: (mode, current, plan) =>
system( system(
@@ -567,16 +565,6 @@ export function createApp(o: AppOptions): App {
system: (m, current, plan) => system(m, current, undefined, undefined, plan), system: (m, current, plan) => system(m, current, undefined, undefined, plan),
persistRule: trusted ? (rule) => persistProjectRule(project, rule) : undefined, persistRule: trusted ? (rule) => persistProjectRule(project, rule) : undefined,
spawn, spawn,
fallback(current, tried) {
for (const ref of current.spec.fallback ?? []) {
if (tried.has(ref)) continue
try {
const m = resolveModel(loaded, ref)
return { model: m, client: clientFor(m), effort: startEffort(m) }
} catch {}
}
return undefined
},
}) })
// MCP tools join the engine's list as servers connect (the engine reads it at each prompt). // MCP tools join the engine's list as servers connect (the engine reads it at each prompt).
@@ -647,18 +635,11 @@ export function createApp(o: AppOptions): App {
const snapshots = o.snapshots === false ? undefined : new Snapshots(project.root, project.gitRoot) const snapshots = o.snapshots === false ? undefined : new Snapshots(project.root, project.gitRoot)
engine.busyInput = loaded.config.busy_input ?? "steer" engine.busyInput = loaded.config.busy_input ?? "steer"
const turns = new Turns(engine, snapshots, store) const turns = new Turns(engine, snapshots, store)
// After a reply, learn the window if nobody configured it (the model is loaded by then).
bus.on((e) => {
if (e.type === "done" && !engine.model.spec.context) void discoverContext(engine.model, engine.client).then((n) => n && engine.usage && bus.emit({ type: "usage", usage: engine.usage, context: n, used: engine.contextUsed() }))
})
// By provider for an instance's models (`deepseek/deepseek-flash`), src/provider/refs.ts. // By provider for an instance's models (`deepseek/deepseek-flash`), src/provider/refs.ts.
const modelRefs = () => loaded.refs.map((r) => r.ref) const modelRefs = () => loaded.refs.map((r) => r.ref)
const switchModel = (ref: string) => { const switchModel = (ref: string) => {
const m = resolveModel(loaded, ref) const m = resolveModel(loaded, ref)
const prev = engine.model
engine.setModel(m, clientFor(m), startEffort(m)) engine.setModel(m, clientFor(m), startEffort(m))
// Moving to another connection frees the old one's memory (llama-swap /unload).
if (prev.connectionName !== m.connectionName) void unload(prev)
} }
const setIn = (key: string, value: unknown) => { const setIn = (key: string, value: unknown) => {
const path = key.split(".") const path = key.split(".")
+7 -21
View File
@@ -12,7 +12,7 @@ import { renderJson, renderPlain, settleJobs, terminalAsker } from "./headless.t
import { enterLocal, localFor, LocalError, localRefusal, localTmp, LOCAL_FLAGS } from "./project/local.ts" import { enterLocal, localFor, LocalError, localRefusal, localTmp, LOCAL_FLAGS } from "./project/local.ts"
import { findProject, setTrust, trustOf } from "./project/root.ts" import { findProject, setTrust, trustOf } from "./project/root.ts"
import { effortsFor } from "./provider/effort.ts" import { effortsFor } from "./provider/effort.ts"
import { clientFor, ModelError, resolveModel } from "./provider/index.ts" import { clientFor, ModelError, NOT_LOGGED_IN, resolveModel } from "./provider/index.ts"
import { Store } from "./session/store.ts" import { Store } from "./session/store.ts"
import { promptTitle } from "./session/title.ts" import { promptTitle } from "./session/title.ts"
import { VERSION } from "./version.ts" import { VERSION } from "./version.ts"
@@ -23,7 +23,7 @@ Usage:
lembas [-l] [-m model] [--mode M] [-c | -s id] lembas [-l] [-m model] [--mode M] [-c | -s id]
the TUI (-c continues the last session here, -s resumes one) the TUI (-c continues the last session here, -s resumes one)
lembas run [options] "…" one prompt, headless lembas run [options] "…" one prompt, headless
lembas models [--discover] list configured models lembas models list the server's models
lembas sessions list recent sessions lembas sessions list recent sessions
lembas sessions delete <id>… delete sessions, here and in the web UI lembas sessions delete <id>… delete sessions, here and in the web UI
lembas sessions prune [--dry-run] lembas sessions prune [--dry-run]
@@ -110,7 +110,7 @@ async function loginCommand(cmd: "login" | "logout", args: string[]) {
if (!name) fail(Object.keys(known).length ? `which one? lembas logout <${Object.keys(known).join("|")}>` : "not logged in to any instance", 2) if (!name) fail(Object.keys(known).length ? `which one? lembas logout <${Object.keys(known).join("|")}>` : "not logged in to any instance", 2)
const r = await logout(name) const r = await logout(name)
console.log(`Signed out of ${name}${r.revoked ? "" : " here (the instance could not be told — revoke it under User → Security → Devices there)"}; its connection is removed.`) console.log(`Signed out of ${name}${r.revoked ? "" : " here (the instance could not be told — revoke it under User → Security → Devices there)"}; its connection is removed.`)
if (r.modelLeft) console.log(`config.yaml still starts on ${r.modelLeft}: choose another with lembas config set model <connection/model>.`) if (r.modelLeft) console.log(`config.yaml still starts on ${r.modelLeft}: choose another with lembas config set model <id>.`)
return return
} }
let address = positionals[0] ?? "" let address = positionals[0] ?? ""
@@ -305,17 +305,11 @@ async function main() {
if (cmd === "models") { if (cmd === "models") {
const loaded = loadConfig() const loaded = loadConfig()
const discover = argv.includes("--discover") if (!Object.keys(loaded.connections).length && !Object.keys(loaded.broken).length) console.log(NOT_LOGGED_IN)
for (const [name, c] of Object.entries(loaded.connections)) { for (const [name, c] of Object.entries(loaded.connections)) {
console.log( console.log(`${name} (${c.webui?.url ?? c.base_url}${c.webui?.name ? ` — ${c.webui.name}` : ""}; models ${c.webui?.fetched_at ? `read ${c.webui.fetched_at.slice(0, 16).replace("T", " ")}` : "not read yet"})`)
c.webui
? `${name} (webui, ${c.webui.url}${c.webui.name ? ` — ${c.webui.name}` : ""}; models ${c.webui.fetched_at ? `read ${c.webui.fetched_at.slice(0, 16).replace("T", " ")}` : "not read yet"})`
: `${name} (${c.dialect}, ${c.base_url})`,
)
for (const [id, spec] of Object.entries(c.models)) { for (const [id, spec] of Object.entries(c.models)) {
// Its ref: an instance's model by provider, anything else connection/model. const m = resolveModel(loaded, id)
const ref = loaded.refs.find((r) => r.connection === name && r.id === id)?.ref ?? `${name}/${id}`
const m = resolveModel(loaded, ref)
const bits = [ const bits = [
spec.context ? `ctx ${spec.context}` : "", spec.context ? `ctx ${spec.context}` : "",
spec.max_output ? `out ${spec.max_output}` : "", spec.max_output ? `out ${spec.max_output}` : "",
@@ -323,15 +317,7 @@ async function main() {
spec.tools === false ? "no-tools" : "", spec.tools === false ? "no-tools" : "",
spec.effort || spec.efforts ? `effort ${spec.effort ?? "off"} of ${effortsFor(m).join("/")}` : "", spec.effort || spec.efforts ? `effort ${spec.effort ?? "off"} of ${effortsFor(m).join("/")}` : "",
].filter(Boolean) ].filter(Boolean)
console.log(` ${ref}${bits.length ? " — " + bits.join(", ") : ""}`) console.log(` ${m.ref}${bits.length ? " — " + bits.join(", ") : ""}`)
}
if (discover) {
try {
const found = await clientFor(resolveModel(loaded, `${name}/${Object.keys(c.models)[0] ?? "?"}`)).listModels()
for (const d of found) if (!c.models[d.id]) console.log(` ${name}/${d.id} (discovered${d.context ? `, ctx ${d.context}` : ""})`)
} catch (e) {
console.log(` (discovery failed: ${(e as Error).message})`)
}
} }
} }
for (const [name, why] of Object.entries(loaded.broken)) console.log(`${name} ✗ ${why}`) for (const [name, why] of Object.entries(loaded.broken)) console.log(`${name} ✗ ${why}`)
+1 -1
View File
@@ -29,7 +29,7 @@ export function jsonSchemas(): Record<keyof typeof SCHEMA_FILES, Record<string,
}) })
return { return {
config: make(Config, "LLeMbas CLI config.yaml", "~/.config/lembas/config.yaml, or a project's .agent/config.yaml. Strings may be {env:NAME} or {file:path}."), config: make(Config, "LLeMbas CLI config.yaml", "~/.config/lembas/config.yaml, or a project's .agent/config.yaml. Strings may be {env:NAME} or {file:path}."),
connections: make(ConnectionsFile, "LLeMbas CLI connections.yaml", "~/.config/lembas/connections.yaml: the LLM endpoints. Global only. Keys as {env:NAME}, {file:path} or key_cmd — never pasted."), connections: make(ConnectionsFile, "LLeMbas CLI connections.yaml", "~/.config/lembas/connections.yaml: the LLeMbas server this machine is logged in to, as lembas login writes it. Global only."),
} }
} }
+31 -30
View File
@@ -3,8 +3,8 @@ import { join } from "node:path"
import { isScalar, parse, parseDocument } from "yaml" import { isScalar, parse, parseDocument } from "yaml"
import { z } from "zod" import { z } from "zod"
import { connectionFiles, isLocal, paths } from "./paths.ts" import { connectionFiles, isLocal, paths } from "./paths.ts"
import { asMode, Config, Connection, ConnectionsFile, McpServer, PERSONALITIES, renamePermissionKeys, WebuiConnection, type PermissionConfig } from "./schema.ts" import { asMode, Config, McpServer, PERSONALITIES, renamePermissionKeys, WebuiConnection, type Connection, type PermissionConfig } from "./schema.ts"
import { runKeyCmd, substituteDeep } from "./substitute.ts" import { substituteDeep } from "./substitute.ts"
import { expandWebui, refreshFailures, type WebuiMark } from "../lembas/webui.ts" import { expandWebui, refreshFailures, type WebuiMark } from "../lembas/webui.ts"
import { defaultLibrary, withInstancePersonalization } from "../lembas/personal.ts" import { defaultLibrary, withInstancePersonalization } from "../lembas/personal.ts"
import { instances } from "../lembas/login.ts" import { instances } from "../lembas/login.ts"
@@ -14,8 +14,8 @@ export class ConfigError extends Error {}
export interface Loaded { export interface Loaded {
config: Config config: Config
/** A `type: webui` entry is here as the openai-chat connection it is spoken to as, with the /** The LLeMbas server this machine is logged in to — at most one: its `webui` entry as the
* instance's models and a `webui` mark. */ * `/v1` it is spoken to at, with its models and a `webui` mark (custom connections are gone). */
connections: Record<string, Connection & { webui?: WebuiMark }> connections: Record<string, Connection & { webui?: WebuiMark }>
/** Connections whose {env:}/{file:} substitution failed, with the reason. Unusable, not fatal. */ /** Connections whose {env:}/{file:} substitution failed, with the reason. Unusable, not fatal. */
broken: Record<string, string> broken: Record<string, string>
@@ -270,41 +270,47 @@ export function loadConfig(opts: { projectConfigDir?: string; trusted?: boolean
const connections: Loaded["connections"] = {} const connections: Loaded["connections"] = {}
const broken: Record<string, string> = {} const broken: Record<string, string> = {}
// Validate each raw file, then substitute each connection on its own: a missing variable for one // The login: a `webui` entry, as `lembas login` writes it. An entry of any other kind is a custom
// connection must not stop the others from working. In local mode, the user's own file first // connection from 1.x, gone in 2.0 (the CLI is LLeMbas-server-specific): said, and read past, so
// (read, never written), then the local one, whose entries win by name. // an old file still starts. In local mode, the user's own file first (read, never written), then
// the local one, whose entries win by name. Each entry is substituted on its own.
const raw: Record<string, unknown> = {} const raw: Record<string, unknown> = {}
for (const connFile of connectionFiles()) { for (const connFile of connectionFiles()) {
files.push(connFile) files.push(connFile)
const mode = statSync(connFile).mode & 0o777 const mode = statSync(connFile).mode & 0o777
if (mode & 0o077) warnings.push(`${connFile} is mode ${mode.toString(8)}; chmod 600 it — it references credentials`) if (mode & 0o077) warnings.push(`${connFile} is mode ${mode.toString(8)}; chmod 600 it — it references credentials`)
Object.assign(raw, validate(ConnectionsFile, readYaml(connFile), connFile).connections) const doc = readYaml(connFile)
const entries = doc && typeof doc === "object" ? (doc as { connections?: unknown }).connections : undefined
if (entries === undefined || entries === null) continue
if (typeof entries !== "object" || Array.isArray(entries)) throw new ConfigError(`${connFile} is invalid:\n connections: expected the login as written by lembas login`)
for (const [name, c] of Object.entries(entries as Record<string, unknown>)) {
if ((c as { type?: string } | null)?.type === "webui") raw[name] = c
else warnings.push(`${connFile}: ${name} is a custom connection, and LLeMbas CLI 2.0 has none — it talks only to the LLeMbas server it is logged in to (lembas login <address>). Ignored; delete it.`)
} }
for (const [name, c] of Object.entries(raw)) { }
// One server: the login made last, where several are left from 1.x.
const logins = instances()
const names = Object.keys(raw)
const chosen = names.length > 1 ? [...names].sort((a, b) => (logins[b]?.logged_in_at ?? "").localeCompare(logins[a]?.logged_in_at ?? ""))[0]! : names[0]
if (names.length > 1) warnings.push(`connections.yaml holds ${names.length} LLeMbas logins (${names.join(", ")}); LLeMbas CLI 2.0 uses one server — ${chosen}, the latest. lembas logout the others.`)
if (chosen) {
const name = chosen
try { try {
if ((c as { type?: string }).type === "webui") { const w = WebuiConnection.safeParse(substituteDeep(raw[name], `connections.${name}`))
const w = WebuiConnection.safeParse(substituteDeep(c, `connections.${name}`))
if (w.success) { if (w.success) {
const c = (connections[name] = expandWebui(name, w.data)) const c = (connections[name] = expandWebui(name, w.data))
if (refreshFailures[name]) if (refreshFailures[name])
warnings.push( warnings.push(
`${name}: the instance could not be asked for its models (${refreshFailures[name]}); ${c.webui.fetched_at ? `using what it said at ${c.webui.fetched_at.slice(0, 16).replace("T", " ")}` : "none known yet"}`, `${name}: the instance could not be asked for its models (${refreshFailures[name]}); ${c.webui.fetched_at ? `using what it said at ${c.webui.fetched_at.slice(0, 16).replace("T", " ")}` : "none known yet"}`,
) )
} } else broken[name] = `once {env:}/{file:} are filled in: ${w.error.issues.map((i) => `${i.path.join(".")}: ${i.message}`).join("; ")}`
else broken[name] = `once {env:}/{file:} are filled in: ${w.error.issues.map((i) => `${i.path.join(".")}: ${i.message}`).join("; ")}`
continue
}
// Again once filled in: "{env:LLM_URL}" passed as a placeholder; what it became must be a URL.
const r = Connection.safeParse(substituteDeep(c, `connections.${name}`))
if (r.success) connections[name] = r.data
else broken[name] = `once {env:}/{file:} are filled in: ${r.error.issues.map((i) => `${i.path.join(".")}: ${i.message}`).join("; ")}`
} catch (e) { } catch (e) {
broken[name] = (e as Error).message broken[name] = (e as Error).message
} }
} }
/** The webui connection a voice half or the search means: the one named, or the only one. A /** The server a voice half or the search goes through: the login (`connection:` may still name
* string says why there is none. */ * it). A string says why there is none. */
const webuiFor = (named?: string): { name: string; c: Loaded["connections"][string] } | string => { const webuiFor = (named?: string): { name: string; c: Loaded["connections"][string] } | string => {
if (named) return connections[named]?.webui ? { name: named, c: connections[named]! } : broken[named] ? `${named} is unusable: ${broken[named]}` : `there is no webui connection named ${named}` if (named) return connections[named]?.webui ? { name: named, c: connections[named]! } : broken[named] ? `${named} is unusable: ${broken[named]}` : `there is no webui connection named ${named}`
const all = Object.entries(connections).filter(([, c]) => c.webui) const all = Object.entries(connections).filter(([, c]) => c.webui)
@@ -408,10 +414,7 @@ export function loadConfig(opts: { projectConfigDir?: string; trusted?: boolean
config = personal.config config = personal.config
if (config.library === undefined && !isLocal() && defaultLibrary()) config = { ...config, library: "lembas" } if (config.library === undefined && !isLocal() && defaultLibrary()) config = { ...config, library: "lembas" }
// An instance that speaks protocol 2 serves `<provider>/<model>` ids. const refs = modelRefs(connections)
const logins = instances()
for (const [name, c] of Object.entries(connections)) if (c.webui && logins[name]?.protocols?.includes(2)) c.webui.v2 = true
const refs = modelRefs(connections, Object.keys(logins))
// A config.yaml naming the model the way it was named before providers (`example/deepseek-flash`) // A config.yaml naming the model the way it was named before providers (`example/deepseek-flash`)
// is rewritten to the new name, once (migrateRefs). Only that form: a pinned `<login>/<provider>/ // is rewritten to the new name, once (migrateRefs). Only that form: a pinned `<login>/<provider>/
// <model>` means what it says, and `embedding` stays as written — its name keys the stored vectors. // <model>` means what it says, and `embedding` stays as written — its name keys the stored vectors.
@@ -480,9 +483,7 @@ function localOnly(config: Config, connections: Loaded["connections"], warnings:
return out return out
} }
/** The API key for a connection: `api_key` (already substituted) or the output of `key_cmd`. */ /** The server's token: the login's `api_key`, already substituted from its key file. */
export function resolveKey(name: string, c: Connection): string | undefined { export function resolveKey(_name: string, c: Pick<Connection, "api_key">): string | undefined {
if (c.api_key) return c.api_key return c.api_key || undefined
if (c.key_cmd) return runKeyCmd(c.key_cmd, `connections.${name}.key_cmd`)
return undefined
} }
+32 -33
View File
@@ -1,6 +1,5 @@
import { z } from "zod" import { z } from "zod"
export const DIALECTS = ["openai-chat", "responses", "anthropic", "gemini", "ollama"] as const
export const EFFORTS = ["minimal", "low", "medium", "high", "xhigh", "max"] as const export const EFFORTS = ["minimal", "low", "medium", "high", "xhigh", "max"] as const
export const MODES = ["manual", "edit", "auto", "plan"] as const export const MODES = ["manual", "edit", "auto", "plan"] as const
/** The personality presets (harness/prompts/personality/, shared with the web UI), then custom. */ /** The personality presets (harness/prompts/personality/, shared with the web UI), then custom. */
@@ -54,35 +53,19 @@ export const ModelSpec = z.strictObject({
top_p: z.number().min(0).max(1).optional().describe("Nucleus sampling; absent = the server's default."), top_p: z.number().min(0).max(1).optional().describe("Nucleus sampling; absent = the server's default."),
efforts: z.array(z.enum(EFFORTS)).optional().describe("The model's own effort vocabulary, in offering order."), efforts: z.array(z.enum(EFFORTS)).optional().describe("The model's own effort vocabulary, in offering order."),
effort: z.union([z.enum(EFFORTS), z.literal("off")]).optional().describe("Default effort; `off` sends none."), effort: z.union([z.enum(EFFORTS), z.literal("off")]).optional().describe("Default effort; `off` sends none."),
effort_style: z.enum(["top", "kwargs", "both"]).optional().describe("openai-chat only: where the effort goes. llama.cpp drops top-level, so `both` is the default."), effort_style: z.enum(["top", "kwargs", "both"]).optional().describe("Where the effort goes in a request: top-level, in chat_template_kwargs, or both (the default: llama.cpp drops top-level)."),
effort_map: z.partialRecord(z.enum(EFFORTS), z.number().int().positive()).optional().describe("anthropic/gemini: effort → thinking budget tokens."),
vision: z.boolean().optional().describe("The model reads images: @image files, pasted paths and view_image."), vision: z.boolean().optional().describe("The model reads images: @image files, pasted paths and view_image."),
tools: z.boolean().optional().describe("false: a model without tool calls — it can only talk."), tools: z.boolean().optional().describe("false: a model without tool calls — it can only talk."),
cache: z.boolean().optional().describe("anthropic: prompt caching breakpoints."),
headers: z.record(z.string(), z.string()).optional().describe("Extra HTTP headers for this model's requests."), headers: z.record(z.string(), z.string()).optional().describe("Extra HTTP headers for this model's requests."),
body: z.record(z.string(), z.unknown()).optional().describe("Merged into every request body for this model."), body: z.record(z.string(), z.unknown()).optional().describe("Merged into every request body for this model."),
notes: z.string().max(300).optional().describe("What this model is good at, in a line: the model in use is told which others there are, for handing work over (task) or switching."), notes: z.string().max(300).optional().describe("What this model is good at, in a line: the model in use is told which others there are, for handing work over (task) or switching."),
fallback: z.array(z.string()).optional().describe("Other models (connection/model), in order, to switch to when this one's server cannot be reached at all — never once a reply has begun."),
single_session: z.boolean().optional().describe("The server serves one request at a time: a subagent on this same model would take its only slot and push the session's cached prompt out, so it is refused."), single_session: z.boolean().optional().describe("The server serves one request at a time: a subagent on this same model would take its only slot and push the session's cached prompt out, so it is refused."),
group: z.string().optional().describe("Models sharing one server that holds one model at a time (llama-swap), named by any string: a subagent on another model of the same group would unload this one, so it is refused. One connection's one_model_at_a_time is the same rule for all its models; this is for models one connection serves from several such servers (a LLeMbas instance)."), group: z.string().optional().describe("Models sharing one GPU server that holds one model at a time (llama-swap), as the LLeMbas server says (capacity.group): a subagent on another model of the same group would unload this one, so it is refused."),
}) })
export type ModelSpec = z.infer<typeof ModelSpec> export type ModelSpec = z.infer<typeof ModelSpec>
export const Connection = z.strictObject({ /** Server oddities; auto is right almost always. */
dialect: z.enum(DIALECTS).describe("The API it speaks: openai-chat (llama.cpp, vLLM, LM Studio, OpenAI…), responses (OpenAI Responses), anthropic, gemini, ollama (its native /api/chat)."), export const Quirks = z
base_url: Url.describe("Where the API is, e.g. https://api.openai.com/v1 or http://localhost:8080/v1."),
api_key: z.string().optional().describe("Never pasted: {env:NAME} or {file:~/path}. Or use key_cmd."),
key_cmd: z.string().optional().describe("A command whose output is the key, e.g. pass show openai."),
auth: z.enum(["bearer", "x-api-key", "x-goog-api-key", "none"]).optional().describe("How the key is sent. Default per dialect: anthropic x-api-key, gemini x-goog-api-key, others bearer."),
headers: z.record(z.string(), z.string()).optional().describe("Extra HTTP headers for every request."),
body: z.record(z.string(), z.unknown()).optional().describe("Merged into every request body."),
tls: Tls.optional().describe("TLS for endpoints behind a private CA. `ca` is a PEM file added to the trusted roots; `insecure` skips verification entirely (last resort). The system store is always trusted."),
timeout: z.number().positive().optional().describe("Seconds of silence allowed: waiting for the response, then between two pieces of a streamed reply. Not a limit on how long a reply may take. Default 600."),
discover: z.boolean().optional().describe("Merge ids and context lengths from GET /models."),
unload_url: Url.optional().describe("llama-swap style: frees VRAM when switching away."),
one_model_at_a_time: z.boolean().optional().describe("The server holds one model at a time (llama-swap in front of one GPU): a subagent on another of its models would unload the session's, so it is refused. The session's own model may still be its own subagent."),
unload_method: z.enum(["GET", "POST"]).optional().describe("How unload_url is called (default POST)."),
quirks: z
.strictObject({ .strictObject({
stream_usage: Tristate.optional().describe("Ask for token usage in the stream (stream_options); auto learns when a server refuses it."), stream_usage: Tristate.optional().describe("Ask for token usage in the stream (stream_options); auto learns when a server refuses it."),
prompt_progress: Tristate.optional().describe("Ask how far the server is through reading the prompt (llama.cpp's return_progress), shown while it reads; auto learns when a server refuses it, and one that ignores it shows nothing."), prompt_progress: Tristate.optional().describe("Ask how far the server is through reading the prompt (llama.cpp's return_progress), shown while it reads; auto learns when a server refuses it, and one that ignores it shows nothing."),
@@ -90,10 +73,24 @@ export const Connection = z.strictObject({
max_tokens_field: z.enum(["max_tokens", "max_completion_tokens"]).optional().describe("OpenAI reasoning models refuse `max_tokens`; most local servers only know it."), max_tokens_field: z.enum(["max_tokens", "max_completion_tokens"]).optional().describe("OpenAI reasoning models refuse `max_tokens`; most local servers only know it."),
}) })
.optional() .optional()
.describe("Server oddities; auto is right almost always."), .describe("Server oddities; auto is right almost always.")
models: z.record(z.string(), ModelSpec).default({}).describe("The models to offer, by the id the server knows them by."),
}) /** The LLeMbas server as LLeMbas CLI speaks to it: its OpenAI-shaped `/v1` with this machine's
export type Connection = z.infer<typeof Connection> * token, and the models it serves. Not something a person writes — custom connections were removed
* in 2.0 (CLI rule 2: the CLI is LLeMbas-server-specific); `lembas login` writes the one `webui`
* entry below, and the loader turns it into this (lembas/webui.ts, expandWebui). */
export interface Connection {
base_url: string
api_key?: string
tls?: z.infer<typeof Tls>
/** Seconds of silence allowed in a reply (default 600). */
timeout?: number
quirks?: z.infer<typeof Quirks>
/** Extra headers and body fields for every request (none from a login; tests use them). */
headers?: Record<string, string>
body?: Record<string, unknown>
models: Record<string, ModelSpec>
}
/** A LLeMbas instance: its address and this machine's key, and nothing else. The models, /** A LLeMbas instance: its address and this machine's key, and nothing else. The models,
* their settings, and — where config.yaml says `provider: webui` — the voice and the web search come * their settings, and — where config.yaml says `provider: webui` — the voice and the web search come
@@ -101,20 +98,22 @@ export type Connection = z.infer<typeof Connection>
export const WebuiConnection = z.strictObject({ export const WebuiConnection = z.strictObject({
type: z.literal("webui").describe("A LLeMbas instance: its models, their settings, its voice and its web search."), type: z.literal("webui").describe("A LLeMbas instance: its models, their settings, its voice and its web search."),
url: Url.describe("The instance's address, e.g. https://ai.example.org (no /v1)."), url: Url.describe("The instance's address, e.g. https://ai.example.org (no /v1)."),
api_key: z.string().optional().describe("This machine's token, as login wrote it: {file:~/.config/lembas/lembas/<name>.key}. Or key_cmd."), api_key: z.string().optional().describe("This machine's token, as login wrote it: {file:~/.config/lembas/lembas/<name>.key}."),
key_cmd: z.string().optional(),
tls: Tls.optional().describe("For an instance behind a private CA: `ca`, the PEM file that signed its certificate."), tls: Tls.optional().describe("For an instance behind a private CA: `ca`, the PEM file that signed its certificate."),
timeout: z.number().positive().optional().describe("Seconds of silence allowed in a reply, as for any connection (default 600)."), timeout: z.number().positive().optional().describe("Seconds of silence allowed in a reply, as for any connection (default 600)."),
quirks: Connection.shape.quirks, quirks: Quirks,
models: z.record(z.string(), ModelSpec).optional().describe("Only to change what the instance says about a model, here: each entry goes over the instance's. Usually absent."), models: z.record(z.string(), ModelSpec).optional().describe("Only to change what the instance says about a model, here: each entry goes over the instance's. Usually absent."),
}) })
export type WebuiConnection = z.infer<typeof WebuiConnection> export type WebuiConnection = z.infer<typeof WebuiConnection>
/** connections.yaml: the login, as `lembas login` writes it. An entry of any other kind — a custom
* connection from 1.x — is read past with a warning (config/load.ts), not refused, so an old file
* still starts. */
export const ConnectionsFile = z.strictObject({ export const ConnectionsFile = z.strictObject({
connections: z connections: z
.record(z.string(), z.union([WebuiConnection, Connection])) .record(z.string(), WebuiConnection)
.default({}) .default({})
.describe("Each LLM endpoint by a name of your choosing; a model is then `name/model-id`. `type: webui` is a LLeMbas instance, whose models come from it."), .describe("The LLeMbas server this machine is logged in to, as `lembas login` writes it: its address and this machine's token. Its models come from it at every start."),
}) })
export type ConnectionsFile = z.infer<typeof ConnectionsFile> export type ConnectionsFile = z.infer<typeof ConnectionsFile>
@@ -231,8 +230,8 @@ export const Update = z.strictObject({
export type Update = z.infer<typeof Update> export type Update = z.infer<typeof Update>
export const Config = z.strictObject({ export const Config = z.strictObject({
model: z.string().optional().describe("The model a session starts with: `connection/model`, a connection from connections.yaml and one of its models."), model: z.string().optional().describe("The model a session starts with: one the LLeMbas server you are logged in to serves, by its id there (`deepseek/deepseek-flash`). Default: the server's default for your account."),
small_model: z.string().optional().describe("`connection/model` for small jobs — session titles, commit messages. Default: the session's model. On a server that holds one model at a time (llama-swap), another model here swaps."), small_model: z.string().optional().describe("The model for small jobs — session titles, commit messages, by its id on the server. Default: the session's model."),
titles: z.enum(["model", "prompt"]).optional().describe("Session titles: model (default) — named from the first prompt, then by the model after its first reply; prompt — only from the first prompt, no extra request."), titles: z.enum(["model", "prompt"]).optional().describe("Session titles: model (default) — named from the first prompt, then by the model after its first reply; prompt — only from the first prompt, no extra request."),
mode: z.enum(MODES).optional().describe("The permission mode a session starts in (default manual)."), mode: z.enum(MODES).optional().describe("The permission mode a session starts in (default manual)."),
theme: z.string().optional().describe("Colours: moria (default), shire, lembas, lembas-light, hermes, mono, or a themes/<name>.yaml of your own."), theme: z.string().optional().describe("Colours: moria (default), shire, lembas, lembas-light, hermes, mono, or a themes/<name>.yaml of your own."),
@@ -293,7 +292,7 @@ export const Config = z.strictObject({
voice: Voice.optional().describe("Voice in and out. Global only — it carries endpoints and keys, like a connection."), voice: Voice.optional().describe("Voice in and out. Global only — it carries endpoints and keys, like a connection."),
mcp: z.record(z.string(), McpServer).optional().describe("MCP servers by name. `{env:}`/`{file:}` are filled in per server (a missing one disables that server only). A project's servers count only once the project is trusted."), mcp: z.record(z.string(), McpServer).optional().describe("MCP servers by name. `{env:}`/`{file:}` are filled in per server (a missing one disables that server only). A project's servers count only once the project is trusted."),
knowledge: z.array(z.string()).optional().describe("The knowledge bases searched (lembas kb); absent: all of them. A project's own config narrows it to what that project needs."), knowledge: z.array(z.string()).optional().describe("The knowledge bases searched (lembas kb); absent: all of them. A project's own config narrows it to what that project needs."),
embedding: z.string().optional().describe("Global config only. connection/model of an embedding model (OpenAI-shaped /embeddings, or Ollama): knowledge bases are then searched by meaning as well as by words. lembas kb reindex after changing it."), embedding: z.string().optional().describe("Global config only. An embedding model the LLeMbas server serves, by its id there: knowledge bases are then searched by meaning as well as by words. lembas kb reindex after changing it."),
effort: z.union([z.enum(EFFORTS), z.literal("off")]).optional().describe("The effort a session starts with, when the model takes it; otherwise the model's own default."), effort: z.union([z.enum(EFFORTS), z.literal("off")]).optional().describe("The effort a session starts with, when the model takes it; otherwise the model's own default."),
settings_tool: z.enum(["ask", "allow", "off"]).optional().describe("Global config only. The agent's settings tool: ask (default) — each change is approved like any tool call; allow — changes go through without asking (a less strict mode still asks); off — no settings tool."), settings_tool: z.enum(["ask", "allow", "off"]).optional().describe("Global config only. The agent's settings tool: ask (default) — each change is approved like any tool call; allow — changes go through without asking (a less strict mode still asks); off — no settings tool."),
update: Update.optional().describe("Global config only. Updating LLeMbas CLI itself."), update: Update.optional().describe("Global config only. Updating LLeMbas CLI itself."),
+1 -1
View File
@@ -19,7 +19,7 @@ export type Source = "default" | "global" | "project" | "session"
export interface SettingDef { export interface SettingDef {
key: string key: string
/** model: a configured `connection/model`; effort: one the model takes, or off; list: comma-separated. */ /** model: a model the server serves, by its id; effort: one the model takes, or off; list: comma-separated. */
kind: "enum" | "model" | "effort" | "number" | "boolean" | "list" | "string" kind: "enum" | "model" | "effort" | "number" | "boolean" | "list" | "string"
values?: readonly string[] values?: readonly string[]
/** The agent may change it (through the settings tool, with approval). */ /** The agent may change it (through the settings tool, with approval). */
+11 -1
View File
@@ -7,7 +7,8 @@
// pasted into the YAML. The models are not written anywhere: a webui connection reads them from the // pasted into the YAML. The models are not written anywhere: a webui connection reads them from the
// instance at every start (webui.ts). `lembas.json` remembers which connections a login made, so // instance at every start (webui.ts). `lembas.json` remembers which connections a login made, so
// `login` again refreshes them and `logout` removes exactly them. Without a login nothing here is // `login` again refreshes them and `logout` removes exactly them. Without a login nothing here is
// read, and LLeMbas CLI runs on its hand-written config as before. // read, and LLeMbas CLI has no model to talk to: 2.0 talks to the server it is logged in to, one at
// a time, and nothing else.
import { chmodSync, existsSync, mkdirSync, readFileSync, rmSync, writeFileSync } from "node:fs" import { chmodSync, existsSync, mkdirSync, readFileSync, rmSync, writeFileSync } from "node:fs"
import { join, resolve } from "node:path" import { join, resolve } from "node:path"
import { parseDocument } from "yaml" import { parseDocument } from "yaml"
@@ -416,6 +417,15 @@ export async function login(address: string, o: LoginOptions): Promise<LoginResu
changes.push("the previous login could not be signed out: revoke it under User → Security → Devices on the instance") changes.push("the previous login could not be signed out: revoke it under User → Security → Devices on the instance")
} }
await handOver(replaced) await handOver(replaced)
// One server at a time (2.0: the CLI is LLeMbas-server-specific): a login to another one signs
// this machine out of the one it was logged in to.
for (const [name, old] of Object.entries(ownInstances()))
if (name !== connection && old.base_url !== discovery.base_url) {
const out = await logout(name).catch(() => undefined)
if (!out) continue
changes.push(`signed out of ${old.instance_name ?? old.base_url} (${name}): LLeMbas CLI uses one server at a time`)
if (out.modelLeft) changes.push(`config.yaml still starts on ${out.modelLeft}, a model of that server: change it with lembas config set model <id>, or remove it to start on this server's default`)
}
// Only logins this mode made: in local mode, the user's own login to the same instance is not // Only logins this mode made: in local mode, the user's own login to the same instance is not
// signed out or replaced — it is not this project's. // signed out or replaced — it is not this project's.
const known = ownInstances() const known = ownInstances()
+2 -14
View File
@@ -19,7 +19,7 @@ import { join } from "node:path"
import { parse } from "yaml" import { parse } from "yaml"
import { connectionFiles, isLocal, paths } from "../config/paths.ts" import { connectionFiles, isLocal, paths } from "../config/paths.ts"
import type { Connection, ModelSpec, WebuiConnection } from "../config/schema.ts" import type { Connection, ModelSpec, WebuiConnection } from "../config/schema.ts"
import { runKeyCmd, substituteDeep } from "../config/substitute.ts" import { substituteDeep } from "../config/substitute.ts"
import { instanceInfo, LembasError, models as listModels, normalise, personalization, type InstanceInfo, type Personalization, type ServedModel, type Tls } from "./client.ts" import { instanceInfo, LembasError, models as listModels, normalise, personalization, type InstanceInfo, type Personalization, type ServedModel, type Tls } from "./client.ts"
/** What a loaded webui connection carries besides an openai-chat connection's fields. */ /** What a loaded webui connection carries besides an openai-chat connection's fields. */
@@ -33,14 +33,9 @@ export interface WebuiMark {
services?: InstanceInfo["services"] services?: InstanceInfo["services"]
/** When the models were last read from the instance; absent: never. */ /** When the models were last read from the instance; absent: never. */
fetched_at?: string fetched_at?: string
/** Each served model's provider, where the instance said it (`provider`). */
providers?: Record<string, string>
/** The served ids of its embedding models, which are not chat models and so not in /** The served ids of its embedding models, which are not chat models and so not in
* `models`: `embedding: llama/nomic-embed` finds its connection by them (library/embed.ts). */ * `models`: `embedding: llama/nomic-embed` finds its connection by them (library/embed.ts). */
embeddings?: string[] embeddings?: string[]
/** The instance speaks link protocol 2: its ids are `<provider>/<model>` even
* where a model does not say its provider. Set by loadConfig from the login. */
v2?: boolean
} }
export type WebuiLoaded = Connection & { webui: WebuiMark } export type WebuiLoaded = Connection & { webui: WebuiMark }
@@ -134,18 +129,12 @@ export function expandWebui(name: string, w: WebuiConnection, cache = readCache(
for (const m of ordered) models[m.id] = { ...specFor(m, name), ...(overrides.get(m.id) ?? {}) } for (const m of ordered) models[m.id] = { ...specFor(m, name), ...(overrides.get(m.id) ?? {}) }
// A model the entry names that the instance does not (yet) list: kept, as written. // A model the entry names that the instance does not (yet) list: kept, as written.
for (const [id, spec] of unmatched) models[id] ??= spec for (const [id, spec] of unmatched) models[id] ??= spec
const providers = Object.fromEntries(served.filter((m) => m.provider && m.id.startsWith(`${m.provider}/`)).map((m) => [m.id, m.provider!]))
return { return {
dialect: "openai-chat",
base_url: `${url}/v1`, base_url: `${url}/v1`,
...(w.api_key !== undefined ? { api_key: w.api_key } : {}), ...(w.api_key !== undefined ? { api_key: w.api_key } : {}),
...(w.key_cmd !== undefined ? { key_cmd: w.key_cmd } : {}),
...(w.tls ? { tls: w.tls } : {}), ...(w.tls ? { tls: w.tls } : {}),
...(w.timeout ? { timeout: w.timeout } : {}), ...(w.timeout ? { timeout: w.timeout } : {}),
...(w.quirks ? { quirks: w.quirks } : {}), ...(w.quirks ? { quirks: w.quirks } : {}),
// The instance decides which models exist: one it serves that the cache does not know yet is
// still a model, not an error.
discover: true,
models, models,
webui: { webui: {
url, url,
@@ -153,7 +142,6 @@ export function expandWebui(name: string, w: WebuiConnection, cache = readCache(
...(cache?.instance?.name ? { name: cache.instance.name } : {}), ...(cache?.instance?.name ? { name: cache.instance.name } : {}),
...(cache?.instance?.services ? { services: cache.instance.services } : {}), ...(cache?.instance?.services ? { services: cache.instance.services } : {}),
...(cache && cache.url === url ? { fetched_at: cache.fetched_at } : {}), ...(cache && cache.url === url ? { fetched_at: cache.fetched_at } : {}),
...(Object.keys(providers).length ? { providers } : {}),
...(embeddings.length ? { embeddings } : {}), ...(embeddings.length ? { embeddings } : {}),
}, },
} }
@@ -200,7 +188,7 @@ export async function refreshWebui(o: { timeoutMs?: number; only?: string } = {}
entries.map(async ([name, c]): Promise<Refreshed> => { entries.map(async ([name, c]): Promise<Refreshed> => {
try { try {
const w = substituteDeep(c, `connections.${name}`) as WebuiConnection const w = substituteDeep(c, `connections.${name}`) as WebuiConnection
const token = w.api_key ?? (w.key_cmd ? runKeyCmd(w.key_cmd, `connections.${name}.key_cmd`) : undefined) const token = w.api_key
if (!token) throw new LembasError("no key (lembas login)") if (!token) throw new LembasError("no key (lembas login)")
const got = await fetchWebui(name, w.url, token.trim(), w.tls, o.timeoutMs ?? 4000) const got = await fetchWebui(name, w.url, token.trim(), w.tls, o.timeoutMs ?? 4000)
delete refreshFailures[name] delete refreshFailures[name]
+17 -31
View File
@@ -1,6 +1,6 @@
// Embeddings for the library: `embedding: connection/model` in config.yaml, over the connection's // Embeddings for the library: `embedding:` in config.yaml names an embedding model of the LLeMbas
// OpenAI-shaped /embeddings (llama.cpp with --embedding, vLLM, LM Studio, OpenAI…) or Ollama's // server, reached at its OpenAI-shaped /v1/embeddings. Vectors are made unit length here, so a dot
// /api/embed. Vectors are made unit length here, so a dot product is the cosine. // product is the cosine.
import { resolveKey, type Loaded } from "../config/load.ts" import { resolveKey, type Loaded } from "../config/load.ts"
import { authHeaders, joinUrl, request, tlsFor } from "../provider/http.ts" import { authHeaders, joinUrl, request, tlsFor } from "../provider/http.ts"
import { resolveModel } from "../provider/index.ts" import { resolveModel } from "../provider/index.ts"
@@ -15,46 +15,32 @@ function unit(v: number[]): Float32Array {
export function embedderFor(loaded: Loaded, ref: string | undefined): Embedder | undefined { export function embedderFor(loaded: Loaded, ref: string | undefined): Embedder | undefined {
if (!ref) return undefined if (!ref) return undefined
// Through the same resolver as every model ref: an instance's model by its provider // An embedding model of the server, by its served id (`llama/nomic-embed`) — not a chat model, so
// (`llama/nomic-embed`) or in the older form (`example/nomic-embed`). The name stored with the // not a model ref (lembas/webui.ts, isChatModel): found among the ids the server says it serves
// vectors stays the ref as written, so a library indexed under it is still found. // for embeddings, or as a ref in any of its forms. The name stored with the vectors stays the ref
// An embedding model need not be listed on its connection (`connection/<any id>`, as before). // as written, so a library indexed under it is still found.
let name: string const [name, c] = Object.entries(loaded.connections)[0] ?? []
if (!name || !c) throw new Error("embedding: not logged in to a LLeMbas server (lembas login)")
let id: string let id: string
if (c.webui?.embeddings?.includes(ref)) id = ref
else if (ref.startsWith(`${name}/`) && c.webui?.embeddings?.includes(ref.slice(name.length + 1))) id = ref.slice(name.length + 1)
else {
try { try {
const resolved = resolveModel(loaded, ref) id = resolveModel(loaded, ref).id
name = resolved.connectionName
id = resolved.id
} catch (e) { } catch (e) {
// An instance's embedding model by its served id (`llama/nomic-embed`): not a throw new Error(`embedding: ${(e as Error).message}`)
// model ref, since only chat models are (lembas/webui.ts, isChatModel), so found by the ids
// its connection says it serves.
const served = Object.entries(loaded.connections).find(([, c]) => (c as { webui?: { embeddings?: string[] } }).webui?.embeddings?.includes(ref))
const slash = ref.indexOf("/")
if (served) {
name = served[0]
id = ref
} else {
if (slash <= 0) throw new Error(`embedding: "${ref}" must be written connection/model`)
name = ref.slice(0, slash)
id = ref.slice(slash + 1)
if (!loaded.connections[name]) throw new Error(`embedding: ${(e as Error).message}`)
} }
} }
const c = loaded.connections[name]!
if (c.dialect === "anthropic" || c.dialect === "gemini") throw new Error(`embedding: the ${c.dialect} dialect has no embeddings here; use an OpenAI-compatible or Ollama connection`)
const ollama = c.dialect === "ollama"
return { return {
model: ref, model: ref,
async embed(texts, signal) { async embed(texts, signal) {
const res = await request( const res = await request(
joinUrl(c.base_url, ollama ? "api/embed" : "embeddings"), joinUrl(c.base_url, "embeddings"),
{ method: "POST", headers: authHeaders(c, resolveKey(name, c)), body: JSON.stringify({ model: id, input: texts }), signal, timeoutMs: (c.timeout ?? 120) * 1000, tls: tlsFor(c) }, { method: "POST", headers: authHeaders(c, resolveKey(name, c)), body: JSON.stringify({ model: id, input: texts }), signal, timeoutMs: (c.timeout ?? 120) * 1000, tls: tlsFor(c) },
`${ref} (embeddings)`, `${ref} (embeddings)`,
) )
const j = (await res.json()) as { data?: { embedding: number[]; index?: number }[]; embeddings?: number[][] } const j = (await res.json()) as { data?: { embedding: number[]; index?: number }[] }
const rows = ollama ? (j.embeddings ?? []) : [...(j.data ?? [])].sort((a, b) => (a.index ?? 0) - (b.index ?? 0)).map((d) => d.embedding) return [...(j.data ?? [])].sort((a, b) => (a.index ?? 0) - (b.index ?? 0)).map((d) => unit(d.embedding))
return rows.map(unit)
}, },
} }
} }
+1 -1
View File
@@ -8,7 +8,7 @@ const CONFIG_TEMPLATE = `# LLeMbas CLI settings for this project. Merged over ~/
# and read only while the project is trusted. Connections cannot be set here. # and read only while the project is trusted. Connections cannot be set here.
# #
# mode: edit # manual | edit | auto | plan # mode: edit # manual | edit | auto | plan
# model: connection/model # model: provider/model
# permission: # permission:
# bash: # bash:
# "npm test *": allow # "npm test *": allow
-228
View File
@@ -1,228 +0,0 @@
// Anthropic Messages (`POST {base}/messages`) — api.anthropic.com, and servers that imitate it
// (vLLM, llama-server). Content blocks both ways; thinking with signatures that must be sent back
// on the next turn; optional prompt-caching breakpoints.
import type { Effort } from "../config/schema.ts"
import { resolveKey } from "../config/load.ts"
import { Parts, record, recordFailure, retrying, statusOf, SwapBanner } from "./common.ts"
import { authHeaders, joinUrl, request, tlsFor } from "./http.ts"
import { sseJson } from "./sse.ts"
import { estimateTokens, estimateValue } from "./tokens.ts"
import { ProviderError, type ChatRequest, type Client, type DiscoveredModel, type Message, type ResolvedModel, type StreamEvent, type ToolCallPart } from "./types.ts"
/** Thinking budgets by effort, when the model's config gives no effort_map. */
export const DEFAULT_BUDGETS: Record<Effort, number> = { minimal: 1024, low: 2048, medium: 8192, high: 16384, xhigh: 32000, max: 48000 }
type Block = Record<string, unknown>
type AMessage = { role: "user" | "assistant"; content: Block[] }
/** Anthropic ids must match ^[a-zA-Z0-9_-]+$; ids from other providers are mapped consistently. */
export const safeId = (id: string) => id.replace(/[^a-zA-Z0-9_-]/g, "_") || "call"
export function toAnthropicMessages(messages: Message[], vision: boolean, thinking: boolean): AMessage[] {
const out: AMessage[] = []
const push = (role: AMessage["role"], blocks: Block[]) => {
if (!blocks.length) return
const last = out[out.length - 1]
// Consecutive turns of one role are merged: tool results and a following note are one user turn.
if (last && last.role === role) last.content.push(...blocks)
else out.push({ role, content: blocks })
}
for (const m of messages) {
if (m.role === "user") {
push(
"user",
m.parts.map((p) =>
p.type === "text"
? { type: "text", text: p.text }
: vision
? { type: "image", source: { type: "base64", media_type: p.mime, data: p.data } }
: { type: "text", text: "[image omitted: this model has no vision]" },
),
)
} else if (m.role === "assistant") {
const blocks: Block[] = []
for (const p of m.parts) {
if (p.type === "reasoning") {
// A thinking block can only go back with its signature, and only while thinking is on.
if (!thinking) continue
const redacted = (p.opaque as { redacted?: string } | undefined)?.redacted
if (redacted) blocks.push({ type: "redacted_thinking", data: redacted })
else if (p.signature) blocks.push({ type: "thinking", thinking: p.text, signature: p.signature })
} else if (p.type === "text") {
if (p.text.trim()) blocks.push({ type: "text", text: p.text })
} else {
let input: unknown = {}
try {
input = p.args.trim() ? JSON.parse(p.args) : {}
} catch {
input = { _unparsed: p.args }
}
blocks.push({ type: "tool_use", id: safeId(p.id), name: p.name, input })
}
}
push("assistant", blocks)
} else {
push("user", [{ type: "tool_result", tool_use_id: safeId(m.callId), content: m.content || "(empty)", ...(m.isError ? { is_error: true } : {}) }])
}
}
return out
}
export class AnthropicClient implements Client {
constructor(private m: ResolvedModel) {}
private headers() {
return authHeaders(this.m.connection, resolveKey(this.m.connectionName, this.m.connection), { "anthropic-version": "2023-06-01", ...this.m.spec.headers })
}
buildBody(req: ChatRequest): Record<string, unknown> {
const { m } = this
const budget = req.effort ? (m.spec.effort_map?.[req.effort] ?? DEFAULT_BUDGETS[req.effort]) : undefined
let maxTokens = m.spec.max_output ?? 8192
if (budget && maxTokens <= budget) maxTokens = budget + Math.min(8192, maxTokens)
const cache = m.spec.cache === true
const body: Record<string, unknown> = {
model: m.id,
max_tokens: maxTokens,
stream: true,
messages: toAnthropicMessages(req.messages, m.spec.vision === true, !!budget),
...m.connection.body,
...m.spec.body,
}
if (req.system) body.system = cache ? [{ type: "text", text: req.system, cache_control: { type: "ephemeral" } }] : req.system
if (req.tools.length && m.spec.tools !== false) {
const tools = req.tools.map((t) => ({ name: t.name, description: t.description, input_schema: t.parameters }))
if (cache) (tools[tools.length - 1] as Block).cache_control = { type: "ephemeral" }
body.tools = tools
}
if (cache) {
const msgs = body.messages as AMessage[]
const last = msgs[msgs.length - 1]?.content
if (last?.length) last[last.length - 1] = { ...last[last.length - 1], cache_control: { type: "ephemeral" } }
}
if (budget) body.thinking = { type: "enabled", budget_tokens: budget }
else {
// Sampling parameters and extended thinking do not mix on this API.
if (m.spec.temperature !== undefined) body.temperature = m.spec.temperature
if (m.spec.top_p !== undefined) body.top_p = m.spec.top_p
}
return body
}
async *stream(req: ChatRequest): AsyncGenerator<StreamEvent> {
const body = this.buildBody(req)
yield* retrying(() => this.attempt(req, body))
}
private async *attempt(req: ChatRequest, body: Record<string, unknown>): AsyncGenerator<StreamEvent> {
const { m } = this
const res = await request(
joinUrl(m.connection.base_url, "messages"),
{ method: "POST", headers: this.headers(), body: JSON.stringify(body), signal: req.signal, timeoutMs: (m.connection.timeout ?? 600) * 1000, tls: tlsFor(m.connection) },
m.ref,
).catch((e) => {
recordFailure(m.ref, body, e)
throw e
})
if (!res.body) throw new ProviderError(`${m.ref}: empty response`)
const out = new Parts()
const banner = new SwapBanner()
type Open = { type: string; text: string; signature?: string; id?: string; name?: string; json: string; data?: string; index: number }
const blocks = new Map<number, Open>()
let toolIndex = 0
let finish = "stop"
let input = 0
let output = 0
let cached: number | undefined
let sawUsage = false
for await (const { data } of sseJson(record(res.body, m.ref, body))) {
const ev = data as any
switch (ev?.type) {
case "message_start": {
const u = ev.message?.usage
if (u) {
sawUsage = true
cached = u.cache_read_input_tokens || undefined
input = (u.input_tokens ?? 0) + (u.cache_read_input_tokens ?? 0) + (u.cache_creation_input_tokens ?? 0)
output = u.output_tokens ?? 0
}
break
}
case "content_block_start": {
const b = ev.content_block ?? {}
blocks.set(ev.index, { type: b.type, text: b.text ?? b.thinking ?? "", id: b.id, name: b.name, json: "", data: b.data, index: ev.index })
if (b.type === "tool_use") yield { type: "tool_call_delta", index: toolIndex, name: b.name, argsDelta: "" }
if (b.type === "text" && b.text) {
out.add("text", b.text)
yield { type: "text", text: b.text }
}
break
}
case "content_block_delta": {
const blk = blocks.get(ev.index)
const d = ev.delta ?? {}
if (!blk) break
if (d.type === "text_delta") {
out.add("text", d.text)
yield { type: "text", text: d.text }
} else if (d.type === "thinking_delta") {
const b = banner.feed(d.thinking ?? "")
if (b.notice) yield { type: "notice", message: b.notice }
if (b.text) {
blk.text += b.text
if (b.text.trim()) out.output = true
yield { type: "reasoning", text: b.text }
}
} else if (d.type === "signature_delta") blk.signature = (blk.signature ?? "") + d.signature
else if (d.type === "input_json_delta") {
blk.json += d.partial_json ?? ""
out.output = true
yield { type: "tool_call_delta", index: toolIndex, argsDelta: d.partial_json ?? "" }
}
break
}
case "content_block_stop": {
const blk = blocks.get(ev.index)
if (!blk) break
if (blk.type === "thinking" && blk.text) out.push({ type: "reasoning", text: blk.text, signature: blk.signature })
else if (blk.type === "redacted_thinking") out.push({ type: "reasoning", text: "", opaque: { redacted: blk.data } })
else if (blk.type === "tool_use") {
const call: ToolCallPart = { type: "tool_call", id: blk.id || `toolu_${toolIndex}`, name: blk.name ?? "", args: blk.json || "{}" }
out.push(call)
toolIndex++
}
blocks.delete(ev.index)
break
}
case "message_delta": {
if (ev.delta?.stop_reason) finish = ev.delta.stop_reason
if (ev.usage?.output_tokens !== undefined) {
sawUsage = true
output = ev.usage.output_tokens
}
break
}
case "error": {
const err = ev.error ?? {}
throw new ProviderError(`${m.ref}: ${err.message ?? JSON.stringify(err)}`, statusOf(err) ?? (err.type === "overloaded_error" ? 529 : undefined), undefined, !out.output)
}
default:
if (ev?.error) throw new ProviderError(`${m.ref}: ${ev.error.message ?? JSON.stringify(ev.error)}`, statusOf(ev.error), undefined, !out.output)
}
}
const message = out.message()
const reason = finish === "max_tokens" ? "length" : finish === "tool_use" || message.parts.some((p) => p.type === "tool_call") ? "tool_calls" : "stop"
yield sawUsage
? { type: "usage", usage: { input, output, cached } }
: { type: "usage", usage: { input: estimateValue(body.messages), output: message.parts.reduce((n, p) => n + estimateTokens(p.type === "tool_call" ? p.args : p.text), 0), estimated: true } }
yield { type: "finish", reason, message }
}
async listModels(): Promise<DiscoveredModel[]> {
const res = await request(joinUrl(this.m.connection.base_url, "models"), { headers: this.headers(), timeoutMs: 15_000, tls: tlsFor(this.m.connection) }, this.m.connectionName)
const j = (await res.json()) as any
return (Array.isArray(j?.data) ? j.data : []).map((x: any) => ({ id: String(x.id), context: x.context_window ?? x.max_input_tokens })).filter((x: DiscoveredModel) => x.id)
}
}
-53
View File
@@ -1,53 +0,0 @@
// A model's context window when the config does not give one. Config always wins; what is found
// here is kept in learned.json. Tried in order: the server's model list; llama-server's /props
// (directly, or through llama-swap's /upstream/<model>/ passthrough); Ollama's /api/show.
import { learned, learnContext } from "./learned.ts"
import { authHeaders, request, tlsFor } from "./http.ts"
import { OllamaClient } from "./ollama.ts"
import type { Client, ResolvedModel } from "./types.ts"
import { resolveKey } from "../config/load.ts"
/** The window to plan with: configured, else learned, else undefined (unknown). */
export function contextWindow(m: ResolvedModel): number | undefined {
return m.spec.context || learned().contexts[m.ref] || undefined
}
export async function discoverContext(m: ResolvedModel, client: Client): Promise<number | undefined> {
if (m.spec.context) return m.spec.context
const known = learned().contexts[m.ref]
if (known) return known
const found = (await fromList(m, client)) ?? (await fromProps(m)) ?? (m.connection.dialect === "ollama" ? await (client as OllamaClient).contextOf(m.id).catch(() => undefined) : undefined)
if (found && found > 0) learnContext(m.ref, found)
return found
}
async function fromList(m: ResolvedModel, client: Client): Promise<number | undefined> {
try {
return (await client.listModels()).find((d) => d.id === m.id)?.context
} catch {
return undefined
}
}
async function fromProps(m: ResolvedModel): Promise<number | undefined> {
const base = m.connection.base_url.replace(/\/+$/, "")
const origin = base.replace(/\/v1$/, "")
const headers = authHeaders(m.connection, resolveKey(m.connectionName, m.connection))
for (const url of [`${origin}/upstream/${encodeURIComponent(m.id)}/props`, `${origin}/props`]) {
try {
const res = await request(url, { headers, timeoutMs: 15_000, tls: tlsFor(m.connection) }, m.ref)
const j = (await res.json()) as any
const n = Number(j?.default_generation_settings?.n_ctx ?? j?.n_ctx)
if (Number.isFinite(n) && n > 0) return n
} catch {}
}
return undefined
}
/** llama-swap and friends: free the connection's memory when the session moves elsewhere. */
export async function unload(m: ResolvedModel): Promise<void> {
if (!m.connection.unload_url) return
try {
await request(m.connection.unload_url, { method: m.connection.unload_method ?? "POST", headers: authHeaders(m.connection, resolveKey(m.connectionName, m.connection)), timeoutMs: 10_000, tls: tlsFor(m.connection) }, m.ref)
} catch {}
}
-316
View File
@@ -1,316 +0,0 @@
// Google Gemini, native (`{base}/models/{id}:streamGenerateContent?alt=sse`), base
// https://generativelanguage.googleapis.com/v1beta. UNTESTED: no Gemini endpoint has been
// available to try it against (see the wiki's Working-notes).
import type { Effort } from "../config/schema.ts"
import { resolveKey } from "../config/load.ts"
import { DEFAULT_BUDGETS } from "./anthropic.ts"
import { Parts, record, recordFailure, retrying, statusOf } from "./common.ts"
import { authHeaders, joinUrl, request, tlsFor } from "./http.ts"
import { sseJson } from "./sse.ts"
import { estimateTokens, estimateValue } from "./tokens.ts"
import { ProviderError, type ChatRequest, type Client, type DiscoveredModel, type Message, type ResolvedModel, type StreamEvent, type ToolCallPart } from "./types.ts"
type Content = { role: "user" | "model"; parts: Record<string, unknown>[] }
// Checked against Hermes Agent (agent/gemini_native_adapter.py, agent/gemini_schema.py) and
// OpenCode (provider/transform.ts). Still untested against a real Gemini endpoint.
/** Gemini's own id, without an aggregator prefix. */
const bare = (id: string) => id.replace(/^(models|google|gemini)\//i, "")
/** Gemini 3 and later: tool calls and results carry ids, and thinking is a level, not a budget. */
export const isGemini3 = (id: string) => Number(/gemini-(\d+)/i.exec(bare(id))?.[1] ?? 0) >= 3
const LEGACY = /gemini-(?:(?:flash|pro)-)?[12](?:[.-]|$)/i
/** A call replayed without a signature (made by another model, or before signatures existed):
* Google's documented sentinel, without which Gemini 3 refuses the whole history. */
export const SKIP_SIGNATURE = "skip_thought_signature_validator"
/** Thought tokens count against maxOutputTokens, and Gemini's own default is low. */
export const DEFAULT_MAX_OUTPUT = 65535
const PLACEHOLDER = "[The previous response was interrupted before it completed.]"
// ── tool schemas ──────────────────────────────────────────────────────────────────────────────
// v1beta takes plain JSON Schema as `parametersJsonSchema`; only same-document $refs have to be
// inlined. Other API versions take `parameters`, an OpenAPI subset where one unknown key 400s the
// whole request, so that path drops everything outside the subset.
const ALLOWED = new Set(["type", "format", "title", "description", "nullable", "enum", "maxItems", "minItems", "properties", "required", "minProperties", "maxProperties", "minLength", "maxLength", "pattern", "example", "anyOf", "propertyOrdering", "default", "items", "minimum", "maximum"])
const STRUCTURAL: Record<string, string[]> = { array: ["items", "minItems", "maxItems"], object: ["properties", "required", "minProperties", "maxProperties", "propertyOrdering"] }
const isObj = (v: unknown): v is Record<string, unknown> => !!v && typeof v === "object" && !Array.isArray(v)
export function geminiSchema(s: unknown): Record<string, unknown> {
if (!isObj(s)) return {}
const out: Record<string, unknown> = {}
for (const [k, v] of Object.entries(s)) {
if (!ALLOWED.has(k)) continue
if (k === "properties") {
if (isObj(v)) out.properties = Object.fromEntries(Object.entries(v).map(([p, sub]) => [p, geminiSchema(sub)]))
} else if (k === "items") out.items = geminiSchema(v)
else if (k === "anyOf") {
if (Array.isArray(v)) out.anyOf = v.filter(isObj).map(geminiSchema)
} else out[k] = v
}
const types = out.type
if (Array.isArray(types)) {
delete out.type
const real = types.filter((t): t is string => typeof t === "string" && t !== "null")
if (!real.length) out.type = types.includes("null") ? "null" : "object"
else if (real.length === 1) out.type = real[0]
else {
// Several types: one branch each, and the structure belongs on the branch it describes.
const moved: Record<string, unknown> = {}
for (const keys of Object.values(STRUCTURAL)) for (const k of keys) if (k in out) (moved[k] = out[k]), delete out[k]
out.anyOf = real.map((t) => geminiSchema({ type: t, ...Object.fromEntries(Object.entries(moved).filter(([k]) => STRUCTURAL[t]?.includes(k))) }))
}
if (types.includes("null")) out.nullable = true
}
// Every enum entry must be a string, whatever the declared type.
if (Array.isArray(out.enum) && (Array.isArray(types) || ["integer", "number", "boolean"].includes(out.type as string))) {
const vals = [...new Set(out.enum.map((e) => (typeof e === "boolean" || (typeof e === "number" && Number.isFinite(e)) ? String(e) : typeof e === "string" ? e : undefined)).filter((e) => e !== undefined))]
if (vals.length) out.enum = vals
else delete out.enum
}
// `required` naming a property that is not there fails the entire request.
if (Array.isArray(out.required)) {
const names = isObj(out.properties) ? out.properties : {}
const req = out.required.filter((r): r is string => typeof r === "string" && r in names)
if (req.length) out.required = req
else delete out.required
}
return out
}
/** For `parameters`: the subset, and never an empty schema. */
export function geminiParameters(s: unknown): Record<string, unknown> {
const out = geminiSchema(s)
return Object.keys(out).length ? out : { type: "object", properties: {} }
}
function inlineRefs(node: unknown, root: Record<string, unknown>, budget: { n: number }, stack: string[] = []): unknown {
if (Array.isArray(node)) return node.map((x) => inlineRefs(x, root, budget, stack))
if (!isObj(node)) return node
const ref = node.$ref
if (typeof ref !== "string") return Object.fromEntries(Object.entries(node).map(([k, v]) => [k, inlineRefs(v, root, budget, stack)]))
if (stack.includes(ref)) throw new Error(`circular $ref ${ref}`)
if (--budget.n < 0) throw new Error("$ref budget exhausted")
if (!ref.startsWith("#/")) throw new Error(`unresolvable $ref ${ref}`)
let target: unknown = root
for (const raw of ref.slice(2).split("/")) {
const part = raw.replace(/~1/g, "/").replace(/~0/g, "~")
target = isObj(target) ? target[part] : undefined
}
if (!isObj(target)) throw new Error(`unresolvable $ref ${ref}`)
const inlined = inlineRefs(target, root, budget, [...stack, ref]) as Record<string, unknown>
// Siblings of a $ref apply alongside it and win.
const { $ref: _, ...siblings } = node
return Object.keys(siblings).length ? { ...inlined, ...(inlineRefs(siblings, root, budget, stack) as object) } : inlined
}
/** For `parametersJsonSchema`: root $schema dropped, local $refs inlined, an object root. A schema
* whose refs cannot all be resolved goes as it is, so the provider names the real problem. */
export function geminiJsonSchema(s: unknown): Record<string, unknown> {
if (!isObj(s) || !Object.keys(s).length) return { type: "object", properties: {} }
const { $schema: _, ...rest } = structuredClone(s)
let out: Record<string, unknown>
try {
out = inlineRefs(rest, rest, { n: 256 }) as Record<string, unknown>
} catch {
return rest
}
delete out.$defs
delete out.definitions
if (!Object.keys(out).length) return { type: "object", properties: {} }
if (out.type === "object" && !("properties" in out)) out.properties = {}
return out
}
/** `parametersJsonSchema` exists only on v1beta. */
export const acceptsJsonSchema = (base: string) => /\/v1beta\/*$/i.test(base.trim())
// ── history ───────────────────────────────────────────────────────────────────────────────────
/** A JSON Schema inside a function response is resolved by Gemini 3, and an unknown $ref 400s. */
const hasRef = (v: unknown): boolean => (Array.isArray(v) ? v.some(hasRef) : isObj(v) ? Object.entries(v).some(([k, x]) => (k === "$ref" && typeof x === "string" && x.startsWith("#/")) || hasRef(x)) : false)
function functionResponse(content: string, isError: boolean | undefined): Record<string, unknown> {
if (isError) return { error: content }
// A JSON object result goes structured; anything else under "output", as Google documents.
if (/^\s*\{/.test(content))
try {
const parsed = JSON.parse(content)
if (isObj(parsed) && !hasRef(parsed)) return parsed
} catch {}
return { output: content }
}
export function toGeminiContents(messages: Message[], vision: boolean, ids = false): Content[] {
const out: Content[] = []
const names = new Map<string, string>()
const results = (c: Content) => c.parts.some((p) => "functionResponse" in p)
const push = (role: Content["role"], parts: Record<string, unknown>[]) => {
if (!parts.length) return
const last = out[out.length - 1]
// Same-role turns must merge (Gemini insists on alternation), except that a person's words are
// never folded into a turn of tool results: Gemini 3 then reads them as part of the result and
// answers with nothing. A placeholder model turn keeps the two apart.
if (last && last.role === role && role === "user" && results(last) !== parts.some((p) => "functionResponse" in p)) out.push({ role: "model", parts: [{ text: PLACEHOLDER }] }, { role, parts })
else if (last && last.role === role) last.parts.push(...parts)
else out.push({ role, parts })
}
for (const m of messages) {
if (m.role === "user")
push(
"user",
m.parts.map((p) => (p.type === "text" ? { text: p.text } : vision ? { inlineData: { mimeType: p.mime, data: p.data } } : { text: "[image omitted: this model has no vision]" })),
)
else if (m.role === "assistant") {
const parts: Record<string, unknown>[] = []
for (const p of m.parts) {
if (p.type === "text" && p.text.trim()) parts.push({ text: p.text })
else if (p.type === "tool_call") {
names.set(p.id, p.name)
let args: unknown = {}
try {
args = p.args.trim() ? JSON.parse(p.args) : {}
} catch {
args = { _raw: p.args }
}
// Gemini 3 wants the thought signature back on the call it came with.
parts.push({ functionCall: { name: p.name, args: isObj(args) ? args : { _value: args }, ...(ids ? { id: p.id } : {}) }, thoughtSignature: p.signature ?? SKIP_SIGNATURE })
}
}
push("model", parts)
} else push("user", [{ functionResponse: { name: names.get(m.callId) ?? m.name, response: functionResponse(m.content, m.isError), ...(ids ? { id: m.callId } : {}) } }])
}
return out
}
// ── thinking ──────────────────────────────────────────────────────────────────────────────────
const LEVELS = ["minimal", "low", "medium", "high"] as const
/** The levels a Gemini 3 model takes (OpenCode's table). */
function levelsOf(id: string): readonly string[] {
const s = id.toLowerCase()
if (s.includes("gemma") || s.includes("flash-image")) return ["minimal", "high"]
if (s.includes("pro-image")) return ["high"]
if (s.includes("flash")) return LEVELS
return ["low", "medium", "high"]
}
export function thinkingConfig(id: string, effort: Effort, map?: Partial<Record<Effort, number>>): Record<string, unknown> {
if (map?.[effort]) return { thinkingBudget: map[effort], includeThoughts: true }
const b = bare(id)
if (LEGACY.test(b)) {
// 2.x takes a budget; 2.5 Pro goes to 32k, the rest to 24k.
const max = /gemini-2[.-]5/i.test(b) && /pro/i.test(b) && !/flash/i.test(b) ? 32768 : 24576
return { thinkingBudget: Math.min(DEFAULT_BUDGETS[effort], max), includeThoughts: true }
}
// 3 and later take a level: the one asked for, else the next one up the model has, else its top.
const i = LEVELS.indexOf(effort as (typeof LEVELS)[number])
const want = i < 0 ? LEVELS.length - 1 : i // xhigh and max: the top
const have = levelsOf(b)
const level = LEVELS.slice(want).find((l) => have.includes(l)) ?? have[have.length - 1]
return { thinkingLevel: level, includeThoughts: true }
}
/** Keys sorted, so a call sent again compares equal to itself (as Hermes does). */
const sortedJson = (v: unknown) => JSON.stringify(v, (_, x) => (isObj(x) ? Object.fromEntries(Object.keys(x).sort().map((k) => [k, x[k]])) : x))
const FINISH: Record<string, string> = { STOP: "stop", MAX_TOKENS: "length", SAFETY: "content_filter", RECITATION: "content_filter", BLOCKLIST: "content_filter", PROHIBITED_CONTENT: "content_filter", SPII: "content_filter" }
export class GeminiClient implements Client {
constructor(private m: ResolvedModel) {}
private headers() {
return authHeaders(this.m.connection, resolveKey(this.m.connectionName, this.m.connection), this.m.spec.headers)
}
buildBody(req: ChatRequest): Record<string, unknown> {
const { m } = this
const generationConfig: Record<string, unknown> = {}
if (m.spec.temperature !== undefined) generationConfig.temperature = m.spec.temperature
generationConfig.maxOutputTokens = m.spec.max_output ?? DEFAULT_MAX_OUTPUT
if (m.spec.top_p !== undefined) generationConfig.topP = m.spec.top_p
if (req.effort) generationConfig.thinkingConfig = thinkingConfig(m.id, req.effort, m.spec.effort_map)
const body: Record<string, unknown> = { contents: toGeminiContents(req.messages, m.spec.vision === true, isGemini3(m.id)), generationConfig, ...m.connection.body, ...m.spec.body }
if (req.system) body.systemInstruction = { parts: [{ text: req.system }] }
if (req.tools.length && m.spec.tools !== false) {
const json = acceptsJsonSchema(m.connection.base_url)
body.tools = [{ functionDeclarations: req.tools.map((t) => ({ name: t.name, description: t.description, ...(json ? { parametersJsonSchema: geminiJsonSchema(t.parameters) } : { parameters: geminiParameters(t.parameters) }) })) }]
}
return body
}
async *stream(req: ChatRequest): AsyncGenerator<StreamEvent> {
const body = this.buildBody(req)
yield* retrying(() => this.attempt(req, body))
}
private async *attempt(req: ChatRequest, body: Record<string, unknown>): AsyncGenerator<StreamEvent> {
const { m } = this
const url = joinUrl(m.connection.base_url, `models/${encodeURIComponent(m.id)}:streamGenerateContent?alt=sse`)
const res = await request(url, { method: "POST", headers: this.headers(), body: JSON.stringify(body), signal: req.signal, timeoutMs: (m.connection.timeout ?? 600) * 1000, tls: tlsFor(m.connection) }, m.ref).catch((e) => {
recordFailure(m.ref, body, e)
throw e
})
if (!res.body) throw new ProviderError(`${m.ref}: empty response`)
const out = new Parts()
let finish = "stop"
let usage: StreamEvent | undefined
// One slot per call. Gemini 3 ids its calls; 2.5 does not, restarts part numbering in every
// event and may send a call again whole, so a payload repeating (or extending) a slot's arguments
// is that call and anything else is a new one.
const slots = new Map<string, { index: number; part: ToolCallPart }>()
const slotFor = (fc: any, sig: string, at: number, args: string) => {
if (typeof fc.id === "string" && fc.id) return `id:${fc.id}`
const key = `${at}:${fc.name}:${sig}`
const s = slots.get(key)
if (!s || args.startsWith(s.part.args)) return key
for (const [k, o] of slots) if (k.startsWith(`${key}#`) && args.startsWith(o.part.args)) return k
return `${key}#${slots.size}`
}
for await (const { data } of sseJson(record(res.body, m.ref, body))) {
const chunk = data as any
if (chunk?.error) throw new ProviderError(`${m.ref}: ${chunk.error.message ?? JSON.stringify(chunk.error)}`, statusOf(chunk.error) ?? chunk.error.code, undefined, !out.output)
const cand = chunk?.candidates?.[0]
const parts: any[] = cand?.content?.parts ?? []
for (const [at, p] of parts.entries()) {
if (typeof p.text === "string" && p.text) {
const kind = p.thought === true ? "reasoning" : "text"
out.add(kind, p.text)
yield { type: kind, text: p.text }
}
const fc = p.functionCall
if (fc?.name) {
const args = sortedJson(fc.args ?? {})
const sig = typeof p.thoughtSignature === "string" ? p.thoughtSignature : ""
const key = slotFor(fc, sig, at, args)
let s = slots.get(key)
if (!s) {
s = { index: slots.size, part: { type: "tool_call", id: typeof fc.id === "string" && fc.id ? fc.id : `gemini_call_${slots.size}`, name: fc.name, args: "" } }
slots.set(key, s)
out.push(s.part)
}
const prev = s.part.args
s.part.args = args
if (sig) s.part.signature = sig
yield { type: "tool_call_delta", index: s.index, name: fc.name, argsDelta: args.startsWith(prev) ? args.slice(prev.length) : args }
}
}
if (cand?.finishReason) finish = FINISH[String(cand.finishReason).toUpperCase()] ?? "stop"
const u = chunk?.usageMetadata
if (u) usage = { type: "usage", usage: { input: u.promptTokenCount ?? 0, output: (u.candidatesTokenCount ?? 0) + (u.thoughtsTokenCount ?? 0), reasoning: u.thoughtsTokenCount, cached: u.cachedContentTokenCount } }
}
const message = out.message()
if (slots.size && finish !== "length") finish = "tool_calls"
yield usage ?? { type: "usage", usage: { input: estimateValue(body.contents), output: message.parts.reduce((n, p) => n + estimateTokens(p.type === "tool_call" ? p.args : p.text), 0), estimated: true } }
yield { type: "finish", reason: finish, message }
}
async listModels(): Promise<DiscoveredModel[]> {
const res = await request(joinUrl(this.m.connection.base_url, "models?pageSize=200"), { headers: this.headers(), timeoutMs: 15_000, tls: tlsFor(this.m.connection) }, this.m.connectionName)
const j = (await res.json()) as any
return (j?.models ?? [])
.filter((x: any) => !x.supportedGenerationMethods || x.supportedGenerationMethods.includes("generateContent"))
.map((x: any) => ({ id: String(x.name ?? "").replace(/^models\//, ""), context: x.inputTokenLimit }))
.filter((x: DiscoveredModel) => x.id)
}
}
+3 -13
View File
@@ -27,20 +27,10 @@ export function tlsFor(c: Pick<Connection, "tls">): { ca?: string; rejectUnautho
return out return out
} }
const DEFAULT_AUTH: Record<Connection["dialect"], NonNullable<Connection["auth"]>> = { /** The server's token as a bearer, plus the connection's and model's own headers. */
"openai-chat": "bearer", export function authHeaders(c: Pick<Connection, "headers">, key: string | undefined, extra: Record<string, string> = {}): Record<string, string> {
responses: "bearer",
anthropic: "x-api-key",
gemini: "x-goog-api-key",
ollama: "bearer",
}
/** Auth plus the connection's and model's own headers. */
export function authHeaders(c: Connection, key: string | undefined, extra: Record<string, string> = {}): Record<string, string> {
const h: Record<string, string> = { "content-type": "application/json", ...extra } const h: Record<string, string> = { "content-type": "application/json", ...extra }
const style = c.auth ?? DEFAULT_AUTH[c.dialect] if (key) h.authorization = `Bearer ${key}`
if (key && style === "bearer") h.authorization = `Bearer ${key}`
else if (key && style !== "none") h[style] = key
return { ...h, ...c.headers } return { ...h, ...c.headers }
} }
+27 -45
View File
@@ -1,64 +1,40 @@
import { findRef, modelRefs } from "./refs.ts" import { findRef, modelRefs } from "./refs.ts"
import type { Loaded } from "../config/load.ts" import type { Loaded } from "../config/load.ts"
import { AnthropicClient } from "./anthropic.ts"
import { OpenAIChatClient } from "./openai-chat.ts" import { OpenAIChatClient } from "./openai-chat.ts"
import { ResponsesClient } from "./responses.ts" import type { Client, ResolvedModel } from "./types.ts"
import { GeminiClient } from "./gemini.ts"
import { OllamaClient } from "./ollama.ts"
import { ProviderError, type Client, type ResolvedModel } from "./types.ts"
export class ModelError extends Error {} export class ModelError extends Error {}
/** `connection/model`. The model part may itself contain slashes (`openrouter/qwen/qwen3-coder`). */ /** What a start without a login says: LLeMbas CLI 2.0 has no model of its own, only the server's. */
export const NOT_LOGGED_IN = "Not logged in to a LLeMbas server. LLeMbas CLI uses the models of the server it is logged in to: lembas login <address> (e.g. lembas login ai.example.org)."
/** A model of the server this machine is logged in to, by its served id (`deepseek/deepseek-flash`)
* or an older form of its name (refs.ts). Without a ref: config.yaml's `model`, else the server's
* default for the account, else its first. */
export function resolveModel(loaded: Loaded, ref: string | undefined): ResolvedModel { export function resolveModel(loaded: Loaded, ref: string | undefined): ResolvedModel {
const refs = loaded.refs ?? modelRefs(loaded.connections) const refs = loaded.refs ?? modelRefs(loaded.connections)
if (!ref) { const [name, connection] = Object.entries(loaded.connections)[0] ?? []
// A LLeMbas instance says which model its account starts on; that, before anything else. if (!name || !connection) {
const instance = Object.entries(loaded.connections).find(([, c]) => c.webui?.default && c.models[c.webui.default]) const why = Object.values(loaded.broken ?? {})[0]
if (instance) ref = refs.find((r) => r.connection === instance[0] && r.id === instance[1].webui!.default)?.ref ?? `${instance[0]}/${instance[1].webui!.default}` throw new ModelError(why ? `The LLeMbas login is unusable: ${why}` : NOT_LOGGED_IN)
} }
if (!ref) { if (loaded.broken?.[name]) throw new ModelError(`The LLeMbas login ${name} is unusable: ${loaded.broken[name]}`)
const first = refs[0] if (!ref) ref = connection.webui?.default && connection.models[connection.webui.default] ? connection.webui.default : refs[0]?.ref
if (!first) throw new ModelError("No model configured. Add a connection to ~/.config/lembas/connections.yaml and set `model:` in config.yaml.") if (!ref) throw new ModelError(`${name} serves no model to this account yet. lembas login ${connection.webui?.url ?? ""} again once it does.`.replace(" ", " "))
ref = first.ref
}
// Any form of the name (`deepseek/x`, or the old `example/x`): the model it names, under
// the ref it has now.
const known = findRef(loaded.connections, refs, ref) const known = findRef(loaded.connections, refs, ref)
if (known) { if (known) return { ref: known.ref, connectionName: name, connection, id: known.id, spec: connection.models[known.id] ?? {} }
const connection = loaded.connections[known.connection]! // A model the server serves that its list did not have when it was read: the server decides
if (loaded.broken?.[known.connection]) throw new ModelError(`Connection "${known.connection}" is unusable: ${loaded.broken?.[known.connection]}`) // which models exist, so it is asked for rather than refused here. The login's own name in front
return { ref: known.ref, connectionName: known.connection, connection, id: known.id, spec: connection.models[known.id] ?? {} } // of it (an older form) is taken off.
} const id = ref.startsWith(`${name}/`) ? ref.slice(name.length + 1) : ref
const slash = ref.indexOf("/") return { ref: id, connectionName: name, connection, id, spec: {} }
if (slash <= 0) throw new ModelError(`Model "${ref}" must be written connection/model`)
const connectionName = ref.slice(0, slash)
const id = ref.slice(slash + 1)
if (loaded.broken?.[connectionName]) throw new ModelError(`Connection "${connectionName}" is unusable: ${loaded.broken[connectionName]}`)
const connection = loaded.connections[connectionName]
if (!connection) throw new ModelError(`No connection named "${connectionName}" (have: ${Object.keys(loaded.connections).join(", ") || "none"})`)
const spec = connection.models[id]
if (!spec && !connection.discover)
throw new ModelError(`Connection "${connectionName}" has no model "${id}" (have: ${Object.keys(connection.models).join(", ") || "none"}). Add it, or set discover: true.`)
return { ref, connectionName, connection, id, spec: spec ?? {} }
} }
export function clientFor(m: ResolvedModel): Client { export function clientFor(m: ResolvedModel): Client {
switch (m.connection.dialect) {
case "openai-chat":
return new OpenAIChatClient(m) return new OpenAIChatClient(m)
case "anthropic":
return new AnthropicClient(m)
case "responses":
return new ResponsesClient(m)
case "gemini":
return new GeminiClient(m)
case "ollama":
return new OllamaClient(m)
}
} }
/** Which prompt family a model gets when the config does not say: anthropic, gpt, gemini, or /** Which prompt family a model gets when the server does not say: anthropic, gpt, gemini, or
* local — every open-weight or unknown model, which gets Hermes' act-don't-describe guidance. */ * local — every open-weight or unknown model, which gets Hermes' act-don't-describe guidance. */
export function familyOf(m: ResolvedModel): string { export function familyOf(m: ResolvedModel): string {
if (m.spec.family) return m.spec.family if (m.spec.family) return m.spec.family
@@ -68,3 +44,9 @@ export function familyOf(m: ResolvedModel): string {
if (/\bgpt|^o[134]\b|^o[134]-|codex/.test(id)) return "gpt" if (/\bgpt|^o[134]\b|^o[134]-|codex/.test(id)) return "gpt"
return "local" return "local"
} }
/** The window to plan with, as the server gives it; undefined when it does not say (no percentage,
* no automatic compaction). */
export function contextWindow(m: ResolvedModel): number | undefined {
return m.spec.context || undefined
}
+5 -10
View File
@@ -6,14 +6,12 @@ import { paths } from "../config/paths.ts"
// What endpoints have taught us, kept across runs in ~/.local/state/lembas/learned.json. // What endpoints have taught us, kept across runs in ~/.local/state/lembas/learned.json.
// Only ever narrows behaviour; deleting the file is always safe. // Only ever narrows behaviour; deleting the file is always safe.
interface Learned { interface Learned {
/** `connection/model` → the efforts it actually accepts. */ /** A model ref → the efforts it actually accepts. */
efforts: Record<string, Effort[]> efforts: Record<string, Effort[]>
/** base URLs that reject `stream_options`. */ /** base URLs that reject `stream_options`. */
noStreamOptions: string[] noStreamOptions: string[]
/** base URLs that reject `return_progress`. */ /** base URLs that reject `return_progress`. */
noProgress: string[] noProgress: string[]
/** `connection/model` → context window found by discovery (config always wins). */
contexts: Record<string, number>
} }
let cache: Learned | undefined let cache: Learned | undefined
@@ -25,9 +23,11 @@ function file() {
export function learned(): Learned { export function learned(): Learned {
if (cache) return cache if (cache) return cache
try { try {
cache = { efforts: {}, noStreamOptions: [], noProgress: [], contexts: {}, ...JSON.parse(readFileSync(file(), "utf8")) } // An older file's `contexts` (what discovery found, before 2.0 took the server's word) is dropped.
const { contexts: _, ...kept } = JSON.parse(readFileSync(file(), "utf8")) as Record<string, unknown>
cache = { efforts: {}, noStreamOptions: [], noProgress: [], ...kept } as Learned
} catch { } catch {
cache = { efforts: {}, noStreamOptions: [], noProgress: [], contexts: {} } cache = { efforts: {}, noStreamOptions: [], noProgress: [] }
} }
return cache! return cache!
} }
@@ -46,11 +46,6 @@ export function learnEfforts(ref: string, efforts: Effort[]) {
save() save()
} }
export function learnContext(ref: string, n: number) {
learned().contexts[ref] = n
save()
}
export function learnNoStreamOptions(baseUrl: string) { export function learnNoStreamOptions(baseUrl: string) {
const l = learned() const l = learned()
if (!l.noStreamOptions.includes(baseUrl)) l.noStreamOptions.push(baseUrl) if (!l.noStreamOptions.includes(baseUrl)) l.noStreamOptions.push(baseUrl)
-178
View File
@@ -1,178 +0,0 @@
// Ollama, native (`POST {base}/api/chat`, NDJSON), base e.g. http://localhost:11434. Its
// OpenAI-compatible /v1 works through openai-chat too; the native API adds num_ctx (without it
// Ollama silently cuts every conversation to its small default window) and `think` levels.
// UNTESTED: no Ollama server has been available to try it against (see the wiki). Checked against
// Hermes Agent's handling (agent/model_metadata.py, agent/reasoning_params.py): /api/show decides
// the window to ask for and whether `think` may be sent at all.
import { resolveKey } from "../config/load.ts"
import { Parts, record, recordFailure, retrying, statusOf } from "./common.ts"
import { authHeaders, joinUrl, request, tlsFor } from "./http.ts"
import { estimateTokens, estimateValue } from "./tokens.ts"
import { ProviderError, type ChatRequest, type Client, type DiscoveredModel, type Message, type ResolvedModel, type StreamEvent } from "./types.ts"
export function toOllamaMessages(system: string, messages: Message[], vision: boolean): Record<string, unknown>[] {
const out: Record<string, unknown>[] = []
if (system) out.push({ role: "system", content: system })
for (const m of messages) {
if (m.role === "user") {
const images = vision ? m.parts.filter((p) => p.type === "image").map((p) => (p as { data: string }).data) : []
const text = m.parts.map((p) => (p.type === "text" ? p.text : vision ? "" : "[image omitted: this model has no vision]")).filter(Boolean).join("\n")
out.push({ role: "user", content: text, ...(images.length ? { images } : {}) })
} else if (m.role === "assistant") {
const text = m.parts.map((p) => (p.type === "text" ? p.text : "")).join("")
const thinking = m.parts.map((p) => (p.type === "reasoning" ? p.text : "")).join("")
const calls = m.parts.flatMap((p) => {
if (p.type !== "tool_call") return []
let args: unknown = {}
try {
args = p.args.trim() ? JSON.parse(p.args) : {}
} catch {}
return [{ function: { name: p.name, arguments: args } }]
})
out.push({ role: "assistant", content: text, ...(thinking ? { thinking } : {}), ...(calls.length ? { tool_calls: calls } : {}) })
} else out.push({ role: "tool", content: m.content, tool_name: m.name })
}
return out
}
/** NDJSON: one JSON object per line. */
async function* ndjson(body: ReadableStream<Uint8Array>): AsyncGenerator<unknown> {
const dec = new TextDecoder()
let buf = ""
for await (const chunk of body) {
buf += dec.decode(chunk, { stream: true })
let nl: number
while ((nl = buf.indexOf("\n")) !== -1) {
const line = buf.slice(0, nl).trim()
buf = buf.slice(nl + 1)
if (line)
try {
yield JSON.parse(line)
} catch {}
}
}
const last = (buf + dec.decode()).trim()
if (last)
try {
yield JSON.parse(last)
} catch {}
}
/** What /api/show says about a model. */
export interface OllamaShow {
/** "completion", "tools", "thinking", "vision"…; absent before Ollama 0.6. */
capabilities?: string[]
/** The runtime window the Modelfile sets (`num_ctx`), which the owner of the server chose. */
numCtx?: number
/** The trained maximum (`model_info["<arch>.context_length"]`). */
trained?: number
}
export function parseShow(j: any): OllamaShow {
const out: OllamaShow = {}
if (Array.isArray(j?.capabilities)) out.capabilities = j.capabilities.map(String)
for (const line of String(j?.parameters ?? "").split("\n")) {
const f = line.trim().split(/\s+/)
if (f[0] === "num_ctx" && Number(f[1]) > 0) out.numCtx = Number(f[1])
}
const info = j?.model_info ?? {}
const key = Object.keys(info).find((k) => k.endsWith(".context_length"))
if (key && Number(info[key]) > 0) out.trained = Number(info[key])
return out
}
export class OllamaClient implements Client {
private shown?: Promise<OllamaShow | undefined>
constructor(private m: ResolvedModel) {}
/** /api/show for this model, asked once; undefined when the server does not answer it. */
show(id = this.m.id): Promise<OllamaShow | undefined> {
const ask = async () => {
const res = await request(joinUrl(this.m.connection.base_url, "api/show"), { method: "POST", headers: this.headers(), body: JSON.stringify({ model: id, name: id }), timeoutMs: 15_000, tls: tlsFor(this.m.connection) }, this.m.connectionName)
return parseShow(await res.json())
}
if (id !== this.m.id) return ask().catch(() => undefined)
return (this.shown ??= ask().catch(() => undefined))
}
private headers() {
return authHeaders(this.m.connection, resolveKey(this.m.connectionName, this.m.connection), this.m.spec.headers)
}
buildBody(req: ChatRequest, show?: OllamaShow): Record<string, unknown> {
const { m } = this
const options: Record<string, unknown> = {}
// Without num_ctx Ollama cuts every conversation to its small default window. Configured wins;
// else the Modelfile's, else the trained maximum (set `context` to cap it to what memory holds).
const ctx = m.spec.context ?? show?.numCtx ?? show?.trained
if (ctx) options.num_ctx = ctx
if (m.spec.max_output) options.num_predict = m.spec.max_output
if (m.spec.temperature !== undefined) options.temperature = m.spec.temperature
if (m.spec.top_p !== undefined) options.top_p = m.spec.top_p
const body: Record<string, unknown> = { model: m.id, messages: toOllamaMessages(req.system, req.messages, m.spec.vision === true), stream: true, options, ...m.connection.body, ...m.spec.body }
if (req.tools.length && m.spec.tools !== false)
body.tools = req.tools.map((t) => ({ type: "function", function: { name: t.name, description: t.description, parameters: t.parameters } }))
// gpt-oss takes a level; other thinking models take a switch. A model /api/show says cannot
// think gets no `think` at all: Ollama refuses the request.
const thinks = !show?.capabilities || show.capabilities.includes("thinking")
if (thinks && req.effort) body.think = /gpt-oss/.test(m.id) ? (req.effort === "minimal" ? "low" : req.effort === "xhigh" || req.effort === "max" ? "high" : req.effort) : true
else if (thinks && m.spec.efforts?.length) body.think = false
return body
}
async *stream(req: ChatRequest): AsyncGenerator<StreamEvent> {
const body = this.buildBody(req, await this.show())
yield* retrying(() => this.attempt(req, body))
}
private async *attempt(req: ChatRequest, body: Record<string, unknown>): AsyncGenerator<StreamEvent> {
const { m } = this
const res = await request(joinUrl(m.connection.base_url, "api/chat"), { method: "POST", headers: this.headers(), body: JSON.stringify(body), signal: req.signal, timeoutMs: (m.connection.timeout ?? 600) * 1000, tls: tlsFor(m.connection) }, m.ref).catch((e) => {
recordFailure(m.ref, body, e)
throw e
})
if (!res.body) throw new ProviderError(`${m.ref}: empty response`)
const out = new Parts()
let finish = "stop"
let usage: StreamEvent | undefined
let calls = 0
for await (const chunk of ndjson(record(res.body, m.ref, body)) as AsyncGenerator<any>) {
if (chunk?.error) throw new ProviderError(`${m.ref}: ${chunk.error}`, statusOf(chunk) ?? 500, undefined, !out.output)
const msg = chunk?.message ?? {}
if (msg.thinking) {
out.add("reasoning", msg.thinking)
yield { type: "reasoning", text: msg.thinking }
}
if (msg.content) {
out.add("text", msg.content)
yield { type: "text", text: msg.content }
}
for (const tc of msg.tool_calls ?? []) {
const args = JSON.stringify(tc.function?.arguments ?? {})
yield { type: "tool_call_delta", index: calls, name: tc.function?.name, argsDelta: args }
out.push({ type: "tool_call", id: tc.id ?? `ollama_call_${calls}`, name: tc.function?.name ?? "", args })
calls++
}
if (chunk?.done) {
finish = chunk.done_reason === "length" ? "length" : "stop"
if (chunk.prompt_eval_count !== undefined || chunk.eval_count !== undefined) usage = { type: "usage", usage: { input: chunk.prompt_eval_count ?? 0, output: chunk.eval_count ?? 0 } }
}
}
const message = out.message()
if (calls && finish !== "length") finish = "tool_calls"
yield usage ?? { type: "usage", usage: { input: estimateValue(body.messages), output: message.parts.reduce((n, p) => n + estimateTokens(p.type === "tool_call" ? p.args : p.text), 0), estimated: true } }
yield { type: "finish", reason: finish, message }
}
async listModels(): Promise<DiscoveredModel[]> {
const res = await request(joinUrl(this.m.connection.base_url, "api/tags"), { headers: this.headers(), timeoutMs: 15_000, tls: tlsFor(this.m.connection) }, this.m.connectionName)
const j = (await res.json()) as any
return (j?.models ?? []).map((x: any) => ({ id: String(x.name ?? x.model ?? "") })).filter((x: DiscoveredModel) => x.id)
}
/** The model's window: the Modelfile's num_ctx, else its trained maximum. */
async contextOf(id: string): Promise<number | undefined> {
const s = await this.show(id)
return s?.numCtx ?? s?.trained
}
}
+18 -59
View File
@@ -1,69 +1,29 @@
// How a model is named: by its provider, not by the login it came through. A LLeMbas // How a model is named: by the id the LLeMbas server serves it under — `deepseek/deepseek-flash`,
// instance that speaks protocol 2 serves every model as `<provider>/<model>` — `deepseek/deepseek-flash`, // `llama/bonsai` — which is its ref here too. LLeMbas CLI 2.0 talks to one server, the one it is
// `llama/bonsai` — and that id is the model's ref here too: each provider is a connection of its // logged in to, so there is nothing to tell apart: no connection prefix, no second login.
// own as far as the person can see, backed by the instance (requests still go to its /v1, with the
// served id). The older form of the ref was `<login connection>/<id>` (`example/deepseek-flash`).
// //
// - An instance that does not speak protocol 2 serves bare ids: those stay `<login>/<id>`. // The older forms still resolve, and a config.yaml that names one is rewritten to the id (load.ts):
// - A connection of the user's own with the provider's name (`deepseek:` in connections.yaml) // `<login>/<id>` (`example/deepseek/deepseek-flash`) and, from before providers, `<login>/<model>`
// keeps `deepseek/x`; the instance's is then `<login>/deepseek/x`. // for the one served `<provider>/<model>` of that name (`example/deepseek-flash`).
// - Two instances serving the same provider: the one logged in to first keeps the short form, the
// other's are `<login>/<provider>/<model>`.
// - The old forms still resolve — `example/deepseek-flash` and `example/deepseek/deepseek-flash`
// are `deepseek/deepseek-flash` — and a config.yaml that names one is rewritten to the new form.
import type { Connection } from "../config/schema.ts" import type { Connection } from "../config/schema.ts"
export interface ModelRef { export interface ModelRef {
/** What the person sees and writes. */ /** What the person sees and writes: the served id. */
ref: string ref: string
/** The connection it is spoken to through, and the id sent there. */ /** The login it is spoken to through, and the id sent there. */
connection: string connection: string
id: string id: string
/** An instance's model: its provider, when known — what the old `<login>/<model>` is matched by. */
provider?: string
} }
type Conns = Record<string, Connection & { webui?: { providers?: Record<string, string>; v2?: boolean } }> type Conns = Record<string, Connection>
/** The provider of a served id: as the instance said it (`provider`), else — from an /** Every model the server serves, in its order, with its ref. */
* instance that speaks protocol 2, whose ids are always `<provider>/<model>` — the part before the export function modelRefs(connections: Conns): ModelRef[] {
* first `/`. An id with a `/` from an older instance is just an id. */ return Object.entries(connections).flatMap(([name, c]) => Object.keys(c.models).map((id) => ({ ref: id, connection: name, id })))
export function providerOf(c: Conns[string], id: string): string | undefined {
if (!c.webui) return undefined
const said = c.webui.providers?.[id]
if (said) return said
return c.webui.v2 && id.indexOf("/") > 0 ? id.slice(0, id.indexOf("/")) : undefined
} }
/** Every model, in config order, with its ref. `logins`: the webui connections in the order they /** A ref as written — the id, or an older form — as the model it names, or undefined. `old`: it
* were logged in to (first keeps a shared provider's short form). */ * was an older form, which config.yaml is rewritten from. */
export function modelRefs(connections: Conns, logins: string[] = []): ModelRef[] {
const names = new Set(Object.keys(connections))
const webui = Object.keys(connections).filter((n) => connections[n]!.webui)
const order = [...logins.filter((n) => webui.includes(n)), ...webui.filter((n) => !logins.includes(n))]
// Which webui connection a provider's short form belongs to.
const owner = new Map<string, string>()
for (const w of order)
for (const id of Object.keys(connections[w]!.models)) {
const provider = providerOf(connections[w]!, id)
if (!provider) continue
// A connection of that name wins it — unless it is this login itself (`llama/bonsai` on a
// login named llama is the same string either way).
if ((names.has(provider) && provider !== w) || owner.has(provider)) continue
owner.set(provider, w)
}
const out: ModelRef[] = []
for (const [name, c] of Object.entries(connections))
for (const id of Object.keys(c.models)) {
const provider = providerOf(c, id)
const short = provider && owner.get(provider) === name
out.push({ ref: short ? id : `${name}/${id}`, connection: name, id, ...(provider ? { provider } : {}) })
}
return out
}
/** A ref as written — any of the forms above — as the model it names, or undefined. `old`: it was
* the form from before providers (`<login>/<model>` for a served `<provider>/<model>`). */
export function findRef(connections: Conns, refs: ModelRef[], ref: string): (ModelRef & { old?: boolean }) | undefined { export function findRef(connections: Conns, refs: ModelRef[], ref: string): (ModelRef & { old?: boolean }) | undefined {
const exact = refs.find((r) => r.ref === ref) const exact = refs.find((r) => r.ref === ref)
if (exact) return exact if (exact) return exact
@@ -71,13 +31,12 @@ export function findRef(connections: Conns, refs: ModelRef[], ref: string): (Mod
if (slash <= 0) return undefined if (slash <= 0) return undefined
const conn = ref.slice(0, slash) const conn = ref.slice(0, slash)
const id = ref.slice(slash + 1) const id = ref.slice(slash + 1)
const c = connections[conn] if (!connections[conn]) return undefined
if (!c) return undefined
const direct = refs.find((r) => r.connection === conn && r.id === id) const direct = refs.find((r) => r.connection === conn && r.id === id)
if (direct) return direct if (direct) return { ...direct, old: true }
// `<login>/<model>` from before providers: the one served model of that name, whatever its provider. // `<login>/<model>` from before providers: the one served model of that name, whatever its provider.
if (c.webui && !id.includes("/")) { if (!id.includes("/")) {
const by = refs.filter((r) => r.connection === conn && r.provider && r.id === `${r.provider}/${id}`) const by = refs.filter((r) => r.connection === conn && r.id.endsWith(`/${id}`) && r.id.indexOf("/") === r.id.length - id.length - 1)
if (by.length === 1) return { ...by[0]!, old: true } if (by.length === 1) return { ...by[0]!, old: true }
} }
return undefined return undefined
-174
View File
@@ -1,174 +0,0 @@
// OpenAI Responses (`POST {base}/responses`) — api.openai.com, and vLLM. Stateless (store: false):
// the whole history is sent as input items, and reasoning items come back opaque (with their
// encrypted content when the server provides it) so a reasoning model keeps its thread.
import { resolveKey } from "../config/load.ts"
import { historyArgs, Parts, record, recordFailure, retrying, statusOf, SwapBanner } from "./common.ts"
import { authHeaders, joinUrl, request, tlsFor } from "./http.ts"
import { contextFrom } from "./openai-chat.ts"
import { sseJson } from "./sse.ts"
import { estimateTokens, estimateValue } from "./tokens.ts"
import { ProviderError, type ChatRequest, type Client, type DiscoveredModel, type Message, type ResolvedModel, type StreamEvent } from "./types.ts"
type Item = Record<string, unknown>
export function toResponsesInput(messages: Message[], vision: boolean): Item[] {
const out: Item[] = []
for (const m of messages) {
if (m.role === "user") {
out.push({
role: "user",
content: m.parts.map((p) =>
p.type === "text"
? { type: "input_text", text: p.text }
: vision
? { type: "input_image", image_url: `data:${p.mime};base64,${p.data}` }
: { type: "input_text", text: "[image omitted: this model has no vision]" },
),
})
} else if (m.role === "assistant") {
let text: string[] = []
const flush = () => {
if (text.join("").trim()) out.push({ role: "assistant", content: [{ type: "output_text", text: text.join("") }] })
text = []
}
for (const p of m.parts) {
if (p.type === "text") text.push(p.text)
else if (p.type === "reasoning") {
// Only a reasoning item this dialect produced can go back; text from elsewhere cannot.
const item = (p.opaque as { responses?: Item } | undefined)?.responses
if (item) {
flush()
out.push(item)
}
} else {
flush()
out.push({ type: "function_call", call_id: p.id, name: p.name, arguments: historyArgs(p.args) })
}
}
flush()
} else out.push({ type: "function_call_output", call_id: m.callId, output: m.content })
}
return out
}
export class ResponsesClient implements Client {
constructor(private m: ResolvedModel) {}
private headers() {
return authHeaders(this.m.connection, resolveKey(this.m.connectionName, this.m.connection), this.m.spec.headers)
}
buildBody(req: ChatRequest): Record<string, unknown> {
const { m } = this
const body: Record<string, unknown> = {
model: m.id,
input: toResponsesInput(req.messages, m.spec.vision === true),
stream: true,
store: false,
...m.connection.body,
...m.spec.body,
}
if (req.system) body.instructions = req.system
if (req.tools.length && m.spec.tools !== false)
body.tools = req.tools.map((t) => ({ type: "function", name: t.name, description: t.description, parameters: t.parameters, strict: false }))
if (m.spec.max_output) body.max_output_tokens = m.spec.max_output
if (req.effort) {
body.reasoning = { effort: req.effort, summary: "auto" }
body.include = ["reasoning.encrypted_content"]
} else {
if (m.spec.temperature !== undefined) body.temperature = m.spec.temperature
if (m.spec.top_p !== undefined) body.top_p = m.spec.top_p
}
return body
}
async *stream(req: ChatRequest): AsyncGenerator<StreamEvent> {
let body = this.buildBody(req)
yield* retrying(
() => this.attempt(req, body),
(e) => {
// Some servers do not know `include`; drop it once rather than fail.
if (body.include && (e.status === 400 || e.status === 422) && /include/i.test(e.message)) {
const { include: _, ...rest } = body
body = rest
return { retry: true }
}
return { retry: false }
},
)
}
private async *attempt(req: ChatRequest, body: Record<string, unknown>): AsyncGenerator<StreamEvent> {
const { m } = this
const res = await request(
joinUrl(m.connection.base_url, "responses"),
{ method: "POST", headers: this.headers(), body: JSON.stringify(body), signal: req.signal, timeoutMs: (m.connection.timeout ?? 600) * 1000, tls: tlsFor(m.connection) },
m.ref,
).catch((e) => {
recordFailure(m.ref, body, e)
throw e
})
if (!res.body) throw new ProviderError(`${m.ref}: empty response`)
const out = new Parts()
const banner = new SwapBanner()
const reasoning = new Map<string, string>()
const callIndex = new Map<string, number>()
let finish = "stop"
let usage: StreamEvent | undefined
for await (const { data } of sseJson(record(res.body, m.ref, body))) {
const ev = data as any
const t: string = ev?.type ?? ""
if (t === "response.output_text.delta") {
out.add("text", ev.delta ?? "")
yield { type: "text", text: ev.delta ?? "" }
} else if (t === "response.reasoning_summary_text.delta" || t === "response.reasoning_text.delta") {
const b = banner.feed(ev.delta ?? "")
if (b.notice) yield { type: "notice", message: b.notice }
if (!b.text) continue
reasoning.set(ev.item_id ?? "", (reasoning.get(ev.item_id ?? "") ?? "") + b.text)
if (b.text.trim()) out.output = true
yield { type: "reasoning", text: b.text }
} else if (t === "response.output_item.added" && ev.item?.type === "function_call") {
const index = callIndex.size
callIndex.set(ev.item.id ?? ev.item.call_id, index)
yield { type: "tool_call_delta", index, name: ev.item.name, argsDelta: "" }
} else if (t === "response.function_call_arguments.delta") {
out.output = true
yield { type: "tool_call_delta", index: callIndex.get(ev.item_id) ?? 0, argsDelta: ev.delta ?? "" }
} else if (t === "response.output_item.done") {
const item = ev.item ?? {}
if (item.type === "function_call") out.push({ type: "tool_call", id: item.call_id ?? item.id, name: item.name, args: item.arguments || "{}" })
else if (item.type === "reasoning") {
const summary = Array.isArray(item.summary) ? item.summary.map((s: any) => s.text ?? "").join("\n") : ""
const text = reasoning.get(item.id ?? "") || summary
// vLLM puts the raw reasoning in `content`, which it will not accept back; keep what can return.
const { content: _c, ...back } = item
if (text || item.encrypted_content) out.push({ type: "reasoning", text, opaque: item.encrypted_content ? { responses: back } : undefined })
}
} else if (t === "response.completed" || t === "response.incomplete") {
const r = ev.response ?? {}
if (t === "response.incomplete" || r.status === "incomplete") finish = "length"
const u = r.usage
if (u) usage = { type: "usage", usage: { input: u.input_tokens ?? 0, output: u.output_tokens ?? 0, reasoning: u.output_tokens_details?.reasoning_tokens, cached: u.input_tokens_details?.cached_tokens || undefined } }
} else if (t === "response.failed" || t === "error" || ev?.error) {
const err = ev.response?.error ?? ev.error ?? ev
throw new ProviderError(`${m.ref}: ${err.message ?? JSON.stringify(err)}`, statusOf(err), undefined, !out.output)
}
}
const message = out.message()
if (finish !== "length" && message.parts.some((p) => p.type === "tool_call")) finish = "tool_calls"
yield usage ?? {
type: "usage",
usage: { input: estimateValue(body.input), output: message.parts.reduce((n, p) => n + estimateTokens(p.type === "tool_call" ? p.args : p.text), 0), estimated: true },
}
yield { type: "finish", reason: finish, message }
}
async listModels(): Promise<DiscoveredModel[]> {
const res = await request(joinUrl(this.m.connection.base_url, "models"), { headers: this.headers(), timeoutMs: 15_000, tls: tlsFor(this.m.connection) }, this.m.connectionName)
const j = (await res.json()) as any
return (Array.isArray(j?.data) ? j.data : []).map((x: any) => ({ id: String(x.id), context: contextFrom(x) })).filter((x: DiscoveredModel) => x.id)
}
}
+1 -1
View File
@@ -56,7 +56,7 @@ export interface ChatRequest {
} }
export interface ResolvedModel { export interface ResolvedModel {
/** `connection/model` */ /** The served id (`deepseek/deepseek-flash`). */
ref: string ref: string
connectionName: string connectionName: string
connection: Connection connection: Connection
+6 -7
View File
@@ -19,8 +19,8 @@ export function capacityRefusal(main: ResolvedModel, helper: ResolvedModel): str
} }
const shared = groupOf(main) const shared = groupOf(main)
if (shared !== undefined && shared === groupOf(helper)) { if (shared !== undefined && shared === groupOf(helper)) {
const where = main.spec.group !== undefined ? `${main.ref}'s server (group ${main.spec.group})` : main.connectionName const where = `${main.ref}'s server (group ${main.spec.group})`
return `${where} holds one model at a time (${main.spec.group !== undefined ? "group" : "one_model_at_a_time"}), so a subagent on ${helper.ref} would unload ${main.ref}, and this session's cached prompt with it, while its turn waits.` return `${where} holds one model at a time, so a subagent on ${helper.ref} would unload ${main.ref}, and this session's cached prompt with it, while its turn waits.`
} }
return "" return ""
} }
@@ -35,15 +35,14 @@ export function capacityBeside(running: ResolvedModel, helper: ResolvedModel): s
} }
const shared = groupOf(running) const shared = groupOf(running)
if (shared !== undefined && shared === groupOf(helper)) { if (shared !== undefined && shared === groupOf(helper)) {
const where = running.spec.group !== undefined ? `${running.ref}'s server (group ${running.spec.group})` : running.connectionName const where = `${running.ref}'s server (group ${running.spec.group})`
return `${where} holds one model at a time, and a subagent is already running on ${running.ref} there; one on ${helper.ref} would unload it.` return `${where} holds one model at a time, and a subagent is already running on ${running.ref} there; one on ${helper.ref} would unload it.`
} }
return "" return ""
} }
/** Which one-model-at-a-time server a model is on: its own `group`, else its connection when the /** Which one-model-at-a-time server a model is on, as the LLeMbas server says (`capacity.group`,
* connection sets one_model_at_a_time, else none. The spec's capacity.group. */ * the spec's), or none. */
function groupOf(m: ResolvedModel): string | undefined { function groupOf(m: ResolvedModel): string | undefined {
if (m.spec.group !== undefined) return `group:${m.spec.group}` return m.spec.group !== undefined ? `group:${m.spec.group}` : undefined
return m.connection.one_model_at_a_time ? `connection:${m.connectionName}` : undefined
} }
+1 -1
View File
@@ -1,7 +1,7 @@
// /context: where the window goes. Estimates (≈4 characters a token) per part, scaled so they add // /context: where the window goes. Estimates (≈4 characters a token) per part, scaled so they add
// up to what the server last reported when it reported anything. // up to what the server last reported when it reported anything.
import type { Engine } from "./engine.ts" import type { Engine } from "./engine.ts"
import { contextWindow } from "../provider/discover.ts" import { contextWindow } from "../provider/index.ts"
import { estimateTokens, imageTokens } from "../provider/tokens.ts" import { estimateTokens, imageTokens } from "../provider/tokens.ts"
import { toSpec, type Tool } from "../tool/tool.ts" import { toSpec, type Tool } from "../tool/tool.ts"
+2 -19
View File
@@ -14,7 +14,7 @@ import { fillPrompt } from "../prompt/fill.ts"
import { splitPurpose, toSpec, ToolError, type Tool, type ToolContext, type ToolResult } from "../tool/tool.ts" import { splitPurpose, toSpec, ToolError, type Tool, type ToolContext, type ToolResult } from "../tool/tool.ts"
import { fill, promptText } from "../prompt/assemble.ts" import { fill, promptText } from "../prompt/assemble.ts"
import { resolveCall } from "../tool/names.ts" import { resolveCall } from "../tool/names.ts"
import { contextWindow } from "../provider/discover.ts" import { contextWindow } from "../provider/index.ts"
import { summaryTurns, type Store } from "./store.ts" import { summaryTurns, type Store } from "./store.ts"
import { estimateTokens, estimateValue } from "../provider/tokens.ts" import { estimateTokens, estimateValue } from "../provider/tokens.ts"
import { duration } from "../duration.ts" import { duration } from "../duration.ts"
@@ -43,9 +43,6 @@ export interface EngineOptions {
compaction?: { autoAt?: number } compaction?: { autoAt?: number }
/** Starts a subagent for the task tool; absent in a subagent. */ /** Starts a subagent for the task tool; absent in a subagent. */
spawn?: ToolContext["spawn"] spawn?: ToolContext["spawn"]
/** The model to switch to when this one's server cannot be reached (its `fallback` list), or
* undefined when there is none left to try. */
fallback?: (current: ResolvedModel, tried: Set<string>) => { model: ResolvedModel; client: Client; effort: Effort | null } | undefined
/** What a message sent mid-turn starts with: the user's words for a session, the main agent's for /** What a message sent mid-turn starts with: the user's words for a session, the main agent's for
* a subagent (loop.json jobs.marks to_subagent). */ * a subagent (loop.json jobs.marks to_subagent). */
steerMark?: string steerMark?: string
@@ -264,7 +261,7 @@ export class Engine {
} }
private droppedThinking(): number { private droppedThinking(): number {
if (!this.usage || this.model.connection.dialect !== "openai-chat") return 0 if (!this.usage) return 0
const reply = this.messages[this.usageAt - 1] const reply = this.messages[this.usageAt - 1]
if (reply?.role !== "assistant") return 0 if (reply?.role !== "assistant") return 0
const estimate = reply.parts.reduce((n, p) => n + (p.type === "reasoning" ? estimateTokens(p.text) : 0), 0) const estimate = reply.parts.reduce((n, p) => n + (p.type === "reasoning" ? estimateTokens(p.text) : 0), 0)
@@ -468,7 +465,6 @@ export class Engine {
let ranOut: { limit: string; said: string } | undefined let ranOut: { limit: string; said: string } | undefined
let nudges = 0 let nudges = 0
let midRetries = 0 let midRetries = 0
const tried = new Set<string>()
for (let step = 1; ; step++) { for (let step = 1; ; step++) {
// Checked between steps, never mid-reply: a reply cut off mid-sentence is worth nothing. // Checked between steps, never mid-reply: a reply cut off mid-sentence is worth nothing.
@@ -519,19 +515,6 @@ export class Engine {
bus.emit({ type: "done", reason: "cancelled" }) bus.emit({ type: "done", reason: "cancelled" })
return "cancelled" return "cancelled"
} }
// The server is not there at all: the model's fallback, if it has one, for the rest of the
// session — said, and shown in the status bar.
if (e instanceof ProviderError && e.unreachable && this.o.fallback) {
tried.add(this.model.ref)
const next = this.o.fallback(this.model, tried)
if (next) {
bus.emit({ type: "notice", message: `${e.message.split("\n")[0]!.slice(0, 160)} — switching to ${next.model.ref}, its fallback` })
this.setModel(next.model, next.client, next.effort)
bus.emit({ type: "setting", key: "model", value: next.model.ref })
step--
continue
}
}
// A server failure part-way through a reply (llama.cpp failing to parse gpt-oss's own // A server failure part-way through a reply (llama.cpp failing to parse gpt-oss's own
// output is the one seen) is worth two more goes. What was already shown is discarded. // output is the one seen) is worth two more goes. What was already shown is discarded.
if (e instanceof ProviderError && !e.beforeOutput && (e.status ?? 0) >= 500 && midRetries < 2) { if (e instanceof ProviderError && !e.beforeOutput && (e.status ?? 0) >= 500 && midRetries < 2) {
+2 -2
View File
@@ -85,7 +85,7 @@ const pendingCodes = new Map<string, (text: string) => void>()
const AGENT_TEMPLATE = (name: string) => `--- const AGENT_TEMPLATE = (name: string) => `---
description: what ${name} is for — the model reads this when it picks a subagent description: what ${name} is for — the model reads this when it picks a subagent
# model: connection/model # run it on another model (default: the session's) # model: provider/model # run it on another of the server's models (default: the session's)
# mode: plan # plan: read-only; edit; manual (asks you) — never looser than yours # mode: plan # plan: read-only; edit; manual (asks you) — never looser than yours
# tools: [read, glob, grep, list, bash, web_search, web_fetch] # default: every tool a subagent may have # tools: [read, glob, grep, list, bash, web_search, web_fetch] # default: every tool a subagent may have
# isolation: worktree # work in a git checkout and branch of its own # isolation: worktree # work in a git checkout and branch of its own
@@ -153,7 +153,7 @@ export const COMMANDS: Command[] = [
}, },
{ {
name: "model", name: "model",
args: "[connection/model]", args: "[model]",
description: "switch model", description: "switch model",
run(c, arg) { run(c, arg) {
const refs = c.app.modelRefs() const refs = c.app.modelRefs()
+8 -32
View File
@@ -1,18 +1,9 @@
import { TextAttributes } from "@opentui/core" import { TextAttributes } from "@opentui/core"
import { useKeyboard } from "@opentui/solid" import { useKeyboard } from "@opentui/solid"
import { join } from "node:path"
import { createSignal, Show } from "solid-js" import { createSignal, Show } from "solid-js"
import { paths } from "../../config/paths.ts" import { ModelError, NOT_LOGGED_IN } from "../../provider/index.ts"
import { ModelError } from "../../provider/index.ts"
import { useTui } from "../context.ts" import { useTui } from "../context.ts"
const EXAMPLE = `connections:
local: # any name; llama.cpp, LM Studio, vLLM, Ollama's /v1…
dialect: openai-chat
base_url: http://127.0.0.1:8080/v1
models:
qwen3: { context: 131072 }`
/** Before there is a session: no model to talk to (a fresh install), or a config file that cannot /** Before there is a session: no model to talk to (a fresh install), or a config file that cannot
* be read. Says what to do and where; r tries again once the files are fixed, q or ctrl+c leaves. * be read. Says what to do and where; r tries again once the files are fixed, q or ctrl+c leaves.
* Not an exception: thrown with the terminal still the TUI's, mouse reporting would stay on, and * Not an exception: thrown with the terminal still the TUI's, mouse reporting would stay on, and
@@ -38,7 +29,7 @@ export function Setup(p: { error: Error; onRetry: () => void; onQuit: () => void
try { try {
const { codeInstructions, login } = await import("../../lembas/login.ts") const { codeInstructions, login } = await import("../../lembas/login.ts")
const r = await login(target, { onCode: (start, d) => setStatus(codeInstructions(start, d)) }) const r = await login(target, { onCode: (start, d) => setStatus(codeInstructions(start, d)) })
setStatus(`Signed in: ${r.models.length} model${r.models.length === 1 ? "" : "s"} as ${r.connection}/…${r.instanceDefault ? `, starting on ${r.connection}/${r.instanceDefault}` : ""}.`) setStatus(`Signed in: ${r.models.length} model${r.models.length === 1 ? "" : "s"}${r.instanceDefault ? `, starting on ${r.instanceDefault}` : ""}.`)
p.onRetry() p.onRetry()
} catch (e) { } catch (e) {
setStatus((e as Error).message) setStatus((e as Error).message)
@@ -65,11 +56,11 @@ export function Setup(p: { error: Error; onRetry: () => void; onQuit: () => void
return ( return (
<box flexDirection="column" width="100%" height="100%" paddingLeft={2} paddingTop={1} paddingRight={2} gap={1}> <box flexDirection="column" width="100%" height="100%" paddingLeft={2} paddingTop={1} paddingRight={2} gap={1}>
<text fg={c().title} attributes={TextAttributes.BOLD}> <text fg={c().title} attributes={TextAttributes.BOLD}>
{theme().branding.agentName} — {noModel() ? "no model to talk to yet" : "the configuration has a mistake"} {theme().branding.agentName} — {p.error.message === NOT_LOGGED_IN ? "not logged in to a LLeMbas server" : noModel() ? "no model to talk to" : "the configuration has a mistake"}
</text> </text>
{/* "No model configured" is what the rest of the screen explains; anything else (a model {/* Not being logged in is what the rest of the screen explains; anything else (a model the
that does not exist, a file that does not parse) is shown as it is. */} server does not serve, a file that does not parse) is shown as it is. */}
<Show when={!p.error.message.startsWith("No model configured")}> <Show when={p.error.message !== NOT_LOGGED_IN}>
<text fg={noModel() ? c().text : c().error} wrapMode="word"> <text fg={noModel() ? c().text : c().error} wrapMode="word">
{p.error.message} {p.error.message}
</text> </text>
@@ -80,7 +71,7 @@ export function Setup(p: { error: Error; onRetry: () => void; onQuit: () => void
when={typing()} when={typing()}
fallback={ fallback={
<text fg={c().muted} wrapMode="word"> <text fg={c().muted} wrapMode="word">
{"Sign in to a LLeMbas instance and use its models: press l (or, from a shell, lembas login <address>)."} {"LLeMbas CLI uses the models of the LLeMbas server it is logged in to. Log in: press l (or, from a shell, lembas login <address>)."}
</text> </text>
} }
> >
@@ -96,24 +87,9 @@ export function Setup(p: { error: Error; onRetry: () => void; onQuit: () => void
{status()} {status()}
</text> </text>
</Show> </Show>
<text fg={c().muted} wrapMode="word">
{`Or, without an instance — nothing is predefined: add the server you use to ${join(paths.config, "connections.yaml")}, for example`}
</text>
<box paddingLeft={2}>
<text fg={c().text}>{EXAMPLE}</text>
</box>
<text fg={c().muted} wrapMode="word">
{`and pick the model in ${join(paths.config, "config.yaml")}:`}
</text>
<box paddingLeft={2}>
<text fg={c().text}>model: local/qwen3</text>
</box>
<text fg={c().muted} wrapMode="word">
A key is never pasted in: api_key: "{"{env:NAME}"}" or "{"{file:~/path}"}". https://llembas.eu/docs/ has the other dialects (Anthropic, OpenAI, Gemini, Ollama).
</text>
</Show> </Show>
<text fg={c().muted} wrapMode="word"> <text fg={c().muted} wrapMode="word">
From a shell, lembas config check reads the files and says what is wrong, and lembas models lists the models it found. From a shell, lembas config check reads the files and says what is wrong, and lembas models lists the server's models.
</text> </text>
<text fg={c().label}>{typing() ? "enter sign in · esc back · ctrl+r try again · ctrl+c quit" : `${noModel() ? "l log in · " : ""}r try again · q quit`}</text> <text fg={c().label}>{typing() ? "enter sign in · esc back · ctrl+r try again · ctrl+c quit" : `${noModel() ? "l log in · " : ""}r try again · q quit`}</text>
</box> </box>
+7 -10
View File
@@ -34,14 +34,11 @@ function pair(): [Transport, Transport] {
return [side(a, b), side(b, a)] return [side(a, b), side(b, a)]
} }
/** The parent's model on `f`; a second connection `g` for a subagent's model, when given. */ /** The parent's model `f/m`; `g/m` for a subagent, when its own script is given (the fake server
function config(url: string, childUrl?: string) { * routes it there). */
function config(url: string, child?: boolean) {
mkdirSync(paths.config, { recursive: true }) mkdirSync(paths.config, { recursive: true })
writeFileSync( writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n type: webui\n url: ${url}\n models: { f/m: {}${child ? ", g/m: {}" : ""} }\n`, { mode: 0o600 })
join(paths.config, "connections.yaml"),
`connections:\n f:\n dialect: openai-chat\n base_url: ${url}\n models: { m: {} }\n${childUrl ? ` g:\n dialect: openai-chat\n base_url: ${childUrl}\n models: { m: {} }\n` : ""}`,
{ mode: 0o600 },
)
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\nmode: auto\n") writeFileSync(join(paths.config, "config.yaml"), "model: f/m\nmode: auto\n")
} }
@@ -145,14 +142,14 @@ test("job/stop stops a command and answers once it has; the -32602 cases: a comm
}, SLOW) }, SLOW)
test("a background subagent: its transcript as history turns; send to a finished one starts it again under the same id", async () => { test("a background subagent: its transcript as history turns; send to a finished one starts it again under the same id", async () => {
const child = (other = fakeProvider([{ chunks: [delta({ content: "calc.py:2 divides by len(v) - 1." })] }, { chunks: [delta({ content: "The tests cover it." })] }]))
fake = fakeProvider([ fake = fakeProvider([
{ chunks: [toolCall(0, "t1", "task", JSON.stringify({ description: "find the bug", prompt: "Find why average is wrong", model: "g/m", background: true }))] }, { chunks: [toolCall(0, "t1", "task", JSON.stringify({ description: "find the bug", prompt: "Find why average is wrong", model: "g/m", background: true }))] },
{ chunks: [delta({ content: "Started it." })] }, { chunks: [delta({ content: "Started it." })] },
{ chunks: [delta({ content: "Got the report." })] }, { chunks: [delta({ content: "Got the report." })] },
{ chunks: [delta({ content: "Got the second report." })] }, { chunks: [delta({ content: "Got the second report." })] },
]) ], { route: (model) => (model.startsWith("g/") ? child : undefined) })
other = fakeProvider([{ chunks: [delta({ content: "calc.py:2 divides by len(v) - 1." })] }, { chunks: [delta({ content: "The tests cover it." })] }]) config(fake.url, true)
config(fake.url, other.url)
const { c, events, sid } = await client() const { c, events, sid } = await client()
await c.request("session/prompt", { sessionId: sid, prompt: [{ type: "text", text: "go" }] }) await c.request("session/prompt", { sessionId: sid, prompt: [{ type: "text", text: "go" }] })
expect(await until(() => events.some((e) => e.type === "job" && e.state === "end"))).toBe(true) expect(await until(() => events.some((e) => e.type === "job" && e.state === "end"))).toBe(true)
+4 -4
View File
@@ -40,7 +40,7 @@ function pair(): [Transport, Transport] {
function config(url: string, extra = "", model = "{}") { function config(url: string, extra = "", model = "{}") {
mkdirSync(paths.config, { recursive: true }) mkdirSync(paths.config, { recursive: true })
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n dialect: openai-chat\n base_url: ${url}\n models: { m: ${model} }\n`, { mode: 0o600 }) writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n type: webui\n url: ${url}\n models: { f/m: ${model} }\n`, { mode: 0o600 })
writeFileSync(join(paths.config, "config.yaml"), `model: f/m\n${extra}`) writeFileSync(join(paths.config, "config.yaml"), `model: f/m\n${extra}`)
} }
@@ -535,7 +535,7 @@ test("answered at the terminal first: the web UI's card is settled", async () =>
test("the model event: said whenever the model or effort changes, forwarded to LLeMbas", async () => { test("the model event: said whenever the model or effort changes, forwarded to LLeMbas", async () => {
fake = fakeProvider([]) fake = fakeProvider([])
mkdirSync(paths.config, { recursive: true }) mkdirSync(paths.config, { recursive: true })
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n example:\n dialect: openai-chat\n base_url: ${fake.url}\n models: { a: { efforts: [low, high] }, b: {} }\n`, { mode: 0o600 }) writeFileSync(join(paths.config, "connections.yaml"), `connections:\n example:\n type: webui\n url: ${fake.url}\n models: { example/a: { efforts: [low, high] }, example/b: {} }\n`, { mode: 0o600 })
writeFileSync(join(paths.config, "config.yaml"), "model: example/a\n") writeFileSync(join(paths.config, "config.yaml"), "model: example/a\n")
const root = project() const root = project()
const { c, events } = await client(limitsFor(root), { instance: "example" }) const { c, events } = await client(limitsFor(root), { instance: "example" })
@@ -771,13 +771,13 @@ test("a command's mode from the web is clamped to remote.max_mode in a terminal
test("a command naming a model gives the session its model and its effort back afterwards", async () => { test("a command naming a model gives the session its model and its effort back afterwards", async () => {
fake = fakeProvider([{ chunks: [delta({ content: "ok" })] }]) fake = fakeProvider([{ chunks: [delta({ content: "ok" })] }])
mkdirSync(paths.config, { recursive: true }) mkdirSync(paths.config, { recursive: true })
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n dialect: openai-chat\n base_url: ${fake.url}\n models: { m: { efforts: [low, high], effort: low }, other: {} }\n`, { mode: 0o600 }) writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n type: webui\n url: ${fake.url}\n models: { f/m: { efforts: [low, high], effort: low }, f/other: {} }\n`, { mode: 0o600 })
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\n") writeFileSync(join(paths.config, "config.yaml"), "model: f/m\n")
const root = project({ ".agent/config.yaml": "", ".agent/commands/quick.md": "---\nmodel: f/other\n---\nQuick." }) const root = project({ ".agent/config.yaml": "", ".agent/commands/quick.md": "---\nmodel: f/other\n---\nQuick." })
const { c } = await client(limitsFor(root)) const { c } = await client(limitsFor(root))
const s: any = await c.request("session/new", { cwd: root, _meta: { effort: "high" } }) const s: any = await c.request("session/new", { cwd: root, _meta: { effort: "high" } })
await c.request("session/prompt", { sessionId: s.sessionId, prompt: [{ type: "text", text: "/quick" }] }) await c.request("session/prompt", { sessionId: s.sessionId, prompt: [{ type: "text", text: "/quick" }] })
expect(fake.requests[0].model).toBe("other") expect(fake.requests[0].model).toBe("f/other")
expect(await c.request<any>("_lembas/configure", { sessionId: s.sessionId })).toEqual({ model: "f/m", effort: "high" }) expect(await c.request<any>("_lembas/configure", { sessionId: s.sessionId })).toEqual({ model: "f/m", effort: "high" })
}) })
+5 -5
View File
@@ -36,7 +36,7 @@ function pair(): [Transport, Transport] {
function config(url: string, extra = "") { function config(url: string, extra = "") {
mkdirSync(paths.config, { recursive: true }) mkdirSync(paths.config, { recursive: true })
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n dialect: openai-chat\n base_url: ${url}\n models: { m: {} }\n`, { mode: 0o600 }) writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n type: webui\n url: ${url}\n models: { f/m: {} }\n`, { mode: 0o600 })
writeFileSync(join(paths.config, "config.yaml"), `model: f/m\n${extra}`) writeFileSync(join(paths.config, "config.yaml"), `model: f/m\n${extra}`)
} }
@@ -210,19 +210,19 @@ test("_lembas/directories is for the link only: an editor's ACP session has no r
test("the chat's model and effort: named by the instance, at the start and between prompts", async () => { test("the chat's model and effort: named by the instance, at the start and between prompts", async () => {
fake = fakeProvider([{ chunks: [delta({ content: "one" })] }, { chunks: [delta({ content: "two" })] }]) fake = fakeProvider([{ chunks: [delta({ content: "one" })] }, { chunks: [delta({ content: "two" })] }])
mkdirSync(paths.config, { recursive: true }) mkdirSync(paths.config, { recursive: true })
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n example:\n dialect: openai-chat\n base_url: ${fake.url}\n models: { bonsai: {}, deepseek-flash: { efforts: [low, medium, high] } }\n`, { mode: 0o600 }) writeFileSync(join(paths.config, "connections.yaml"), `connections:\n example:\n type: webui\n url: ${fake.url}\n models: { bonsai: {}, deepseek-flash: { efforts: [low, medium, high] } }\n`, { mode: 0o600 })
writeFileSync(join(paths.config, "config.yaml"), "model: example/bonsai\n") writeFileSync(join(paths.config, "config.yaml"), "model: bonsai\n")
const root = project() const root = project()
const { c } = await client({ roots: [root], maxMode: "auto", approvalTimeoutMs: 50, requireTrust: true }, undefined, "example") const { c } = await client({ roots: [root], maxMode: "auto", approvalTimeoutMs: 50, requireTrust: true }, undefined, "example")
// Not the device's own default: the model the chat was started on there. // Not the device's own default: the model the chat was started on there.
const s: any = await c.request("session/new", { cwd: root, _meta: { mode: "auto", lembas_model: "deepseek-flash", effort: "high" } }) const s: any = await c.request("session/new", { cwd: root, _meta: { mode: "auto", lembas_model: "deepseek-flash", effort: "high" } })
expect(s._meta.lembas).toMatchObject({ model: "example/deepseek-flash", effort: "high" }) expect(s._meta.lembas).toMatchObject({ model: "deepseek-flash", effort: "high" })
await c.request("session/prompt", { sessionId: s.sessionId, prompt: [{ type: "text", text: "hi" }] }) await c.request("session/prompt", { sessionId: s.sessionId, prompt: [{ type: "text", text: "hi" }] })
expect(fake.requests[0].model).toBe("deepseek-flash") expect(fake.requests[0].model).toBe("deepseek-flash")
expect(JSON.stringify(fake.requests[0])).toContain('"reasoning_effort":"high"') expect(JSON.stringify(fake.requests[0])).toContain('"reasoning_effort":"high"')
// Changed in the web UI between two prompts. // Changed in the web UI between two prompts.
const changed: any = await c.request("_lembas/configure", { sessionId: s.sessionId, model: "bonsai", effort: "off" }) const changed: any = await c.request("_lembas/configure", { sessionId: s.sessionId, model: "bonsai", effort: "off" })
expect(changed).toEqual({ model: "example/bonsai", effort: "off" }) expect(changed).toEqual({ model: "bonsai", effort: "off" })
await c.request("session/prompt", { sessionId: s.sessionId, prompt: [{ type: "text", text: "again" }] }) await c.request("session/prompt", { sessionId: s.sessionId, prompt: [{ type: "text", text: "again" }] })
expect(fake.requests.at(-1).model).toBe("bonsai") expect(fake.requests.at(-1).model).toBe("bonsai")
expect(JSON.stringify(fake.requests.at(-1))).not.toContain("reasoning_effort") expect(JSON.stringify(fake.requests.at(-1))).not.toContain("reasoning_effort")
-115
View File
@@ -1,115 +0,0 @@
import { afterEach, describe, expect, test } from "bun:test"
import { readFileSync } from "node:fs"
import { join } from "node:path"
import { AnthropicClient, toAnthropicMessages } from "../src/provider/anthropic.ts"
import type { Message, ResolvedModel, StreamEvent } from "../src/provider/types.ts"
import { fakeProvider, type Fake } from "./fake-provider.ts"
let fake: Fake | undefined
afterEach(() => fake?.stop())
const model = (url: string, spec: ResolvedModel["spec"] = {}): ResolvedModel => ({
ref: "a/m",
connectionName: "a",
id: "m",
spec,
connection: { dialect: "anthropic", base_url: url, api_key: "k", models: {} },
})
const sse = (events: Record<string, unknown>[]) => events.map((e) => `event: ${e.type}\ndata: ${JSON.stringify(e)}\n\n`).join("")
async function run(c: AnthropicClient, messages: Message[] = [{ role: "user", parts: [{ type: "text", text: "hi" }] }], effort: any = null) {
const ev: StreamEvent[] = []
for await (const e of c.stream({ system: "sys", messages, tools: [{ name: "read", description: "read", parameters: { type: "object" } }], effort })) ev.push(e)
return ev
}
describe("anthropic dialect", () => {
test("a real vLLM stream: thinking with its signature, then a tool call", async () => {
fake = fakeProvider([{ chunks: [], raw: readFileSync(join(import.meta.dir, "fixtures/sse/vllm-anthropic-thinking-tool.sse"), "utf8") }])
const ev = await run(new AnthropicClient(model(fake.url.replace(/\/v1$/, ""))))
const fin = ev.find((e) => e.type === "finish")!
if (fin.type !== "finish") throw new Error()
expect(fin.reason).toBe("tool_calls")
const thinking = fin.message.parts.find((p) => p.type === "reasoning")
expect(thinking?.type === "reasoning" && thinking.signature).toBeTruthy()
expect(fin.message.parts.some((p) => p.type === "tool_call")).toBe(true)
expect(ev.some((e) => e.type === "reasoning")).toBe(true)
})
test("request: headers, max_tokens above the thinking budget, no sampling with thinking, cache breakpoints", async () => {
fake = fakeProvider([{ chunks: [], raw: sse([{ type: "message_start", message: { usage: { input_tokens: 5 } } }, { type: "message_stop" }]) }])
const c = new AnthropicClient(model(fake.url, { max_output: 4000, temperature: 0.5, cache: true, effort_map: { high: 10000 } }))
await run(c, undefined, "high")
const r = fake.requests[0]
expect(r.thinking).toEqual({ type: "enabled", budget_tokens: 10000 })
expect(r.max_tokens).toBeGreaterThan(10000)
expect(r.temperature).toBeUndefined()
expect(r.system[0].cache_control).toEqual({ type: "ephemeral" })
expect(r.tools.at(-1).cache_control).toEqual({ type: "ephemeral" })
expect(r.messages.at(-1).content.at(-1).cache_control).toEqual({ type: "ephemeral" })
})
test("stream: text, redacted thinking, tool input in pieces, cache usage, stop reasons", async () => {
fake = fakeProvider([
{
chunks: [],
raw: sse([
{ type: "message_start", message: { usage: { input_tokens: 10, cache_read_input_tokens: 90, output_tokens: 1 } } },
{ type: "content_block_start", index: 0, content_block: { type: "redacted_thinking", data: "OPAQUE" } },
{ type: "content_block_stop", index: 0 },
{ type: "content_block_start", index: 1, content_block: { type: "text", text: "" } },
{ type: "content_block_delta", index: 1, delta: { type: "text_delta", text: "Reading." } },
{ type: "content_block_stop", index: 1 },
{ type: "content_block_start", index: 2, content_block: { type: "tool_use", id: "toolu_1", name: "read", input: {} } },
{ type: "content_block_delta", index: 2, delta: { type: "input_json_delta", partial_json: '{"pa' } },
{ type: "content_block_delta", index: 2, delta: { type: "input_json_delta", partial_json: 'th":"a"}' } },
{ type: "content_block_stop", index: 2 },
{ type: "message_delta", delta: { stop_reason: "tool_use" }, usage: { output_tokens: 42 } },
{ type: "message_stop" },
]),
},
])
const ev = await run(new AnthropicClient(model(fake.url)))
const fin = ev.find((e) => e.type === "finish")
expect(fin?.type === "finish" && fin.message.parts).toEqual([
{ type: "reasoning", text: "", opaque: { redacted: "OPAQUE" } },
{ type: "text", text: "Reading." },
{ type: "tool_call", id: "toolu_1", name: "read", args: '{"path":"a"}' },
])
expect(ev.find((e) => e.type === "usage")).toEqual({ type: "usage", usage: { input: 100, output: 42, cached: 90 } })
expect(fake.requests[0]).toMatchObject({ model: "m", stream: true })
})
test("overloaded before any output is retried once", async () => {
fake = fakeProvider([
{ chunks: [], raw: sse([{ type: "message_start", message: { usage: { input_tokens: 1 } } }, { type: "error", error: { type: "overloaded_error", message: "Overloaded" } }]) },
{ chunks: [], raw: sse([{ type: "content_block_start", index: 0, content_block: { type: "text", text: "ok" } }, { type: "content_block_stop", index: 0 }, { type: "message_stop" }]) },
])
const ev = await run(new AnthropicClient(model(fake.url)))
expect(ev.some((e) => e.type === "notice" && e.message.includes("Overloaded"))).toBe(true)
const fin = ev.find((e) => e.type === "finish")
expect(fin?.type === "finish" && fin.message.parts).toEqual([{ type: "text", text: "ok" }])
})
test("history: tool results grouped as a user turn, ids made safe, thinking only with a signature and only while thinking", () => {
const msgs: Message[] = [
{ role: "user", parts: [{ type: "text", text: "go" }] },
{ role: "assistant", parts: [{ type: "reasoning", text: "unsigned (from another provider)" }, { type: "reasoning", text: "signed", signature: "SIG" }, { type: "tool_call", id: "call:1", name: "read", args: '{"path":"a"}' }, { type: "tool_call", id: "call:2", name: "read", args: "{}" }] },
{ role: "tool", callId: "call:1", name: "read", content: "A" },
{ role: "tool", callId: "call:2", name: "read", content: "", isError: true },
{ role: "user", parts: [{ type: "text", text: "and?" }] },
]
const on = toAnthropicMessages(msgs, false, true)
expect(on.map((m) => m.role)).toEqual(["user", "assistant", "user"])
expect(on[1]!.content).toEqual([
{ type: "thinking", thinking: "signed", signature: "SIG" },
{ type: "tool_use", id: "call_1", name: "read", input: { path: "a" } },
{ type: "tool_use", id: "call_2", name: "read", input: {} },
])
expect(on[2]!.content).toEqual([
{ type: "tool_result", tool_use_id: "call_1", content: "A" },
{ type: "tool_result", tool_use_id: "call_2", content: "(empty)", is_error: true },
{ type: "text", text: "and?" },
])
expect(toAnthropicMessages(msgs, false, false)[1]!.content.some((b) => b.type === "thinking")).toBe(false)
})
})
+1 -1
View File
@@ -16,7 +16,7 @@ afterEach(() => fake?.stop())
function app(edit: string, config = "") { function app(edit: string, config = "") {
fake = fakeProvider([{ chunks: [toolCall(0, "b1", "bash", '{"command":"echo one"}')] }, { chunks: [delta({ content: "ok" }, "stop")] }]) fake = fakeProvider([{ chunks: [toolCall(0, "b1", "bash", '{"command":"echo one"}')] }, { chunks: [delta({ content: "ok" }, "stop")] }])
mkdirSync(paths.config, { recursive: true }) mkdirSync(paths.config, { recursive: true })
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n dialect: openai-chat\n base_url: ${fake.url}\n models: { m: {} }\n`, { mode: 0o600 }) writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n type: webui\n url: ${fake.url}\n models: { f/m: {} }\n`, { mode: 0o600 })
writeFileSync(join(paths.config, "config.yaml"), `model: f/m\ntitles: prompt\n${config}`) writeFileSync(join(paths.config, "config.yaml"), `model: f/m\ntitles: prompt\n${config}`)
return createApp({ cwd: mkdtempSync(join(tmpdir(), "ph-approve-")), mode: "manual", store: false, snapshots: false, asker: { ask: async (): Promise<AskReply> => ({ kind: "once", command: edit }) } }) return createApp({ cwd: mkdtempSync(join(tmpdir(), "ph-approve-")), mode: "manual", store: false, snapshots: false, asker: { ask: async (): Promise<AskReply> => ({ kind: "once", command: edit }) } })
} }
+1 -1
View File
@@ -33,7 +33,7 @@ describe("the project board", () => {
{ chunks: [delta({ content: "next" })] }, { chunks: [delta({ content: "next" })] },
]) ])
mkdirSync(paths.config, { recursive: true }) mkdirSync(paths.config, { recursive: true })
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n dialect: openai-chat\n base_url: ${fake.url}\n models: { m: {} }\n`, { mode: 0o600 }) writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n type: webui\n url: ${fake.url}\n models: { f/m: {} }\n`, { mode: 0o600 })
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\n") writeFileSync(join(paths.config, "config.yaml"), "model: f/m\n")
const cwd = mkdtempSync(join(tmpdir(), "ph-board-")) const cwd = mkdtempSync(join(tmpdir(), "ph-board-"))
mkdirSync(join(cwd, ".agent")) mkdirSync(join(cwd, ".agent"))
+1 -1
View File
@@ -16,7 +16,7 @@ afterEach(() => fake?.stop())
function app(limits: string, script: Parameters<typeof fakeProvider>[0], ask: () => Promise<AskReply> = async () => ({ kind: "once" })) { function app(limits: string, script: Parameters<typeof fakeProvider>[0], ask: () => Promise<AskReply> = async () => ({ kind: "once" })) {
fake = fakeProvider(script) fake = fakeProvider(script)
mkdirSync(paths.config, { recursive: true }) mkdirSync(paths.config, { recursive: true })
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n dialect: openai-chat\n base_url: ${fake.url}\n models: { m: {} }\n`, { mode: 0o600 }) writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n type: webui\n url: ${fake.url}\n models: { f/m: {} }\n`, { mode: 0o600 })
writeFileSync(join(paths.config, "config.yaml"), `model: f/m\ntitles: prompt\nlimits: { ${limits} }\n`) writeFileSync(join(paths.config, "config.yaml"), `model: f/m\ntitles: prompt\nlimits: { ${limits} }\n`)
const cwd = mkdtempSync(join(tmpdir(), "ph-budget-")) const cwd = mkdtempSync(join(tmpdir(), "ph-budget-"))
writeFileSync(join(cwd, "big.txt"), "x".repeat(5000) + "\n") writeFileSync(join(cwd, "big.txt"), "x".repeat(5000) + "\n")
+7 -16
View File
@@ -14,24 +14,15 @@ import { delta, fakeProvider, toolCall, type Fake } from "./fake-provider.ts"
let fake: Fake | undefined let fake: Fake | undefined
afterEach(() => fake?.stop()) afterEach(() => fake?.stop())
const model = (conn: string, id: string, o: { one?: boolean; single?: boolean } = {}): ResolvedModel => ({ const model = (conn: string, id: string, o: { single?: boolean } = {}): ResolvedModel => ({
ref: `${conn}/${id}`, ref: `${conn}/${id}`,
connectionName: conn, connectionName: "ai",
connection: { dialect: "openai-chat", base_url: "http://x/v1", models: {}, ...(o.one ? { one_model_at_a_time: true } : {}) } as ResolvedModel["connection"], connection: { base_url: "http://x/v1", models: {} },
id, id: `${conn}/${id}`,
spec: o.single ? { single_session: true } : {}, spec: o.single ? { single_session: true } : {},
}) })
test("one model at a time: another model of the same server is refused, the session's own is not", () => { test("a model's group (the server's capacity.group): another model of the same GPU is refused, the session's own is not", () => {
const main = model("swap", "qwen", { one: true })
expect(capacityRefusal(main, model("swap", "gemma", { one: true }))).toContain("holds one model at a time")
expect(capacityRefusal(main, main)).toBe("")
expect(capacityRefusal(main, model("cloud", "big"))).toBe("")
// Without the flag, as before.
expect(capacityRefusal(model("swap", "qwen"), model("swap", "gemma"))).toBe("")
})
test("a model's group: models one connection serves from several servers are told apart", () => {
const grouped = (id: string, group?: string): ResolvedModel => ({ ...model("ai", id), spec: group ? { group } : {} }) const grouped = (id: string, group?: string): ResolvedModel => ({ ...model("ai", id), spec: group ? { group } : {} })
const main = grouped("qwen", "gpu") const main = grouped("qwen", "gpu")
expect(capacityRefusal(main, grouped("gemma", "gpu"))).toContain("holds one model at a time") expect(capacityRefusal(main, grouped("gemma", "gpu"))).toContain("holds one model at a time")
@@ -55,14 +46,14 @@ test("task on a model the server cannot hold beside the session's is refused, an
mkdirSync(paths.config, { recursive: true }) mkdirSync(paths.config, { recursive: true })
writeFileSync( writeFileSync(
join(paths.config, "connections.yaml"), join(paths.config, "connections.yaml"),
`connections:\n f:\n dialect: openai-chat\n base_url: ${fake.url}\n one_model_at_a_time: true\n models: { m: {}, other: {} }\n`, `connections:\n f:\n type: webui\n url: ${fake.url}\n models: { f/m: { group: gpu }, f/other: { group: gpu } }\n`,
{ mode: 0o600 }, { mode: 0o600 },
) )
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\n") writeFileSync(join(paths.config, "config.yaml"), "model: f/m\n")
const a = createApp({ cwd: mkdtempSync(join(tmpdir(), "ph-cap-")), mode: "edit", store: false, asker: { ask: async (): Promise<AskReply> => ({ kind: "once" }) } }) const a = createApp({ cwd: mkdtempSync(join(tmpdir(), "ph-cap-")), mode: "edit", store: false, asker: { ask: async (): Promise<AskReply> => ({ kind: "once" }) } })
await a.engine.prompt("get a second opinion") await a.engine.prompt("get a second opinion")
// Two requests: the session's, then its next step — never one for f/other. // Two requests: the session's, then its next step — never one for f/other.
expect(fake.requests.map((r: any) => r.model)).toEqual(["m", "m"]) expect(fake.requests.map((r: any) => r.model)).toEqual(["f/m", "f/m"])
const result = fake.requests[1].messages.at(-1).content as string const result = fake.requests[1].messages.at(-1).content as string
expect(result).toContain("holds one model at a time") expect(result).toContain("holds one model at a time")
expect(result).toContain("Do this part yourself") expect(result).toContain("Do this part yourself")
+29 -42
View File
@@ -4,7 +4,7 @@ import { tmpdir } from "node:os"
import { join } from "node:path" import { join } from "node:path"
import { paths } from "../src/config/paths.ts" import { paths } from "../src/config/paths.ts"
import { loadConfig, resolveKey } from "../src/config/load.ts" import { loadConfig, resolveKey } from "../src/config/load.ts"
import { resolveModel } from "../src/provider/index.ts" import { NOT_LOGGED_IN, resolveModel } from "../src/provider/index.ts"
const write = (file: string, text: string) => { const write = (file: string, text: string) => {
mkdirSync(join(file, ".."), { recursive: true }) mkdirSync(join(file, ".."), { recursive: true })
@@ -17,7 +17,7 @@ beforeEach(() => {
}) })
describe("config", () => { describe("config", () => {
test("connections: substitution, a broken one does not break the others, mode warning", () => { test("connections: the login, its key from a file; a custom connection is read past with a warning (2.0)", () => {
write(join(paths.config, "keyfile"), "sk-from-file\n") write(join(paths.config, "keyfile"), "sk-from-file\n")
write( write(
join(paths.config, "connections.yaml"), join(paths.config, "connections.yaml"),
@@ -27,32 +27,40 @@ describe("config", () => {
base_url: http://llm.example/v1 base_url: http://llm.example/v1
api_key: "{env:PH_TEST_KEY}" api_key: "{env:PH_TEST_KEY}"
models: models:
qwen: { context: 131072, max_output: 8192, efforts: [low, high], effort: high } qwen: { context: 131072 }
file: example:
dialect: openai-chat type: webui
base_url: http://x/v1 url: https://ai.example.org
api_key: "{file:${join(paths.config, "keyfile")}}" api_key: "{file:${join(paths.config, "keyfile")}}"
models: { a: {} } models:
broken: llama/qwen: { context: 131072, max_output: 8192, efforts: [low, high], effort: high }
dialect: openai-chat
base_url: http://y/v1
api_key: "{env:PH_MISSING}"
models: { b: {} }
`, `,
) )
chmodSync(join(paths.config, "connections.yaml"), 0o644) chmodSync(join(paths.config, "connections.yaml"), 0o644)
const l = loadConfig() const l = loadConfig()
expect(l.connections.swap!.api_key).toBe("sk-test") expect(Object.keys(l.connections)).toEqual(["example"])
expect(resolveKey("file", l.connections.file!)).toBe("sk-from-file") expect(l.connections.example!.base_url).toBe("https://ai.example.org/v1")
expect(l.broken.broken).toContain("PH_MISSING") expect(resolveKey("example", l.connections.example!)).toBe("sk-from-file")
expect(l.warnings.some((w) => w.includes("swap is a custom connection") && w.includes("lembas login"))).toBe(true)
expect(l.warnings.some((w) => w.includes("chmod 600"))).toBe(true) expect(l.warnings.some((w) => w.includes("chmod 600"))).toBe(true)
expect(resolveModel(l, "swap/qwen").spec.context).toBe(131072) expect(resolveModel(l, "llama/qwen").spec.context).toBe(131072)
expect(() => resolveModel(l, "broken/b")).toThrow("unusable") // The older form, the login's name in front, names the same model.
expect(() => resolveModel(l, "swap/nope")).toThrow('no model "nope"') expect(resolveModel(l, "example/llama/qwen").ref).toBe("llama/qwen")
// One the server's list does not have yet is asked for, not refused here: the server decides.
expect(resolveModel(l, "llama/new")).toMatchObject({ ref: "llama/new", id: "llama/new", spec: {} })
}) })
test("key_cmd", () => { test("no login: no model, and the error says how to log in", () => {
expect(resolveKey("k", { dialect: "openai-chat", base_url: "http://x", key_cmd: "echo sk-cmd", models: {} })).toBe("sk-cmd") write(join(paths.config, "connections.yaml"), "connections: {}\n")
expect(() => resolveModel(loadConfig(), undefined)).toThrow(NOT_LOGGED_IN)
expect(NOT_LOGGED_IN).toContain("lembas login <address>")
})
test("a login whose key file is gone is unusable, said as such", () => {
write(join(paths.config, "connections.yaml"), `connections:\n example:\n type: webui\n url: https://ai.example.org\n api_key: "{env:PH_MISSING}"\n`)
const l = loadConfig()
expect(l.broken.example).toContain("PH_MISSING")
expect(() => resolveModel(l, undefined)).toThrow("unusable")
}) })
test("project config: merged when trusted, ignored when not, global-only keys refused", () => { test("project config: merged when trusted, ignored when not, global-only keys refused", () => {
@@ -76,34 +84,13 @@ describe("config", () => {
}) })
}) })
test("effort_map may name only some efforts (zod 4 records keyed by an enum are exhaustive)", async () => {
const { ModelSpec } = await import("../src/config/schema.ts")
expect(ModelSpec.safeParse({ effort_map: { high: 10000 } }).success).toBe(true)
expect(ModelSpec.safeParse({ effort_map: { huge: 1 } }).success).toBe(false)
})
test("a URL may be {env:}/{file:}: accepted as written, and checked once filled in", () => { test("a URL may be {env:}/{file:}: accepted as written, and checked once filled in", () => {
process.env.PH_URL = "http://llm.example/v1" process.env.PH_URL = "http://llm.example/v1"
process.env.PH_NOT_URL = "not a url" process.env.PH_NOT_URL = "not a url"
write( write(join(paths.config, "connections.yaml"), "connections: {}\n")
join(paths.config, "connections.yaml"),
`connections:
ok: { dialect: openai-chat, base_url: "{env:PH_URL}", models: { m: {} } }
bad: { dialect: openai-chat, base_url: "{env:PH_NOT_URL}", models: { m: {} } }
typo: { dialect: openai-chat, base_url: "{env:PH_URL", models: { m: {} } }
`,
)
chmodSync(join(paths.config, "connections.yaml"), 0o600) chmodSync(join(paths.config, "connections.yaml"), 0o600)
write(join(paths.config, "config.yaml"), 'search:\n searxng: { base_url: "{env:PH_URL}" }\n firecrawl: { base_url: "{env:PH_NOT_URL}" }\n') write(join(paths.config, "config.yaml"), 'search:\n searxng: { base_url: "{env:PH_URL}" }\n firecrawl: { base_url: "{env:PH_NOT_URL}" }\n')
expect(() => loadConfig()).toThrow(/typo\.base_url: Invalid URL/)
write(join(paths.config, "connections.yaml"), `connections:
ok: { dialect: openai-chat, base_url: "{env:PH_URL}", models: { m: {} } }
bad: { dialect: openai-chat, base_url: "{env:PH_NOT_URL}", models: { m: {} } }
`)
chmodSync(join(paths.config, "connections.yaml"), 0o600)
const l = loadConfig() const l = loadConfig()
expect(l.connections.ok!.base_url).toBe("http://llm.example/v1")
expect(l.broken.bad).toContain("base_url: Invalid URL")
expect(l.config.search?.searxng?.base_url).toBe("http://llm.example/v1") expect(l.config.search?.searxng?.base_url).toBe("http://llm.example/v1")
expect(l.config.search?.firecrawl).toBeUndefined() expect(l.config.search?.firecrawl).toBeUndefined()
expect(l.warnings.some((w) => w.includes("firecrawl is off"))).toBe(true) expect(l.warnings.some((w) => w.includes("firecrawl is off"))).toBe(true)
+3 -21
View File
@@ -5,10 +5,7 @@ import { join } from "node:path"
import { createApp } from "../src/app.ts" import { createApp } from "../src/app.ts"
import type { AskReply, Event } from "../src/bus/index.ts" import type { AskReply, Event } from "../src/bus/index.ts"
import { paths } from "../src/config/paths.ts" import { paths } from "../src/config/paths.ts"
import { discoverContext } from "../src/provider/discover.ts"
import { resetLearned } from "../src/provider/learned.ts" import { resetLearned } from "../src/provider/learned.ts"
import { OpenAIChatClient } from "../src/provider/openai-chat.ts"
import type { ResolvedModel } from "../src/provider/types.ts"
import { contextBreakdown, renderBreakdown } from "../src/session/context.ts" import { contextBreakdown, renderBreakdown } from "../src/session/context.ts"
import { delta, fakeProvider, toolCall, usage, type Fake } from "./fake-provider.ts" import { delta, fakeProvider, toolCall, usage, type Fake } from "./fake-provider.ts"
@@ -18,7 +15,6 @@ afterEach(() => {
resetLearned() resetLearned()
}) })
const rm = (url: string, id: string): ResolvedModel => ({ ref: `c/${id}`, connectionName: "c", id, spec: {}, connection: { dialect: "openai-chat", base_url: url, models: {} } })
function app(yaml: string, script: Parameters<typeof fakeProvider>[0], config = "") { function app(yaml: string, script: Parameters<typeof fakeProvider>[0], config = "") {
fake = fakeProvider(script) fake = fakeProvider(script)
@@ -31,24 +27,10 @@ function app(yaml: string, script: Parameters<typeof fakeProvider>[0], config =
} }
describe("context", () => { describe("context", () => {
test("discovery: the model list first, then llama-server /props through llama-swap's passthrough", async () => {
fake = fakeProvider([])
expect(await discoverContext(rm(fake.url, "m1"), new OpenAIChatClient(rm(fake.url, "m1")))).toBe(32768)
expect(await discoverContext(rm(fake.url, "swapped"), new OpenAIChatClient(rm(fake.url, "swapped")))).toBe(16384)
expect(fake.calls.map((c) => c.path)).toContain("/upstream/swapped/props")
})
test("switching to another connection calls the old one's unload_url", async () => {
const a = app(`connections:\n f:\n dialect: openai-chat\n base_url: URL\n unload_url: URL/../unload\n unload_method: GET\n models: { m: {} }\n g:\n dialect: openai-chat\n base_url: URL\n models: { n: {} }\n`, [])
a.switchModel("g/n")
await Bun.sleep(100)
expect(fake!.calls.some((c) => c.path.endsWith("/unload"))).toBe(true)
})
test("past auto_at: nothing already sent is cut; the conversation is compacted, with a table of the reads", async () => { test("past auto_at: nothing already sent is cut; the conversation is compacted, with a table of the reads", async () => {
// window 16000, auto_at 0.6 → limit 9600. Harness spec 3.0.0 (context.md): old tool outputs are // window 16000, auto_at 0.6 → limit 9600. Harness spec 3.0.0 (context.md): old tool outputs are
// never pruned in place — that would make the server read everything after them again. // never pruned in place — that would make the server read everything after them again.
const a = app(`connections:\n f:\n dialect: openai-chat\n base_url: URL\n models: { m: { context: 16000 } }\n`, [ const a = app(`connections:\n f:\n type: webui\n url: URL\n models: { f/m: { context: 16000 } }\n`, [
{ chunks: [toolCall(0, "c1", "read", '{"path":"big.txt"}'), usage(1000, 10)] }, { chunks: [toolCall(0, "c1", "read", '{"path":"big.txt"}'), usage(1000, 10)] },
{ chunks: [toolCall(0, "c2", "read", '{"path":"big2.txt"}'), usage(6600, 10)] }, { chunks: [toolCall(0, "c2", "read", '{"path":"big2.txt"}'), usage(6600, 10)] },
{ chunks: [delta({ content: "## Summary\nRead two files." })] }, { chunks: [delta({ content: "## Summary\nRead two files." })] },
@@ -75,7 +57,7 @@ describe("context", () => {
}) })
test("too full: compacted mid-task, then told to carry on", async () => { test("too full: compacted mid-task, then told to carry on", async () => {
const a = app(`connections:\n f:\n dialect: openai-chat\n base_url: URL\n models: { m: { context: 4000 } }\n`, [ const a = app(`connections:\n f:\n type: webui\n url: URL\n models: { f/m: { context: 4000 } }\n`, [
{ chunks: [delta({ content: "y".repeat(8000) }), toolCall(0, "c1", "list", "{}"), usage(3900, 2000)] }, { chunks: [delta({ content: "y".repeat(8000) }), toolCall(0, "c1", "list", "{}"), usage(3900, 2000)] },
{ chunks: [delta({ content: "## What we are doing\nA long task." })] }, { chunks: [delta({ content: "## What we are doing\nA long task." })] },
{ chunks: [delta({ content: "carried on" }), usage(500, 5)] }, { chunks: [delta({ content: "carried on" }), usage(500, 5)] },
@@ -93,7 +75,7 @@ describe("context", () => {
}) })
test("/context: parts of the system prompt, tools, the conversation, against the window", async () => { test("/context: parts of the system prompt, tools, the conversation, against the window", async () => {
const a = app(`connections:\n f:\n dialect: openai-chat\n base_url: URL\n models: { m: { context: 32768 } }\n`, [{ chunks: [toolCall(0, "c1", "read", '{"path":"big.txt"}'), usage(1500, 20)] }, { chunks: [delta({ content: "ok" }), usage(3100, 5)] }]) const a = app(`connections:\n f:\n type: webui\n url: URL\n models: { f/m: { context: 32768 } }\n`, [{ chunks: [toolCall(0, "c1", "read", '{"path":"big.txt"}'), usage(1500, 20)] }, { chunks: [delta({ content: "ok" }), usage(3100, 5)] }])
await a.engine.prompt("read it") await a.engine.prompt("read it")
const system = a.engine.o.system(a.engine.mode, a.engine.model) const system = a.engine.o.system(a.engine.mode, a.engine.model)
const text = renderBreakdown(contextBreakdown(a.engine, system, [], undefined, a.engine.o.tools)) const text = renderBreakdown(contextBreakdown(a.engine, system, [], undefined, a.engine.o.tools))
+2 -2
View File
@@ -19,7 +19,7 @@ function setup(script: Parameters<typeof fakeProvider>[0]) {
mkdirSync(paths.config, { recursive: true }) mkdirSync(paths.config, { recursive: true })
writeFileSync( writeFileSync(
join(paths.config, "connections.yaml"), join(paths.config, "connections.yaml"),
`connections:\n fake:\n dialect: openai-chat\n base_url: ${fake.url}\n models:\n m: { context: 32768, max_output: 1000 }\n`, `connections:\n fake:\n type: webui\n url: ${fake.url}\n models:\n fake/m: { context: 32768, max_output: 1000 }\n`,
{ mode: 0o600 }, { mode: 0o600 },
) )
writeFileSync(join(paths.config, "config.yaml"), "model: fake/m\n") writeFileSync(join(paths.config, "config.yaml"), "model: fake/m\n")
@@ -113,7 +113,7 @@ describe("the connection timeout", () => {
const dir = setup(script) const dir = setup(script)
writeFileSync( writeFileSync(
join(paths.config, "connections.yaml"), join(paths.config, "connections.yaml"),
`connections:\n fake:\n dialect: openai-chat\n base_url: ${fake!.url}\n timeout: 0.4\n models:\n m: { context: 32768 }\n`, `connections:\n fake:\n type: webui\n url: ${fake!.url}\n timeout: 0.4\n models:\n fake/m: { context: 32768 }\n`,
{ mode: 0o600 }, { mode: 0o600 },
) )
return dir return dir
+1 -1
View File
@@ -17,7 +17,7 @@ function setup(script: Parameters<typeof fakeProvider>[0]) {
mkdirSync(paths.config, { recursive: true }) mkdirSync(paths.config, { recursive: true })
writeFileSync( writeFileSync(
join(paths.config, "connections.yaml"), join(paths.config, "connections.yaml"),
`connections:\n fake:\n dialect: openai-chat\n base_url: ${fake.url}\n models:\n m: { context: 32768 }\n`, `connections:\n fake:\n type: webui\n url: ${fake.url}\n models:\n fake/m: { context: 32768 }\n`,
{ mode: 0o600 }, { mode: 0o600 },
) )
writeFileSync(join(paths.config, "config.yaml"), "model: fake/m\n") writeFileSync(join(paths.config, "config.yaml"), "model: fake/m\n")
+11 -5
View File
@@ -11,8 +11,12 @@ export interface Fake {
stop(): void stop(): void
} }
/** `show`: what Ollama's /api/show answers (404 when not given); it takes no scripted response. */ /** `route`: another fake that answers for a model (by the id in the request), as one LLeMbas server
export function fakeProvider(script: Scripted[], opts: { show?: unknown } = {}): Fake { * serves models from several backends — the request is passed to it whole, and recorded there. */
/** `served`: what GET /v1/models lists, as a LLeMbas server lists its models (default two
* placeholders). Any other GET — the instance's own description, its personalization — is a 404,
* which a client reads as "this server does not have it". */
export function fakeProvider(script: Scripted[], opts: { route?: (model: string) => Fake | undefined; served?: Record<string, unknown>[] } = {}): Fake {
const requests: any[] = [] const requests: any[] = []
const calls: { path: string; headers: Record<string, string> }[] = [] const calls: { path: string; headers: Record<string, string> }[] = []
let i = 0 let i = 0
@@ -21,11 +25,13 @@ export function fakeProvider(script: Scripted[], opts: { show?: unknown } = {}):
async fetch(req) { async fetch(req) {
const url = new URL(req.url) const url = new URL(req.url)
if (req.method !== "POST" || url.pathname.endsWith("/unload")) calls.push({ path: url.pathname + url.search, headers: Object.fromEntries(req.headers.entries()) }) if (req.method !== "POST" || url.pathname.endsWith("/unload")) calls.push({ path: url.pathname + url.search, headers: Object.fromEntries(req.headers.entries()) })
if (url.pathname.endsWith("/models")) return Response.json({ data: [{ id: "m1", meta: { n_ctx: 32768 } }, { id: "m2", max_model_len: 8192 }] }) if (url.pathname.endsWith("/models")) return Response.json({ data: opts.served ?? [{ id: "m1", meta: { n_ctx: 32768 } }, { id: "m2", max_model_len: 8192 }] })
if (url.pathname.endsWith("/props")) return url.pathname.includes("/upstream/swapped/") ? Response.json({ default_generation_settings: { n_ctx: 16384 } }) : new Response("no", { status: 404 }) if (url.pathname.endsWith("/props")) return url.pathname.includes("/upstream/swapped/") ? Response.json({ default_generation_settings: { n_ctx: 16384 } }) : new Response("no", { status: 404 })
if (url.pathname.endsWith("/unload")) return new Response("ok") if (url.pathname.endsWith("/unload")) return new Response("ok")
if (url.pathname.endsWith("/api/show")) return opts.show ? Response.json(opts.show) : new Response("not found", { status: 404 }) if (req.method === "GET") return new Response("not found", { status: 404 })
const body = await req.json() const body = (await req.json()) as any
const to = typeof body?.model === "string" ? opts.route?.(body.model) : undefined
if (to) return fetch(to.url.replace(/\/v1$/, "") + url.pathname + url.search, { method: "POST", headers: req.headers, body: JSON.stringify(body) })
requests.push(body) requests.push(body)
calls.push({ path: url.pathname + url.search, headers: Object.fromEntries(req.headers.entries()) }) calls.push({ path: url.pathname + url.search, headers: Object.fromEntries(req.headers.entries()) })
const r = script[i++] const r = script[i++]
-225
View File
@@ -1,225 +0,0 @@
// Gemini and Ollama are UNTESTED against real servers (the wiki says so). These fixtures follow the
// documented shapes and what Hermes Agent and OpenCode handle.
import { afterEach, describe, expect, test } from "bun:test"
import { DEFAULT_MAX_OUTPUT, GeminiClient, geminiJsonSchema, geminiSchema, SKIP_SIGNATURE, thinkingConfig, toGeminiContents } from "../src/provider/gemini.ts"
import { OllamaClient, parseShow } from "../src/provider/ollama.ts"
import type { Message, ResolvedModel, StreamEvent } from "../src/provider/types.ts"
import { BUILTIN_TOOLS } from "../src/tool/registry.ts"
import { toSpec } from "../src/tool/tool.ts"
import { fakeProvider, type Fake } from "./fake-provider.ts"
let fake: Fake | undefined
afterEach(() => fake?.stop())
const m = (dialect: "gemini" | "ollama", url: string, id: string, spec: ResolvedModel["spec"] = {}): ResolvedModel => ({
ref: `x/${id}`,
connectionName: "x",
id,
spec,
connection: { dialect, base_url: url, api_key: "KEY", models: {} },
})
async function run(c: GeminiClient | OllamaClient, messages: Message[] = [{ role: "user", parts: [{ type: "text", text: "hi" }] }], effort: any = null) {
const ev: StreamEvent[] = []
for await (const e of c.stream({ system: "sys", messages, tools: BUILTIN_TOOLS.map(toSpec).slice(0, 2), effort })) ev.push(e)
return ev
}
const finishOf = (ev: StreamEvent[]) => {
const f = ev.find((e) => e.type === "finish")
if (!f || f.type !== "finish") throw new Error("no finish")
return f
}
describe("gemini", () => {
test("every built-in tool's schema is reduced to what Gemini accepts", () => {
const allowed = new Set(["type", "format", "title", "description", "nullable", "enum", "maxItems", "minItems", "properties", "required", "minProperties", "maxProperties", "minLength", "maxLength", "pattern", "example", "anyOf", "propertyOrdering", "default", "items", "minimum", "maximum"])
const walk = (s: any, path: string) => {
if (!s || typeof s !== "object") return
for (const [k, v] of Object.entries(s)) {
if (k === "properties") for (const [p, sub] of Object.entries(v as object)) walk(sub, `${path}.${p}`)
else {
expect(allowed.has(k) ? k : `${path}: ${k}`).toBe(k)
if (k === "items" || k === "anyOf") walk(v, path)
}
}
}
for (const t of BUILTIN_TOOLS.map(toSpec)) walk(geminiSchema(t.parameters), t.name)
expect(geminiSchema({ type: ["string", "null"], additionalProperties: false })).toEqual({ type: "string", nullable: true })
})
test("subset schema: unions keep every branch, enums become strings, required only names what exists", () => {
expect(geminiSchema({ type: ["array", "string", "null"], items: { type: "string", $comment: "x" }, minItems: 1 })).toEqual({
anyOf: [{ type: "array", items: { type: "string" }, minItems: 1 }, { type: "string" }],
nullable: true,
})
expect(geminiSchema({ type: "integer", enum: [1, 2, 2, null] })).toEqual({ type: "integer", enum: ["1", "2"] })
expect(geminiSchema({ type: "object", properties: { a: { type: "string" } }, required: ["a", "b"] })).toEqual({ type: "object", properties: { a: { type: "string" } }, required: ["a"] })
expect(geminiSchema({ type: "object", required: ["b"] })).toEqual({ type: "object" })
})
test("JSON Schema (v1beta): local $refs inlined with their siblings, $defs and $schema gone; a circular one goes as it is", () => {
const s = { $schema: "x", type: "object", $defs: { P: { type: "string", description: "p" } }, properties: { a: { $ref: "#/$defs/P", description: "mine" } } }
expect(geminiJsonSchema(s)).toEqual({ type: "object", properties: { a: { type: "string", description: "mine" } } })
const loop = { type: "object", $defs: { N: { type: "object", properties: { next: { $ref: "#/$defs/N" } } } }, properties: { n: { $ref: "#/$defs/N" } } }
expect(geminiJsonSchema(loop)).toEqual(loop)
expect(geminiJsonSchema({ type: "object" })).toEqual({ type: "object", properties: {} })
expect(geminiJsonSchema(undefined)).toEqual({ type: "object", properties: {} })
})
test("thinking: budgets for 2.x (2.5 Pro to 32k), levels for 3 fitted to what each model has", () => {
expect(thinkingConfig("gemini-2.5-pro", "max")).toEqual({ thinkingBudget: 32768, includeThoughts: true })
expect(thinkingConfig("models/gemini-2.5-flash", "max")).toEqual({ thinkingBudget: 24576, includeThoughts: true })
expect(thinkingConfig("gemini-2.5-flash", "low", { low: 500 })).toEqual({ thinkingBudget: 500, includeThoughts: true })
const level = (id: string, e: any) => thinkingConfig(id, e).thinkingLevel
expect(level("gemini-3-pro", "minimal")).toBe("low")
expect(level("gemini-3-pro", "medium")).toBe("medium")
expect(level("gemini-3-flash", "minimal")).toBe("minimal")
expect(level("gemini-3.1-pro", "xhigh")).toBe("high")
expect(level("gemma-4-31b-it", "medium")).toBe("high")
})
test("request: URL, key header, system, thinking budget (2.5) or level (3); stream: thoughts, text, a call with its signature", async () => {
const frames = [
{ candidates: [{ content: { role: "model", parts: [{ text: "Let me look.", thought: true }] } }] },
{ candidates: [{ content: { role: "model", parts: [{ text: "Reading the file." }] } }] },
{ candidates: [{ content: { role: "model", parts: [{ functionCall: { name: "read", args: { path: "a.ts" } }, thoughtSignature: "SIG" }] }, finishReason: "STOP" }], usageMetadata: { promptTokenCount: 50, candidatesTokenCount: 10, thoughtsTokenCount: 5, cachedContentTokenCount: 20 } },
]
fake = fakeProvider([{ chunks: frames }, { chunks: frames }])
const base = fake.url.replace(/\/v1$/, "/v1beta")
const ev = await run(new GeminiClient(m("gemini", base, "gemini-2.5-pro", { max_output: 1000 })), undefined, "medium")
expect(fake.calls[0]!.path).toBe("/v1beta/models/gemini-2.5-pro:streamGenerateContent?alt=sse")
expect(fake.calls[0]!.headers["x-goog-api-key"]).toBe("KEY")
expect(fake.calls[0]!.headers.authorization).toBeUndefined()
const r = fake.requests[0]
expect(r.systemInstruction).toEqual({ parts: [{ text: "sys" }] })
expect(r.generationConfig).toEqual({ maxOutputTokens: 1000, thinkingConfig: { thinkingBudget: 8192, includeThoughts: true } })
expect(r.tools[0].functionDeclarations[0].name).toBe("read")
expect(r.tools[0].functionDeclarations[0].parametersJsonSchema.type).toBe("object")
expect(r.tools[0].functionDeclarations[0].parameters).toBeUndefined()
const fin = finishOf(ev)
expect(fin.reason).toBe("tool_calls")
expect(fin.message.parts).toEqual([
{ type: "reasoning", text: "Let me look." },
{ type: "text", text: "Reading the file." },
{ type: "tool_call", id: "gemini_call_0", name: "read", args: '{"path":"a.ts"}', signature: "SIG" },
])
expect(ev.find((e) => e.type === "usage")).toEqual({ type: "usage", usage: { input: 50, output: 15, reasoning: 5, cached: 20 } })
await run(new GeminiClient(m("gemini", base, "gemini-3-pro")), undefined, "high")
expect(fake.requests[1].generationConfig).toEqual({ maxOutputTokens: DEFAULT_MAX_OUTPUT, thinkingConfig: { thinkingLevel: "high", includeThoughts: true } })
})
test("another API version gets the OpenAPI subset as `parameters`", async () => {
fake = fakeProvider([{ chunks: [{ candidates: [{ content: { parts: [{ text: "ok" }] }, finishReason: "STOP" }] }] }])
await run(new GeminiClient(m("gemini", fake.url, "gemini-2.0-flash")))
const d = fake.requests[0].tools[0].functionDeclarations[0]
expect(d.parametersJsonSchema).toBeUndefined()
expect(d.parameters.type).toBe("object")
})
test("stream: a call sent again stays one call; a different one is a new call; Gemini 3 ids are kept", async () => {
const call = (args: unknown, extra: Record<string, unknown> = {}) => ({ candidates: [{ content: { parts: [{ functionCall: { name: "read", args, ...extra } }] } }] })
fake = fakeProvider([
{ chunks: [call({ path: "a", limit: 5 }), call({ limit: 5, path: "a" }), call({ path: "b" }), { candidates: [{ finishReason: "STOP" }] }] },
{ chunks: [call({ path: "c" }, { id: "fc_1" }), { candidates: [{ finishReason: "SAFETY" }] }] },
])
const base = fake.url.replace(/\/v1$/, "/v1beta")
const fin = finishOf(await run(new GeminiClient(m("gemini", base, "gemini-2.5-flash"))))
expect(fin.message.parts).toEqual([
{ type: "tool_call", id: "gemini_call_0", name: "read", args: '{"limit":5,"path":"a"}' },
{ type: "tool_call", id: "gemini_call_1", name: "read", args: '{"path":"b"}' },
])
const fin3 = finishOf(await run(new GeminiClient(m("gemini", base, "gemini-3-pro"))))
expect(fin3.message.parts).toEqual([{ type: "tool_call", id: "fc_1", name: "read", args: '{"path":"c"}' }])
expect(fin3.reason).toBe("tool_calls")
})
test("history: the signature rides back on its call; results answer by name; roles merge", () => {
const c = toGeminiContents(
[
{ role: "user", parts: [{ type: "text", text: "go" }] },
{ role: "assistant", parts: [{ type: "reasoning", text: "x" }, { type: "tool_call", id: "gemini_call_0", name: "read", args: '{"path":"a"}', signature: "SIG" }] },
{ role: "tool", callId: "gemini_call_0", name: "read", content: "A" },
{ role: "user", parts: [{ type: "text", text: "and?" }] },
],
false,
)
expect(c).toEqual([
{ role: "user", parts: [{ text: "go" }] },
{ role: "model", parts: [{ functionCall: { name: "read", args: { path: "a" } }, thoughtSignature: "SIG" }] },
{ role: "user", parts: [{ functionResponse: { name: "read", response: { output: "A" } } }] },
// never folded into the tool result: Gemini 3 would read it as part of it
{ role: "model", parts: [{ text: "[The previous response was interrupted before it completed.]" }] },
{ role: "user", parts: [{ text: "and?" }] },
])
})
test("history for Gemini 3: ids on both sides, the sentinel for an unsigned call, JSON results structured unless they hold a $ref", () => {
const c = toGeminiContents(
[
{ role: "user", parts: [{ type: "text", text: "go" }] },
{ role: "assistant", parts: [{ type: "tool_call", id: "c1", name: "read", args: "{}" }, { type: "tool_call", id: "c2", name: "grep", args: "{}" }] },
{ role: "tool", callId: "c1", name: "read", content: '{"lines": 3}' },
{ role: "tool", callId: "c2", name: "grep", content: '{"$ref": "#/$defs/X"}', isError: false },
],
false,
true,
)
expect(c[1]!.parts[0]).toEqual({ functionCall: { name: "read", args: {}, id: "c1" }, thoughtSignature: SKIP_SIGNATURE })
expect(c[2]).toEqual({
role: "user",
parts: [{ functionResponse: { name: "read", response: { lines: 3 }, id: "c1" } }, { functionResponse: { name: "grep", response: { output: '{"$ref": "#/$defs/X"}' }, id: "c2" } }],
})
})
})
describe("ollama", () => {
const nd = (lines: unknown[]) => lines.map((l) => JSON.stringify(l)).join("\n") + "\n"
test("NDJSON: thinking, text, whole tool calls, counts; num_ctx and think in the request", async () => {
fake = fakeProvider([
{
chunks: [],
raw: nd([
{ message: { role: "assistant", content: "", thinking: "Hmm." }, done: false },
{ message: { role: "assistant", content: "Reading." }, done: false },
{ message: { role: "assistant", content: "", tool_calls: [{ function: { name: "read", arguments: { path: "a.ts" } } }] }, done: false },
{ message: { role: "assistant", content: "" }, done: true, done_reason: "stop", prompt_eval_count: 30, eval_count: 7 },
]),
},
])
const ev = await run(new OllamaClient(m("ollama", fake.url.replace(/\/v1$/, ""), "gpt-oss:20b", { context: 32768, max_output: 2000 })), undefined, "high")
expect(fake.calls[0]!.path).toBe("/api/chat")
expect(fake.requests[0]).toMatchObject({ model: "gpt-oss:20b", stream: true, think: "high", options: { num_ctx: 32768, num_predict: 2000 } })
expect(fake.requests[0].messages[0]).toEqual({ role: "system", content: "sys" })
const fin = finishOf(ev)
expect(fin.message.parts).toEqual([
{ type: "reasoning", text: "Hmm." },
{ type: "text", text: "Reading." },
{ type: "tool_call", id: "ollama_call_0", name: "read", args: '{"path":"a.ts"}' },
])
expect(fin.reason).toBe("tool_calls")
expect(ev.find((e) => e.type === "usage")).toEqual({ type: "usage", usage: { input: 30, output: 7 } })
})
test("think: a switch for other models, off when efforts exist but none is chosen; an error line is retried once", async () => {
fake = fakeProvider([{ chunks: [], raw: nd([{ error: "model runner has unexpectedly stopped" }]) }, { chunks: [], raw: nd([{ message: { content: "ok" }, done: true }]) }, { chunks: [], raw: nd([{ message: { content: "ok" }, done: true }]) }])
const base = fake.url.replace(/\/v1$/, "")
const ev = await run(new OllamaClient(m("ollama", base, "qwen3:8b", { efforts: ["low", "high"] })), undefined, "low")
expect(fake.requests[0].think).toBe(true)
expect(ev.some((e) => e.type === "notice" && e.message.includes("unexpectedly stopped"))).toBe(true)
await run(new OllamaClient(m("ollama", base, "qwen3:8b", { efforts: ["low", "high"] })))
expect(fake.requests[2].think).toBe(false)
})
test("/api/show: no `think` for a model that cannot think; num_ctx from the Modelfile, else the trained maximum", async () => {
const show = { capabilities: ["completion", "tools"], parameters: "stop \"<|im_end|>\"\nnum_ctx 16384", model_info: { "qwen3.context_length": 40960 } }
expect(parseShow(show)).toEqual({ capabilities: ["completion", "tools"], numCtx: 16384, trained: 40960 })
fake = fakeProvider([{ chunks: [], raw: nd([{ message: { content: "ok" }, done: true }]) }], { show })
const base = fake.url.replace(/\/v1$/, "")
const c = new OllamaClient(m("ollama", base, "llama3:8b", { efforts: ["low", "high"] }))
await run(c, undefined, "high")
expect(fake.requests[0].think).toBeUndefined()
expect(fake.requests[0].options.num_ctx).toBe(16384)
expect(await c.contextOf("llama3:8b")).toBe(16384)
expect(parseShow({ model_info: { "llama.context_length": 8192 } })).toEqual({ trained: 8192 })
})
})
+1 -1
View File
@@ -190,7 +190,7 @@ test("every {{variable}} in a description is declared, and none reaches a model"
expect(used.sort()).toEqual(Object.keys(s.variables ?? {}).sort()) expect(used.sort()).toEqual(Object.keys(s.variables ?? {}).sort())
} }
mkdirSync(paths.config, { recursive: true }) mkdirSync(paths.config, { recursive: true })
writeFileSync(join(paths.config, "connections.yaml"), "connections:\n f:\n dialect: openai-chat\n base_url: http://127.0.0.1:9/v1\n models: { m: {} }\n", { mode: 0o600 }) writeFileSync(join(paths.config, "connections.yaml"), "connections:\n f:\n type: webui\n url: http://127.0.0.1:9/v1\n models: { f/m: {} }\n", { mode: 0o600 })
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\nlimits: { bash_timeout: 90 }\n") writeFileSync(join(paths.config, "config.yaml"), "model: f/m\nlimits: { bash_timeout: 90 }\n")
const app = createApp({ cwd: mkdtempSync(join(tmpdir(), "ph-vars-")), store: false, snapshots: false, asker: { ask: async () => ({ kind: "once" }) } }) const app = createApp({ cwd: mkdtempSync(join(tmpdir(), "ph-vars-")), store: false, snapshots: false, asker: { ask: async () => ({ kind: "once" }) } })
for (const t of app.engine.o.tools) { for (const t of app.engine.o.tools) {
+1 -1
View File
@@ -18,7 +18,7 @@ afterEach(() => fake?.stop())
function app(script: Parameters<typeof fakeProvider>[0]) { function app(script: Parameters<typeof fakeProvider>[0]) {
fake = fakeProvider(script) fake = fakeProvider(script)
mkdirSync(paths.config, { recursive: true }) mkdirSync(paths.config, { recursive: true })
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n dialect: openai-chat\n base_url: ${fake.url}\n models: { m: {} }\n`, { mode: 0o600 }) writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n type: webui\n url: ${fake.url}\n models: { f/m: {} }\n`, { mode: 0o600 })
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\n") writeFileSync(join(paths.config, "config.yaml"), "model: f/m\n")
return createApp({ cwd: mkdtempSync(join(tmpdir(), "ph-hl-")), mode: "auto", store: false, snapshots: false, asker: { ask: async (): Promise<AskReply> => ({ kind: "once" }) } }) return createApp({ cwd: mkdtempSync(join(tmpdir(), "ph-hl-")), mode: "auto", store: false, snapshots: false, asker: { ask: async (): Promise<AskReply> => ({ kind: "once" }) } })
} }
+1 -1
View File
@@ -38,7 +38,7 @@ function pair(): [Transport, Transport] {
function config(url: string, extra = "") { function config(url: string, extra = "") {
mkdirSync(paths.config, { recursive: true }) mkdirSync(paths.config, { recursive: true })
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n dialect: openai-chat\n base_url: ${url}\n models: { m: {} }\n`, { mode: 0o600 }) writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n type: webui\n url: ${url}\n models: { f/m: {} }\n`, { mode: 0o600 })
writeFileSync(join(paths.config, "config.yaml"), `model: f/m\n${extra}`) writeFileSync(join(paths.config, "config.yaml"), `model: f/m\n${extra}`)
} }
+1 -1
View File
@@ -51,7 +51,7 @@ describe("an image counts by its dimensions, never its encoding", () => {
{ chunks: [delta({ content: "a screenshot" }), usage(56_000, 5)] }, { chunks: [delta({ content: "a screenshot" }), usage(56_000, 5)] },
]) ])
mkdirSync(paths.config, { recursive: true }) mkdirSync(paths.config, { recursive: true })
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n dialect: openai-chat\n base_url: ${fake.url}\n models:\n m: { vision: true, context: 262144 }\n`, { mode: 0o600 }) writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n type: webui\n url: ${fake.url}\n models:\n f/m: { vision: true, context: 262144 }\n`, { mode: 0o600 })
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\n") writeFileSync(join(paths.config, "config.yaml"), "model: f/m\n")
const root = mkdtempSync(join(tmpdir(), "ph-img-")) const root = mkdtempSync(join(tmpdir(), "ph-img-"))
writeFileSync(join(root, "shot.png"), png(2560, 1440, 1_500_000)) writeFileSync(join(root, "shot.png"), png(2560, 1440, 1_500_000))
+1 -1
View File
@@ -33,7 +33,7 @@ describe("images", () => {
function setup(vision: boolean, script: Parameters<typeof fakeProvider>[0]) { function setup(vision: boolean, script: Parameters<typeof fakeProvider>[0]) {
fake = fakeProvider(script) fake = fakeProvider(script)
mkdirSync(paths.config, { recursive: true }) mkdirSync(paths.config, { recursive: true })
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n dialect: openai-chat\n base_url: ${fake.url}\n models:\n m: { vision: ${vision} }\n`, { mode: 0o600 }) writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n type: webui\n url: ${fake.url}\n models:\n f/m: { vision: ${vision} }\n`, { mode: 0o600 })
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\n") writeFileSync(join(paths.config, "config.yaml"), "model: f/m\n")
return createApp({ cwd: project(), mode: "edit", store: false, asker: { ask: async (): Promise<AskReply> => ({ kind: "once" }) } }) return createApp({ cwd: project(), mode: "edit", store: false, asker: { ask: async (): Promise<AskReply> => ({ kind: "once" }) } })
} }
+1 -1
View File
@@ -17,7 +17,7 @@ afterEach(() => fake?.stop())
function app(url: string) { function app(url: string) {
mkdirSync(paths.config, { recursive: true }) mkdirSync(paths.config, { recursive: true })
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n dialect: openai-chat\n base_url: ${url}\n models: { m: {} }\n`, { mode: 0o600 }) writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n type: webui\n url: ${url}\n models: { f/m: {} }\n`, { mode: 0o600 })
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\n") writeFileSync(join(paths.config, "config.yaml"), "model: f/m\n")
return createApp({ cwd: mkdtempSync(join(tmpdir(), "ph-wake-")), mode: "auto", store: false, asker: { ask: async (): Promise<AskReply> => ({ kind: "once" }) } }) return createApp({ cwd: mkdtempSync(join(tmpdir(), "ph-wake-")), mode: "auto", store: false, asker: { ask: async (): Promise<AskReply> => ({ kind: "once" }) } })
} }
+1 -1
View File
@@ -155,7 +155,7 @@ describe("in an app", () => {
function app(script: Parameters<typeof fakeProvider>[0], store?: Store) { function app(script: Parameters<typeof fakeProvider>[0], store?: Store) {
fake = fakeProvider(script) fake = fakeProvider(script)
mkdirSync(paths.config, { recursive: true }) mkdirSync(paths.config, { recursive: true })
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n dialect: openai-chat\n base_url: ${fake.url}\n models: { m: {} }\n`, { mode: 0o600 }) writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n type: webui\n url: ${fake.url}\n models: { f/m: {} }\n`, { mode: 0o600 })
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\ntitles: prompt\n") writeFileSync(join(paths.config, "config.yaml"), "model: f/m\ntitles: prompt\n")
return createApp({ cwd: dir(), mode: "auto", store: store ?? false, snapshots: false, asker: { ask: async (): Promise<AskReply> => ({ kind: "once" }) } }) return createApp({ cwd: dir(), mode: "auto", store: store ?? false, snapshots: false, asker: { ask: async (): Promise<AskReply> => ({ kind: "once" }) } })
} }
+14 -16
View File
@@ -25,14 +25,16 @@ test("draft-07, described, and no string held to a format ({env:} stands in for
const props = (schemas.config as any).properties const props = (schemas.config as any).properties
expect(props.mode.enum).toEqual(["manual", "edit", "auto", "plan"]) expect(props.mode.enum).toEqual(["manual", "edit", "auto", "plan"])
expect(props.small_model.description).toContain("session titles") expect(props.small_model.description).toContain("session titles")
const kinds = (schemas.connections as any).properties.connections.additionalProperties.anyOf // 2.0: the login is the only kind of entry.
expect(kinds[0].properties.type.const ?? kinds[0].properties.type.enum[0]).toBe("webui") const login = (schemas.connections as any).properties.connections.additionalProperties
expect(kinds[1].properties.base_url.description).toContain("Where the API is") expect(login.properties.type.const ?? login.properties.type.enum[0]).toBe("webui")
expect(login.properties.url.description).toContain("The instance's address")
expect(login.properties.dialect).toBeUndefined()
}) })
const goodConfig = ` const goodConfig = `
model: local/qwen3 model: llama/qwen3
small_model: local/tiny small_model: llama/tiny
titles: prompt titles: prompt
mode: edit mode: edit
icons: plain icons: plain
@@ -59,19 +61,14 @@ mcp:
` `
const goodConnections = ` const goodConnections = `
connections: connections:
local: example:
dialect: openai-chat type: webui
base_url: "{env:LLM_URL}" url: https://ai.example.org
api_key: "{file:~/.config/lembas/lembas/example.key}"
tls: { ca: ~/ca.crt } tls: { ca: ~/ca.crt }
quirks: { stream_usage: off } quirks: { stream_usage: off }
models: models:
qwen3: { context: 131072, max_output: 65536, efforts: [low, high], effort: low, vision: true } llama/qwen3: { context: 131072, max_output: 65536, efforts: [low, high], effort: low, vision: true }
anthropic:
dialect: anthropic
base_url: https://api.anthropic.com
api_key: "{env:ANTHROPIC_API_KEY}"
models:
claude: { effort_map: { low: 4000, high: 32000 }, cache: true }
` `
test("what the loader accepts, the schema accepts", () => { test("what the loader accepts, the schema accepts", () => {
@@ -94,7 +91,8 @@ test("…and what it refuses, the schema marks", () => {
expect(Config.safeParse(bad).success).toBe(false) expect(Config.safeParse(bad).success).toBe(false)
expect(checkConfig(bad)).toBe(false) expect(checkConfig(bad)).toBe(false)
} }
for (const bad of [{ connections: { x: { dialect: "openai", base_url: "http://h/v1" } } }, { connections: { x: { dialect: "anthropic" } } }, { connections: { x: { dialect: "anthropic", base_url: "u", models: { m: { context: -1 } } } } }]) { // A custom connection (1.x): the loader reads past it with a warning, and the schema marks it.
for (const bad of [{ connections: { x: { dialect: "openai-chat", base_url: "http://h/v1" } } }, { connections: { x: { type: "webui" } } }, { connections: { x: { type: "webui", url: "https://h", models: { m: { context: -1 } } } } }, { connections: { x: { type: "webui", url: "https://h", key_cmd: "pass x" } } }]) {
expect(checkConnections(bad)).toBe(false) expect(checkConnections(bad)).toBe(false)
} }
}) })
+3 -3
View File
@@ -51,7 +51,7 @@ test("the instance's library replaces this machine's, under the tools' own names
mkdirSync(join(paths.config, "lembas"), { recursive: true }) mkdirSync(join(paths.config, "lembas"), { recursive: true })
writeFileSync(join(paths.config, "lembas.json"), JSON.stringify({ instances: { ai: { base_url: lembas.base, connection: "ai", logged_in_at: "" } } })) writeFileSync(join(paths.config, "lembas.json"), JSON.stringify({ instances: { ai: { base_url: lembas.base, connection: "ai", logged_in_at: "" } } }))
writeFileSync(join(paths.config, "lembas", "ai.key"), "lmb_lib\n", { mode: 0o600 }) writeFileSync(join(paths.config, "lembas", "ai.key"), "lmb_lib\n", { mode: 0o600 })
writeFileSync(join(paths.config, "connections.yaml"), "connections:\n f:\n dialect: openai-chat\n base_url: http://127.0.0.1:1/v1\n models: { m: {} }\n", { mode: 0o600 }) writeFileSync(join(paths.config, "connections.yaml"), "connections:\n f:\n type: webui\n url: http://127.0.0.1:1/v1\n models: { f/m: {} }\n", { mode: 0o600 })
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\nlibrary: lembas\n") writeFileSync(join(paths.config, "config.yaml"), "model: f/m\nlibrary: lembas\n")
const app = createApp({ cwd: mkdtempSync(join(tmpdir(), "ph-lib-")), store: false, asker: { ask: async (): Promise<AskReply> => ({ kind: "once" }) } }) const app = createApp({ cwd: mkdtempSync(join(tmpdir(), "ph-lib-")), store: false, asker: { ask: async (): Promise<AskReply> => ({ kind: "once" }) } })
await app.mcpReady await app.mcpReady
@@ -74,7 +74,7 @@ test("a session the client gave a library scope sends its id on every call to th
mkdirSync(join(paths.config, "lembas"), { recursive: true }) mkdirSync(join(paths.config, "lembas"), { recursive: true })
writeFileSync(join(paths.config, "lembas.json"), JSON.stringify({ instances: { ai: { base_url: lembas.base, connection: "ai", logged_in_at: "" } } })) writeFileSync(join(paths.config, "lembas.json"), JSON.stringify({ instances: { ai: { base_url: lembas.base, connection: "ai", logged_in_at: "" } } }))
writeFileSync(join(paths.config, "lembas", "ai.key"), "lmb_lib\n", { mode: 0o600 }) writeFileSync(join(paths.config, "lembas", "ai.key"), "lmb_lib\n", { mode: 0o600 })
writeFileSync(join(paths.config, "connections.yaml"), "connections:\n f:\n dialect: openai-chat\n base_url: http://127.0.0.1:1/v1\n models: { m: {} }\n", { mode: 0o600 }) writeFileSync(join(paths.config, "connections.yaml"), "connections:\n f:\n type: webui\n url: http://127.0.0.1:1/v1\n models: { f/m: {} }\n", { mode: 0o600 })
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\nlibrary: lembas\n") writeFileSync(join(paths.config, "config.yaml"), "model: f/m\nlibrary: lembas\n")
const app = createApp({ cwd: mkdtempSync(join(tmpdir(), "ph-lib-")), asker: { ask: async (): Promise<AskReply> => ({ kind: "once" }) } }) const app = createApp({ cwd: mkdtempSync(join(tmpdir(), "ph-lib-")), asker: { ask: async (): Promise<AskReply> => ({ kind: "once" }) } })
stops.push(() => void app.close()) stops.push(() => void app.close())
@@ -104,7 +104,7 @@ test("a session the client gave a library scope sends its id on every call to th
test("without a login it says so and keeps this machine's library", () => { test("without a login it says so and keeps this machine's library", () => {
writeFileSync(join(paths.config, "lembas.json"), JSON.stringify({ instances: {} })) writeFileSync(join(paths.config, "lembas.json"), JSON.stringify({ instances: {} }))
writeFileSync(join(paths.config, "connections.yaml"), "connections:\n f:\n dialect: openai-chat\n base_url: http://127.0.0.1:1/v1\n models: { m: {} }\n", { mode: 0o600 }) writeFileSync(join(paths.config, "connections.yaml"), "connections:\n f:\n type: webui\n url: http://127.0.0.1:1/v1\n models: { f/m: {} }\n", { mode: 0o600 })
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\nlibrary: lembas\n") writeFileSync(join(paths.config, "config.yaml"), "model: f/m\nlibrary: lembas\n")
const app = createApp({ cwd: mkdtempSync(join(tmpdir(), "ph-lib-")), store: false, asker: { ask: async (): Promise<AskReply> => ({ kind: "once" }) } }) const app = createApp({ cwd: mkdtempSync(join(tmpdir(), "ph-lib-")), store: false, asker: { ask: async (): Promise<AskReply> => ({ kind: "once" }) } })
expect(app.loaded.warnings.join(" ")).toContain("not logged in") expect(app.loaded.warnings.join(" ")).toContain("not logged in")
+4 -4
View File
@@ -187,14 +187,14 @@ test("login writes one webui connection named after the instance, and no models"
const loaded = loadConfig() const loaded = loadConfig()
const c = loaded.connections.example! const c = loaded.connections.example!
expect(c.dialect).toBe("openai-chat") expect(c.webui?.url).toBe(fake.url)
expect(c.base_url).toBe(`${fake.url}/v1`) expect(c.base_url).toBe(`${fake.url}/v1`)
expect(c.api_key).toBe("lmb_secret") expect(c.api_key).toBe("lmb_secret")
// The instance's models with its settings, the default first; the group named after the connection. // The instance's models with its settings, the default first; the group named after the connection.
expect(Object.keys(c.models)).toEqual(["qwen", "gemma"]) expect(Object.keys(c.models)).toEqual(["qwen", "gemma"])
expect(c.models.qwen).toEqual({ name: "Qwen", context: 131072, temperature: 0.6, top_p: 0.95, efforts: ["low", "high"], tools: true, group: "example:gpu" }) expect(c.models.qwen).toEqual({ name: "Qwen", context: 131072, temperature: 0.6, top_p: 0.95, efforts: ["low", "high"], tools: true, group: "example:gpu" })
expect(c.models.gemma).toEqual({ context: 32768, vision: true, single_session: true, group: "example:gpu" }) expect(c.models.gemma).toEqual({ context: 32768, vision: true, single_session: true, group: "example:gpu" })
expect(resolveModel(loaded, undefined).ref).toBe("example/qwen") expect(resolveModel(loaded, undefined).ref).toBe("qwen")
}) })
test("a model changed on the instance is here at the next start, without logging in again", async () => { test("a model changed on the instance is here at the next start, without logging in again", async () => {
@@ -205,7 +205,7 @@ test("a model changed on the instance is here at the next start, without logging
expect(r).toEqual([{ name: "example", ok: true, models: 1 }]) expect(r).toEqual([{ name: "example", ok: true, models: 1 }])
const loaded = loadConfig() const loaded = loadConfig()
expect(Object.keys(loaded.connections.example!.models)).toEqual(["llama"]) expect(Object.keys(loaded.connections.example!.models)).toEqual(["llama"])
expect(resolveModel(loaded, undefined).ref).toBe("example/llama") expect(resolveModel(loaded, undefined).ref).toBe("llama")
// A model the instance does not list yet is still asked for, not refused here. // A model the instance does not list yet is still asked for, not refused here.
expect(resolveModel(loaded, "example/brand-new").id).toBe("brand-new") expect(resolveModel(loaded, "example/brand-new").id).toBe("brand-new")
}) })
@@ -266,7 +266,7 @@ test("an older kind of login is replaced: the old connection, its key and its pi
const c = cfg() const c = cfg()
// It was the instance's default, so the instance decides it now; the rest follow the new name. // It was the instance's default, so the instance decides it now; the rest follow the new name.
expect(c.model).toBeUndefined() expect(c.model).toBeUndefined()
expect(c.small_model).toBe("example/gemma") expect(c.small_model).toBe("gemma")
expect(r.changes.join("\n")).toContain("the connection ai from an earlier login is now example") expect(r.changes.join("\n")).toContain("the connection ai from an earlier login is now example")
}) })
+6 -3
View File
@@ -96,10 +96,13 @@ test("an OpenAI-shaped /embeddings: in order, unit length", async () => {
return Response.json({ data: input.map((_, i) => ({ index: i, embedding: [3, 4] })).reverse() }) return Response.json({ data: input.map((_, i) => ({ index: i, embedding: [3, 4] })).reverse() })
}, },
}) })
const e = embedderFor({ connections: { emb: { dialect: "openai-chat", base_url: `http://127.0.0.1:${server.port}/v1`, models: {} } } } as never, "emb/nomic")! // The server's embedding model, by its served id.
const loaded = { connections: { emb: { base_url: `http://127.0.0.1:${server.port}/v1`, models: {}, webui: { url: `http://127.0.0.1:${server.port}`, embeddings: ["llama/nomic"] } } }, refs: [] } as never
const e = embedderFor(loaded, "llama/nomic")!
const [v] = await e.embed(["x", "y"]) const [v] = await e.embed(["x", "y"])
expect(Array.from(v!)).toEqual([0.6000000238418579, 0.800000011920929]) expect(Array.from(v!)).toEqual([0.6000000238418579, 0.800000011920929])
expect(() => embedderFor({ connections: { a: { dialect: "anthropic", base_url: "http://x", models: {} } } } as never, "a/b")).toThrow("no embeddings") // Not logged in: said.
expect(() => embedderFor({ connections: {}, refs: [] } as never, "llama/nomic")).toThrow("not logged in")
}) })
test("ingest: text and code; HTML as markdown; binaries skipped; long text cut and said; links out of the directory ignored", async () => { test("ingest: text and code; HTML as markdown; binaries skipped; long text cut and said; links out of the directory ignored", async () => {
@@ -135,7 +138,7 @@ test("the tools: knowledge_search is offered only with something to search; a pr
{ chunks: [delta({ content: "ok" }, "stop")] }, { chunks: [delta({ content: "ok" }, "stop")] },
]) ])
mkdirSync(paths.config, { recursive: true }) mkdirSync(paths.config, { recursive: true })
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n dialect: openai-chat\n base_url: ${fake.url}\n models: { m: {} }\n`, { mode: 0o600 }) writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n type: webui\n url: ${fake.url}\n models: { f/m: {} }\n`, { mode: 0o600 })
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\ntitles: prompt\n") writeFileSync(join(paths.config, "config.yaml"), "model: f/m\ntitles: prompt\n")
const cwd = mkdtempSync(join(tmpdir(), "ph-libtools-")) const cwd = mkdtempSync(join(tmpdir(), "ph-libtools-"))
mkdirSync(join(cwd, ".agent")) mkdirSync(join(cwd, ".agent"))
+5 -5
View File
@@ -34,7 +34,7 @@ function repo(): string {
/** The user's own files: a connection and a config.yaml local mode must not read. */ /** The user's own files: a connection and a config.yaml local mode must not read. */
function userFiles(extraConfig = "") { function userFiles(extraConfig = "") {
mkdirSync(globalPaths.config, { recursive: true }) mkdirSync(globalPaths.config, { recursive: true })
writeFileSync(join(globalPaths.config, "connections.yaml"), "connections:\n mine:\n dialect: openai-chat\n base_url: http://127.0.0.1:9/v1\n models: { m: {} }\n", { mode: 0o600 }) writeFileSync(join(globalPaths.config, "connections.yaml"), "connections:\n mine:\n type: webui\n url: http://127.0.0.1:9/v1\n models: { mine/m: {} }\n", { mode: 0o600 })
writeFileSync(join(globalPaths.config, "config.yaml"), `model: mine/m\ntheme: forest\n${extraConfig}`) writeFileSync(join(globalPaths.config, "config.yaml"), `model: mine/m\ntheme: forest\n${extraConfig}`)
} }
@@ -67,7 +67,7 @@ test("a local directory that came with the repository is refused", () => {
expect(isLocal()).toBe(false) expect(isLocal()).toBe(false)
}) })
test("the user's config.yaml is not read; their connections are, and a local one wins by name", () => { test("the user's config.yaml is not read; their login is, and a local one wins by name", () => {
userFiles() userFiles()
const dir = repo() const dir = repo()
enterLocal(dir) enterLocal(dir)
@@ -76,10 +76,10 @@ test("the user's config.yaml is not read; their connections are, and a local one
expect(loaded.config.theme).toBeUndefined() expect(loaded.config.theme).toBeUndefined()
expect(Object.keys(loaded.connections)).toEqual(["mine"]) expect(Object.keys(loaded.connections)).toEqual(["mine"])
writeFileSync(join(paths.config, "config.yaml"), "model: mine/m\n") writeFileSync(join(paths.config, "config.yaml"), "model: mine/m\n")
writeFileSync(join(paths.config, "connections.yaml"), "connections:\n mine:\n dialect: openai-chat\n base_url: http://127.0.0.1:8/v1\n models: { n: {} }\n", { mode: 0o600 }) writeFileSync(join(paths.config, "connections.yaml"), "connections:\n mine:\n type: webui\n url: http://127.0.0.1:8/v1\n models: { mine/n: {} }\n", { mode: 0o600 })
loaded = loadConfig() loaded = loadConfig()
expect(loaded.config.model).toBe("mine/m") expect(loaded.config.model).toBe("mine/m")
expect(Object.keys(loaded.connections.mine!.models)).toEqual(["n"]) expect(Object.keys(loaded.connections.mine!.models)).toEqual(["mine/n"])
}) })
test("a login made outside local mode is read, never written or signed out", async () => { test("a login made outside local mode is read, never written or signed out", async () => {
@@ -152,7 +152,7 @@ test("the CLI: -l refuses what it turns off, and a run writes nothing outside th
const home = realpathSync(mkdtempSync(join(tmpdir(), "lembas-local-home-"))) const home = realpathSync(mkdtempSync(join(tmpdir(), "lembas-local-home-")))
const tmp = realpathSync(mkdtempSync(join(tmpdir(), "lembas-local-tmp-"))) const tmp = realpathSync(mkdtempSync(join(tmpdir(), "lembas-local-tmp-")))
mkdirSync(join(home, ".config/lembas"), { recursive: true }) mkdirSync(join(home, ".config/lembas"), { recursive: true })
writeFileSync(join(home, ".config/lembas/connections.yaml"), `connections:\n fake:\n dialect: openai-chat\n base_url: ${fake.url}\n models: { m: {} }\n`, { mode: 0o600 }) writeFileSync(join(home, ".config/lembas/connections.yaml"), `connections:\n fake:\n type: webui\n url: ${fake.url}\n models: { fake/m: {} }\n`, { mode: 0o600 })
writeFileSync(join(home, ".config/lembas/config.yaml"), "model: nowhere/x\n") writeFileSync(join(home, ".config/lembas/config.yaml"), "model: nowhere/x\n")
const dir = repo() const dir = repo()
const env = { ...process.env, LEMBAS_HOME: home, HOME: home, TMPDIR: tmp, XDG_DATA_HOME: join(home, ".local/share"), XDG_STATE_HOME: join(home, ".local/state") } const env = { ...process.env, LEMBAS_HOME: home, HOME: home, TMPDIR: tmp, XDG_DATA_HOME: join(home, ".local/share"), XDG_STATE_HOME: join(home, ".local/state") }
+1 -1
View File
@@ -181,7 +181,7 @@ describe("through the engine", () => {
{ chunks: [delta({ content: "It said it." }, "stop"), usage(20, 5)] }, { chunks: [delta({ content: "It said it." }, "stop"), usage(20, 5)] },
]) ])
mkdirSync(paths.config, { recursive: true }) mkdirSync(paths.config, { recursive: true })
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n fake:\n dialect: openai-chat\n base_url: ${fake.url}\n models:\n m: { context: 32768 }\n`, { mode: 0o600 }) writeFileSync(join(paths.config, "connections.yaml"), `connections:\n fake:\n type: webui\n url: ${fake.url}\n models:\n fake/m: { context: 32768 }\n`, { mode: 0o600 })
writeFileSync(join(paths.config, "config.yaml"), `model: fake/m\nmcp:\n fix:\n command: ["${BUN}", "${SERVER}"]\n`) writeFileSync(join(paths.config, "config.yaml"), `model: fake/m\nmcp:\n fix:\n command: ["${BUN}", "${SERVER}"]\n`)
const dir = mkdtempSync(join(tmpdir(), "lembas-mcp-")) const dir = mkdtempSync(join(tmpdir(), "lembas-mcp-"))
const asked: string[] = [] const asked: string[] = []
+2 -32
View File
@@ -24,7 +24,7 @@ function setup(script: Parameters<typeof fakeProvider>[0], connections: (url: st
} }
const two = (url: string) => const two = (url: string) =>
`connections:\n f:\n dialect: openai-chat\n base_url: ${url}\n models:\n main: {}\n big: { name: Big One, notes: slow but careful; best at reviews }\n` `connections:\n f:\n type: webui\n url: ${url}\n models:\n f/main: {}\n f/big: { name: Big One, notes: slow but careful; best at reviews }\n`
test("the model is told which other models there are, with their notes", async () => { test("the model is told which other models there are, with their notes", async () => {
const { app } = setup([{ chunks: [delta({ content: "ok" }, "stop")] }], two) const { app } = setup([{ chunks: [delta({ content: "ok" }, "stop")] }], two)
@@ -44,38 +44,8 @@ test("task runs a subagent on the model asked for", async () => {
two, two,
) )
expect(await app.engine.prompt("ask the big one")).toBe("stop") expect(await app.engine.prompt("ask the big one")).toBe("stop")
expect(fake!.requests.map((r) => r.model)).toEqual(["main", "big", "main"]) expect(fake!.requests.map((r) => r.model)).toEqual(["f/main", "f/big", "f/main"])
const result = app.engine.messages.find((m) => m.role === "tool") const result = app.engine.messages.find((m) => m.role === "tool")
expect(result && result.role === "tool" && result.content).toContain("Yes.") expect(result && result.role === "tool" && result.content).toContain("Yes.")
}) })
test("a server that cannot be reached: the model's fallback takes over, said and kept", async () => {
const { app, events } = setup([{ chunks: [delta({ content: "from the fallback" }, "stop")] }], (url) =>
[
"connections:",
" gone:",
" dialect: openai-chat",
" base_url: http://127.0.0.1:9/v1",
" timeout: 5",
" models:",
" main: { fallback: [f/main] }",
" f:",
" dialect: openai-chat",
` base_url: ${url}`,
" models:",
" main: {}",
"",
].join("\n"),
)
app.settings.set("model", "gone/main")
expect(await app.engine.prompt("hi")).toBe("stop")
expect(app.engine.model.ref).toBe("f/main")
expect(events.some((e) => e.type === "notice" && e.message.includes("switching to f/main, its fallback"))).toBe(true)
expect(events.some((e) => e.type === "setting" && e.key === "model")).toBe(true)
})
test("no fallback: the error stands", async () => {
const { app } = setup([], (url) => `connections:\n gone:\n dialect: openai-chat\n base_url: http://127.0.0.1:9/v1\n models:\n main: {}\n f:\n dialect: openai-chat\n base_url: ${url}\n models:\n main: {}\n`)
app.settings.set("model", "gone/main")
expect(await app.engine.prompt("hi")).toBe("error")
})
+1 -1
View File
@@ -85,7 +85,7 @@ test("a model calling notes with an action still gets its note written, and read
{ chunks: [delta({ content: "ok" }, "stop")] }, { chunks: [delta({ content: "ok" }, "stop")] },
]) ])
mkdirSync(paths.config, { recursive: true }) mkdirSync(paths.config, { recursive: true })
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n dialect: openai-chat\n base_url: ${fake.url}\n models: { m: {} }\n`, { mode: 0o600 }) writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n type: webui\n url: ${fake.url}\n models: { f/m: {} }\n`, { mode: 0o600 })
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\ntitles: prompt\n") writeFileSync(join(paths.config, "config.yaml"), "model: f/m\ntitles: prompt\n")
const app = createApp({ cwd: mkdtempSync(join(tmpdir(), "ph-names-")), store: false, snapshots: false, asker: { ask: async () => ({ kind: "once" }) } }) const app = createApp({ cwd: mkdtempSync(join(tmpdir(), "ph-names-")), store: false, snapshots: false, asker: { ask: async () => ({ kind: "once" }) } })
for (const n of app.library.notes(["global"])) app.library.deleteNote(n.id) for (const n of app.library.notes(["global"])) app.library.deleteNote(n.id)
+2 -2
View File
@@ -137,7 +137,7 @@ test("logged in, the account's personalization is used instead of the local keys
test("/settings writes the personalization to the account, not to config.yaml", async () => { test("/settings writes the personalization to the account, not to config.yaml", async () => {
inst = instance() inst = instance()
model = fakeProvider([]) model = fakeProvider([])
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n dialect: openai-chat\n base_url: ${model.url}\n models: { m: {} }\n`, { mode: 0o600 }) writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n type: webui\n url: ${model.url}\n models: { f/m: {} }\n`, { mode: 0o600 })
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\nlibrary: local\n") writeFileSync(join(paths.config, "config.yaml"), "model: f/m\nlibrary: local\n")
loggedIn(inst) loggedIn(inst)
await fetchWebui("example", inst.url, "lmb_key", undefined) await fetchWebui("example", inst.url, "lmb_key", undefined)
@@ -188,7 +188,7 @@ test("the library: the account's by default when the login may read it; an expli
test("opted out on the web, or a preset the CLI does not know: setting one key here wipes nothing there", async () => { test("opted out on the web, or a preset the CLI does not know: setting one key here wipes nothing there", async () => {
inst = instance({ personal: { enabled: false, available: true, personality: "pirate", personality_custom: "Arr.", instructions: "Answer in Slovak." } }) inst = instance({ personal: { enabled: false, available: true, personality: "pirate", personality_custom: "Arr.", instructions: "Answer in Slovak." } })
model = fakeProvider([]) model = fakeProvider([])
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n dialect: openai-chat\n base_url: ${model.url}\n models: { m: {} }\n`, { mode: 0o600 }) writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n type: webui\n url: ${model.url}\n models: { f/m: {} }\n`, { mode: 0o600 })
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\nlibrary: local\n") writeFileSync(join(paths.config, "config.yaml"), "model: f/m\nlibrary: local\n")
loggedIn(inst) loggedIn(inst)
await fetchWebui("example", inst.url, "lmb_key", undefined) await fetchWebui("example", inst.url, "lmb_key", undefined)
+1 -1
View File
@@ -68,7 +68,7 @@ test("the engine keeps the plan with the session: said on the bus, stored, back
{ chunks: [delta({ content: "ok" })] }, { chunks: [delta({ content: "ok" })] },
]) ])
mkdirSync(paths.config, { recursive: true }) mkdirSync(paths.config, { recursive: true })
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n dialect: openai-chat\n base_url: ${fake.url}\n models: { m: {} }\n`, { mode: 0o600 }) writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n type: webui\n url: ${fake.url}\n models: { f/m: {} }\n`, { mode: 0o600 })
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\ntitles: prompt\n") writeFileSync(join(paths.config, "config.yaml"), "model: f/m\ntitles: prompt\n")
const cwd = mkdtempSync(join(tmpdir(), "ph-planstore-")) const cwd = mkdtempSync(join(tmpdir(), "ph-planstore-"))
const store = new Store(join(cwd, "sessions.db")) const store = new Store(join(cwd, "sessions.db"))
+1 -1
View File
@@ -16,7 +16,7 @@ const RENDERED = "# Fix the average\nObjectives: [o1] Tests pass (open)\n## p1 C
function app(script: Parameters<typeof fakeProvider>[0], plan?: (r: PlanRequest) => Promise<PlanReply>) { function app(script: Parameters<typeof fakeProvider>[0], plan?: (r: PlanRequest) => Promise<PlanReply>) {
fake = fakeProvider(script) fake = fakeProvider(script)
mkdirSync(paths.config, { recursive: true }) mkdirSync(paths.config, { recursive: true })
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n dialect: openai-chat\n base_url: ${fake.url}\n models: { m: {} }\n`, { mode: 0o600 }) writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n type: webui\n url: ${fake.url}\n models: { f/m: {} }\n`, { mode: 0o600 })
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\n") writeFileSync(join(paths.config, "config.yaml"), "model: f/m\n")
const cwd = mkdtempSync(join(tmpdir(), "ph-plan-")) const cwd = mkdtempSync(join(tmpdir(), "ph-plan-"))
Bun.spawnSync(["git", "init", "-q", cwd]) Bun.spawnSync(["git", "init", "-q", cwd])
+26 -57
View File
@@ -1,8 +1,8 @@
// Models from a LLeMbas instance named by their provider: `deepseek/deepseek-flash`, not // Models of the LLeMbas server named by the id it serves them under: `deepseek/deepseek-flash`, not
// `example/deepseek-flash`. The old forms still resolve, a config naming one is rewritten, a // `example/deepseek-flash`. LLeMbas CLI 2.0 talks to one server, so the id is the ref; the older
// connection of the user's own wins its name, two instances share a provider by login order, and an // forms still resolve and a config naming one is rewritten; a custom connection beside the login is
// instance that does not speak protocol 2 (bare ids) keeps `<login>/<id>`. On the link, the model event and the // read past. On the link, the model event and the announcement say whether the session's model is
// announcement say whether the session's model is this link's instance's. // this link's instance's.
import { afterEach, beforeEach, expect, test } from "bun:test" import { afterEach, beforeEach, expect, test } from "bun:test"
import { existsSync, mkdirSync, mkdtempSync, readFileSync, realpathSync, rmSync, writeFileSync } from "node:fs" import { existsSync, mkdirSync, mkdtempSync, readFileSync, realpathSync, rmSync, writeFileSync } from "node:fs"
import { tmpdir } from "node:os" import { tmpdir } from "node:os"
@@ -35,27 +35,19 @@ function clean() {
rmSync(join(paths.state, "webui"), { recursive: true, force: true }) rmSync(join(paths.state, "webui"), { recursive: true, force: true })
} }
const conns = (o: Record<string, { webui?: boolean; models: string[] }>) => const conns = (o: Record<string, { models: string[] }>) =>
Object.fromEntries(Object.entries(o).map(([n, c]) => [n, { dialect: "openai-chat", base_url: "http://x/v1", models: Object.fromEntries(c.models.map((m) => [m, {}])), ...(c.webui ? { webui: { url: "http://x", v2: true } } : {}) }])) as never Object.fromEntries(Object.entries(o).map(([n, c]) => [n, { base_url: "http://x/v1", models: Object.fromEntries(c.models.map((m) => [m, {}])), webui: { url: "http://x" } }])) as never
test("refs: by provider; a connection of the user's own wins its name; a bare id keeps the login", () => { test("refs: the served id; the older forms name the same model", () => {
const c = conns({ example: { webui: true, models: ["deepseek/deepseek-flash", "llama/bonsai", "old"] }, llama: { models: ["mine"] } }) const c = conns({ example: { models: ["deepseek/deepseek-flash", "llama/bonsai", "old"] } })
const refs = modelRefs(c, ["example"]) const refs = modelRefs(c)
expect(refs.map((r) => r.ref)).toEqual(["deepseek/deepseek-flash", "example/llama/bonsai", "example/old", "llama/mine"]) expect(refs.map((r) => r.ref)).toEqual(["deepseek/deepseek-flash", "llama/bonsai", "old"])
// Every old way of writing them. expect(findRef(c, refs, "example/deepseek-flash")).toMatchObject({ ref: "deepseek/deepseek-flash", old: true })
expect(findRef(c, refs, "example/deepseek-flash")?.ref).toBe("deepseek/deepseek-flash") expect(findRef(c, refs, "example/deepseek/deepseek-flash")).toMatchObject({ ref: "deepseek/deepseek-flash", old: true })
expect(findRef(c, refs, "example/deepseek/deepseek-flash")?.ref).toBe("deepseek/deepseek-flash") expect(findRef(c, refs, "example/bonsai")?.ref).toBe("llama/bonsai")
expect(findRef(c, refs, "example/bonsai")?.ref).toBe("example/llama/bonsai") expect(findRef(c, refs, "example/old")?.ref).toBe("old")
expect(findRef(c, refs, "llama/mine")).toMatchObject({ connection: "llama", id: "mine" }) expect(findRef(c, refs, "llama/bonsai")?.old).toBeUndefined()
expect(findRef(c, refs, "llama/bonsai")).toBeUndefined() expect(findRef(c, refs, "other/bonsai")).toBeUndefined()
})
test("refs: two instances with one provider — the first logged in to keeps the short form", () => {
const c = conns({ example: { webui: true, models: ["deepseek/a"] }, work: { webui: true, models: ["deepseek/a", "openai/b"] } })
expect(modelRefs(c, ["example", "work"]).map((r) => r.ref)).toEqual(["deepseek/a", "work/deepseek/a", "openai/b"])
expect(modelRefs(c, ["work", "example"]).map((r) => r.ref)).toEqual(["example/deepseek/a", "deepseek/a", "openai/b"])
// A login named after its own provider: the same string either way.
expect(modelRefs(conns({ llama: { webui: true, models: ["llama/bonsai"] } }), ["llama"]).map((r) => r.ref)).toEqual(["llama/bonsai"])
}) })
/** Logged in to an instance served by `fake`, whose models are `models` (as /v1/models said them; /** Logged in to an instance served by `fake`, whose models are `models` (as /v1/models said them;
@@ -94,23 +86,13 @@ test("logged in: the model is deepseek/deepseek-flash everywhere, requests go to
await app.close() await app.close()
}) })
test("an instance that does not speak protocol 2 (bare ids): <login>/<id>", () => { test("a custom connection beside the login is read past, and said (2.0 has none)", () => {
fake = fakeProvider([])
loggedIn(["deepseek-flash"])
writeFileSync(join(paths.config, "config.yaml"), "model: example/deepseek-flash\n")
const loaded = loadConfig()
expect(loaded.refs.map((r) => r.ref)).toEqual(["example/deepseek-flash"])
expect(resolveModel(loaded, undefined).ref).toBe("example/deepseek-flash")
expect(readFileSync(join(paths.config, "config.yaml"), "utf8")).toBe("model: example/deepseek-flash\n")
})
test("a connection of the user's own named like a provider keeps it; the instance's is <login>/<provider>/<model>", () => {
fake = fakeProvider([]) fake = fakeProvider([])
loggedIn(["deepseek/deepseek-flash"], ` deepseek:\n dialect: openai-chat\n base_url: ${"http://127.0.0.1:9/v1"}\n models: { deepseek-flash: {} }\n`) loggedIn(["deepseek/deepseek-flash"], ` deepseek:\n dialect: openai-chat\n base_url: ${"http://127.0.0.1:9/v1"}\n models: { deepseek-flash: {} }\n`)
const loaded = loadConfig() const loaded = loadConfig()
expect(loaded.refs.map((r) => r.ref)).toEqual(["example/deepseek/deepseek-flash", "deepseek/deepseek-flash"]) expect(loaded.refs.map((r) => r.ref)).toEqual(["deepseek/deepseek-flash"])
expect(resolveModel(loaded, "deepseek/deepseek-flash").connectionName).toBe("deepseek") expect(resolveModel(loaded, "deepseek/deepseek-flash").connectionName).toBe("example")
expect(resolveModel(loaded, "example/deepseek/deepseek-flash").connectionName).toBe("example") expect(loaded.warnings.some((w) => w.includes("deepseek is a custom connection"))).toBe(true)
}) })
function pair(): [Transport, Transport] { function pair(): [Transport, Transport] {
@@ -145,13 +127,13 @@ test("on the link: the instance names its model as served, the session reports t
// The audit of 568923c. // The audit of 568923c.
test("only the old <login>/<model> is rewritten, once, in place: a pinned long form, anchors and comments stay", () => { test("an older form is rewritten, once, in place: anchors and comments stay", () => {
fake = fakeProvider([]) fake = fakeProvider([])
loggedIn(["deepseek/deepseek-flash", "llama/nomic-embed"]) loggedIn(["deepseek/deepseek-flash", "llama/nomic-embed"])
// Pinned on purpose: means what it says, and is left alone. // The login's name in front: one server, so it is the same model, written the old way.
writeFileSync(join(paths.config, "config.yaml"), "model: example/deepseek/deepseek-flash\n") writeFileSync(join(paths.config, "config.yaml"), "model: example/deepseek/deepseek-flash\n")
expect(loadConfig().config.model).toBe("example/deepseek/deepseek-flash") expect(loadConfig().config.model).toBe("deepseek/deepseek-flash")
expect(readFileSync(join(paths.config, "config.yaml"), "utf8")).toBe("model: example/deepseek/deepseek-flash\n") expect(readFileSync(join(paths.config, "config.yaml"), "utf8")).toBe("model: deepseek/deepseek-flash\n")
// The old form behind an anchor, with a comment: the scalar changes, nothing else. // The old form behind an anchor, with a comment: the scalar changes, nothing else.
writeFileSync(join(paths.config, "config.yaml"), "model: &m example/deepseek-flash # my pick\nsmall_model: *m\nembedding: example/nomic-embed\n") writeFileSync(join(paths.config, "config.yaml"), "model: &m example/deepseek-flash # my pick\nsmall_model: *m\nembedding: example/nomic-embed\n")
const l = loadConfig() const l = loadConfig()
@@ -168,25 +150,12 @@ test("only the old <login>/<model> is rewritten, once, in place: a pinned long f
expect(existsSync(join(paths.state, "migrations.json"))).toBe(true) expect(existsSync(join(paths.state, "migrations.json"))).toBe(true)
}) })
test("the provider: as the instance says it; from a protocol-2 instance, before the /; else the id is just an id", () => {
fake = fakeProvider([])
// Not protocol 2: an id with a / and no provider is `<login>/<id>`.
loggedIn(["org/model-a"], "", { noProvider: true })
expect(loadConfig().refs.map((r) => r.ref)).toEqual(["example/org/model-a"])
// The instance speaks protocol 2: its ids are `<provider>/<model>`.
loggedIn(["org/model-a"], "", { noProvider: true, protocols: [1, 2] })
expect(loadConfig().refs.map((r) => r.ref)).toEqual(["org/model-a"])
// `provider` said: used, whatever the protocol.
loggedIn(["org/model-a"])
expect(loadConfig().refs.map((r) => r.ref)).toEqual(["org/model-a"])
})
test("a models: override keyed by the old bare name lands on the served model; no phantom appears", () => { test("a models: override keyed by the old bare name lands on the served model; no phantom appears", () => {
fake = fakeProvider([]) fake = fakeProvider([])
loggedIn(["deepseek/deepseek-flash"], " models: { deepseek-flash: { context: 1000 }, upcoming: { context: 5 } }\n") loggedIn(["deepseek/deepseek-flash"], " models: { deepseek-flash: { context: 1000 }, upcoming: { context: 5 } }\n")
writeFileSync(join(paths.config, "config.yaml"), "model: example/deepseek-flash\n") writeFileSync(join(paths.config, "config.yaml"), "model: example/deepseek-flash\n")
const l = loadConfig() const l = loadConfig()
expect(l.refs.map((r) => r.ref)).toEqual(["deepseek/deepseek-flash", "example/upcoming"]) expect(l.refs.map((r) => r.ref)).toEqual(["deepseek/deepseek-flash", "upcoming"])
const m = resolveModel(l, l.config.model) const m = resolveModel(l, l.config.model)
expect(m).toMatchObject({ id: "deepseek/deepseek-flash", spec: { context: 1000 } }) expect(m).toMatchObject({ id: "deepseek/deepseek-flash", spec: { context: 1000 } })
}) })
+1 -1
View File
@@ -13,7 +13,7 @@ afterEach(() => {
}) })
function model(url: string, spec: ResolvedModel["spec"] = {}, conn: Partial<Connection> = {}): ResolvedModel { function model(url: string, spec: ResolvedModel["spec"] = {}, conn: Partial<Connection> = {}): ResolvedModel {
const connection: Connection = { dialect: "openai-chat", base_url: url, models: { m: spec }, ...conn } const connection: Connection = { base_url: url, models: { m: spec }, ...conn }
return { ref: `c${Math.random().toString(36).slice(2, 7)}/m`, connectionName: "c", connection, id: "m", spec } return { ref: `c${Math.random().toString(36).slice(2, 7)}/m`, connectionName: "c", connection, id: "m", spec }
} }
+1 -1
View File
@@ -51,7 +51,7 @@ test("the asker is told the purpose, and the tool runs without it", async () =>
{ chunks: [delta({ content: "ok" }, "stop")] }, { chunks: [delta({ content: "ok" }, "stop")] },
]) ])
mkdirSync(paths.config, { recursive: true }) mkdirSync(paths.config, { recursive: true })
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n dialect: openai-chat\n base_url: ${fake.url}\n models: { m: {} }\n`, { mode: 0o600 }) writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n type: webui\n url: ${fake.url}\n models: { f/m: {} }\n`, { mode: 0o600 })
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\n") writeFileSync(join(paths.config, "config.yaml"), "model: f/m\n")
const asked: (string | undefined)[] = [] const asked: (string | undefined)[] = []
const app = createApp({ cwd: mkdtempSync(join(tmpdir(), "ph-purpose-")), store: false, snapshots: false, asker: { ask: async (r): Promise<AskReply> => (asked.push(r.purpose), { kind: "once" }) } }) const app = createApp({ cwd: mkdtempSync(join(tmpdir(), "ph-purpose-")), store: false, snapshots: false, asker: { ask: async (r): Promise<AskReply> => (asked.push(r.purpose), { kind: "once" }) } })
+5 -5
View File
@@ -18,7 +18,7 @@ afterEach(() => fake?.stop())
async function replay(name: string) { async function replay(name: string) {
fake = fakeProvider([{ chunks: [], raw: fixture(name) }]) fake = fakeProvider([{ chunks: [], raw: fixture(name) }])
const m: ResolvedModel = { ref: "swap/x", connectionName: "swap", id: "x", spec: {}, connection: { dialect: "openai-chat", base_url: fake.url, models: {} } } const m: ResolvedModel = { ref: "swap/x", connectionName: "swap", id: "x", spec: {}, connection: { base_url: fake.url, models: {} } }
const events: StreamEvent[] = [] const events: StreamEvent[] = []
for await (const e of new OpenAIChatClient(m).stream({ system: "", messages: [{ role: "user", parts: [{ type: "text", text: "x" }] }], tools: [] })) for await (const e of new OpenAIChatClient(m).stream({ system: "", messages: [{ role: "user", parts: [{ type: "text", text: "x" }] }], tools: [] }))
events.push(e) events.push(e)
@@ -48,7 +48,7 @@ describe("recorded llama-swap streams", () => {
test("qwen35: an answer given only inside reasoning is followed by one request for the answer", async () => { test("qwen35: an answer given only inside reasoning is followed by one request for the answer", async () => {
fake = fakeProvider([{ chunks: [], raw: fixture("qwen35-answer-only-in-reasoning.sse") }, { chunks: [delta({ content: "The divisor was off by one." })] }]) fake = fakeProvider([{ chunks: [], raw: fixture("qwen35-answer-only-in-reasoning.sse") }, { chunks: [delta({ content: "The divisor was off by one." })] }])
mkdirSync(paths.config, { recursive: true }) mkdirSync(paths.config, { recursive: true })
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n dialect: openai-chat\n base_url: ${fake.url}\n models: { m: {} }\n`, { mode: 0o600 }) writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n type: webui\n url: ${fake.url}\n models: { f/m: {} }\n`, { mode: 0o600 })
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\n") writeFileSync(join(paths.config, "config.yaml"), "model: f/m\n")
const dir = mkdtempSync(join(tmpdir(), "lembas-replay-")) const dir = mkdtempSync(join(tmpdir(), "lembas-replay-"))
const app = createApp({ cwd: dir, mode: "edit", store: false, asker: { ask: async (): Promise<AskReply> => ({ kind: "deny" }) } }) const app = createApp({ cwd: dir, mode: "edit", store: false, asker: { ask: async (): Promise<AskReply> => ({ kind: "deny" }) } })
@@ -95,7 +95,7 @@ describe("effort refused inside a 200 stream (llama-swap loading, then the templ
connectionName: "swap", connectionName: "swap",
id: "bonsai", id: "bonsai",
spec: { efforts: ["low", "medium", "high"] }, spec: { efforts: ["low", "medium", "high"] },
connection: { dialect: "openai-chat", base_url: fake.url, models: {} }, connection: { base_url: fake.url, models: {} },
} }
const events: StreamEvent[] = [] const events: StreamEvent[] = []
for await (const e of new OpenAIChatClient(m).stream({ system: "", messages: [{ role: "user", parts: [{ type: "text", text: "x" }] }], tools: [], effort: "high" })) for await (const e of new OpenAIChatClient(m).stream({ system: "", messages: [{ role: "user", parts: [{ type: "text", text: "x" }] }], tools: [], effort: "high" }))
@@ -111,7 +111,7 @@ describe("effort refused inside a 200 stream (llama-swap loading, then the templ
test("an error after output has started is not retried", async () => { test("an error after output has started is not retried", async () => {
fake = fakeProvider([{ chunks: [], raw: 'data: {"choices":[{"delta":{"content":"partial"}}]}\n\nerror: {"error":{"code":500,"message":"Unexpected reasoning effort high"}}\n\n' }]) fake = fakeProvider([{ chunks: [], raw: 'data: {"choices":[{"delta":{"content":"partial"}}]}\n\nerror: {"error":{"code":500,"message":"Unexpected reasoning effort high"}}\n\n' }])
const m: ResolvedModel = { ref: "swap/late", connectionName: "swap", id: "x", spec: { efforts: ["high"] }, connection: { dialect: "openai-chat", base_url: fake.url, models: {} } } const m: ResolvedModel = { ref: "swap/late", connectionName: "swap", id: "x", spec: { efforts: ["high"] }, connection: { base_url: fake.url, models: {} } }
const run = async () => { const run = async () => {
for await (const _ of new OpenAIChatClient(m).stream({ system: "", messages: [], tools: [], effort: "high" })) void _ for await (const _ of new OpenAIChatClient(m).stream({ system: "", messages: [], tools: [], effort: "high" })) void _
} }
@@ -123,7 +123,7 @@ describe("effort refused inside a 200 stream (llama-swap loading, then the templ
describe("llama-swap unloads the model under a running request", () => { describe("llama-swap unloads the model under a running request", () => {
test("'group: model unloaded' (a string code) is a server error: retried once, and the reply arrives", async () => { test("'group: model unloaded' (a string code) is a server error: retried once, and the reply arrives", async () => {
fake = fakeProvider([{ chunks: [], raw: fixture("llama-swap-model-unloaded.sse") }, { chunks: [delta({ content: "second try" })] }]) fake = fakeProvider([{ chunks: [], raw: fixture("llama-swap-model-unloaded.sse") }, { chunks: [delta({ content: "second try" })] }])
const m: ResolvedModel = { ref: "swap/x", connectionName: "swap", id: "x", spec: {}, connection: { dialect: "openai-chat", base_url: fake.url, models: {} } } const m: ResolvedModel = { ref: "swap/x", connectionName: "swap", id: "x", spec: {}, connection: { base_url: fake.url, models: {} } }
const events: StreamEvent[] = [] const events: StreamEvent[] = []
for await (const e of new OpenAIChatClient(m).stream({ system: "", messages: [{ role: "user", parts: [{ type: "text", text: "x" }] }], tools: [] })) events.push(e) for await (const e of new OpenAIChatClient(m).stream({ system: "", messages: [{ role: "user", parts: [{ type: "text", text: "x" }] }], tools: [] })) events.push(e)
expect(events.some((e) => e.type === "notice" && e.message.includes("group: model unloaded"))).toBe(true) expect(events.some((e) => e.type === "notice" && e.message.includes("group: model unloaded"))).toBe(true)
-83
View File
@@ -1,83 +0,0 @@
import { afterEach, describe, expect, test } from "bun:test"
import { readFileSync } from "node:fs"
import { join } from "node:path"
import { ResponsesClient, toResponsesInput } from "../src/provider/responses.ts"
import type { Message, ResolvedModel, StreamEvent } from "../src/provider/types.ts"
import { fakeProvider, type Fake } from "./fake-provider.ts"
let fake: Fake | undefined
afterEach(() => fake?.stop())
const model = (url: string, spec: ResolvedModel["spec"] = {}): ResolvedModel => ({ ref: "r/m", connectionName: "r", id: "m", spec, connection: { dialect: "responses", base_url: url, models: {} } })
const sse = (events: Record<string, unknown>[]) => events.map((e) => `data: ${JSON.stringify(e)}\n\n`).join("")
async function run(c: ResponsesClient, messages: Message[] = [{ role: "user", parts: [{ type: "text", text: "hi" }] }], effort: any = null) {
const ev: StreamEvent[] = []
for await (const e of c.stream({ system: "sys", messages, tools: [{ name: "list", description: "d", parameters: { type: "object" } }], effort })) ev.push(e)
return ev
}
describe("responses dialect", () => {
test("a real vLLM stream: reasoning text, then a function call, with usage", async () => {
fake = fakeProvider([{ chunks: [], raw: readFileSync(join(import.meta.dir, "fixtures/sse/vllm-responses-reasoning-tool.sse"), "utf8") }])
const ev = await run(new ResponsesClient(model(fake.url)))
const fin = ev.find((e) => e.type === "finish")!
if (fin.type !== "finish") throw new Error()
expect(fin.reason).toBe("tool_calls")
expect(fin.message.parts[0]).toMatchObject({ type: "reasoning" })
expect(fin.message.parts[1]).toMatchObject({ type: "tool_call", name: "list" })
expect(ev.find((e) => e.type === "usage")).toMatchObject({ usage: { input: 4788, output: 111, reasoning: 21 } })
})
test("request shape; an encrypted reasoning item goes back verbatim on the next turn", async () => {
const reasoningItem = { id: "rs_1", type: "reasoning", summary: [{ type: "summary_text", text: "plan" }], encrypted_content: "ENC" }
fake = fakeProvider([
{
chunks: [],
raw: sse([
{ type: "response.reasoning_summary_text.delta", item_id: "rs_1", delta: "plan" },
{ type: "response.output_item.done", item: reasoningItem },
{ type: "response.output_item.added", item: { type: "function_call", id: "fc_1", call_id: "call_1", name: "list" } },
{ type: "response.function_call_arguments.delta", item_id: "fc_1", delta: "{}" },
{ type: "response.output_item.done", item: { type: "function_call", id: "fc_1", call_id: "call_1", name: "list", arguments: "{}" } },
{ type: "response.completed", response: { status: "completed", usage: { input_tokens: 10, output_tokens: 5, input_tokens_details: { cached_tokens: 4 } } } },
]),
},
{ chunks: [], raw: sse([{ type: "response.output_text.delta", delta: "done" }, { type: "response.completed", response: { status: "completed" } }]) },
])
const c = new ResponsesClient(model(fake.url, { max_output: 999 }))
const first = await run(c, undefined, "high")
const fin = first.find((e) => e.type === "finish")!
if (fin.type !== "finish") throw new Error()
expect(first.find((e) => e.type === "usage")).toEqual({ type: "usage", usage: { input: 10, output: 5, reasoning: undefined, cached: 4 } })
expect(fake.requests[0]).toMatchObject({ model: "m", store: false, instructions: "sys", max_output_tokens: 999, reasoning: { effort: "high", summary: "auto" }, include: ["reasoning.encrypted_content"] })
expect(fake.requests[0].tools[0]).toMatchObject({ type: "function", name: "list" })
await run(c, [{ role: "user", parts: [{ type: "text", text: "hi" }] }, fin.message, { role: "tool", callId: "call_1", name: "list", content: "a.txt" }], "high")
expect(fake.requests[1].input).toEqual([
{ role: "user", content: [{ type: "input_text", text: "hi" }] },
reasoningItem,
{ type: "function_call", call_id: "call_1", name: "list", arguments: "{}" },
{ type: "function_call_output", call_id: "call_1", output: "a.txt" },
])
})
test("a server that does not know `include` gets one retry without it", async () => {
fake = fakeProvider([{ status: 400, body: '{"error":{"message":"Unknown parameter: include"}}' }, { chunks: [], raw: sse([{ type: "response.output_text.delta", delta: "ok" }]) }])
await run(new ResponsesClient(model(fake.url)), undefined, "low")
expect(fake.requests[1].include).toBeUndefined()
expect(fake.requests[1].reasoning).toEqual({ effort: "low", summary: "auto" })
})
test("history from another dialect: reasoning without an item is not sent; images only with vision", () => {
const input = toResponsesInput(
[
{ role: "user", parts: [{ type: "text", text: "see" }, { type: "image", mime: "image/png", data: "AAA" }] },
{ role: "assistant", parts: [{ type: "reasoning", text: "thought elsewhere" }, { type: "text", text: "ok" }] },
],
true,
)
expect(input).toEqual([
{ role: "user", content: [{ type: "input_text", text: "see" }, { type: "input_image", image_url: "data:image/png;base64,AAA" }] },
{ role: "assistant", content: [{ type: "output_text", text: "ok" }] },
])
})
})
+1 -1
View File
@@ -19,7 +19,7 @@ const yes = { ask: async (): Promise<AskReply> => ({ kind: "once" }) }
function setup(script: Parameters<typeof fakeProvider>[0]) { function setup(script: Parameters<typeof fakeProvider>[0]) {
fake = fakeProvider(script) fake = fakeProvider(script)
mkdirSync(paths.config, { recursive: true }) mkdirSync(paths.config, { recursive: true })
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n fake:\n dialect: openai-chat\n base_url: ${fake.url}\n models:\n m: { context: 32768 }\n`, { mode: 0o600 }) writeFileSync(join(paths.config, "connections.yaml"), `connections:\n fake:\n type: webui\n url: ${fake.url}\n models:\n fake/m: { context: 32768 }\n`, { mode: 0o600 })
writeFileSync(join(paths.config, "config.yaml"), "model: fake/m\n") writeFileSync(join(paths.config, "config.yaml"), "model: fake/m\n")
const dir = mkdtempSync(join(tmpdir(), "lembas-results-")) const dir = mkdtempSync(join(tmpdir(), "lembas-results-"))
writeFileSync(join(dir, "a.txt"), "one\ntwo\nthree\n") writeFileSync(join(dir, "a.txt"), "one\ntwo\nthree\n")
+1 -1
View File
@@ -41,7 +41,7 @@ function pair(): [Transport, Transport] {
function config(url: string) { function config(url: string) {
mkdirSync(paths.config, { recursive: true }) mkdirSync(paths.config, { recursive: true })
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n dialect: openai-chat\n base_url: ${url}\n models: { m: {} }\n`, { mode: 0o600 }) writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n type: webui\n url: ${url}\n models: { f/m: {} }\n`, { mode: 0o600 })
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\n") writeFileSync(join(paths.config, "config.yaml"), "model: f/m\n")
} }
+2 -3
View File
@@ -21,7 +21,7 @@ function home(config = "model: f/m\n", script: Parameters<typeof fakeProvider>[0
mkdirSync(paths.config, { recursive: true }) mkdirSync(paths.config, { recursive: true })
writeFileSync( writeFileSync(
join(paths.config, "connections.yaml"), join(paths.config, "connections.yaml"),
`connections:\n f:\n dialect: openai-chat\n base_url: ${fake.url}\n models:\n m: { efforts: [low, medium, high], effort: medium }\n big: {}\n`, `connections:\n f:\n type: webui\n url: ${fake.url}\n models:\n f/m: { efforts: [low, medium, high], effort: medium }\n f/big: {}\n`,
{ mode: 0o600 }, { mode: 0o600 },
) )
writeFileSync(join(paths.config, "config.yaml"), config) writeFileSync(join(paths.config, "config.yaml"), config)
@@ -68,7 +68,6 @@ test("session, global and project scope: what is written where, and where a valu
expect(readFileSync(join(cwd, ".agent/config.yaml"), "utf8")).toContain("auto_at: 0.6") expect(readFileSync(join(cwd, ".agent/config.yaml"), "utf8")).toContain("auto_at: 0.6")
expect(s.get("compaction.auto_at")).toMatchObject({ value: 0.6, source: "project" }) expect(s.get("compaction.auto_at")).toMatchObject({ value: 0.6, source: "project" })
expect(() => s.set("model", "f/nope")).toThrow()
expect(() => s.set("effort", "max")).toThrow("takes") expect(() => s.set("effort", "max")).toThrow("takes")
expect(() => s.set("update.channel", "beta", "project")).toThrow("can be set for: global") expect(() => s.set("update.channel", "beta", "project")).toThrow("can be set for: global")
}) })
@@ -92,7 +91,7 @@ test("the agent changes a setting through the tool, with approval; a model chang
expect(asked[0]!.always).toEqual(["session effort=*"]) expect(asked[0]!.always).toEqual(["session effort=*"])
expect(app.engine.model.ref).toBe("f/big") expect(app.engine.model.ref).toBe("f/big")
// The third request went to the new model. // The third request went to the new model.
expect(fake!.requests[2]!.model).toBe("big") expect(fake!.requests[2]!.model).toBe("f/big")
}) })
test("settings_tool: allow — no asking; but a less strict mode is asked every time, with no 'always'", async () => { test("settings_tool: allow — no asking; but a less strict mode is asked every time, with no 'always'", async () => {
+1 -1
View File
@@ -22,7 +22,7 @@ function setup(script: Parameters<typeof fakeProvider>[0], config = "") {
mkdirSync(paths.config, { recursive: true }) mkdirSync(paths.config, { recursive: true })
writeFileSync( writeFileSync(
join(paths.config, "connections.yaml"), join(paths.config, "connections.yaml"),
`connections:\n f:\n dialect: openai-chat\n base_url: ${fake.url}\n models:\n main: { context: 32768 }\n a: {}\n`, `connections:\n f:\n type: webui\n url: ${fake.url}\n models:\n f/main: { context: 32768 }\n f/a: {}\n`,
{ mode: 0o600 }, { mode: 0o600 },
) )
writeFileSync(join(paths.config, "config.yaml"), `model: f/main\ntitles: prompt\n${config}`) writeFileSync(join(paths.config, "config.yaml"), `model: f/main\ntitles: prompt\n${config}`)
+11 -12
View File
@@ -13,17 +13,15 @@ let fake: Fake | undefined
let other: Fake | undefined let other: Fake | undefined
afterEach(() => (fake?.stop(), other?.stop(), (other = undefined))) afterEach(() => (fake?.stop(), other?.stop(), (other = undefined)))
/** Two scripted servers: the parent's model on `f`, the child's on `g` (task's `model: "g/m"`) — /** Two scripts: the parent's model `f/m`, the child's `g/…` (task's `model: "g/m"`), routed by the
* a shared script would hand a reply to whichever engine asked first. */ * fake server — a shared script would hand a reply to whichever engine asked first. */
function pair(parent: Parameters<typeof fakeProvider>[0], child: Parameters<typeof fakeProvider>[0], o: { oneAtATime?: boolean; config?: string; asker?: Asker } = {}) { function pair(parent: Parameters<typeof fakeProvider>[0], child: Parameters<typeof fakeProvider>[0], o: { oneAtATime?: boolean; config?: string; asker?: Asker } = {}) {
fake = fakeProvider(parent) const child$ = (other = fakeProvider(child))
other = fakeProvider(child) fake = fakeProvider(parent, { route: (model) => (model.startsWith("g/") ? child$ : undefined) })
mkdirSync(paths.config, { recursive: true }) mkdirSync(paths.config, { recursive: true })
writeFileSync( // One server; its g models answered by the child's script, sharing one GPU when asked to.
join(paths.config, "connections.yaml"), const g = o.oneAtATime ? "{ group: gpu }" : "{}"
`connections:\n f:\n dialect: openai-chat\n base_url: ${fake.url}\n models: { m: {} }\n g:\n dialect: openai-chat\n base_url: ${other.url}\n${o.oneAtATime ? " one_model_at_a_time: true\n" : ""} models: { m: {}, n: {} }\n`, writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n type: webui\n url: ${fake.url}\n models: { f/m: {}, g/m: ${g}, g/n: ${g} }\n`, { mode: 0o600 })
{ mode: 0o600 },
)
writeFileSync(join(paths.config, "config.yaml"), `model: f/m\n${o.config ?? ""}`) writeFileSync(join(paths.config, "config.yaml"), `model: f/m\n${o.config ?? ""}`)
const cwd = mkdtempSync(join(tmpdir(), "ph-sub-")) const cwd = mkdtempSync(join(tmpdir(), "ph-sub-"))
writeFileSync(join(cwd, "calc.py"), "def average(v):\n return sum(v) / (len(v) - 1)\n") writeFileSync(join(cwd, "calc.py"), "def average(v):\n return sum(v) / (len(v) - 1)\n")
@@ -40,7 +38,7 @@ const settled = async (a: ReturnType<typeof createApp>, n: number) => {
function app(script: Parameters<typeof fakeProvider>[0]) { function app(script: Parameters<typeof fakeProvider>[0]) {
fake = fakeProvider(script) fake = fakeProvider(script)
mkdirSync(paths.config, { recursive: true }) mkdirSync(paths.config, { recursive: true })
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n dialect: openai-chat\n base_url: ${fake.url}\n models: { m: {} }\n`, { mode: 0o600 }) writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n type: webui\n url: ${fake.url}\n models: { f/m: {} }\n`, { mode: 0o600 })
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\n") writeFileSync(join(paths.config, "config.yaml"), "model: f/m\n")
const cwd = mkdtempSync(join(tmpdir(), "ph-sub-")) const cwd = mkdtempSync(join(tmpdir(), "ph-sub-"))
writeFileSync(join(cwd, "calc.py"), "def average(v):\n return sum(v) / (len(v) - 1)\n") writeFileSync(join(cwd, "calc.py"), "def average(v):\n return sum(v) / (len(v) - 1)\n")
@@ -176,7 +174,7 @@ test("a worktree subagent cannot touch the main checkout, even unrestricted", as
{ chunks: [delta({ content: "tried" })] }, { chunks: [delta({ content: "tried" })] },
{ chunks: [delta({ content: "ok" })] }, { chunks: [delta({ content: "ok" })] },
]) ])
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n dialect: openai-chat\n base_url: ${fake.url}\n models: { m: {} }\n`, { mode: 0o600 }) writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n type: webui\n url: ${fake.url}\n models: { f/m: {} }\n`, { mode: 0o600 })
for (const args of [["init", "-q", "-b", "main"], ["config", "user.email", "t@e"], ["config", "user.name", "T"], ["config", "commit.gpgsign", "false"], ["add", "calc.py"], ["commit", "-q", "-m", "init"]]) git(root, args) for (const args of [["init", "-q", "-b", "main"], ["config", "user.email", "t@e"], ["config", "user.name", "T"], ["config", "commit.gpgsign", "false"], ["add", "calc.py"], ["commit", "-q", "-m", "init"]]) git(root, args)
const b = createApp({ cwd: root, mode: "auto", store: false, asker: { ask: async (): Promise<AskReply> => ({ kind: "once" }) } }) const b = createApp({ cwd: root, mode: "auto", store: false, asker: { ask: async (): Promise<AskReply> => ({ kind: "once" }) } })
await b.engine.prompt("go") await b.engine.prompt("go")
@@ -300,8 +298,9 @@ describe("background subagents (spec 2.0.0)", () => {
const results = fake!.requests[1].messages.filter((m: any) => m.role === "tool").map((m: any) => m.content as string) const results = fake!.requests[1].messages.filter((m: any) => m.role === "tool").map((m: any) => m.content as string)
expect(results[0]).toStartWith("[#0001]\nStarted subagent explore as job1") expect(results[0]).toStartWith("[#0001]\nStarted subagent explore as job1")
expect(results[1]).toContain("holds one model at a time, and a subagent is already running on g/m there") expect(results[1]).toContain("holds one model at a time, and a subagent is already running on g/m there")
expect(other!.requests.map((r: any) => r.model)).toEqual(["m"])
await settled(a, 3) await settled(a, 3)
for (let i = 0; i < 100 && !other!.requests.length; i++) await Bun.sleep(20)
expect(other!.requests.map((r: any) => r.model)).toEqual(["g/m"])
}) })
test("a broken agent file is said at start, and the agent is still offered with its body", () => { test("a broken agent file is said at start, and the agent is still offered with its body", () => {
+3 -3
View File
@@ -18,7 +18,7 @@ function setup(script: Parameters<typeof fakeProvider>[0], config = "") {
mkdirSync(paths.config, { recursive: true }) mkdirSync(paths.config, { recursive: true })
writeFileSync( writeFileSync(
join(paths.config, "connections.yaml"), join(paths.config, "connections.yaml"),
`connections:\n fake:\n dialect: openai-chat\n base_url: ${fake.url}\n models:\n m: { context: 32768 }\n tiny: { context: 8192 }\n`, `connections:\n fake:\n type: webui\n url: ${fake.url}\n models:\n fake/m: { context: 32768 }\n fake/tiny: { context: 8192 }\n`,
{ mode: 0o600 }, { mode: 0o600 },
) )
writeFileSync(join(paths.config, "config.yaml"), `model: fake/m\n${config}`) writeFileSync(join(paths.config, "config.yaml"), `model: fake/m\n${config}`)
@@ -111,8 +111,8 @@ describe("a session gets a title", () => {
const app = createApp({ cwd: dir, asker: noAsk, store, snapshots: false }) const app = createApp({ cwd: dir, asker: noAsk, store, snapshots: false })
await app.engine.prompt("something") await app.engine.prompt("something")
await until(() => store.session(app.engine.sessionId!)!.title === "Tiny title") await until(() => store.session(app.engine.sessionId!)!.title === "Tiny title")
expect(fake!.requests[0].model).toBe("m") expect(fake!.requests[0].model).toBe("fake/m")
expect(fake!.requests[1].model).toBe("tiny") expect(fake!.requests[1].model).toBe("fake/tiny")
}) })
test("a failed title request leaves the prompt's title", async () => { test("a failed title request leaves the prompt's title", async () => {
+1 -1
View File
@@ -31,7 +31,7 @@ const model = (tls?: { ca?: string; insecure?: boolean }): ResolvedModel => ({
connectionName: "t", connectionName: "t",
id: "m", id: "m",
spec: {}, spec: {},
connection: { dialect: "openai-chat", base_url: `https://127.0.0.1:${server.port}/v1`, tls, models: {} }, connection: { base_url: `https://127.0.0.1:${server.port}/v1`, tls, models: {} },
}) })
test("a private CA is refused by default, with advice", async () => { test("a private CA is refused by default, with advice", async () => {
+1 -1
View File
@@ -39,7 +39,7 @@ test("The CLI's own files under .agent that are links are refused, not written t
test("an untrusted project's task board is neither in the prompt nor writable", async () => { test("an untrusted project's task board is neither in the prompt nor writable", async () => {
mkdirSync(paths.config, { recursive: true }) mkdirSync(paths.config, { recursive: true })
writeFileSync(join(paths.config, "connections.yaml"), "connections:\n f:\n dialect: openai-chat\n base_url: http://127.0.0.1:9/v1\n models: { m: {} }\n", { mode: 0o600 }) writeFileSync(join(paths.config, "connections.yaml"), "connections:\n f:\n type: webui\n url: http://127.0.0.1:9/v1\n models: { f/m: {} }\n", { mode: 0o600 })
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\n") writeFileSync(join(paths.config, "config.yaml"), "model: f/m\n")
const root = mkdtempSync(join(tmpdir(), "ph-untrusted-")) const root = mkdtempSync(join(tmpdir(), "ph-untrusted-"))
mkdirSync(join(root, ".agent")) mkdirSync(join(root, ".agent"))
+1 -1
View File
@@ -14,7 +14,7 @@ afterEach(() => setup?.renderer.destroy())
async function boot(cwd: string) { async function boot(cwd: string) {
const fake = fakeProvider([]) const fake = fakeProvider([])
mkdirSync(paths.config, { recursive: true }) mkdirSync(paths.config, { recursive: true })
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n fake:\n dialect: openai-chat\n base_url: ${fake.url}\n models: { m: {} }\n`, { mode: 0o600 }) writeFileSync(join(paths.config, "connections.yaml"), `connections:\n fake:\n type: webui\n url: ${fake.url}\n models: { fake/m: {} }\n`, { mode: 0o600 })
writeFileSync(join(paths.config, "config.yaml"), "model: fake/m\n") writeFileSync(join(paths.config, "config.yaml"), "model: fake/m\n")
let quit = 0 let quit = 0
setup = await testRender(() => <Boot options={{ cwd, store: false }} onQuit={() => quit++} />, { width: 100, height: 30 }) setup = await testRender(() => <Boot options={{ cwd, store: false }} onQuit={() => quit++} />, { width: 100, height: 30 })
+1 -1
View File
@@ -16,7 +16,7 @@ export async function tui(script: Scripted[], size = { width: 100, height: 30 },
mkdirSync(paths.config, { recursive: true }) mkdirSync(paths.config, { recursive: true })
writeFileSync( writeFileSync(
join(paths.config, "connections.yaml"), join(paths.config, "connections.yaml"),
`connections:\n fake:\n dialect: openai-chat\n base_url: ${fake.url}\n models:\n coder: { context: 32768, efforts: [low, high], effort: low }\n other: { context: 8192 }\n`, `connections:\n fake:\n type: webui\n url: ${fake.url}\n models:\n fake/coder: { context: 32768, efforts: [low, high], effort: low }\n fake/other: { context: 8192 }\n`,
{ mode: 0o600 }, { mode: 0o600 },
) )
writeFileSync(join(paths.config, "config.yaml"), `model: fake/coder\n${extraConfig}`) writeFileSync(join(paths.config, "config.yaml"), `model: fake/coder\n${extraConfig}`)
+4 -4
View File
@@ -31,16 +31,16 @@ test("no model yet: the sign-in is first and focused; ctrl+r opens the session o
return setup!.captureCharFrame() return setup!.captureCharFrame()
} }
let f = await frame() let f = await frame()
expect(f).toContain("no model to talk to yet") // 2.0: the server's models or none — the screen is the sign-in, and nothing else to fill in.
expect(f).toContain("not logged in to a LLeMbas server")
expect(f).toContain("Log in to a LLeMbas instance") expect(f).toContain("Log in to a LLeMbas instance")
// Before the file instructions: the screen opens on the sign-in. expect(f).not.toContain("connections.yaml")
expect(f.indexOf("Log in to a LLeMbas instance")).toBeLessThan(f.indexOf("connections.yaml"))
// Focused: what is typed is the address, without pressing l first. // Focused: what is typed is the address, without pressing l first.
await setup.mockInput.typeText("ai.example") await setup.mockInput.typeText("ai.example")
f = await frame() f = await frame()
expect(f).toContain("Address: ai.example") expect(f).toContain("Address: ai.example")
// A model added meanwhile: ctrl+r, a letter-free key, opens the session. // A model added meanwhile: ctrl+r, a letter-free key, opens the session.
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n fake:\n dialect: openai-chat\n base_url: ${fake.url}\n models: { m: {} }\n`, { mode: 0o600 }) writeFileSync(join(paths.config, "connections.yaml"), `connections:\n fake:\n type: webui\n url: ${fake.url}\n models: { fake/m: {} }\n`, { mode: 0o600 })
writeFileSync(join(paths.config, "config.yaml"), "model: fake/m\n") writeFileSync(join(paths.config, "config.yaml"), "model: fake/m\n")
setup.mockInput.pressKey("r", { ctrl: true }) setup.mockInput.pressKey("r", { ctrl: true })
f = await frame() f = await frame()