Custom connections are gone: the CLI talks to the LLeMbas server it is logged in to
LLeMbas CLI 2.0 is LLeMbas-server-specific (CLI rule 2). Removed: the anthropic, gemini, ollama and responses clients, context discovery and llama-swap unloading (discover.ts), the Connection schema with its dialect, auth, key_cmd, discover, unload_url and one_model_at_a_time, the model `fallback` (it only ever moved to another server), effort_map and cache, and the multi-connection refs. Kept: openai-chat, http, sse, think, effort, tokens and all of src/lembas. - connections.yaml holds the login (`type: webui`, as lembas login writes it). A custom entry from 1.x is read past with a warning that says how to log in, so an old file still starts; with several logins left, the latest is used and the others named. - A model's ref is the id the server serves it under; the older forms (`<login>/<id>`, `<login>/<model>`) still resolve and config.yaml is rewritten once. Without a login there is no model, and the error (and the first-run screen, now the sign-in alone) says `lembas login <address>`. - A login to another server signs this machine out of the previous one. - The window is the server's word (no discovery); a capacity group comes from the server's `capacity.group` alone; embeddings use the server's /v1/embeddings. - `lembas models` lists the server's models; install.sh's starter files, the README and the JSON schemas say the same. Tests: every fixture is a login now (the fake server lists models, answers other GETs with 404, and can hand one model to another script, as a server with several backends); the dialect suites and the cross-connection tests are gone with their code. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
80 files changed
+587
-2575
No files matched your search
@@ -4,12 +4,11 @@
|
|||||||
|
|
||||||
# LLeMbas CLI
|
# LLeMbas CLI
|
||||||
|
|
||||||
A terminal coding agent and project manager: the `lembas` command. It talks to any LLM API
|
A terminal coding agent and project manager: the `lembas` command. It signs in to a
|
||||||
(OpenAI-compatible servers such as llama.cpp, vLLM or LM Studio, the OpenAI Responses API,
|
[LLeMbas](https://github.com/LLeMbas/LLeMbas) server and uses its models, memory and search, works
|
||||||
Anthropic, Gemini, Ollama), works in your project with four permission modes and a git snapshot of
|
in your project with four permission modes and a git snapshot of every turn, keeps its own notes,
|
||||||
every turn, remembers what it learns across sessions, and can sign in to a
|
knowledge bases and skills on your machine, and lets the server's web UI work the same sessions.
|
||||||
[LLeMbas](https://github.com/LLeMbas/LLeMbas) server to use its models, library and search — and
|
Linux, x64 and arm64.
|
||||||
let its web UI work the same sessions. Linux, x64 and arm64.
|
|
||||||
|
|
||||||
## Install
|
## Install
|
||||||
|
|
||||||
@@ -24,12 +23,12 @@ checksum before installing anything ([SECURITY.md](SECURITY.md) shows how to che
|
|||||||
## First steps
|
## First steps
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
cd your-project && lembas # the TUI
|
lembas login https://your-server # sign in to your LLeMbas server
|
||||||
lembas login https://your-server # sign in to a LLeMbas server and use its models
|
cd your-project && lembas # the TUI, on the server's models
|
||||||
```
|
```
|
||||||
|
|
||||||
Without a server, add a connection to `~/.config/lembas/connections.yaml` and set `model:` in
|
From 2.0 the CLI talks only to the LLeMbas server it is logged in to; the custom connections of 1.x
|
||||||
`~/.config/lembas/config.yaml`.
|
(`dialect:` entries in `connections.yaml`) are gone.
|
||||||
|
|
||||||
`lembas` updates itself (`lembas update` does it now); `lembas uninstall` removes it, and
|
`lembas` updates itself (`lembas update` does it now); `lembas uninstall` removes it, and
|
||||||
`--purge` takes settings and sessions too.
|
`--purge` takes settings and sessions too.
|
||||||
|
|||||||
+7
-19
@@ -221,8 +221,9 @@ if [ "$starter" -eq 1 ] && [ ! -e "$CONFIG/config.yaml" ]; then
|
|||||||
# The full list: https://llembas.eu/docs/. An editor with a YAML language server
|
# The full list: https://llembas.eu/docs/. An editor with a YAML language server
|
||||||
# completes and checks keys as you type (the first line points it at the schema).
|
# completes and checks keys as you type (the first line points it at the schema).
|
||||||
|
|
||||||
# The model to start with, as connection/model from connections.yaml.
|
# The model to start with, by its id on the LLeMbas server you log in to (lembas models lists
|
||||||
# model: local/qwen3
|
# them). Unset: the server's default for your account.
|
||||||
|
# model: deepseek/deepseek-flash
|
||||||
|
|
||||||
# manual (asks before changes) | edit (changes files in the project freely) | plan
|
# manual (asks before changes) | edit (changes files in the project freely) | plan
|
||||||
mode: manual
|
mode: manual
|
||||||
@@ -237,27 +238,14 @@ fi
|
|||||||
if [ "$starter" -eq 1 ] && [ ! -e "$CONFIG/connections.yaml" ]; then
|
if [ "$starter" -eq 1 ] && [ ! -e "$CONFIG/connections.yaml" ]; then
|
||||||
(umask 077 && cat >"$CONFIG/connections.yaml" <<'YAML'
|
(umask 077 && cat >"$CONFIG/connections.yaml" <<'YAML'
|
||||||
# yaml-language-server: $schema=schema/connections.schema.json
|
# yaml-language-server: $schema=schema/connections.schema.json
|
||||||
# Where the models are. Global only: a project can never point LLeMbas CLI elsewhere.
|
# The LLeMbas server this machine is logged in to. Global only: a project can never point
|
||||||
# Keys are never written here in plain text: {env:VAR}, {file:~/path}, or key_cmd.
|
# LLeMbas CLI elsewhere. Written by `lembas login <address>`, not by hand; the models come from
|
||||||
# Then set `model: <connection>/<model>` in config.yaml, and check with `lembas models`.
|
# the server at every start (`lembas models` lists them).
|
||||||
connections: {}
|
connections: {}
|
||||||
#
|
|
||||||
# connections:
|
|
||||||
# local: # llama.cpp / llama-swap / vLLM / LM Studio — OpenAI-compatible
|
|
||||||
# dialect: openai-chat
|
|
||||||
# base_url: http://127.0.0.1:8080/v1
|
|
||||||
# models:
|
|
||||||
# qwen3: { context: 131072, max_output: 65536 }
|
|
||||||
# anthropic:
|
|
||||||
# dialect: anthropic
|
|
||||||
# base_url: https://api.anthropic.com
|
|
||||||
# api_key: "{env:ANTHROPIC_API_KEY}"
|
|
||||||
# models:
|
|
||||||
# claude-sonnet-5: { context: 200000 }
|
|
||||||
YAML
|
YAML
|
||||||
)
|
)
|
||||||
chmod 600 "$CONFIG/connections.yaml"
|
chmod 600 "$CONFIG/connections.yaml"
|
||||||
say " wrote $CONFIG/connections.yaml (0600) — add a connection there"
|
say " wrote $CONFIG/connections.yaml (0600) — lembas login <address> fills it in"
|
||||||
fi
|
fi
|
||||||
# The schemas for editors follow the binary, so they are rewritten on every install.
|
# The schemas for editors follow the binary, so they are rewritten on every install.
|
||||||
"$TARGET" config schema >/dev/null && say " wrote $CONFIG/schema/ (JSON Schemas for editors)"
|
"$TARGET" config schema >/dev/null && say " wrote $CONFIG/schema/ (JSON Schemas for editors)"
|
||||||
|
|||||||
+1
-1
@@ -1,7 +1,7 @@
|
|||||||
{
|
{
|
||||||
"name": "lembas-cli",
|
"name": "lembas-cli",
|
||||||
"version": "1.5.2",
|
"version": "1.5.2",
|
||||||
"description": "LLeMbas CLI: a terminal coding agent and project manager — OpenTUI, any LLM API, memory, skills, MCP, git-native.",
|
"description": "LLeMbas CLI: a terminal coding agent and project manager for a LLeMbas server — OpenTUI, memory, skills, MCP, git-native.",
|
||||||
"license": "MIT",
|
"license": "MIT",
|
||||||
"private": true,
|
"private": true,
|
||||||
"type": "module",
|
"type": "module",
|
||||||
|
|||||||
@@ -3,11 +3,11 @@
|
|||||||
"type": "object",
|
"type": "object",
|
||||||
"properties": {
|
"properties": {
|
||||||
"model": {
|
"model": {
|
||||||
"description": "The model a session starts with: `connection/model`, a connection from connections.yaml and one of its models.",
|
"description": "The model a session starts with: one the LLeMbas server you are logged in to serves, by its id there (`deepseek/deepseek-flash`). Default: the server's default for your account.",
|
||||||
"type": "string"
|
"type": "string"
|
||||||
},
|
},
|
||||||
"small_model": {
|
"small_model": {
|
||||||
"description": "`connection/model` for small jobs — session titles, commit messages. Default: the session's model. On a server that holds one model at a time (llama-swap), another model here swaps.",
|
"description": "The model for small jobs — session titles, commit messages, by its id on the server. Default: the session's model.",
|
||||||
"type": "string"
|
"type": "string"
|
||||||
},
|
},
|
||||||
"titles": {
|
"titles": {
|
||||||
@@ -866,7 +866,7 @@
|
|||||||
}
|
}
|
||||||
},
|
},
|
||||||
"embedding": {
|
"embedding": {
|
||||||
"description": "Global config only. connection/model of an embedding model (OpenAI-shaped /embeddings, or Ollama): knowledge bases are then searched by meaning as well as by words. lembas kb reindex after changing it.",
|
"description": "Global config only. An embedding model the LLeMbas server serves, by its id there: knowledge bases are then searched by meaning as well as by words. lembas kb reindex after changing it.",
|
||||||
"type": "string"
|
"type": "string"
|
||||||
},
|
},
|
||||||
"effort": {
|
"effort": {
|
||||||
|
|||||||
+188
-527
@@ -4,564 +4,225 @@
|
|||||||
"properties": {
|
"properties": {
|
||||||
"connections": {
|
"connections": {
|
||||||
"default": {},
|
"default": {},
|
||||||
"description": "Each LLM endpoint by a name of your choosing; a model is then `name/model-id`. `type: webui` is a LLeMbas instance, whose models come from it.",
|
"description": "The LLeMbas server this machine is logged in to, as `lembas login` writes it: its address and this machine's token. Its models come from it at every start.",
|
||||||
"type": "object",
|
"type": "object",
|
||||||
"propertyNames": {
|
"propertyNames": {
|
||||||
"type": "string"
|
"type": "string"
|
||||||
},
|
},
|
||||||
"additionalProperties": {
|
"additionalProperties": {
|
||||||
"anyOf": [
|
"type": "object",
|
||||||
{
|
"properties": {
|
||||||
|
"type": {
|
||||||
|
"type": "string",
|
||||||
|
"const": "webui",
|
||||||
|
"description": "A LLeMbas instance: its models, their settings, its voice and its web search."
|
||||||
|
},
|
||||||
|
"url": {
|
||||||
|
"type": "string",
|
||||||
|
"description": "The instance's address, e.g. https://ai.example.org (no /v1)."
|
||||||
|
},
|
||||||
|
"api_key": {
|
||||||
|
"description": "This machine's token, as login wrote it: {file:~/.config/lembas/lembas/<name>.key}.",
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"tls": {
|
||||||
|
"description": "For an instance behind a private CA: `ca`, the PEM file that signed its certificate.",
|
||||||
"type": "object",
|
"type": "object",
|
||||||
"properties": {
|
"properties": {
|
||||||
"type": {
|
"ca": {
|
||||||
"type": "string",
|
|
||||||
"const": "webui",
|
|
||||||
"description": "A LLeMbas instance: its models, their settings, its voice and its web search."
|
|
||||||
},
|
|
||||||
"url": {
|
|
||||||
"type": "string",
|
|
||||||
"description": "The instance's address, e.g. https://ai.example.org (no /v1)."
|
|
||||||
},
|
|
||||||
"api_key": {
|
|
||||||
"description": "This machine's token, as login wrote it: {file:~/.config/lembas/lembas/<name>.key}. Or key_cmd.",
|
|
||||||
"type": "string"
|
"type": "string"
|
||||||
},
|
},
|
||||||
"key_cmd": {
|
"insecure": {
|
||||||
"type": "string"
|
"type": "boolean"
|
||||||
},
|
|
||||||
"tls": {
|
|
||||||
"description": "For an instance behind a private CA: `ca`, the PEM file that signed its certificate.",
|
|
||||||
"type": "object",
|
|
||||||
"properties": {
|
|
||||||
"ca": {
|
|
||||||
"type": "string"
|
|
||||||
},
|
|
||||||
"insecure": {
|
|
||||||
"type": "boolean"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"additionalProperties": false
|
|
||||||
},
|
|
||||||
"timeout": {
|
|
||||||
"description": "Seconds of silence allowed in a reply, as for any connection (default 600).",
|
|
||||||
"type": "number",
|
|
||||||
"exclusiveMinimum": 0
|
|
||||||
},
|
|
||||||
"quirks": {
|
|
||||||
"description": "Server oddities; auto is right almost always.",
|
|
||||||
"type": "object",
|
|
||||||
"properties": {
|
|
||||||
"stream_usage": {
|
|
||||||
"description": "Ask for token usage in the stream (stream_options); auto learns when a server refuses it.",
|
|
||||||
"type": "string",
|
|
||||||
"enum": [
|
|
||||||
"auto",
|
|
||||||
"on",
|
|
||||||
"off"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
"prompt_progress": {
|
|
||||||
"description": "Ask how far the server is through reading the prompt (llama.cpp's return_progress), shown while it reads; auto learns when a server refuses it, and one that ignores it shows nothing.",
|
|
||||||
"type": "string",
|
|
||||||
"enum": [
|
|
||||||
"auto",
|
|
||||||
"on",
|
|
||||||
"off"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
"think_tags": {
|
|
||||||
"description": "<think>…</think> in a reply is taken as reasoning; off leaves it as text.",
|
|
||||||
"type": "string",
|
|
||||||
"enum": [
|
|
||||||
"auto",
|
|
||||||
"on",
|
|
||||||
"off"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
"max_tokens_field": {
|
|
||||||
"description": "OpenAI reasoning models refuse `max_tokens`; most local servers only know it.",
|
|
||||||
"type": "string",
|
|
||||||
"enum": [
|
|
||||||
"max_tokens",
|
|
||||||
"max_completion_tokens"
|
|
||||||
]
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"additionalProperties": false
|
|
||||||
},
|
|
||||||
"models": {
|
|
||||||
"description": "Only to change what the instance says about a model, here: each entry goes over the instance's. Usually absent.",
|
|
||||||
"type": "object",
|
|
||||||
"propertyNames": {
|
|
||||||
"type": "string"
|
|
||||||
},
|
|
||||||
"additionalProperties": {
|
|
||||||
"type": "object",
|
|
||||||
"properties": {
|
|
||||||
"name": {
|
|
||||||
"description": "A display name; the id is what is sent.",
|
|
||||||
"type": "string"
|
|
||||||
},
|
|
||||||
"family": {
|
|
||||||
"description": "Picks the prompt overlay: anthropic | gpt | gemini | local. Guessed from the id when absent.",
|
|
||||||
"type": "string"
|
|
||||||
},
|
|
||||||
"context": {
|
|
||||||
"description": "Total context window in tokens. 0 / absent = unknown: no percentage, no auto-compaction.",
|
|
||||||
"type": "integer",
|
|
||||||
"minimum": 0,
|
|
||||||
"maximum": 9007199254740991
|
|
||||||
},
|
|
||||||
"max_output": {
|
|
||||||
"description": "Max tokens per reply.",
|
|
||||||
"type": "integer",
|
|
||||||
"exclusiveMinimum": 0,
|
|
||||||
"maximum": 9007199254740991
|
|
||||||
},
|
|
||||||
"temperature": {
|
|
||||||
"description": "Sampling temperature; absent = the server's default.",
|
|
||||||
"type": "number",
|
|
||||||
"minimum": 0,
|
|
||||||
"maximum": 2
|
|
||||||
},
|
|
||||||
"top_p": {
|
|
||||||
"description": "Nucleus sampling; absent = the server's default.",
|
|
||||||
"type": "number",
|
|
||||||
"minimum": 0,
|
|
||||||
"maximum": 1
|
|
||||||
},
|
|
||||||
"efforts": {
|
|
||||||
"description": "The model's own effort vocabulary, in offering order.",
|
|
||||||
"type": "array",
|
|
||||||
"items": {
|
|
||||||
"type": "string",
|
|
||||||
"enum": [
|
|
||||||
"minimal",
|
|
||||||
"low",
|
|
||||||
"medium",
|
|
||||||
"high",
|
|
||||||
"xhigh",
|
|
||||||
"max"
|
|
||||||
]
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"effort": {
|
|
||||||
"description": "Default effort; `off` sends none.",
|
|
||||||
"anyOf": [
|
|
||||||
{
|
|
||||||
"type": "string",
|
|
||||||
"enum": [
|
|
||||||
"minimal",
|
|
||||||
"low",
|
|
||||||
"medium",
|
|
||||||
"high",
|
|
||||||
"xhigh",
|
|
||||||
"max"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"type": "string",
|
|
||||||
"const": "off"
|
|
||||||
}
|
|
||||||
]
|
|
||||||
},
|
|
||||||
"effort_style": {
|
|
||||||
"description": "openai-chat only: where the effort goes. llama.cpp drops top-level, so `both` is the default.",
|
|
||||||
"type": "string",
|
|
||||||
"enum": [
|
|
||||||
"top",
|
|
||||||
"kwargs",
|
|
||||||
"both"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
"effort_map": {
|
|
||||||
"description": "anthropic/gemini: effort → thinking budget tokens.",
|
|
||||||
"type": "object",
|
|
||||||
"propertyNames": {
|
|
||||||
"type": "string",
|
|
||||||
"enum": [
|
|
||||||
"minimal",
|
|
||||||
"low",
|
|
||||||
"medium",
|
|
||||||
"high",
|
|
||||||
"xhigh",
|
|
||||||
"max"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
"additionalProperties": {
|
|
||||||
"type": "integer",
|
|
||||||
"exclusiveMinimum": 0,
|
|
||||||
"maximum": 9007199254740991
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"vision": {
|
|
||||||
"description": "The model reads images: @image files, pasted paths and view_image.",
|
|
||||||
"type": "boolean"
|
|
||||||
},
|
|
||||||
"tools": {
|
|
||||||
"description": "false: a model without tool calls — it can only talk.",
|
|
||||||
"type": "boolean"
|
|
||||||
},
|
|
||||||
"cache": {
|
|
||||||
"description": "anthropic: prompt caching breakpoints.",
|
|
||||||
"type": "boolean"
|
|
||||||
},
|
|
||||||
"headers": {
|
|
||||||
"description": "Extra HTTP headers for this model's requests.",
|
|
||||||
"type": "object",
|
|
||||||
"propertyNames": {
|
|
||||||
"type": "string"
|
|
||||||
},
|
|
||||||
"additionalProperties": {
|
|
||||||
"type": "string"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"body": {
|
|
||||||
"description": "Merged into every request body for this model.",
|
|
||||||
"type": "object",
|
|
||||||
"propertyNames": {
|
|
||||||
"type": "string"
|
|
||||||
},
|
|
||||||
"additionalProperties": {}
|
|
||||||
},
|
|
||||||
"notes": {
|
|
||||||
"description": "What this model is good at, in a line: the model in use is told which others there are, for handing work over (task) or switching.",
|
|
||||||
"type": "string",
|
|
||||||
"maxLength": 300
|
|
||||||
},
|
|
||||||
"fallback": {
|
|
||||||
"description": "Other models (connection/model), in order, to switch to when this one's server cannot be reached at all — never once a reply has begun.",
|
|
||||||
"type": "array",
|
|
||||||
"items": {
|
|
||||||
"type": "string"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"single_session": {
|
|
||||||
"description": "The server serves one request at a time: a subagent on this same model would take its only slot and push the session's cached prompt out, so it is refused.",
|
|
||||||
"type": "boolean"
|
|
||||||
},
|
|
||||||
"group": {
|
|
||||||
"description": "Models sharing one server that holds one model at a time (llama-swap), named by any string: a subagent on another model of the same group would unload this one, so it is refused. One connection's one_model_at_a_time is the same rule for all its models; this is for models one connection serves from several such servers (a LLeMbas instance).",
|
|
||||||
"type": "string"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"additionalProperties": false
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
"required": [
|
|
||||||
"type",
|
|
||||||
"url"
|
|
||||||
],
|
|
||||||
"additionalProperties": false
|
"additionalProperties": false
|
||||||
},
|
},
|
||||||
{
|
"timeout": {
|
||||||
|
"description": "Seconds of silence allowed in a reply, as for any connection (default 600).",
|
||||||
|
"type": "number",
|
||||||
|
"exclusiveMinimum": 0
|
||||||
|
},
|
||||||
|
"quirks": {
|
||||||
|
"description": "Server oddities; auto is right almost always.",
|
||||||
"type": "object",
|
"type": "object",
|
||||||
"properties": {
|
"properties": {
|
||||||
"dialect": {
|
"stream_usage": {
|
||||||
|
"description": "Ask for token usage in the stream (stream_options); auto learns when a server refuses it.",
|
||||||
"type": "string",
|
"type": "string",
|
||||||
"enum": [
|
"enum": [
|
||||||
"openai-chat",
|
"auto",
|
||||||
"responses",
|
"on",
|
||||||
"anthropic",
|
"off"
|
||||||
"gemini",
|
|
||||||
"ollama"
|
|
||||||
],
|
|
||||||
"description": "The API it speaks: openai-chat (llama.cpp, vLLM, LM Studio, OpenAI…), responses (OpenAI Responses), anthropic, gemini, ollama (its native /api/chat)."
|
|
||||||
},
|
|
||||||
"base_url": {
|
|
||||||
"type": "string",
|
|
||||||
"description": "Where the API is, e.g. https://api.openai.com/v1 or http://localhost:8080/v1."
|
|
||||||
},
|
|
||||||
"api_key": {
|
|
||||||
"description": "Never pasted: {env:NAME} or {file:~/path}. Or use key_cmd.",
|
|
||||||
"type": "string"
|
|
||||||
},
|
|
||||||
"key_cmd": {
|
|
||||||
"description": "A command whose output is the key, e.g. pass show openai.",
|
|
||||||
"type": "string"
|
|
||||||
},
|
|
||||||
"auth": {
|
|
||||||
"description": "How the key is sent. Default per dialect: anthropic x-api-key, gemini x-goog-api-key, others bearer.",
|
|
||||||
"type": "string",
|
|
||||||
"enum": [
|
|
||||||
"bearer",
|
|
||||||
"x-api-key",
|
|
||||||
"x-goog-api-key",
|
|
||||||
"none"
|
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
"headers": {
|
"prompt_progress": {
|
||||||
"description": "Extra HTTP headers for every request.",
|
"description": "Ask how far the server is through reading the prompt (llama.cpp's return_progress), shown while it reads; auto learns when a server refuses it, and one that ignores it shows nothing.",
|
||||||
"type": "object",
|
|
||||||
"propertyNames": {
|
|
||||||
"type": "string"
|
|
||||||
},
|
|
||||||
"additionalProperties": {
|
|
||||||
"type": "string"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"body": {
|
|
||||||
"description": "Merged into every request body.",
|
|
||||||
"type": "object",
|
|
||||||
"propertyNames": {
|
|
||||||
"type": "string"
|
|
||||||
},
|
|
||||||
"additionalProperties": {}
|
|
||||||
},
|
|
||||||
"tls": {
|
|
||||||
"description": "TLS for endpoints behind a private CA. `ca` is a PEM file added to the trusted roots; `insecure` skips verification entirely (last resort). The system store is always trusted.",
|
|
||||||
"type": "object",
|
|
||||||
"properties": {
|
|
||||||
"ca": {
|
|
||||||
"type": "string"
|
|
||||||
},
|
|
||||||
"insecure": {
|
|
||||||
"type": "boolean"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"additionalProperties": false
|
|
||||||
},
|
|
||||||
"timeout": {
|
|
||||||
"description": "Seconds of silence allowed: waiting for the response, then between two pieces of a streamed reply. Not a limit on how long a reply may take. Default 600.",
|
|
||||||
"type": "number",
|
|
||||||
"exclusiveMinimum": 0
|
|
||||||
},
|
|
||||||
"discover": {
|
|
||||||
"description": "Merge ids and context lengths from GET /models.",
|
|
||||||
"type": "boolean"
|
|
||||||
},
|
|
||||||
"unload_url": {
|
|
||||||
"description": "llama-swap style: frees VRAM when switching away.",
|
|
||||||
"type": "string"
|
|
||||||
},
|
|
||||||
"one_model_at_a_time": {
|
|
||||||
"description": "The server holds one model at a time (llama-swap in front of one GPU): a subagent on another of its models would unload the session's, so it is refused. The session's own model may still be its own subagent.",
|
|
||||||
"type": "boolean"
|
|
||||||
},
|
|
||||||
"unload_method": {
|
|
||||||
"description": "How unload_url is called (default POST).",
|
|
||||||
"type": "string",
|
"type": "string",
|
||||||
"enum": [
|
"enum": [
|
||||||
"GET",
|
"auto",
|
||||||
"POST"
|
"on",
|
||||||
|
"off"
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
"quirks": {
|
"think_tags": {
|
||||||
"description": "Server oddities; auto is right almost always.",
|
"description": "<think>…</think> in a reply is taken as reasoning; off leaves it as text.",
|
||||||
"type": "object",
|
"type": "string",
|
||||||
"properties": {
|
"enum": [
|
||||||
"stream_usage": {
|
"auto",
|
||||||
"description": "Ask for token usage in the stream (stream_options); auto learns when a server refuses it.",
|
"on",
|
||||||
"type": "string",
|
"off"
|
||||||
"enum": [
|
]
|
||||||
"auto",
|
|
||||||
"on",
|
|
||||||
"off"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
"prompt_progress": {
|
|
||||||
"description": "Ask how far the server is through reading the prompt (llama.cpp's return_progress), shown while it reads; auto learns when a server refuses it, and one that ignores it shows nothing.",
|
|
||||||
"type": "string",
|
|
||||||
"enum": [
|
|
||||||
"auto",
|
|
||||||
"on",
|
|
||||||
"off"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
"think_tags": {
|
|
||||||
"description": "<think>…</think> in a reply is taken as reasoning; off leaves it as text.",
|
|
||||||
"type": "string",
|
|
||||||
"enum": [
|
|
||||||
"auto",
|
|
||||||
"on",
|
|
||||||
"off"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
"max_tokens_field": {
|
|
||||||
"description": "OpenAI reasoning models refuse `max_tokens`; most local servers only know it.",
|
|
||||||
"type": "string",
|
|
||||||
"enum": [
|
|
||||||
"max_tokens",
|
|
||||||
"max_completion_tokens"
|
|
||||||
]
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"additionalProperties": false
|
|
||||||
},
|
},
|
||||||
"models": {
|
"max_tokens_field": {
|
||||||
"default": {},
|
"description": "OpenAI reasoning models refuse `max_tokens`; most local servers only know it.",
|
||||||
"description": "The models to offer, by the id the server knows them by.",
|
"type": "string",
|
||||||
"type": "object",
|
"enum": [
|
||||||
"propertyNames": {
|
"max_tokens",
|
||||||
"type": "string"
|
"max_completion_tokens"
|
||||||
},
|
]
|
||||||
"additionalProperties": {
|
|
||||||
"type": "object",
|
|
||||||
"properties": {
|
|
||||||
"name": {
|
|
||||||
"description": "A display name; the id is what is sent.",
|
|
||||||
"type": "string"
|
|
||||||
},
|
|
||||||
"family": {
|
|
||||||
"description": "Picks the prompt overlay: anthropic | gpt | gemini | local. Guessed from the id when absent.",
|
|
||||||
"type": "string"
|
|
||||||
},
|
|
||||||
"context": {
|
|
||||||
"description": "Total context window in tokens. 0 / absent = unknown: no percentage, no auto-compaction.",
|
|
||||||
"type": "integer",
|
|
||||||
"minimum": 0,
|
|
||||||
"maximum": 9007199254740991
|
|
||||||
},
|
|
||||||
"max_output": {
|
|
||||||
"description": "Max tokens per reply.",
|
|
||||||
"type": "integer",
|
|
||||||
"exclusiveMinimum": 0,
|
|
||||||
"maximum": 9007199254740991
|
|
||||||
},
|
|
||||||
"temperature": {
|
|
||||||
"description": "Sampling temperature; absent = the server's default.",
|
|
||||||
"type": "number",
|
|
||||||
"minimum": 0,
|
|
||||||
"maximum": 2
|
|
||||||
},
|
|
||||||
"top_p": {
|
|
||||||
"description": "Nucleus sampling; absent = the server's default.",
|
|
||||||
"type": "number",
|
|
||||||
"minimum": 0,
|
|
||||||
"maximum": 1
|
|
||||||
},
|
|
||||||
"efforts": {
|
|
||||||
"description": "The model's own effort vocabulary, in offering order.",
|
|
||||||
"type": "array",
|
|
||||||
"items": {
|
|
||||||
"type": "string",
|
|
||||||
"enum": [
|
|
||||||
"minimal",
|
|
||||||
"low",
|
|
||||||
"medium",
|
|
||||||
"high",
|
|
||||||
"xhigh",
|
|
||||||
"max"
|
|
||||||
]
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"effort": {
|
|
||||||
"description": "Default effort; `off` sends none.",
|
|
||||||
"anyOf": [
|
|
||||||
{
|
|
||||||
"type": "string",
|
|
||||||
"enum": [
|
|
||||||
"minimal",
|
|
||||||
"low",
|
|
||||||
"medium",
|
|
||||||
"high",
|
|
||||||
"xhigh",
|
|
||||||
"max"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"type": "string",
|
|
||||||
"const": "off"
|
|
||||||
}
|
|
||||||
]
|
|
||||||
},
|
|
||||||
"effort_style": {
|
|
||||||
"description": "openai-chat only: where the effort goes. llama.cpp drops top-level, so `both` is the default.",
|
|
||||||
"type": "string",
|
|
||||||
"enum": [
|
|
||||||
"top",
|
|
||||||
"kwargs",
|
|
||||||
"both"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
"effort_map": {
|
|
||||||
"description": "anthropic/gemini: effort → thinking budget tokens.",
|
|
||||||
"type": "object",
|
|
||||||
"propertyNames": {
|
|
||||||
"type": "string",
|
|
||||||
"enum": [
|
|
||||||
"minimal",
|
|
||||||
"low",
|
|
||||||
"medium",
|
|
||||||
"high",
|
|
||||||
"xhigh",
|
|
||||||
"max"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
"additionalProperties": {
|
|
||||||
"type": "integer",
|
|
||||||
"exclusiveMinimum": 0,
|
|
||||||
"maximum": 9007199254740991
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"vision": {
|
|
||||||
"description": "The model reads images: @image files, pasted paths and view_image.",
|
|
||||||
"type": "boolean"
|
|
||||||
},
|
|
||||||
"tools": {
|
|
||||||
"description": "false: a model without tool calls — it can only talk.",
|
|
||||||
"type": "boolean"
|
|
||||||
},
|
|
||||||
"cache": {
|
|
||||||
"description": "anthropic: prompt caching breakpoints.",
|
|
||||||
"type": "boolean"
|
|
||||||
},
|
|
||||||
"headers": {
|
|
||||||
"description": "Extra HTTP headers for this model's requests.",
|
|
||||||
"type": "object",
|
|
||||||
"propertyNames": {
|
|
||||||
"type": "string"
|
|
||||||
},
|
|
||||||
"additionalProperties": {
|
|
||||||
"type": "string"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"body": {
|
|
||||||
"description": "Merged into every request body for this model.",
|
|
||||||
"type": "object",
|
|
||||||
"propertyNames": {
|
|
||||||
"type": "string"
|
|
||||||
},
|
|
||||||
"additionalProperties": {}
|
|
||||||
},
|
|
||||||
"notes": {
|
|
||||||
"description": "What this model is good at, in a line: the model in use is told which others there are, for handing work over (task) or switching.",
|
|
||||||
"type": "string",
|
|
||||||
"maxLength": 300
|
|
||||||
},
|
|
||||||
"fallback": {
|
|
||||||
"description": "Other models (connection/model), in order, to switch to when this one's server cannot be reached at all — never once a reply has begun.",
|
|
||||||
"type": "array",
|
|
||||||
"items": {
|
|
||||||
"type": "string"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"single_session": {
|
|
||||||
"description": "The server serves one request at a time: a subagent on this same model would take its only slot and push the session's cached prompt out, so it is refused.",
|
|
||||||
"type": "boolean"
|
|
||||||
},
|
|
||||||
"group": {
|
|
||||||
"description": "Models sharing one server that holds one model at a time (llama-swap), named by any string: a subagent on another model of the same group would unload this one, so it is refused. One connection's one_model_at_a_time is the same rule for all its models; this is for models one connection serves from several such servers (a LLeMbas instance).",
|
|
||||||
"type": "string"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"additionalProperties": false
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
"required": [
|
|
||||||
"dialect",
|
|
||||||
"base_url"
|
|
||||||
],
|
|
||||||
"additionalProperties": false
|
"additionalProperties": false
|
||||||
|
},
|
||||||
|
"models": {
|
||||||
|
"description": "Only to change what the instance says about a model, here: each entry goes over the instance's. Usually absent.",
|
||||||
|
"type": "object",
|
||||||
|
"propertyNames": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"additionalProperties": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"name": {
|
||||||
|
"description": "A display name; the id is what is sent.",
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"family": {
|
||||||
|
"description": "Picks the prompt overlay: anthropic | gpt | gemini | local. Guessed from the id when absent.",
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"context": {
|
||||||
|
"description": "Total context window in tokens. 0 / absent = unknown: no percentage, no auto-compaction.",
|
||||||
|
"type": "integer",
|
||||||
|
"minimum": 0,
|
||||||
|
"maximum": 9007199254740991
|
||||||
|
},
|
||||||
|
"max_output": {
|
||||||
|
"description": "Max tokens per reply.",
|
||||||
|
"type": "integer",
|
||||||
|
"exclusiveMinimum": 0,
|
||||||
|
"maximum": 9007199254740991
|
||||||
|
},
|
||||||
|
"temperature": {
|
||||||
|
"description": "Sampling temperature; absent = the server's default.",
|
||||||
|
"type": "number",
|
||||||
|
"minimum": 0,
|
||||||
|
"maximum": 2
|
||||||
|
},
|
||||||
|
"top_p": {
|
||||||
|
"description": "Nucleus sampling; absent = the server's default.",
|
||||||
|
"type": "number",
|
||||||
|
"minimum": 0,
|
||||||
|
"maximum": 1
|
||||||
|
},
|
||||||
|
"efforts": {
|
||||||
|
"description": "The model's own effort vocabulary, in offering order.",
|
||||||
|
"type": "array",
|
||||||
|
"items": {
|
||||||
|
"type": "string",
|
||||||
|
"enum": [
|
||||||
|
"minimal",
|
||||||
|
"low",
|
||||||
|
"medium",
|
||||||
|
"high",
|
||||||
|
"xhigh",
|
||||||
|
"max"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"effort": {
|
||||||
|
"description": "Default effort; `off` sends none.",
|
||||||
|
"anyOf": [
|
||||||
|
{
|
||||||
|
"type": "string",
|
||||||
|
"enum": [
|
||||||
|
"minimal",
|
||||||
|
"low",
|
||||||
|
"medium",
|
||||||
|
"high",
|
||||||
|
"xhigh",
|
||||||
|
"max"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "string",
|
||||||
|
"const": "off"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"effort_style": {
|
||||||
|
"description": "Where the effort goes in a request: top-level, in chat_template_kwargs, or both (the default: llama.cpp drops top-level).",
|
||||||
|
"type": "string",
|
||||||
|
"enum": [
|
||||||
|
"top",
|
||||||
|
"kwargs",
|
||||||
|
"both"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"vision": {
|
||||||
|
"description": "The model reads images: @image files, pasted paths and view_image.",
|
||||||
|
"type": "boolean"
|
||||||
|
},
|
||||||
|
"tools": {
|
||||||
|
"description": "false: a model without tool calls — it can only talk.",
|
||||||
|
"type": "boolean"
|
||||||
|
},
|
||||||
|
"headers": {
|
||||||
|
"description": "Extra HTTP headers for this model's requests.",
|
||||||
|
"type": "object",
|
||||||
|
"propertyNames": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"additionalProperties": {
|
||||||
|
"type": "string"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"body": {
|
||||||
|
"description": "Merged into every request body for this model.",
|
||||||
|
"type": "object",
|
||||||
|
"propertyNames": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"additionalProperties": {}
|
||||||
|
},
|
||||||
|
"notes": {
|
||||||
|
"description": "What this model is good at, in a line: the model in use is told which others there are, for handing work over (task) or switching.",
|
||||||
|
"type": "string",
|
||||||
|
"maxLength": 300
|
||||||
|
},
|
||||||
|
"single_session": {
|
||||||
|
"description": "The server serves one request at a time: a subagent on this same model would take its only slot and push the session's cached prompt out, so it is refused.",
|
||||||
|
"type": "boolean"
|
||||||
|
},
|
||||||
|
"group": {
|
||||||
|
"description": "Models sharing one GPU server that holds one model at a time (llama-swap), as the LLeMbas server says (capacity.group): a subagent on another model of the same group would unload this one, so it is refused.",
|
||||||
|
"type": "string"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"additionalProperties": false
|
||||||
|
}
|
||||||
}
|
}
|
||||||
]
|
},
|
||||||
|
"required": [
|
||||||
|
"type",
|
||||||
|
"url"
|
||||||
|
],
|
||||||
|
"additionalProperties": false
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
"additionalProperties": false,
|
"additionalProperties": false,
|
||||||
"title": "LLeMbas CLI connections.yaml",
|
"title": "LLeMbas CLI connections.yaml",
|
||||||
"description": "~/.config/lembas/connections.yaml: the LLM endpoints. Global only. Keys as {env:NAME}, {file:path} or key_cmd — never pasted."
|
"description": "~/.config/lembas/connections.yaml: the LLeMbas server this machine is logged in to, as lembas login writes it. Global only."
|
||||||
}
|
}
|
||||||
+2
-2
@@ -68,12 +68,12 @@ export async function drive(opts: { cols: number; rows: number; bin?: string; ho
|
|||||||
}
|
}
|
||||||
|
|
||||||
export function fakeHome(script: Parameters<typeof fakeProvider>[0]) {
|
export function fakeHome(script: Parameters<typeof fakeProvider>[0]) {
|
||||||
const fake = fakeProvider(script)
|
const fake = fakeProvider(script, { served: [{ id: "fake/coder" }, { id: "fake/other" }] })
|
||||||
const home = mkdtempSync(join(tmpdir(), "lembas-drive-"))
|
const home = mkdtempSync(join(tmpdir(), "lembas-drive-"))
|
||||||
mkdirSync(join(home, ".config/lembas"), { recursive: true })
|
mkdirSync(join(home, ".config/lembas"), { recursive: true })
|
||||||
writeFileSync(
|
writeFileSync(
|
||||||
join(home, ".config/lembas/connections.yaml"),
|
join(home, ".config/lembas/connections.yaml"),
|
||||||
`connections:\n fake:\n dialect: openai-chat\n base_url: ${fake.url}\n models:\n coder: { context: 32768, efforts: [low, high], effort: low }\n other: { context: 8192 }\n`,
|
`connections:\n fake:\n type: webui\n url: ${fake.url}\n api_key: smoke\n models:\n fake/coder: { context: 32768, efforts: [low, high], effort: low }\n fake/other: { context: 8192 }\n`,
|
||||||
{ mode: 0o600 },
|
{ mode: 0o600 },
|
||||||
)
|
)
|
||||||
writeFileSync(join(home, ".config/lembas/config.yaml"), "model: fake/coder\nupdate: { auto: off }\n")
|
writeFileSync(join(home, ".config/lembas/config.yaml"), "model: fake/coder\nupdate: { auto: off }\n")
|
||||||
|
|||||||
+5
-5
@@ -26,11 +26,11 @@ const v = run(["--version"])
|
|||||||
check("--version", v.code === 0 && /^\d+\.\d+\.\d+/.test(v.out), v.err)
|
check("--version", v.code === 0 && /^\d+\.\d+\.\d+/.test(v.out), v.err)
|
||||||
|
|
||||||
// 2. a headless run against a fake endpoint, with the session store on (FTS5 inside the binary)
|
// 2. a headless run against a fake endpoint, with the session store on (FTS5 inside the binary)
|
||||||
const fake = fakeProvider([{ chunks: [delta({ content: "smoke ok" }), usage(5, 2)] }])
|
const fake = fakeProvider([{ chunks: [delta({ content: "smoke ok" }), usage(5, 2)] }], { served: [{ id: "fake/m" }] })
|
||||||
mkdirSync(join(home, ".config/lembas"), { recursive: true })
|
mkdirSync(join(home, ".config/lembas"), { recursive: true })
|
||||||
writeFileSync(
|
writeFileSync(
|
||||||
join(home, ".config/lembas/connections.yaml"),
|
join(home, ".config/lembas/connections.yaml"),
|
||||||
`connections:\n fake:\n dialect: openai-chat\n base_url: ${fake.url}\n models:\n m: {}\n`,
|
`connections:\n fake:\n type: webui\n url: ${fake.url}\n api_key: smoke\n models:\n fake/m: {}\n`,
|
||||||
{ mode: 0o600 },
|
{ mode: 0o600 },
|
||||||
)
|
)
|
||||||
writeFileSync(join(home, ".config/lembas/config.yaml"), "model: fake/m\nupdate: { auto: off }\n")
|
writeFileSync(join(home, ".config/lembas/config.yaml"), "model: fake/m\nupdate: { auto: off }\n")
|
||||||
@@ -46,10 +46,10 @@ check("sessions → recorded (sqlite + FTS5 in the binary)", s.code === 0 && s.o
|
|||||||
// 2b. serve --stdio: the ACP agent inside the binary answers initialize, opens a session and
|
// 2b. serve --stdio: the ACP agent inside the binary answers initialize, opens a session and
|
||||||
// streams a reply — the code the LLeMbas link runs too.
|
// streams a reply — the code the LLeMbas link runs too.
|
||||||
{
|
{
|
||||||
const f = fakeProvider([{ chunks: [delta({ content: "acp ok" })] }])
|
const f = fakeProvider([{ chunks: [delta({ content: "acp ok" })] }], { served: [{ id: "fake/m" }] })
|
||||||
writeFileSync(
|
writeFileSync(
|
||||||
join(home, ".config/lembas/connections.yaml"),
|
join(home, ".config/lembas/connections.yaml"),
|
||||||
`connections:\n fake:\n dialect: openai-chat\n base_url: ${f.url}\n models:\n m: {}\n`,
|
`connections:\n fake:\n type: webui\n url: ${f.url}\n api_key: smoke\n models:\n fake/m: {}\n`,
|
||||||
{ mode: 0o600 },
|
{ mode: 0o600 },
|
||||||
)
|
)
|
||||||
const p = Bun.spawn([bin, "serve", "--stdio"], { cwd: home, env, stdin: "pipe", stdout: "pipe", stderr: "pipe" })
|
const p = Bun.spawn([bin, "serve", "--stdio"], { cwd: home, env, stdin: "pipe", stdout: "pipe", stderr: "pipe" })
|
||||||
@@ -181,7 +181,7 @@ for (const [cols, rows] of [
|
|||||||
const d = await drive({ cols: 100, rows: 30, bin, home: h, cwd })
|
const d = await drive({ cols: 100, rows: 30, bin, home: h, cwd })
|
||||||
await d.waitFor("Trust this directory?")
|
await d.waitFor("Trust this directory?")
|
||||||
await d.key("\r")
|
await d.key("\r")
|
||||||
const setup = await d.waitFor("no model to talk to yet")
|
const setup = await d.waitFor("not logged in to a LLeMbas server")
|
||||||
await Bun.write(join(cfgDir, "connections.yaml"), connections)
|
await Bun.write(join(cfgDir, "connections.yaml"), connections)
|
||||||
await Bun.write(join(cfgDir, "config.yaml"), "model: fake/coder\nupdate: { auto: off }\n")
|
await Bun.write(join(cfgDir, "config.yaml"), "model: fake/coder\nupdate: { auto: off }\n")
|
||||||
// The screen opens on the sign-in, its address focused: ctrl+r tries again from there.
|
// The screen opens on the sign-in, its address focused: ctrl+r tries again from there.
|
||||||
|
|||||||
+1
-20
@@ -17,7 +17,6 @@ import { DEFAULT_RULES, toRules } from "./permission/evaluate.ts"
|
|||||||
import { hardlineRules } from "./permission/hardline.ts"
|
import { hardlineRules } from "./permission/hardline.ts"
|
||||||
import { defaultEffort } from "./provider/effort.ts"
|
import { defaultEffort } from "./provider/effort.ts"
|
||||||
import { clientFor, familyOf, resolveModel } from "./provider/index.ts"
|
import { clientFor, familyOf, resolveModel } from "./provider/index.ts"
|
||||||
import { discoverContext, unload } from "./provider/discover.ts"
|
|
||||||
import { assembleSystem, instructionFiles, knowledgeScope } from "./prompt/assemble.ts"
|
import { assembleSystem, instructionFiles, knowledgeScope } from "./prompt/assemble.ts"
|
||||||
import { describeRepo, repoState } from "./git/repo.ts"
|
import { describeRepo, repoState } from "./git/repo.ts"
|
||||||
import { findProject, persistProjectRule, trustOf, type Project } from "./project/root.ts"
|
import { findProject, persistProjectRule, trustOf, type Project } from "./project/root.ts"
|
||||||
@@ -100,7 +99,7 @@ export interface App {
|
|||||||
/** The subagents the task tool offers (built-in, global, and a trusted project's), as of start. */
|
/** The subagents the task tool offers (built-in, global, and a trusted project's), as of start. */
|
||||||
agents: import("./project/agents.ts").AgentSpec[]
|
agents: import("./project/agents.ts").AgentSpec[]
|
||||||
store: Store | undefined
|
store: Store | undefined
|
||||||
/** Every configured `connection/model`, in config order. */
|
/** Every model the server serves, by its id there, in its order. */
|
||||||
modelRefs(): string[]
|
modelRefs(): string[]
|
||||||
/** A ref in any form it may be written in (the old `<login>/<model>` too) as the ref the
|
/** A ref in any form it may be written in (the old `<login>/<model>` too) as the ref the
|
||||||
* model has now; the ref itself when it names nothing. */
|
* model has now; the ref itself when it names nothing. */
|
||||||
@@ -497,7 +496,6 @@ export function createApp(o: AppOptions): App {
|
|||||||
...(at ? { root: at.root, cwd: at.cwd, projectDir: undefined } : {}),
|
...(at ? { root: at.root, cwd: at.cwd, projectDir: undefined } : {}),
|
||||||
jobs: { registry, owner: () => ({ sessionId: childSessionId, job }) },
|
jobs: { registry, owner: () => ({ sessionId: childSessionId, job }) },
|
||||||
},
|
},
|
||||||
fallback: engine.o.fallback,
|
|
||||||
steerMark: MARKS.to_subagent,
|
steerMark: MARKS.to_subagent,
|
||||||
system: (mode, current, plan) =>
|
system: (mode, current, plan) =>
|
||||||
system(
|
system(
|
||||||
@@ -567,16 +565,6 @@ export function createApp(o: AppOptions): App {
|
|||||||
system: (m, current, plan) => system(m, current, undefined, undefined, plan),
|
system: (m, current, plan) => system(m, current, undefined, undefined, plan),
|
||||||
persistRule: trusted ? (rule) => persistProjectRule(project, rule) : undefined,
|
persistRule: trusted ? (rule) => persistProjectRule(project, rule) : undefined,
|
||||||
spawn,
|
spawn,
|
||||||
fallback(current, tried) {
|
|
||||||
for (const ref of current.spec.fallback ?? []) {
|
|
||||||
if (tried.has(ref)) continue
|
|
||||||
try {
|
|
||||||
const m = resolveModel(loaded, ref)
|
|
||||||
return { model: m, client: clientFor(m), effort: startEffort(m) }
|
|
||||||
} catch {}
|
|
||||||
}
|
|
||||||
return undefined
|
|
||||||
},
|
|
||||||
})
|
})
|
||||||
|
|
||||||
// MCP tools join the engine's list as servers connect (the engine reads it at each prompt).
|
// MCP tools join the engine's list as servers connect (the engine reads it at each prompt).
|
||||||
@@ -647,18 +635,11 @@ export function createApp(o: AppOptions): App {
|
|||||||
const snapshots = o.snapshots === false ? undefined : new Snapshots(project.root, project.gitRoot)
|
const snapshots = o.snapshots === false ? undefined : new Snapshots(project.root, project.gitRoot)
|
||||||
engine.busyInput = loaded.config.busy_input ?? "steer"
|
engine.busyInput = loaded.config.busy_input ?? "steer"
|
||||||
const turns = new Turns(engine, snapshots, store)
|
const turns = new Turns(engine, snapshots, store)
|
||||||
// After a reply, learn the window if nobody configured it (the model is loaded by then).
|
|
||||||
bus.on((e) => {
|
|
||||||
if (e.type === "done" && !engine.model.spec.context) void discoverContext(engine.model, engine.client).then((n) => n && engine.usage && bus.emit({ type: "usage", usage: engine.usage, context: n, used: engine.contextUsed() }))
|
|
||||||
})
|
|
||||||
// By provider for an instance's models (`deepseek/deepseek-flash`), src/provider/refs.ts.
|
// By provider for an instance's models (`deepseek/deepseek-flash`), src/provider/refs.ts.
|
||||||
const modelRefs = () => loaded.refs.map((r) => r.ref)
|
const modelRefs = () => loaded.refs.map((r) => r.ref)
|
||||||
const switchModel = (ref: string) => {
|
const switchModel = (ref: string) => {
|
||||||
const m = resolveModel(loaded, ref)
|
const m = resolveModel(loaded, ref)
|
||||||
const prev = engine.model
|
|
||||||
engine.setModel(m, clientFor(m), startEffort(m))
|
engine.setModel(m, clientFor(m), startEffort(m))
|
||||||
// Moving to another connection frees the old one's memory (llama-swap /unload).
|
|
||||||
if (prev.connectionName !== m.connectionName) void unload(prev)
|
|
||||||
}
|
}
|
||||||
const setIn = (key: string, value: unknown) => {
|
const setIn = (key: string, value: unknown) => {
|
||||||
const path = key.split(".")
|
const path = key.split(".")
|
||||||
|
|||||||
+7
-21
@@ -12,7 +12,7 @@ import { renderJson, renderPlain, settleJobs, terminalAsker } from "./headless.t
|
|||||||
import { enterLocal, localFor, LocalError, localRefusal, localTmp, LOCAL_FLAGS } from "./project/local.ts"
|
import { enterLocal, localFor, LocalError, localRefusal, localTmp, LOCAL_FLAGS } from "./project/local.ts"
|
||||||
import { findProject, setTrust, trustOf } from "./project/root.ts"
|
import { findProject, setTrust, trustOf } from "./project/root.ts"
|
||||||
import { effortsFor } from "./provider/effort.ts"
|
import { effortsFor } from "./provider/effort.ts"
|
||||||
import { clientFor, ModelError, resolveModel } from "./provider/index.ts"
|
import { clientFor, ModelError, NOT_LOGGED_IN, resolveModel } from "./provider/index.ts"
|
||||||
import { Store } from "./session/store.ts"
|
import { Store } from "./session/store.ts"
|
||||||
import { promptTitle } from "./session/title.ts"
|
import { promptTitle } from "./session/title.ts"
|
||||||
import { VERSION } from "./version.ts"
|
import { VERSION } from "./version.ts"
|
||||||
@@ -23,7 +23,7 @@ Usage:
|
|||||||
lembas [-l] [-m model] [--mode M] [-c | -s id]
|
lembas [-l] [-m model] [--mode M] [-c | -s id]
|
||||||
the TUI (-c continues the last session here, -s resumes one)
|
the TUI (-c continues the last session here, -s resumes one)
|
||||||
lembas run [options] "…" one prompt, headless
|
lembas run [options] "…" one prompt, headless
|
||||||
lembas models [--discover] list configured models
|
lembas models list the server's models
|
||||||
lembas sessions list recent sessions
|
lembas sessions list recent sessions
|
||||||
lembas sessions delete <id>… delete sessions, here and in the web UI
|
lembas sessions delete <id>… delete sessions, here and in the web UI
|
||||||
lembas sessions prune [--dry-run]
|
lembas sessions prune [--dry-run]
|
||||||
@@ -110,7 +110,7 @@ async function loginCommand(cmd: "login" | "logout", args: string[]) {
|
|||||||
if (!name) fail(Object.keys(known).length ? `which one? lembas logout <${Object.keys(known).join("|")}>` : "not logged in to any instance", 2)
|
if (!name) fail(Object.keys(known).length ? `which one? lembas logout <${Object.keys(known).join("|")}>` : "not logged in to any instance", 2)
|
||||||
const r = await logout(name)
|
const r = await logout(name)
|
||||||
console.log(`Signed out of ${name}${r.revoked ? "" : " here (the instance could not be told — revoke it under User → Security → Devices there)"}; its connection is removed.`)
|
console.log(`Signed out of ${name}${r.revoked ? "" : " here (the instance could not be told — revoke it under User → Security → Devices there)"}; its connection is removed.`)
|
||||||
if (r.modelLeft) console.log(`config.yaml still starts on ${r.modelLeft}: choose another with lembas config set model <connection/model>.`)
|
if (r.modelLeft) console.log(`config.yaml still starts on ${r.modelLeft}: choose another with lembas config set model <id>.`)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
let address = positionals[0] ?? ""
|
let address = positionals[0] ?? ""
|
||||||
@@ -305,17 +305,11 @@ async function main() {
|
|||||||
|
|
||||||
if (cmd === "models") {
|
if (cmd === "models") {
|
||||||
const loaded = loadConfig()
|
const loaded = loadConfig()
|
||||||
const discover = argv.includes("--discover")
|
if (!Object.keys(loaded.connections).length && !Object.keys(loaded.broken).length) console.log(NOT_LOGGED_IN)
|
||||||
for (const [name, c] of Object.entries(loaded.connections)) {
|
for (const [name, c] of Object.entries(loaded.connections)) {
|
||||||
console.log(
|
console.log(`${name} (${c.webui?.url ?? c.base_url}${c.webui?.name ? ` — ${c.webui.name}` : ""}; models ${c.webui?.fetched_at ? `read ${c.webui.fetched_at.slice(0, 16).replace("T", " ")}` : "not read yet"})`)
|
||||||
c.webui
|
|
||||||
? `${name} (webui, ${c.webui.url}${c.webui.name ? ` — ${c.webui.name}` : ""}; models ${c.webui.fetched_at ? `read ${c.webui.fetched_at.slice(0, 16).replace("T", " ")}` : "not read yet"})`
|
|
||||||
: `${name} (${c.dialect}, ${c.base_url})`,
|
|
||||||
)
|
|
||||||
for (const [id, spec] of Object.entries(c.models)) {
|
for (const [id, spec] of Object.entries(c.models)) {
|
||||||
// Its ref: an instance's model by provider, anything else connection/model.
|
const m = resolveModel(loaded, id)
|
||||||
const ref = loaded.refs.find((r) => r.connection === name && r.id === id)?.ref ?? `${name}/${id}`
|
|
||||||
const m = resolveModel(loaded, ref)
|
|
||||||
const bits = [
|
const bits = [
|
||||||
spec.context ? `ctx ${spec.context}` : "",
|
spec.context ? `ctx ${spec.context}` : "",
|
||||||
spec.max_output ? `out ${spec.max_output}` : "",
|
spec.max_output ? `out ${spec.max_output}` : "",
|
||||||
@@ -323,15 +317,7 @@ async function main() {
|
|||||||
spec.tools === false ? "no-tools" : "",
|
spec.tools === false ? "no-tools" : "",
|
||||||
spec.effort || spec.efforts ? `effort ${spec.effort ?? "off"} of ${effortsFor(m).join("/")}` : "",
|
spec.effort || spec.efforts ? `effort ${spec.effort ?? "off"} of ${effortsFor(m).join("/")}` : "",
|
||||||
].filter(Boolean)
|
].filter(Boolean)
|
||||||
console.log(` ${ref}${bits.length ? " — " + bits.join(", ") : ""}`)
|
console.log(` ${m.ref}${bits.length ? " — " + bits.join(", ") : ""}`)
|
||||||
}
|
|
||||||
if (discover) {
|
|
||||||
try {
|
|
||||||
const found = await clientFor(resolveModel(loaded, `${name}/${Object.keys(c.models)[0] ?? "?"}`)).listModels()
|
|
||||||
for (const d of found) if (!c.models[d.id]) console.log(` ${name}/${d.id} (discovered${d.context ? `, ctx ${d.context}` : ""})`)
|
|
||||||
} catch (e) {
|
|
||||||
console.log(` (discovery failed: ${(e as Error).message})`)
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
for (const [name, why] of Object.entries(loaded.broken)) console.log(`${name} ✗ ${why}`)
|
for (const [name, why] of Object.entries(loaded.broken)) console.log(`${name} ✗ ${why}`)
|
||||||
|
|||||||
@@ -29,7 +29,7 @@ export function jsonSchemas(): Record<keyof typeof SCHEMA_FILES, Record<string,
|
|||||||
})
|
})
|
||||||
return {
|
return {
|
||||||
config: make(Config, "LLeMbas CLI config.yaml", "~/.config/lembas/config.yaml, or a project's .agent/config.yaml. Strings may be {env:NAME} or {file:path}."),
|
config: make(Config, "LLeMbas CLI config.yaml", "~/.config/lembas/config.yaml, or a project's .agent/config.yaml. Strings may be {env:NAME} or {file:path}."),
|
||||||
connections: make(ConnectionsFile, "LLeMbas CLI connections.yaml", "~/.config/lembas/connections.yaml: the LLM endpoints. Global only. Keys as {env:NAME}, {file:path} or key_cmd — never pasted."),
|
connections: make(ConnectionsFile, "LLeMbas CLI connections.yaml", "~/.config/lembas/connections.yaml: the LLeMbas server this machine is logged in to, as lembas login writes it. Global only."),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
+37
-36
@@ -3,8 +3,8 @@ import { join } from "node:path"
|
|||||||
import { isScalar, parse, parseDocument } from "yaml"
|
import { isScalar, parse, parseDocument } from "yaml"
|
||||||
import { z } from "zod"
|
import { z } from "zod"
|
||||||
import { connectionFiles, isLocal, paths } from "./paths.ts"
|
import { connectionFiles, isLocal, paths } from "./paths.ts"
|
||||||
import { asMode, Config, Connection, ConnectionsFile, McpServer, PERSONALITIES, renamePermissionKeys, WebuiConnection, type PermissionConfig } from "./schema.ts"
|
import { asMode, Config, McpServer, PERSONALITIES, renamePermissionKeys, WebuiConnection, type Connection, type PermissionConfig } from "./schema.ts"
|
||||||
import { runKeyCmd, substituteDeep } from "./substitute.ts"
|
import { substituteDeep } from "./substitute.ts"
|
||||||
import { expandWebui, refreshFailures, type WebuiMark } from "../lembas/webui.ts"
|
import { expandWebui, refreshFailures, type WebuiMark } from "../lembas/webui.ts"
|
||||||
import { defaultLibrary, withInstancePersonalization } from "../lembas/personal.ts"
|
import { defaultLibrary, withInstancePersonalization } from "../lembas/personal.ts"
|
||||||
import { instances } from "../lembas/login.ts"
|
import { instances } from "../lembas/login.ts"
|
||||||
@@ -14,8 +14,8 @@ export class ConfigError extends Error {}
|
|||||||
|
|
||||||
export interface Loaded {
|
export interface Loaded {
|
||||||
config: Config
|
config: Config
|
||||||
/** A `type: webui` entry is here as the openai-chat connection it is spoken to as, with the
|
/** The LLeMbas server this machine is logged in to — at most one: its `webui` entry as the
|
||||||
* instance's models and a `webui` mark. */
|
* `/v1` it is spoken to at, with its models and a `webui` mark (custom connections are gone). */
|
||||||
connections: Record<string, Connection & { webui?: WebuiMark }>
|
connections: Record<string, Connection & { webui?: WebuiMark }>
|
||||||
/** Connections whose {env:}/{file:} substitution failed, with the reason. Unusable, not fatal. */
|
/** Connections whose {env:}/{file:} substitution failed, with the reason. Unusable, not fatal. */
|
||||||
broken: Record<string, string>
|
broken: Record<string, string>
|
||||||
@@ -270,41 +270,47 @@ export function loadConfig(opts: { projectConfigDir?: string; trusted?: boolean
|
|||||||
|
|
||||||
const connections: Loaded["connections"] = {}
|
const connections: Loaded["connections"] = {}
|
||||||
const broken: Record<string, string> = {}
|
const broken: Record<string, string> = {}
|
||||||
// Validate each raw file, then substitute each connection on its own: a missing variable for one
|
// The login: a `webui` entry, as `lembas login` writes it. An entry of any other kind is a custom
|
||||||
// connection must not stop the others from working. In local mode, the user's own file first
|
// connection from 1.x, gone in 2.0 (the CLI is LLeMbas-server-specific): said, and read past, so
|
||||||
// (read, never written), then the local one, whose entries win by name.
|
// an old file still starts. In local mode, the user's own file first (read, never written), then
|
||||||
|
// the local one, whose entries win by name. Each entry is substituted on its own.
|
||||||
const raw: Record<string, unknown> = {}
|
const raw: Record<string, unknown> = {}
|
||||||
for (const connFile of connectionFiles()) {
|
for (const connFile of connectionFiles()) {
|
||||||
files.push(connFile)
|
files.push(connFile)
|
||||||
const mode = statSync(connFile).mode & 0o777
|
const mode = statSync(connFile).mode & 0o777
|
||||||
if (mode & 0o077) warnings.push(`${connFile} is mode ${mode.toString(8)}; chmod 600 it — it references credentials`)
|
if (mode & 0o077) warnings.push(`${connFile} is mode ${mode.toString(8)}; chmod 600 it — it references credentials`)
|
||||||
Object.assign(raw, validate(ConnectionsFile, readYaml(connFile), connFile).connections)
|
const doc = readYaml(connFile)
|
||||||
|
const entries = doc && typeof doc === "object" ? (doc as { connections?: unknown }).connections : undefined
|
||||||
|
if (entries === undefined || entries === null) continue
|
||||||
|
if (typeof entries !== "object" || Array.isArray(entries)) throw new ConfigError(`${connFile} is invalid:\n connections: expected the login as written by lembas login`)
|
||||||
|
for (const [name, c] of Object.entries(entries as Record<string, unknown>)) {
|
||||||
|
if ((c as { type?: string } | null)?.type === "webui") raw[name] = c
|
||||||
|
else warnings.push(`${connFile}: ${name} is a custom connection, and LLeMbas CLI 2.0 has none — it talks only to the LLeMbas server it is logged in to (lembas login <address>). Ignored; delete it.`)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
for (const [name, c] of Object.entries(raw)) {
|
// One server: the login made last, where several are left from 1.x.
|
||||||
|
const logins = instances()
|
||||||
|
const names = Object.keys(raw)
|
||||||
|
const chosen = names.length > 1 ? [...names].sort((a, b) => (logins[b]?.logged_in_at ?? "").localeCompare(logins[a]?.logged_in_at ?? ""))[0]! : names[0]
|
||||||
|
if (names.length > 1) warnings.push(`connections.yaml holds ${names.length} LLeMbas logins (${names.join(", ")}); LLeMbas CLI 2.0 uses one server — ${chosen}, the latest. lembas logout the others.`)
|
||||||
|
if (chosen) {
|
||||||
|
const name = chosen
|
||||||
try {
|
try {
|
||||||
if ((c as { type?: string }).type === "webui") {
|
const w = WebuiConnection.safeParse(substituteDeep(raw[name], `connections.${name}`))
|
||||||
const w = WebuiConnection.safeParse(substituteDeep(c, `connections.${name}`))
|
if (w.success) {
|
||||||
if (w.success) {
|
const c = (connections[name] = expandWebui(name, w.data))
|
||||||
const c = (connections[name] = expandWebui(name, w.data))
|
if (refreshFailures[name])
|
||||||
if (refreshFailures[name])
|
warnings.push(
|
||||||
warnings.push(
|
`${name}: the instance could not be asked for its models (${refreshFailures[name]}); ${c.webui.fetched_at ? `using what it said at ${c.webui.fetched_at.slice(0, 16).replace("T", " ")}` : "none known yet"}`,
|
||||||
`${name}: the instance could not be asked for its models (${refreshFailures[name]}); ${c.webui.fetched_at ? `using what it said at ${c.webui.fetched_at.slice(0, 16).replace("T", " ")}` : "none known yet"}`,
|
)
|
||||||
)
|
} else broken[name] = `once {env:}/{file:} are filled in: ${w.error.issues.map((i) => `${i.path.join(".")}: ${i.message}`).join("; ")}`
|
||||||
}
|
|
||||||
else broken[name] = `once {env:}/{file:} are filled in: ${w.error.issues.map((i) => `${i.path.join(".")}: ${i.message}`).join("; ")}`
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
// Again once filled in: "{env:LLM_URL}" passed as a placeholder; what it became must be a URL.
|
|
||||||
const r = Connection.safeParse(substituteDeep(c, `connections.${name}`))
|
|
||||||
if (r.success) connections[name] = r.data
|
|
||||||
else broken[name] = `once {env:}/{file:} are filled in: ${r.error.issues.map((i) => `${i.path.join(".")}: ${i.message}`).join("; ")}`
|
|
||||||
} catch (e) {
|
} catch (e) {
|
||||||
broken[name] = (e as Error).message
|
broken[name] = (e as Error).message
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/** The webui connection a voice half or the search means: the one named, or the only one. A
|
/** The server a voice half or the search goes through: the login (`connection:` may still name
|
||||||
* string says why there is none. */
|
* it). A string says why there is none. */
|
||||||
const webuiFor = (named?: string): { name: string; c: Loaded["connections"][string] } | string => {
|
const webuiFor = (named?: string): { name: string; c: Loaded["connections"][string] } | string => {
|
||||||
if (named) return connections[named]?.webui ? { name: named, c: connections[named]! } : broken[named] ? `${named} is unusable: ${broken[named]}` : `there is no webui connection named ${named}`
|
if (named) return connections[named]?.webui ? { name: named, c: connections[named]! } : broken[named] ? `${named} is unusable: ${broken[named]}` : `there is no webui connection named ${named}`
|
||||||
const all = Object.entries(connections).filter(([, c]) => c.webui)
|
const all = Object.entries(connections).filter(([, c]) => c.webui)
|
||||||
@@ -408,10 +414,7 @@ export function loadConfig(opts: { projectConfigDir?: string; trusted?: boolean
|
|||||||
config = personal.config
|
config = personal.config
|
||||||
if (config.library === undefined && !isLocal() && defaultLibrary()) config = { ...config, library: "lembas" }
|
if (config.library === undefined && !isLocal() && defaultLibrary()) config = { ...config, library: "lembas" }
|
||||||
|
|
||||||
// An instance that speaks protocol 2 serves `<provider>/<model>` ids.
|
const refs = modelRefs(connections)
|
||||||
const logins = instances()
|
|
||||||
for (const [name, c] of Object.entries(connections)) if (c.webui && logins[name]?.protocols?.includes(2)) c.webui.v2 = true
|
|
||||||
const refs = modelRefs(connections, Object.keys(logins))
|
|
||||||
// A config.yaml naming the model the way it was named before providers (`example/deepseek-flash`)
|
// A config.yaml naming the model the way it was named before providers (`example/deepseek-flash`)
|
||||||
// is rewritten to the new name, once (migrateRefs). Only that form: a pinned `<login>/<provider>/
|
// is rewritten to the new name, once (migrateRefs). Only that form: a pinned `<login>/<provider>/
|
||||||
// <model>` means what it says, and `embedding` stays as written — its name keys the stored vectors.
|
// <model>` means what it says, and `embedding` stays as written — its name keys the stored vectors.
|
||||||
@@ -480,9 +483,7 @@ function localOnly(config: Config, connections: Loaded["connections"], warnings:
|
|||||||
return out
|
return out
|
||||||
}
|
}
|
||||||
|
|
||||||
/** The API key for a connection: `api_key` (already substituted) or the output of `key_cmd`. */
|
/** The server's token: the login's `api_key`, already substituted from its key file. */
|
||||||
export function resolveKey(name: string, c: Connection): string | undefined {
|
export function resolveKey(_name: string, c: Pick<Connection, "api_key">): string | undefined {
|
||||||
if (c.api_key) return c.api_key
|
return c.api_key || undefined
|
||||||
if (c.key_cmd) return runKeyCmd(c.key_cmd, `connections.${name}.key_cmd`)
|
|
||||||
return undefined
|
|
||||||
}
|
}
|
||||||
+39
-40
@@ -1,6 +1,5 @@
|
|||||||
import { z } from "zod"
|
import { z } from "zod"
|
||||||
|
|
||||||
export const DIALECTS = ["openai-chat", "responses", "anthropic", "gemini", "ollama"] as const
|
|
||||||
export const EFFORTS = ["minimal", "low", "medium", "high", "xhigh", "max"] as const
|
export const EFFORTS = ["minimal", "low", "medium", "high", "xhigh", "max"] as const
|
||||||
export const MODES = ["manual", "edit", "auto", "plan"] as const
|
export const MODES = ["manual", "edit", "auto", "plan"] as const
|
||||||
/** The personality presets (harness/prompts/personality/, shared with the web UI), then custom. */
|
/** The personality presets (harness/prompts/personality/, shared with the web UI), then custom. */
|
||||||
@@ -54,46 +53,44 @@ export const ModelSpec = z.strictObject({
|
|||||||
top_p: z.number().min(0).max(1).optional().describe("Nucleus sampling; absent = the server's default."),
|
top_p: z.number().min(0).max(1).optional().describe("Nucleus sampling; absent = the server's default."),
|
||||||
efforts: z.array(z.enum(EFFORTS)).optional().describe("The model's own effort vocabulary, in offering order."),
|
efforts: z.array(z.enum(EFFORTS)).optional().describe("The model's own effort vocabulary, in offering order."),
|
||||||
effort: z.union([z.enum(EFFORTS), z.literal("off")]).optional().describe("Default effort; `off` sends none."),
|
effort: z.union([z.enum(EFFORTS), z.literal("off")]).optional().describe("Default effort; `off` sends none."),
|
||||||
effort_style: z.enum(["top", "kwargs", "both"]).optional().describe("openai-chat only: where the effort goes. llama.cpp drops top-level, so `both` is the default."),
|
effort_style: z.enum(["top", "kwargs", "both"]).optional().describe("Where the effort goes in a request: top-level, in chat_template_kwargs, or both (the default: llama.cpp drops top-level)."),
|
||||||
effort_map: z.partialRecord(z.enum(EFFORTS), z.number().int().positive()).optional().describe("anthropic/gemini: effort → thinking budget tokens."),
|
|
||||||
vision: z.boolean().optional().describe("The model reads images: @image files, pasted paths and view_image."),
|
vision: z.boolean().optional().describe("The model reads images: @image files, pasted paths and view_image."),
|
||||||
tools: z.boolean().optional().describe("false: a model without tool calls — it can only talk."),
|
tools: z.boolean().optional().describe("false: a model without tool calls — it can only talk."),
|
||||||
cache: z.boolean().optional().describe("anthropic: prompt caching breakpoints."),
|
|
||||||
headers: z.record(z.string(), z.string()).optional().describe("Extra HTTP headers for this model's requests."),
|
headers: z.record(z.string(), z.string()).optional().describe("Extra HTTP headers for this model's requests."),
|
||||||
body: z.record(z.string(), z.unknown()).optional().describe("Merged into every request body for this model."),
|
body: z.record(z.string(), z.unknown()).optional().describe("Merged into every request body for this model."),
|
||||||
notes: z.string().max(300).optional().describe("What this model is good at, in a line: the model in use is told which others there are, for handing work over (task) or switching."),
|
notes: z.string().max(300).optional().describe("What this model is good at, in a line: the model in use is told which others there are, for handing work over (task) or switching."),
|
||||||
fallback: z.array(z.string()).optional().describe("Other models (connection/model), in order, to switch to when this one's server cannot be reached at all — never once a reply has begun."),
|
|
||||||
single_session: z.boolean().optional().describe("The server serves one request at a time: a subagent on this same model would take its only slot and push the session's cached prompt out, so it is refused."),
|
single_session: z.boolean().optional().describe("The server serves one request at a time: a subagent on this same model would take its only slot and push the session's cached prompt out, so it is refused."),
|
||||||
group: z.string().optional().describe("Models sharing one server that holds one model at a time (llama-swap), named by any string: a subagent on another model of the same group would unload this one, so it is refused. One connection's one_model_at_a_time is the same rule for all its models; this is for models one connection serves from several such servers (a LLeMbas instance)."),
|
group: z.string().optional().describe("Models sharing one GPU server that holds one model at a time (llama-swap), as the LLeMbas server says (capacity.group): a subagent on another model of the same group would unload this one, so it is refused."),
|
||||||
})
|
})
|
||||||
export type ModelSpec = z.infer<typeof ModelSpec>
|
export type ModelSpec = z.infer<typeof ModelSpec>
|
||||||
|
|
||||||
export const Connection = z.strictObject({
|
/** Server oddities; auto is right almost always. */
|
||||||
dialect: z.enum(DIALECTS).describe("The API it speaks: openai-chat (llama.cpp, vLLM, LM Studio, OpenAI…), responses (OpenAI Responses), anthropic, gemini, ollama (its native /api/chat)."),
|
export const Quirks = z
|
||||||
base_url: Url.describe("Where the API is, e.g. https://api.openai.com/v1 or http://localhost:8080/v1."),
|
.strictObject({
|
||||||
api_key: z.string().optional().describe("Never pasted: {env:NAME} or {file:~/path}. Or use key_cmd."),
|
stream_usage: Tristate.optional().describe("Ask for token usage in the stream (stream_options); auto learns when a server refuses it."),
|
||||||
key_cmd: z.string().optional().describe("A command whose output is the key, e.g. pass show openai."),
|
prompt_progress: Tristate.optional().describe("Ask how far the server is through reading the prompt (llama.cpp's return_progress), shown while it reads; auto learns when a server refuses it, and one that ignores it shows nothing."),
|
||||||
auth: z.enum(["bearer", "x-api-key", "x-goog-api-key", "none"]).optional().describe("How the key is sent. Default per dialect: anthropic x-api-key, gemini x-goog-api-key, others bearer."),
|
think_tags: Tristate.optional().describe("<think>…</think> in a reply is taken as reasoning; off leaves it as text."),
|
||||||
headers: z.record(z.string(), z.string()).optional().describe("Extra HTTP headers for every request."),
|
max_tokens_field: z.enum(["max_tokens", "max_completion_tokens"]).optional().describe("OpenAI reasoning models refuse `max_tokens`; most local servers only know it."),
|
||||||
body: z.record(z.string(), z.unknown()).optional().describe("Merged into every request body."),
|
})
|
||||||
tls: Tls.optional().describe("TLS for endpoints behind a private CA. `ca` is a PEM file added to the trusted roots; `insecure` skips verification entirely (last resort). The system store is always trusted."),
|
.optional()
|
||||||
timeout: z.number().positive().optional().describe("Seconds of silence allowed: waiting for the response, then between two pieces of a streamed reply. Not a limit on how long a reply may take. Default 600."),
|
.describe("Server oddities; auto is right almost always.")
|
||||||
discover: z.boolean().optional().describe("Merge ids and context lengths from GET /models."),
|
|
||||||
unload_url: Url.optional().describe("llama-swap style: frees VRAM when switching away."),
|
/** The LLeMbas server as LLeMbas CLI speaks to it: its OpenAI-shaped `/v1` with this machine's
|
||||||
one_model_at_a_time: z.boolean().optional().describe("The server holds one model at a time (llama-swap in front of one GPU): a subagent on another of its models would unload the session's, so it is refused. The session's own model may still be its own subagent."),
|
* token, and the models it serves. Not something a person writes — custom connections were removed
|
||||||
unload_method: z.enum(["GET", "POST"]).optional().describe("How unload_url is called (default POST)."),
|
* in 2.0 (CLI rule 2: the CLI is LLeMbas-server-specific); `lembas login` writes the one `webui`
|
||||||
quirks: z
|
* entry below, and the loader turns it into this (lembas/webui.ts, expandWebui). */
|
||||||
.strictObject({
|
export interface Connection {
|
||||||
stream_usage: Tristate.optional().describe("Ask for token usage in the stream (stream_options); auto learns when a server refuses it."),
|
base_url: string
|
||||||
prompt_progress: Tristate.optional().describe("Ask how far the server is through reading the prompt (llama.cpp's return_progress), shown while it reads; auto learns when a server refuses it, and one that ignores it shows nothing."),
|
api_key?: string
|
||||||
think_tags: Tristate.optional().describe("<think>…</think> in a reply is taken as reasoning; off leaves it as text."),
|
tls?: z.infer<typeof Tls>
|
||||||
max_tokens_field: z.enum(["max_tokens", "max_completion_tokens"]).optional().describe("OpenAI reasoning models refuse `max_tokens`; most local servers only know it."),
|
/** Seconds of silence allowed in a reply (default 600). */
|
||||||
})
|
timeout?: number
|
||||||
.optional()
|
quirks?: z.infer<typeof Quirks>
|
||||||
.describe("Server oddities; auto is right almost always."),
|
/** Extra headers and body fields for every request (none from a login; tests use them). */
|
||||||
models: z.record(z.string(), ModelSpec).default({}).describe("The models to offer, by the id the server knows them by."),
|
headers?: Record<string, string>
|
||||||
})
|
body?: Record<string, unknown>
|
||||||
export type Connection = z.infer<typeof Connection>
|
models: Record<string, ModelSpec>
|
||||||
|
}
|
||||||
|
|
||||||
/** A LLeMbas instance: its address and this machine's key, and nothing else. The models,
|
/** A LLeMbas instance: its address and this machine's key, and nothing else. The models,
|
||||||
* their settings, and — where config.yaml says `provider: webui` — the voice and the web search come
|
* their settings, and — where config.yaml says `provider: webui` — the voice and the web search come
|
||||||
@@ -101,20 +98,22 @@ export type Connection = z.infer<typeof Connection>
|
|||||||
export const WebuiConnection = z.strictObject({
|
export const WebuiConnection = z.strictObject({
|
||||||
type: z.literal("webui").describe("A LLeMbas instance: its models, their settings, its voice and its web search."),
|
type: z.literal("webui").describe("A LLeMbas instance: its models, their settings, its voice and its web search."),
|
||||||
url: Url.describe("The instance's address, e.g. https://ai.example.org (no /v1)."),
|
url: Url.describe("The instance's address, e.g. https://ai.example.org (no /v1)."),
|
||||||
api_key: z.string().optional().describe("This machine's token, as login wrote it: {file:~/.config/lembas/lembas/<name>.key}. Or key_cmd."),
|
api_key: z.string().optional().describe("This machine's token, as login wrote it: {file:~/.config/lembas/lembas/<name>.key}."),
|
||||||
key_cmd: z.string().optional(),
|
|
||||||
tls: Tls.optional().describe("For an instance behind a private CA: `ca`, the PEM file that signed its certificate."),
|
tls: Tls.optional().describe("For an instance behind a private CA: `ca`, the PEM file that signed its certificate."),
|
||||||
timeout: z.number().positive().optional().describe("Seconds of silence allowed in a reply, as for any connection (default 600)."),
|
timeout: z.number().positive().optional().describe("Seconds of silence allowed in a reply, as for any connection (default 600)."),
|
||||||
quirks: Connection.shape.quirks,
|
quirks: Quirks,
|
||||||
models: z.record(z.string(), ModelSpec).optional().describe("Only to change what the instance says about a model, here: each entry goes over the instance's. Usually absent."),
|
models: z.record(z.string(), ModelSpec).optional().describe("Only to change what the instance says about a model, here: each entry goes over the instance's. Usually absent."),
|
||||||
})
|
})
|
||||||
export type WebuiConnection = z.infer<typeof WebuiConnection>
|
export type WebuiConnection = z.infer<typeof WebuiConnection>
|
||||||
|
|
||||||
|
/** connections.yaml: the login, as `lembas login` writes it. An entry of any other kind — a custom
|
||||||
|
* connection from 1.x — is read past with a warning (config/load.ts), not refused, so an old file
|
||||||
|
* still starts. */
|
||||||
export const ConnectionsFile = z.strictObject({
|
export const ConnectionsFile = z.strictObject({
|
||||||
connections: z
|
connections: z
|
||||||
.record(z.string(), z.union([WebuiConnection, Connection]))
|
.record(z.string(), WebuiConnection)
|
||||||
.default({})
|
.default({})
|
||||||
.describe("Each LLM endpoint by a name of your choosing; a model is then `name/model-id`. `type: webui` is a LLeMbas instance, whose models come from it."),
|
.describe("The LLeMbas server this machine is logged in to, as `lembas login` writes it: its address and this machine's token. Its models come from it at every start."),
|
||||||
})
|
})
|
||||||
export type ConnectionsFile = z.infer<typeof ConnectionsFile>
|
export type ConnectionsFile = z.infer<typeof ConnectionsFile>
|
||||||
|
|
||||||
@@ -231,8 +230,8 @@ export const Update = z.strictObject({
|
|||||||
export type Update = z.infer<typeof Update>
|
export type Update = z.infer<typeof Update>
|
||||||
|
|
||||||
export const Config = z.strictObject({
|
export const Config = z.strictObject({
|
||||||
model: z.string().optional().describe("The model a session starts with: `connection/model`, a connection from connections.yaml and one of its models."),
|
model: z.string().optional().describe("The model a session starts with: one the LLeMbas server you are logged in to serves, by its id there (`deepseek/deepseek-flash`). Default: the server's default for your account."),
|
||||||
small_model: z.string().optional().describe("`connection/model` for small jobs — session titles, commit messages. Default: the session's model. On a server that holds one model at a time (llama-swap), another model here swaps."),
|
small_model: z.string().optional().describe("The model for small jobs — session titles, commit messages, by its id on the server. Default: the session's model."),
|
||||||
titles: z.enum(["model", "prompt"]).optional().describe("Session titles: model (default) — named from the first prompt, then by the model after its first reply; prompt — only from the first prompt, no extra request."),
|
titles: z.enum(["model", "prompt"]).optional().describe("Session titles: model (default) — named from the first prompt, then by the model after its first reply; prompt — only from the first prompt, no extra request."),
|
||||||
mode: z.enum(MODES).optional().describe("The permission mode a session starts in (default manual)."),
|
mode: z.enum(MODES).optional().describe("The permission mode a session starts in (default manual)."),
|
||||||
theme: z.string().optional().describe("Colours: moria (default), shire, lembas, lembas-light, hermes, mono, or a themes/<name>.yaml of your own."),
|
theme: z.string().optional().describe("Colours: moria (default), shire, lembas, lembas-light, hermes, mono, or a themes/<name>.yaml of your own."),
|
||||||
@@ -293,7 +292,7 @@ export const Config = z.strictObject({
|
|||||||
voice: Voice.optional().describe("Voice in and out. Global only — it carries endpoints and keys, like a connection."),
|
voice: Voice.optional().describe("Voice in and out. Global only — it carries endpoints and keys, like a connection."),
|
||||||
mcp: z.record(z.string(), McpServer).optional().describe("MCP servers by name. `{env:}`/`{file:}` are filled in per server (a missing one disables that server only). A project's servers count only once the project is trusted."),
|
mcp: z.record(z.string(), McpServer).optional().describe("MCP servers by name. `{env:}`/`{file:}` are filled in per server (a missing one disables that server only). A project's servers count only once the project is trusted."),
|
||||||
knowledge: z.array(z.string()).optional().describe("The knowledge bases searched (lembas kb); absent: all of them. A project's own config narrows it to what that project needs."),
|
knowledge: z.array(z.string()).optional().describe("The knowledge bases searched (lembas kb); absent: all of them. A project's own config narrows it to what that project needs."),
|
||||||
embedding: z.string().optional().describe("Global config only. connection/model of an embedding model (OpenAI-shaped /embeddings, or Ollama): knowledge bases are then searched by meaning as well as by words. lembas kb reindex after changing it."),
|
embedding: z.string().optional().describe("Global config only. An embedding model the LLeMbas server serves, by its id there: knowledge bases are then searched by meaning as well as by words. lembas kb reindex after changing it."),
|
||||||
effort: z.union([z.enum(EFFORTS), z.literal("off")]).optional().describe("The effort a session starts with, when the model takes it; otherwise the model's own default."),
|
effort: z.union([z.enum(EFFORTS), z.literal("off")]).optional().describe("The effort a session starts with, when the model takes it; otherwise the model's own default."),
|
||||||
settings_tool: z.enum(["ask", "allow", "off"]).optional().describe("Global config only. The agent's settings tool: ask (default) — each change is approved like any tool call; allow — changes go through without asking (a less strict mode still asks); off — no settings tool."),
|
settings_tool: z.enum(["ask", "allow", "off"]).optional().describe("Global config only. The agent's settings tool: ask (default) — each change is approved like any tool call; allow — changes go through without asking (a less strict mode still asks); off — no settings tool."),
|
||||||
update: Update.optional().describe("Global config only. Updating LLeMbas CLI itself."),
|
update: Update.optional().describe("Global config only. Updating LLeMbas CLI itself."),
|
||||||
|
|||||||
@@ -19,7 +19,7 @@ export type Source = "default" | "global" | "project" | "session"
|
|||||||
|
|
||||||
export interface SettingDef {
|
export interface SettingDef {
|
||||||
key: string
|
key: string
|
||||||
/** model: a configured `connection/model`; effort: one the model takes, or off; list: comma-separated. */
|
/** model: a model the server serves, by its id; effort: one the model takes, or off; list: comma-separated. */
|
||||||
kind: "enum" | "model" | "effort" | "number" | "boolean" | "list" | "string"
|
kind: "enum" | "model" | "effort" | "number" | "boolean" | "list" | "string"
|
||||||
values?: readonly string[]
|
values?: readonly string[]
|
||||||
/** The agent may change it (through the settings tool, with approval). */
|
/** The agent may change it (through the settings tool, with approval). */
|
||||||
|
|||||||
+11
-1
@@ -7,7 +7,8 @@
|
|||||||
// pasted into the YAML. The models are not written anywhere: a webui connection reads them from the
|
// pasted into the YAML. The models are not written anywhere: a webui connection reads them from the
|
||||||
// instance at every start (webui.ts). `lembas.json` remembers which connections a login made, so
|
// instance at every start (webui.ts). `lembas.json` remembers which connections a login made, so
|
||||||
// `login` again refreshes them and `logout` removes exactly them. Without a login nothing here is
|
// `login` again refreshes them and `logout` removes exactly them. Without a login nothing here is
|
||||||
// read, and LLeMbas CLI runs on its hand-written config as before.
|
// read, and LLeMbas CLI has no model to talk to: 2.0 talks to the server it is logged in to, one at
|
||||||
|
// a time, and nothing else.
|
||||||
import { chmodSync, existsSync, mkdirSync, readFileSync, rmSync, writeFileSync } from "node:fs"
|
import { chmodSync, existsSync, mkdirSync, readFileSync, rmSync, writeFileSync } from "node:fs"
|
||||||
import { join, resolve } from "node:path"
|
import { join, resolve } from "node:path"
|
||||||
import { parseDocument } from "yaml"
|
import { parseDocument } from "yaml"
|
||||||
@@ -416,6 +417,15 @@ export async function login(address: string, o: LoginOptions): Promise<LoginResu
|
|||||||
changes.push("the previous login could not be signed out: revoke it under User → Security → Devices on the instance")
|
changes.push("the previous login could not be signed out: revoke it under User → Security → Devices on the instance")
|
||||||
}
|
}
|
||||||
await handOver(replaced)
|
await handOver(replaced)
|
||||||
|
// One server at a time (2.0: the CLI is LLeMbas-server-specific): a login to another one signs
|
||||||
|
// this machine out of the one it was logged in to.
|
||||||
|
for (const [name, old] of Object.entries(ownInstances()))
|
||||||
|
if (name !== connection && old.base_url !== discovery.base_url) {
|
||||||
|
const out = await logout(name).catch(() => undefined)
|
||||||
|
if (!out) continue
|
||||||
|
changes.push(`signed out of ${old.instance_name ?? old.base_url} (${name}): LLeMbas CLI uses one server at a time`)
|
||||||
|
if (out.modelLeft) changes.push(`config.yaml still starts on ${out.modelLeft}, a model of that server: change it with lembas config set model <id>, or remove it to start on this server's default`)
|
||||||
|
}
|
||||||
// Only logins this mode made: in local mode, the user's own login to the same instance is not
|
// Only logins this mode made: in local mode, the user's own login to the same instance is not
|
||||||
// signed out or replaced — it is not this project's.
|
// signed out or replaced — it is not this project's.
|
||||||
const known = ownInstances()
|
const known = ownInstances()
|
||||||
|
|||||||
+2
-14
@@ -19,7 +19,7 @@ import { join } from "node:path"
|
|||||||
import { parse } from "yaml"
|
import { parse } from "yaml"
|
||||||
import { connectionFiles, isLocal, paths } from "../config/paths.ts"
|
import { connectionFiles, isLocal, paths } from "../config/paths.ts"
|
||||||
import type { Connection, ModelSpec, WebuiConnection } from "../config/schema.ts"
|
import type { Connection, ModelSpec, WebuiConnection } from "../config/schema.ts"
|
||||||
import { runKeyCmd, substituteDeep } from "../config/substitute.ts"
|
import { substituteDeep } from "../config/substitute.ts"
|
||||||
import { instanceInfo, LembasError, models as listModels, normalise, personalization, type InstanceInfo, type Personalization, type ServedModel, type Tls } from "./client.ts"
|
import { instanceInfo, LembasError, models as listModels, normalise, personalization, type InstanceInfo, type Personalization, type ServedModel, type Tls } from "./client.ts"
|
||||||
|
|
||||||
/** What a loaded webui connection carries besides an openai-chat connection's fields. */
|
/** What a loaded webui connection carries besides an openai-chat connection's fields. */
|
||||||
@@ -33,14 +33,9 @@ export interface WebuiMark {
|
|||||||
services?: InstanceInfo["services"]
|
services?: InstanceInfo["services"]
|
||||||
/** When the models were last read from the instance; absent: never. */
|
/** When the models were last read from the instance; absent: never. */
|
||||||
fetched_at?: string
|
fetched_at?: string
|
||||||
/** Each served model's provider, where the instance said it (`provider`). */
|
|
||||||
providers?: Record<string, string>
|
|
||||||
/** The served ids of its embedding models, which are not chat models and so not in
|
/** The served ids of its embedding models, which are not chat models and so not in
|
||||||
* `models`: `embedding: llama/nomic-embed` finds its connection by them (library/embed.ts). */
|
* `models`: `embedding: llama/nomic-embed` finds its connection by them (library/embed.ts). */
|
||||||
embeddings?: string[]
|
embeddings?: string[]
|
||||||
/** The instance speaks link protocol 2: its ids are `<provider>/<model>` even
|
|
||||||
* where a model does not say its provider. Set by loadConfig from the login. */
|
|
||||||
v2?: boolean
|
|
||||||
}
|
}
|
||||||
|
|
||||||
export type WebuiLoaded = Connection & { webui: WebuiMark }
|
export type WebuiLoaded = Connection & { webui: WebuiMark }
|
||||||
@@ -134,18 +129,12 @@ export function expandWebui(name: string, w: WebuiConnection, cache = readCache(
|
|||||||
for (const m of ordered) models[m.id] = { ...specFor(m, name), ...(overrides.get(m.id) ?? {}) }
|
for (const m of ordered) models[m.id] = { ...specFor(m, name), ...(overrides.get(m.id) ?? {}) }
|
||||||
// A model the entry names that the instance does not (yet) list: kept, as written.
|
// A model the entry names that the instance does not (yet) list: kept, as written.
|
||||||
for (const [id, spec] of unmatched) models[id] ??= spec
|
for (const [id, spec] of unmatched) models[id] ??= spec
|
||||||
const providers = Object.fromEntries(served.filter((m) => m.provider && m.id.startsWith(`${m.provider}/`)).map((m) => [m.id, m.provider!]))
|
|
||||||
return {
|
return {
|
||||||
dialect: "openai-chat",
|
|
||||||
base_url: `${url}/v1`,
|
base_url: `${url}/v1`,
|
||||||
...(w.api_key !== undefined ? { api_key: w.api_key } : {}),
|
...(w.api_key !== undefined ? { api_key: w.api_key } : {}),
|
||||||
...(w.key_cmd !== undefined ? { key_cmd: w.key_cmd } : {}),
|
|
||||||
...(w.tls ? { tls: w.tls } : {}),
|
...(w.tls ? { tls: w.tls } : {}),
|
||||||
...(w.timeout ? { timeout: w.timeout } : {}),
|
...(w.timeout ? { timeout: w.timeout } : {}),
|
||||||
...(w.quirks ? { quirks: w.quirks } : {}),
|
...(w.quirks ? { quirks: w.quirks } : {}),
|
||||||
// The instance decides which models exist: one it serves that the cache does not know yet is
|
|
||||||
// still a model, not an error.
|
|
||||||
discover: true,
|
|
||||||
models,
|
models,
|
||||||
webui: {
|
webui: {
|
||||||
url,
|
url,
|
||||||
@@ -153,7 +142,6 @@ export function expandWebui(name: string, w: WebuiConnection, cache = readCache(
|
|||||||
...(cache?.instance?.name ? { name: cache.instance.name } : {}),
|
...(cache?.instance?.name ? { name: cache.instance.name } : {}),
|
||||||
...(cache?.instance?.services ? { services: cache.instance.services } : {}),
|
...(cache?.instance?.services ? { services: cache.instance.services } : {}),
|
||||||
...(cache && cache.url === url ? { fetched_at: cache.fetched_at } : {}),
|
...(cache && cache.url === url ? { fetched_at: cache.fetched_at } : {}),
|
||||||
...(Object.keys(providers).length ? { providers } : {}),
|
|
||||||
...(embeddings.length ? { embeddings } : {}),
|
...(embeddings.length ? { embeddings } : {}),
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
@@ -200,7 +188,7 @@ export async function refreshWebui(o: { timeoutMs?: number; only?: string } = {}
|
|||||||
entries.map(async ([name, c]): Promise<Refreshed> => {
|
entries.map(async ([name, c]): Promise<Refreshed> => {
|
||||||
try {
|
try {
|
||||||
const w = substituteDeep(c, `connections.${name}`) as WebuiConnection
|
const w = substituteDeep(c, `connections.${name}`) as WebuiConnection
|
||||||
const token = w.api_key ?? (w.key_cmd ? runKeyCmd(w.key_cmd, `connections.${name}.key_cmd`) : undefined)
|
const token = w.api_key
|
||||||
if (!token) throw new LembasError("no key (lembas login)")
|
if (!token) throw new LembasError("no key (lembas login)")
|
||||||
const got = await fetchWebui(name, w.url, token.trim(), w.tls, o.timeoutMs ?? 4000)
|
const got = await fetchWebui(name, w.url, token.trim(), w.tls, o.timeoutMs ?? 4000)
|
||||||
delete refreshFailures[name]
|
delete refreshFailures[name]
|
||||||
|
|||||||
+19
-33
@@ -1,6 +1,6 @@
|
|||||||
// Embeddings for the library: `embedding: connection/model` in config.yaml, over the connection's
|
// Embeddings for the library: `embedding:` in config.yaml names an embedding model of the LLeMbas
|
||||||
// OpenAI-shaped /embeddings (llama.cpp with --embedding, vLLM, LM Studio, OpenAI…) or Ollama's
|
// server, reached at its OpenAI-shaped /v1/embeddings. Vectors are made unit length here, so a dot
|
||||||
// /api/embed. Vectors are made unit length here, so a dot product is the cosine.
|
// product is the cosine.
|
||||||
import { resolveKey, type Loaded } from "../config/load.ts"
|
import { resolveKey, type Loaded } from "../config/load.ts"
|
||||||
import { authHeaders, joinUrl, request, tlsFor } from "../provider/http.ts"
|
import { authHeaders, joinUrl, request, tlsFor } from "../provider/http.ts"
|
||||||
import { resolveModel } from "../provider/index.ts"
|
import { resolveModel } from "../provider/index.ts"
|
||||||
@@ -15,46 +15,32 @@ function unit(v: number[]): Float32Array {
|
|||||||
|
|
||||||
export function embedderFor(loaded: Loaded, ref: string | undefined): Embedder | undefined {
|
export function embedderFor(loaded: Loaded, ref: string | undefined): Embedder | undefined {
|
||||||
if (!ref) return undefined
|
if (!ref) return undefined
|
||||||
// Through the same resolver as every model ref: an instance's model by its provider
|
// An embedding model of the server, by its served id (`llama/nomic-embed`) — not a chat model, so
|
||||||
// (`llama/nomic-embed`) or in the older form (`example/nomic-embed`). The name stored with the
|
// not a model ref (lembas/webui.ts, isChatModel): found among the ids the server says it serves
|
||||||
// vectors stays the ref as written, so a library indexed under it is still found.
|
// for embeddings, or as a ref in any of its forms. The name stored with the vectors stays the ref
|
||||||
// An embedding model need not be listed on its connection (`connection/<any id>`, as before).
|
// as written, so a library indexed under it is still found.
|
||||||
let name: string
|
const [name, c] = Object.entries(loaded.connections)[0] ?? []
|
||||||
|
if (!name || !c) throw new Error("embedding: not logged in to a LLeMbas server (lembas login)")
|
||||||
let id: string
|
let id: string
|
||||||
try {
|
if (c.webui?.embeddings?.includes(ref)) id = ref
|
||||||
const resolved = resolveModel(loaded, ref)
|
else if (ref.startsWith(`${name}/`) && c.webui?.embeddings?.includes(ref.slice(name.length + 1))) id = ref.slice(name.length + 1)
|
||||||
name = resolved.connectionName
|
else {
|
||||||
id = resolved.id
|
try {
|
||||||
} catch (e) {
|
id = resolveModel(loaded, ref).id
|
||||||
// An instance's embedding model by its served id (`llama/nomic-embed`): not a
|
} catch (e) {
|
||||||
// model ref, since only chat models are (lembas/webui.ts, isChatModel), so found by the ids
|
throw new Error(`embedding: ${(e as Error).message}`)
|
||||||
// its connection says it serves.
|
|
||||||
const served = Object.entries(loaded.connections).find(([, c]) => (c as { webui?: { embeddings?: string[] } }).webui?.embeddings?.includes(ref))
|
|
||||||
const slash = ref.indexOf("/")
|
|
||||||
if (served) {
|
|
||||||
name = served[0]
|
|
||||||
id = ref
|
|
||||||
} else {
|
|
||||||
if (slash <= 0) throw new Error(`embedding: "${ref}" must be written connection/model`)
|
|
||||||
name = ref.slice(0, slash)
|
|
||||||
id = ref.slice(slash + 1)
|
|
||||||
if (!loaded.connections[name]) throw new Error(`embedding: ${(e as Error).message}`)
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
const c = loaded.connections[name]!
|
|
||||||
if (c.dialect === "anthropic" || c.dialect === "gemini") throw new Error(`embedding: the ${c.dialect} dialect has no embeddings here; use an OpenAI-compatible or Ollama connection`)
|
|
||||||
const ollama = c.dialect === "ollama"
|
|
||||||
return {
|
return {
|
||||||
model: ref,
|
model: ref,
|
||||||
async embed(texts, signal) {
|
async embed(texts, signal) {
|
||||||
const res = await request(
|
const res = await request(
|
||||||
joinUrl(c.base_url, ollama ? "api/embed" : "embeddings"),
|
joinUrl(c.base_url, "embeddings"),
|
||||||
{ method: "POST", headers: authHeaders(c, resolveKey(name, c)), body: JSON.stringify({ model: id, input: texts }), signal, timeoutMs: (c.timeout ?? 120) * 1000, tls: tlsFor(c) },
|
{ method: "POST", headers: authHeaders(c, resolveKey(name, c)), body: JSON.stringify({ model: id, input: texts }), signal, timeoutMs: (c.timeout ?? 120) * 1000, tls: tlsFor(c) },
|
||||||
`${ref} (embeddings)`,
|
`${ref} (embeddings)`,
|
||||||
)
|
)
|
||||||
const j = (await res.json()) as { data?: { embedding: number[]; index?: number }[]; embeddings?: number[][] }
|
const j = (await res.json()) as { data?: { embedding: number[]; index?: number }[] }
|
||||||
const rows = ollama ? (j.embeddings ?? []) : [...(j.data ?? [])].sort((a, b) => (a.index ?? 0) - (b.index ?? 0)).map((d) => d.embedding)
|
return [...(j.data ?? [])].sort((a, b) => (a.index ?? 0) - (b.index ?? 0)).map((d) => unit(d.embedding))
|
||||||
return rows.map(unit)
|
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
+1
-1
@@ -8,7 +8,7 @@ const CONFIG_TEMPLATE = `# LLeMbas CLI settings for this project. Merged over ~/
|
|||||||
# and read only while the project is trusted. Connections cannot be set here.
|
# and read only while the project is trusted. Connections cannot be set here.
|
||||||
#
|
#
|
||||||
# mode: edit # manual | edit | auto | plan
|
# mode: edit # manual | edit | auto | plan
|
||||||
# model: connection/model
|
# model: provider/model
|
||||||
# permission:
|
# permission:
|
||||||
# bash:
|
# bash:
|
||||||
# "npm test *": allow
|
# "npm test *": allow
|
||||||
|
|||||||
@@ -1,228 +0,0 @@
|
|||||||
// Anthropic Messages (`POST {base}/messages`) — api.anthropic.com, and servers that imitate it
|
|
||||||
// (vLLM, llama-server). Content blocks both ways; thinking with signatures that must be sent back
|
|
||||||
// on the next turn; optional prompt-caching breakpoints.
|
|
||||||
import type { Effort } from "../config/schema.ts"
|
|
||||||
import { resolveKey } from "../config/load.ts"
|
|
||||||
import { Parts, record, recordFailure, retrying, statusOf, SwapBanner } from "./common.ts"
|
|
||||||
import { authHeaders, joinUrl, request, tlsFor } from "./http.ts"
|
|
||||||
import { sseJson } from "./sse.ts"
|
|
||||||
import { estimateTokens, estimateValue } from "./tokens.ts"
|
|
||||||
import { ProviderError, type ChatRequest, type Client, type DiscoveredModel, type Message, type ResolvedModel, type StreamEvent, type ToolCallPart } from "./types.ts"
|
|
||||||
|
|
||||||
/** Thinking budgets by effort, when the model's config gives no effort_map. */
|
|
||||||
export const DEFAULT_BUDGETS: Record<Effort, number> = { minimal: 1024, low: 2048, medium: 8192, high: 16384, xhigh: 32000, max: 48000 }
|
|
||||||
|
|
||||||
type Block = Record<string, unknown>
|
|
||||||
type AMessage = { role: "user" | "assistant"; content: Block[] }
|
|
||||||
|
|
||||||
/** Anthropic ids must match ^[a-zA-Z0-9_-]+$; ids from other providers are mapped consistently. */
|
|
||||||
export const safeId = (id: string) => id.replace(/[^a-zA-Z0-9_-]/g, "_") || "call"
|
|
||||||
|
|
||||||
export function toAnthropicMessages(messages: Message[], vision: boolean, thinking: boolean): AMessage[] {
|
|
||||||
const out: AMessage[] = []
|
|
||||||
const push = (role: AMessage["role"], blocks: Block[]) => {
|
|
||||||
if (!blocks.length) return
|
|
||||||
const last = out[out.length - 1]
|
|
||||||
// Consecutive turns of one role are merged: tool results and a following note are one user turn.
|
|
||||||
if (last && last.role === role) last.content.push(...blocks)
|
|
||||||
else out.push({ role, content: blocks })
|
|
||||||
}
|
|
||||||
for (const m of messages) {
|
|
||||||
if (m.role === "user") {
|
|
||||||
push(
|
|
||||||
"user",
|
|
||||||
m.parts.map((p) =>
|
|
||||||
p.type === "text"
|
|
||||||
? { type: "text", text: p.text }
|
|
||||||
: vision
|
|
||||||
? { type: "image", source: { type: "base64", media_type: p.mime, data: p.data } }
|
|
||||||
: { type: "text", text: "[image omitted: this model has no vision]" },
|
|
||||||
),
|
|
||||||
)
|
|
||||||
} else if (m.role === "assistant") {
|
|
||||||
const blocks: Block[] = []
|
|
||||||
for (const p of m.parts) {
|
|
||||||
if (p.type === "reasoning") {
|
|
||||||
// A thinking block can only go back with its signature, and only while thinking is on.
|
|
||||||
if (!thinking) continue
|
|
||||||
const redacted = (p.opaque as { redacted?: string } | undefined)?.redacted
|
|
||||||
if (redacted) blocks.push({ type: "redacted_thinking", data: redacted })
|
|
||||||
else if (p.signature) blocks.push({ type: "thinking", thinking: p.text, signature: p.signature })
|
|
||||||
} else if (p.type === "text") {
|
|
||||||
if (p.text.trim()) blocks.push({ type: "text", text: p.text })
|
|
||||||
} else {
|
|
||||||
let input: unknown = {}
|
|
||||||
try {
|
|
||||||
input = p.args.trim() ? JSON.parse(p.args) : {}
|
|
||||||
} catch {
|
|
||||||
input = { _unparsed: p.args }
|
|
||||||
}
|
|
||||||
blocks.push({ type: "tool_use", id: safeId(p.id), name: p.name, input })
|
|
||||||
}
|
|
||||||
}
|
|
||||||
push("assistant", blocks)
|
|
||||||
} else {
|
|
||||||
push("user", [{ type: "tool_result", tool_use_id: safeId(m.callId), content: m.content || "(empty)", ...(m.isError ? { is_error: true } : {}) }])
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return out
|
|
||||||
}
|
|
||||||
|
|
||||||
export class AnthropicClient implements Client {
|
|
||||||
constructor(private m: ResolvedModel) {}
|
|
||||||
|
|
||||||
private headers() {
|
|
||||||
return authHeaders(this.m.connection, resolveKey(this.m.connectionName, this.m.connection), { "anthropic-version": "2023-06-01", ...this.m.spec.headers })
|
|
||||||
}
|
|
||||||
|
|
||||||
buildBody(req: ChatRequest): Record<string, unknown> {
|
|
||||||
const { m } = this
|
|
||||||
const budget = req.effort ? (m.spec.effort_map?.[req.effort] ?? DEFAULT_BUDGETS[req.effort]) : undefined
|
|
||||||
let maxTokens = m.spec.max_output ?? 8192
|
|
||||||
if (budget && maxTokens <= budget) maxTokens = budget + Math.min(8192, maxTokens)
|
|
||||||
const cache = m.spec.cache === true
|
|
||||||
const body: Record<string, unknown> = {
|
|
||||||
model: m.id,
|
|
||||||
max_tokens: maxTokens,
|
|
||||||
stream: true,
|
|
||||||
messages: toAnthropicMessages(req.messages, m.spec.vision === true, !!budget),
|
|
||||||
...m.connection.body,
|
|
||||||
...m.spec.body,
|
|
||||||
}
|
|
||||||
if (req.system) body.system = cache ? [{ type: "text", text: req.system, cache_control: { type: "ephemeral" } }] : req.system
|
|
||||||
if (req.tools.length && m.spec.tools !== false) {
|
|
||||||
const tools = req.tools.map((t) => ({ name: t.name, description: t.description, input_schema: t.parameters }))
|
|
||||||
if (cache) (tools[tools.length - 1] as Block).cache_control = { type: "ephemeral" }
|
|
||||||
body.tools = tools
|
|
||||||
}
|
|
||||||
if (cache) {
|
|
||||||
const msgs = body.messages as AMessage[]
|
|
||||||
const last = msgs[msgs.length - 1]?.content
|
|
||||||
if (last?.length) last[last.length - 1] = { ...last[last.length - 1], cache_control: { type: "ephemeral" } }
|
|
||||||
}
|
|
||||||
if (budget) body.thinking = { type: "enabled", budget_tokens: budget }
|
|
||||||
else {
|
|
||||||
// Sampling parameters and extended thinking do not mix on this API.
|
|
||||||
if (m.spec.temperature !== undefined) body.temperature = m.spec.temperature
|
|
||||||
if (m.spec.top_p !== undefined) body.top_p = m.spec.top_p
|
|
||||||
}
|
|
||||||
return body
|
|
||||||
}
|
|
||||||
|
|
||||||
async *stream(req: ChatRequest): AsyncGenerator<StreamEvent> {
|
|
||||||
const body = this.buildBody(req)
|
|
||||||
yield* retrying(() => this.attempt(req, body))
|
|
||||||
}
|
|
||||||
|
|
||||||
private async *attempt(req: ChatRequest, body: Record<string, unknown>): AsyncGenerator<StreamEvent> {
|
|
||||||
const { m } = this
|
|
||||||
const res = await request(
|
|
||||||
joinUrl(m.connection.base_url, "messages"),
|
|
||||||
{ method: "POST", headers: this.headers(), body: JSON.stringify(body), signal: req.signal, timeoutMs: (m.connection.timeout ?? 600) * 1000, tls: tlsFor(m.connection) },
|
|
||||||
m.ref,
|
|
||||||
).catch((e) => {
|
|
||||||
recordFailure(m.ref, body, e)
|
|
||||||
throw e
|
|
||||||
})
|
|
||||||
if (!res.body) throw new ProviderError(`${m.ref}: empty response`)
|
|
||||||
|
|
||||||
const out = new Parts()
|
|
||||||
const banner = new SwapBanner()
|
|
||||||
type Open = { type: string; text: string; signature?: string; id?: string; name?: string; json: string; data?: string; index: number }
|
|
||||||
const blocks = new Map<number, Open>()
|
|
||||||
let toolIndex = 0
|
|
||||||
let finish = "stop"
|
|
||||||
let input = 0
|
|
||||||
let output = 0
|
|
||||||
let cached: number | undefined
|
|
||||||
let sawUsage = false
|
|
||||||
|
|
||||||
for await (const { data } of sseJson(record(res.body, m.ref, body))) {
|
|
||||||
const ev = data as any
|
|
||||||
switch (ev?.type) {
|
|
||||||
case "message_start": {
|
|
||||||
const u = ev.message?.usage
|
|
||||||
if (u) {
|
|
||||||
sawUsage = true
|
|
||||||
cached = u.cache_read_input_tokens || undefined
|
|
||||||
input = (u.input_tokens ?? 0) + (u.cache_read_input_tokens ?? 0) + (u.cache_creation_input_tokens ?? 0)
|
|
||||||
output = u.output_tokens ?? 0
|
|
||||||
}
|
|
||||||
break
|
|
||||||
}
|
|
||||||
case "content_block_start": {
|
|
||||||
const b = ev.content_block ?? {}
|
|
||||||
blocks.set(ev.index, { type: b.type, text: b.text ?? b.thinking ?? "", id: b.id, name: b.name, json: "", data: b.data, index: ev.index })
|
|
||||||
if (b.type === "tool_use") yield { type: "tool_call_delta", index: toolIndex, name: b.name, argsDelta: "" }
|
|
||||||
if (b.type === "text" && b.text) {
|
|
||||||
out.add("text", b.text)
|
|
||||||
yield { type: "text", text: b.text }
|
|
||||||
}
|
|
||||||
break
|
|
||||||
}
|
|
||||||
case "content_block_delta": {
|
|
||||||
const blk = blocks.get(ev.index)
|
|
||||||
const d = ev.delta ?? {}
|
|
||||||
if (!blk) break
|
|
||||||
if (d.type === "text_delta") {
|
|
||||||
out.add("text", d.text)
|
|
||||||
yield { type: "text", text: d.text }
|
|
||||||
} else if (d.type === "thinking_delta") {
|
|
||||||
const b = banner.feed(d.thinking ?? "")
|
|
||||||
if (b.notice) yield { type: "notice", message: b.notice }
|
|
||||||
if (b.text) {
|
|
||||||
blk.text += b.text
|
|
||||||
if (b.text.trim()) out.output = true
|
|
||||||
yield { type: "reasoning", text: b.text }
|
|
||||||
}
|
|
||||||
} else if (d.type === "signature_delta") blk.signature = (blk.signature ?? "") + d.signature
|
|
||||||
else if (d.type === "input_json_delta") {
|
|
||||||
blk.json += d.partial_json ?? ""
|
|
||||||
out.output = true
|
|
||||||
yield { type: "tool_call_delta", index: toolIndex, argsDelta: d.partial_json ?? "" }
|
|
||||||
}
|
|
||||||
break
|
|
||||||
}
|
|
||||||
case "content_block_stop": {
|
|
||||||
const blk = blocks.get(ev.index)
|
|
||||||
if (!blk) break
|
|
||||||
if (blk.type === "thinking" && blk.text) out.push({ type: "reasoning", text: blk.text, signature: blk.signature })
|
|
||||||
else if (blk.type === "redacted_thinking") out.push({ type: "reasoning", text: "", opaque: { redacted: blk.data } })
|
|
||||||
else if (blk.type === "tool_use") {
|
|
||||||
const call: ToolCallPart = { type: "tool_call", id: blk.id || `toolu_${toolIndex}`, name: blk.name ?? "", args: blk.json || "{}" }
|
|
||||||
out.push(call)
|
|
||||||
toolIndex++
|
|
||||||
}
|
|
||||||
blocks.delete(ev.index)
|
|
||||||
break
|
|
||||||
}
|
|
||||||
case "message_delta": {
|
|
||||||
if (ev.delta?.stop_reason) finish = ev.delta.stop_reason
|
|
||||||
if (ev.usage?.output_tokens !== undefined) {
|
|
||||||
sawUsage = true
|
|
||||||
output = ev.usage.output_tokens
|
|
||||||
}
|
|
||||||
break
|
|
||||||
}
|
|
||||||
case "error": {
|
|
||||||
const err = ev.error ?? {}
|
|
||||||
throw new ProviderError(`${m.ref}: ${err.message ?? JSON.stringify(err)}`, statusOf(err) ?? (err.type === "overloaded_error" ? 529 : undefined), undefined, !out.output)
|
|
||||||
}
|
|
||||||
default:
|
|
||||||
if (ev?.error) throw new ProviderError(`${m.ref}: ${ev.error.message ?? JSON.stringify(ev.error)}`, statusOf(ev.error), undefined, !out.output)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
const message = out.message()
|
|
||||||
const reason = finish === "max_tokens" ? "length" : finish === "tool_use" || message.parts.some((p) => p.type === "tool_call") ? "tool_calls" : "stop"
|
|
||||||
yield sawUsage
|
|
||||||
? { type: "usage", usage: { input, output, cached } }
|
|
||||||
: { type: "usage", usage: { input: estimateValue(body.messages), output: message.parts.reduce((n, p) => n + estimateTokens(p.type === "tool_call" ? p.args : p.text), 0), estimated: true } }
|
|
||||||
yield { type: "finish", reason, message }
|
|
||||||
}
|
|
||||||
|
|
||||||
async listModels(): Promise<DiscoveredModel[]> {
|
|
||||||
const res = await request(joinUrl(this.m.connection.base_url, "models"), { headers: this.headers(), timeoutMs: 15_000, tls: tlsFor(this.m.connection) }, this.m.connectionName)
|
|
||||||
const j = (await res.json()) as any
|
|
||||||
return (Array.isArray(j?.data) ? j.data : []).map((x: any) => ({ id: String(x.id), context: x.context_window ?? x.max_input_tokens })).filter((x: DiscoveredModel) => x.id)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,53 +0,0 @@
|
|||||||
// A model's context window when the config does not give one. Config always wins; what is found
|
|
||||||
// here is kept in learned.json. Tried in order: the server's model list; llama-server's /props
|
|
||||||
// (directly, or through llama-swap's /upstream/<model>/ passthrough); Ollama's /api/show.
|
|
||||||
import { learned, learnContext } from "./learned.ts"
|
|
||||||
import { authHeaders, request, tlsFor } from "./http.ts"
|
|
||||||
import { OllamaClient } from "./ollama.ts"
|
|
||||||
import type { Client, ResolvedModel } from "./types.ts"
|
|
||||||
import { resolveKey } from "../config/load.ts"
|
|
||||||
|
|
||||||
/** The window to plan with: configured, else learned, else undefined (unknown). */
|
|
||||||
export function contextWindow(m: ResolvedModel): number | undefined {
|
|
||||||
return m.spec.context || learned().contexts[m.ref] || undefined
|
|
||||||
}
|
|
||||||
|
|
||||||
export async function discoverContext(m: ResolvedModel, client: Client): Promise<number | undefined> {
|
|
||||||
if (m.spec.context) return m.spec.context
|
|
||||||
const known = learned().contexts[m.ref]
|
|
||||||
if (known) return known
|
|
||||||
const found = (await fromList(m, client)) ?? (await fromProps(m)) ?? (m.connection.dialect === "ollama" ? await (client as OllamaClient).contextOf(m.id).catch(() => undefined) : undefined)
|
|
||||||
if (found && found > 0) learnContext(m.ref, found)
|
|
||||||
return found
|
|
||||||
}
|
|
||||||
|
|
||||||
async function fromList(m: ResolvedModel, client: Client): Promise<number | undefined> {
|
|
||||||
try {
|
|
||||||
return (await client.listModels()).find((d) => d.id === m.id)?.context
|
|
||||||
} catch {
|
|
||||||
return undefined
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
async function fromProps(m: ResolvedModel): Promise<number | undefined> {
|
|
||||||
const base = m.connection.base_url.replace(/\/+$/, "")
|
|
||||||
const origin = base.replace(/\/v1$/, "")
|
|
||||||
const headers = authHeaders(m.connection, resolveKey(m.connectionName, m.connection))
|
|
||||||
for (const url of [`${origin}/upstream/${encodeURIComponent(m.id)}/props`, `${origin}/props`]) {
|
|
||||||
try {
|
|
||||||
const res = await request(url, { headers, timeoutMs: 15_000, tls: tlsFor(m.connection) }, m.ref)
|
|
||||||
const j = (await res.json()) as any
|
|
||||||
const n = Number(j?.default_generation_settings?.n_ctx ?? j?.n_ctx)
|
|
||||||
if (Number.isFinite(n) && n > 0) return n
|
|
||||||
} catch {}
|
|
||||||
}
|
|
||||||
return undefined
|
|
||||||
}
|
|
||||||
|
|
||||||
/** llama-swap and friends: free the connection's memory when the session moves elsewhere. */
|
|
||||||
export async function unload(m: ResolvedModel): Promise<void> {
|
|
||||||
if (!m.connection.unload_url) return
|
|
||||||
try {
|
|
||||||
await request(m.connection.unload_url, { method: m.connection.unload_method ?? "POST", headers: authHeaders(m.connection, resolveKey(m.connectionName, m.connection)), timeoutMs: 10_000, tls: tlsFor(m.connection) }, m.ref)
|
|
||||||
} catch {}
|
|
||||||
}
|
|
||||||
@@ -1,316 +0,0 @@
|
|||||||
// Google Gemini, native (`{base}/models/{id}:streamGenerateContent?alt=sse`), base
|
|
||||||
// https://generativelanguage.googleapis.com/v1beta. UNTESTED: no Gemini endpoint has been
|
|
||||||
// available to try it against (see the wiki's Working-notes).
|
|
||||||
import type { Effort } from "../config/schema.ts"
|
|
||||||
import { resolveKey } from "../config/load.ts"
|
|
||||||
import { DEFAULT_BUDGETS } from "./anthropic.ts"
|
|
||||||
import { Parts, record, recordFailure, retrying, statusOf } from "./common.ts"
|
|
||||||
import { authHeaders, joinUrl, request, tlsFor } from "./http.ts"
|
|
||||||
import { sseJson } from "./sse.ts"
|
|
||||||
import { estimateTokens, estimateValue } from "./tokens.ts"
|
|
||||||
import { ProviderError, type ChatRequest, type Client, type DiscoveredModel, type Message, type ResolvedModel, type StreamEvent, type ToolCallPart } from "./types.ts"
|
|
||||||
|
|
||||||
type Content = { role: "user" | "model"; parts: Record<string, unknown>[] }
|
|
||||||
|
|
||||||
// Checked against Hermes Agent (agent/gemini_native_adapter.py, agent/gemini_schema.py) and
|
|
||||||
// OpenCode (provider/transform.ts). Still untested against a real Gemini endpoint.
|
|
||||||
|
|
||||||
/** Gemini's own id, without an aggregator prefix. */
|
|
||||||
const bare = (id: string) => id.replace(/^(models|google|gemini)\//i, "")
|
|
||||||
/** Gemini 3 and later: tool calls and results carry ids, and thinking is a level, not a budget. */
|
|
||||||
export const isGemini3 = (id: string) => Number(/gemini-(\d+)/i.exec(bare(id))?.[1] ?? 0) >= 3
|
|
||||||
const LEGACY = /gemini-(?:(?:flash|pro)-)?[12](?:[.-]|$)/i
|
|
||||||
/** A call replayed without a signature (made by another model, or before signatures existed):
|
|
||||||
* Google's documented sentinel, without which Gemini 3 refuses the whole history. */
|
|
||||||
export const SKIP_SIGNATURE = "skip_thought_signature_validator"
|
|
||||||
/** Thought tokens count against maxOutputTokens, and Gemini's own default is low. */
|
|
||||||
export const DEFAULT_MAX_OUTPUT = 65535
|
|
||||||
const PLACEHOLDER = "[The previous response was interrupted before it completed.]"
|
|
||||||
|
|
||||||
// ── tool schemas ──────────────────────────────────────────────────────────────────────────────
|
|
||||||
// v1beta takes plain JSON Schema as `parametersJsonSchema`; only same-document $refs have to be
|
|
||||||
// inlined. Other API versions take `parameters`, an OpenAPI subset where one unknown key 400s the
|
|
||||||
// whole request, so that path drops everything outside the subset.
|
|
||||||
|
|
||||||
const ALLOWED = new Set(["type", "format", "title", "description", "nullable", "enum", "maxItems", "minItems", "properties", "required", "minProperties", "maxProperties", "minLength", "maxLength", "pattern", "example", "anyOf", "propertyOrdering", "default", "items", "minimum", "maximum"])
|
|
||||||
const STRUCTURAL: Record<string, string[]> = { array: ["items", "minItems", "maxItems"], object: ["properties", "required", "minProperties", "maxProperties", "propertyOrdering"] }
|
|
||||||
const isObj = (v: unknown): v is Record<string, unknown> => !!v && typeof v === "object" && !Array.isArray(v)
|
|
||||||
|
|
||||||
export function geminiSchema(s: unknown): Record<string, unknown> {
|
|
||||||
if (!isObj(s)) return {}
|
|
||||||
const out: Record<string, unknown> = {}
|
|
||||||
for (const [k, v] of Object.entries(s)) {
|
|
||||||
if (!ALLOWED.has(k)) continue
|
|
||||||
if (k === "properties") {
|
|
||||||
if (isObj(v)) out.properties = Object.fromEntries(Object.entries(v).map(([p, sub]) => [p, geminiSchema(sub)]))
|
|
||||||
} else if (k === "items") out.items = geminiSchema(v)
|
|
||||||
else if (k === "anyOf") {
|
|
||||||
if (Array.isArray(v)) out.anyOf = v.filter(isObj).map(geminiSchema)
|
|
||||||
} else out[k] = v
|
|
||||||
}
|
|
||||||
const types = out.type
|
|
||||||
if (Array.isArray(types)) {
|
|
||||||
delete out.type
|
|
||||||
const real = types.filter((t): t is string => typeof t === "string" && t !== "null")
|
|
||||||
if (!real.length) out.type = types.includes("null") ? "null" : "object"
|
|
||||||
else if (real.length === 1) out.type = real[0]
|
|
||||||
else {
|
|
||||||
// Several types: one branch each, and the structure belongs on the branch it describes.
|
|
||||||
const moved: Record<string, unknown> = {}
|
|
||||||
for (const keys of Object.values(STRUCTURAL)) for (const k of keys) if (k in out) (moved[k] = out[k]), delete out[k]
|
|
||||||
out.anyOf = real.map((t) => geminiSchema({ type: t, ...Object.fromEntries(Object.entries(moved).filter(([k]) => STRUCTURAL[t]?.includes(k))) }))
|
|
||||||
}
|
|
||||||
if (types.includes("null")) out.nullable = true
|
|
||||||
}
|
|
||||||
// Every enum entry must be a string, whatever the declared type.
|
|
||||||
if (Array.isArray(out.enum) && (Array.isArray(types) || ["integer", "number", "boolean"].includes(out.type as string))) {
|
|
||||||
const vals = [...new Set(out.enum.map((e) => (typeof e === "boolean" || (typeof e === "number" && Number.isFinite(e)) ? String(e) : typeof e === "string" ? e : undefined)).filter((e) => e !== undefined))]
|
|
||||||
if (vals.length) out.enum = vals
|
|
||||||
else delete out.enum
|
|
||||||
}
|
|
||||||
// `required` naming a property that is not there fails the entire request.
|
|
||||||
if (Array.isArray(out.required)) {
|
|
||||||
const names = isObj(out.properties) ? out.properties : {}
|
|
||||||
const req = out.required.filter((r): r is string => typeof r === "string" && r in names)
|
|
||||||
if (req.length) out.required = req
|
|
||||||
else delete out.required
|
|
||||||
}
|
|
||||||
return out
|
|
||||||
}
|
|
||||||
|
|
||||||
/** For `parameters`: the subset, and never an empty schema. */
|
|
||||||
export function geminiParameters(s: unknown): Record<string, unknown> {
|
|
||||||
const out = geminiSchema(s)
|
|
||||||
return Object.keys(out).length ? out : { type: "object", properties: {} }
|
|
||||||
}
|
|
||||||
|
|
||||||
function inlineRefs(node: unknown, root: Record<string, unknown>, budget: { n: number }, stack: string[] = []): unknown {
|
|
||||||
if (Array.isArray(node)) return node.map((x) => inlineRefs(x, root, budget, stack))
|
|
||||||
if (!isObj(node)) return node
|
|
||||||
const ref = node.$ref
|
|
||||||
if (typeof ref !== "string") return Object.fromEntries(Object.entries(node).map(([k, v]) => [k, inlineRefs(v, root, budget, stack)]))
|
|
||||||
if (stack.includes(ref)) throw new Error(`circular $ref ${ref}`)
|
|
||||||
if (--budget.n < 0) throw new Error("$ref budget exhausted")
|
|
||||||
if (!ref.startsWith("#/")) throw new Error(`unresolvable $ref ${ref}`)
|
|
||||||
let target: unknown = root
|
|
||||||
for (const raw of ref.slice(2).split("/")) {
|
|
||||||
const part = raw.replace(/~1/g, "/").replace(/~0/g, "~")
|
|
||||||
target = isObj(target) ? target[part] : undefined
|
|
||||||
}
|
|
||||||
if (!isObj(target)) throw new Error(`unresolvable $ref ${ref}`)
|
|
||||||
const inlined = inlineRefs(target, root, budget, [...stack, ref]) as Record<string, unknown>
|
|
||||||
// Siblings of a $ref apply alongside it and win.
|
|
||||||
const { $ref: _, ...siblings } = node
|
|
||||||
return Object.keys(siblings).length ? { ...inlined, ...(inlineRefs(siblings, root, budget, stack) as object) } : inlined
|
|
||||||
}
|
|
||||||
|
|
||||||
/** For `parametersJsonSchema`: root $schema dropped, local $refs inlined, an object root. A schema
|
|
||||||
* whose refs cannot all be resolved goes as it is, so the provider names the real problem. */
|
|
||||||
export function geminiJsonSchema(s: unknown): Record<string, unknown> {
|
|
||||||
if (!isObj(s) || !Object.keys(s).length) return { type: "object", properties: {} }
|
|
||||||
const { $schema: _, ...rest } = structuredClone(s)
|
|
||||||
let out: Record<string, unknown>
|
|
||||||
try {
|
|
||||||
out = inlineRefs(rest, rest, { n: 256 }) as Record<string, unknown>
|
|
||||||
} catch {
|
|
||||||
return rest
|
|
||||||
}
|
|
||||||
delete out.$defs
|
|
||||||
delete out.definitions
|
|
||||||
if (!Object.keys(out).length) return { type: "object", properties: {} }
|
|
||||||
if (out.type === "object" && !("properties" in out)) out.properties = {}
|
|
||||||
return out
|
|
||||||
}
|
|
||||||
|
|
||||||
/** `parametersJsonSchema` exists only on v1beta. */
|
|
||||||
export const acceptsJsonSchema = (base: string) => /\/v1beta\/*$/i.test(base.trim())
|
|
||||||
|
|
||||||
// ── history ───────────────────────────────────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
/** A JSON Schema inside a function response is resolved by Gemini 3, and an unknown $ref 400s. */
|
|
||||||
const hasRef = (v: unknown): boolean => (Array.isArray(v) ? v.some(hasRef) : isObj(v) ? Object.entries(v).some(([k, x]) => (k === "$ref" && typeof x === "string" && x.startsWith("#/")) || hasRef(x)) : false)
|
|
||||||
|
|
||||||
function functionResponse(content: string, isError: boolean | undefined): Record<string, unknown> {
|
|
||||||
if (isError) return { error: content }
|
|
||||||
// A JSON object result goes structured; anything else under "output", as Google documents.
|
|
||||||
if (/^\s*\{/.test(content))
|
|
||||||
try {
|
|
||||||
const parsed = JSON.parse(content)
|
|
||||||
if (isObj(parsed) && !hasRef(parsed)) return parsed
|
|
||||||
} catch {}
|
|
||||||
return { output: content }
|
|
||||||
}
|
|
||||||
|
|
||||||
export function toGeminiContents(messages: Message[], vision: boolean, ids = false): Content[] {
|
|
||||||
const out: Content[] = []
|
|
||||||
const names = new Map<string, string>()
|
|
||||||
const results = (c: Content) => c.parts.some((p) => "functionResponse" in p)
|
|
||||||
const push = (role: Content["role"], parts: Record<string, unknown>[]) => {
|
|
||||||
if (!parts.length) return
|
|
||||||
const last = out[out.length - 1]
|
|
||||||
// Same-role turns must merge (Gemini insists on alternation), except that a person's words are
|
|
||||||
// never folded into a turn of tool results: Gemini 3 then reads them as part of the result and
|
|
||||||
// answers with nothing. A placeholder model turn keeps the two apart.
|
|
||||||
if (last && last.role === role && role === "user" && results(last) !== parts.some((p) => "functionResponse" in p)) out.push({ role: "model", parts: [{ text: PLACEHOLDER }] }, { role, parts })
|
|
||||||
else if (last && last.role === role) last.parts.push(...parts)
|
|
||||||
else out.push({ role, parts })
|
|
||||||
}
|
|
||||||
for (const m of messages) {
|
|
||||||
if (m.role === "user")
|
|
||||||
push(
|
|
||||||
"user",
|
|
||||||
m.parts.map((p) => (p.type === "text" ? { text: p.text } : vision ? { inlineData: { mimeType: p.mime, data: p.data } } : { text: "[image omitted: this model has no vision]" })),
|
|
||||||
)
|
|
||||||
else if (m.role === "assistant") {
|
|
||||||
const parts: Record<string, unknown>[] = []
|
|
||||||
for (const p of m.parts) {
|
|
||||||
if (p.type === "text" && p.text.trim()) parts.push({ text: p.text })
|
|
||||||
else if (p.type === "tool_call") {
|
|
||||||
names.set(p.id, p.name)
|
|
||||||
let args: unknown = {}
|
|
||||||
try {
|
|
||||||
args = p.args.trim() ? JSON.parse(p.args) : {}
|
|
||||||
} catch {
|
|
||||||
args = { _raw: p.args }
|
|
||||||
}
|
|
||||||
// Gemini 3 wants the thought signature back on the call it came with.
|
|
||||||
parts.push({ functionCall: { name: p.name, args: isObj(args) ? args : { _value: args }, ...(ids ? { id: p.id } : {}) }, thoughtSignature: p.signature ?? SKIP_SIGNATURE })
|
|
||||||
}
|
|
||||||
}
|
|
||||||
push("model", parts)
|
|
||||||
} else push("user", [{ functionResponse: { name: names.get(m.callId) ?? m.name, response: functionResponse(m.content, m.isError), ...(ids ? { id: m.callId } : {}) } }])
|
|
||||||
}
|
|
||||||
return out
|
|
||||||
}
|
|
||||||
|
|
||||||
// ── thinking ──────────────────────────────────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
const LEVELS = ["minimal", "low", "medium", "high"] as const
|
|
||||||
/** The levels a Gemini 3 model takes (OpenCode's table). */
|
|
||||||
function levelsOf(id: string): readonly string[] {
|
|
||||||
const s = id.toLowerCase()
|
|
||||||
if (s.includes("gemma") || s.includes("flash-image")) return ["minimal", "high"]
|
|
||||||
if (s.includes("pro-image")) return ["high"]
|
|
||||||
if (s.includes("flash")) return LEVELS
|
|
||||||
return ["low", "medium", "high"]
|
|
||||||
}
|
|
||||||
|
|
||||||
export function thinkingConfig(id: string, effort: Effort, map?: Partial<Record<Effort, number>>): Record<string, unknown> {
|
|
||||||
if (map?.[effort]) return { thinkingBudget: map[effort], includeThoughts: true }
|
|
||||||
const b = bare(id)
|
|
||||||
if (LEGACY.test(b)) {
|
|
||||||
// 2.x takes a budget; 2.5 Pro goes to 32k, the rest to 24k.
|
|
||||||
const max = /gemini-2[.-]5/i.test(b) && /pro/i.test(b) && !/flash/i.test(b) ? 32768 : 24576
|
|
||||||
return { thinkingBudget: Math.min(DEFAULT_BUDGETS[effort], max), includeThoughts: true }
|
|
||||||
}
|
|
||||||
// 3 and later take a level: the one asked for, else the next one up the model has, else its top.
|
|
||||||
const i = LEVELS.indexOf(effort as (typeof LEVELS)[number])
|
|
||||||
const want = i < 0 ? LEVELS.length - 1 : i // xhigh and max: the top
|
|
||||||
const have = levelsOf(b)
|
|
||||||
const level = LEVELS.slice(want).find((l) => have.includes(l)) ?? have[have.length - 1]
|
|
||||||
return { thinkingLevel: level, includeThoughts: true }
|
|
||||||
}
|
|
||||||
|
|
||||||
/** Keys sorted, so a call sent again compares equal to itself (as Hermes does). */
|
|
||||||
const sortedJson = (v: unknown) => JSON.stringify(v, (_, x) => (isObj(x) ? Object.fromEntries(Object.keys(x).sort().map((k) => [k, x[k]])) : x))
|
|
||||||
|
|
||||||
const FINISH: Record<string, string> = { STOP: "stop", MAX_TOKENS: "length", SAFETY: "content_filter", RECITATION: "content_filter", BLOCKLIST: "content_filter", PROHIBITED_CONTENT: "content_filter", SPII: "content_filter" }
|
|
||||||
|
|
||||||
export class GeminiClient implements Client {
|
|
||||||
constructor(private m: ResolvedModel) {}
|
|
||||||
|
|
||||||
private headers() {
|
|
||||||
return authHeaders(this.m.connection, resolveKey(this.m.connectionName, this.m.connection), this.m.spec.headers)
|
|
||||||
}
|
|
||||||
|
|
||||||
buildBody(req: ChatRequest): Record<string, unknown> {
|
|
||||||
const { m } = this
|
|
||||||
const generationConfig: Record<string, unknown> = {}
|
|
||||||
if (m.spec.temperature !== undefined) generationConfig.temperature = m.spec.temperature
|
|
||||||
generationConfig.maxOutputTokens = m.spec.max_output ?? DEFAULT_MAX_OUTPUT
|
|
||||||
if (m.spec.top_p !== undefined) generationConfig.topP = m.spec.top_p
|
|
||||||
if (req.effort) generationConfig.thinkingConfig = thinkingConfig(m.id, req.effort, m.spec.effort_map)
|
|
||||||
const body: Record<string, unknown> = { contents: toGeminiContents(req.messages, m.spec.vision === true, isGemini3(m.id)), generationConfig, ...m.connection.body, ...m.spec.body }
|
|
||||||
if (req.system) body.systemInstruction = { parts: [{ text: req.system }] }
|
|
||||||
if (req.tools.length && m.spec.tools !== false) {
|
|
||||||
const json = acceptsJsonSchema(m.connection.base_url)
|
|
||||||
body.tools = [{ functionDeclarations: req.tools.map((t) => ({ name: t.name, description: t.description, ...(json ? { parametersJsonSchema: geminiJsonSchema(t.parameters) } : { parameters: geminiParameters(t.parameters) }) })) }]
|
|
||||||
}
|
|
||||||
return body
|
|
||||||
}
|
|
||||||
|
|
||||||
async *stream(req: ChatRequest): AsyncGenerator<StreamEvent> {
|
|
||||||
const body = this.buildBody(req)
|
|
||||||
yield* retrying(() => this.attempt(req, body))
|
|
||||||
}
|
|
||||||
|
|
||||||
private async *attempt(req: ChatRequest, body: Record<string, unknown>): AsyncGenerator<StreamEvent> {
|
|
||||||
const { m } = this
|
|
||||||
const url = joinUrl(m.connection.base_url, `models/${encodeURIComponent(m.id)}:streamGenerateContent?alt=sse`)
|
|
||||||
const res = await request(url, { method: "POST", headers: this.headers(), body: JSON.stringify(body), signal: req.signal, timeoutMs: (m.connection.timeout ?? 600) * 1000, tls: tlsFor(m.connection) }, m.ref).catch((e) => {
|
|
||||||
recordFailure(m.ref, body, e)
|
|
||||||
throw e
|
|
||||||
})
|
|
||||||
if (!res.body) throw new ProviderError(`${m.ref}: empty response`)
|
|
||||||
const out = new Parts()
|
|
||||||
let finish = "stop"
|
|
||||||
let usage: StreamEvent | undefined
|
|
||||||
// One slot per call. Gemini 3 ids its calls; 2.5 does not, restarts part numbering in every
|
|
||||||
// event and may send a call again whole, so a payload repeating (or extending) a slot's arguments
|
|
||||||
// is that call and anything else is a new one.
|
|
||||||
const slots = new Map<string, { index: number; part: ToolCallPart }>()
|
|
||||||
const slotFor = (fc: any, sig: string, at: number, args: string) => {
|
|
||||||
if (typeof fc.id === "string" && fc.id) return `id:${fc.id}`
|
|
||||||
const key = `${at}:${fc.name}:${sig}`
|
|
||||||
const s = slots.get(key)
|
|
||||||
if (!s || args.startsWith(s.part.args)) return key
|
|
||||||
for (const [k, o] of slots) if (k.startsWith(`${key}#`) && args.startsWith(o.part.args)) return k
|
|
||||||
return `${key}#${slots.size}`
|
|
||||||
}
|
|
||||||
for await (const { data } of sseJson(record(res.body, m.ref, body))) {
|
|
||||||
const chunk = data as any
|
|
||||||
if (chunk?.error) throw new ProviderError(`${m.ref}: ${chunk.error.message ?? JSON.stringify(chunk.error)}`, statusOf(chunk.error) ?? chunk.error.code, undefined, !out.output)
|
|
||||||
const cand = chunk?.candidates?.[0]
|
|
||||||
const parts: any[] = cand?.content?.parts ?? []
|
|
||||||
for (const [at, p] of parts.entries()) {
|
|
||||||
if (typeof p.text === "string" && p.text) {
|
|
||||||
const kind = p.thought === true ? "reasoning" : "text"
|
|
||||||
out.add(kind, p.text)
|
|
||||||
yield { type: kind, text: p.text }
|
|
||||||
}
|
|
||||||
const fc = p.functionCall
|
|
||||||
if (fc?.name) {
|
|
||||||
const args = sortedJson(fc.args ?? {})
|
|
||||||
const sig = typeof p.thoughtSignature === "string" ? p.thoughtSignature : ""
|
|
||||||
const key = slotFor(fc, sig, at, args)
|
|
||||||
let s = slots.get(key)
|
|
||||||
if (!s) {
|
|
||||||
s = { index: slots.size, part: { type: "tool_call", id: typeof fc.id === "string" && fc.id ? fc.id : `gemini_call_${slots.size}`, name: fc.name, args: "" } }
|
|
||||||
slots.set(key, s)
|
|
||||||
out.push(s.part)
|
|
||||||
}
|
|
||||||
const prev = s.part.args
|
|
||||||
s.part.args = args
|
|
||||||
if (sig) s.part.signature = sig
|
|
||||||
yield { type: "tool_call_delta", index: s.index, name: fc.name, argsDelta: args.startsWith(prev) ? args.slice(prev.length) : args }
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if (cand?.finishReason) finish = FINISH[String(cand.finishReason).toUpperCase()] ?? "stop"
|
|
||||||
const u = chunk?.usageMetadata
|
|
||||||
if (u) usage = { type: "usage", usage: { input: u.promptTokenCount ?? 0, output: (u.candidatesTokenCount ?? 0) + (u.thoughtsTokenCount ?? 0), reasoning: u.thoughtsTokenCount, cached: u.cachedContentTokenCount } }
|
|
||||||
}
|
|
||||||
const message = out.message()
|
|
||||||
if (slots.size && finish !== "length") finish = "tool_calls"
|
|
||||||
yield usage ?? { type: "usage", usage: { input: estimateValue(body.contents), output: message.parts.reduce((n, p) => n + estimateTokens(p.type === "tool_call" ? p.args : p.text), 0), estimated: true } }
|
|
||||||
yield { type: "finish", reason: finish, message }
|
|
||||||
}
|
|
||||||
|
|
||||||
async listModels(): Promise<DiscoveredModel[]> {
|
|
||||||
const res = await request(joinUrl(this.m.connection.base_url, "models?pageSize=200"), { headers: this.headers(), timeoutMs: 15_000, tls: tlsFor(this.m.connection) }, this.m.connectionName)
|
|
||||||
const j = (await res.json()) as any
|
|
||||||
return (j?.models ?? [])
|
|
||||||
.filter((x: any) => !x.supportedGenerationMethods || x.supportedGenerationMethods.includes("generateContent"))
|
|
||||||
.map((x: any) => ({ id: String(x.name ?? "").replace(/^models\//, ""), context: x.inputTokenLimit }))
|
|
||||||
.filter((x: DiscoveredModel) => x.id)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
+3
-13
@@ -27,20 +27,10 @@ export function tlsFor(c: Pick<Connection, "tls">): { ca?: string; rejectUnautho
|
|||||||
return out
|
return out
|
||||||
}
|
}
|
||||||
|
|
||||||
const DEFAULT_AUTH: Record<Connection["dialect"], NonNullable<Connection["auth"]>> = {
|
/** The server's token as a bearer, plus the connection's and model's own headers. */
|
||||||
"openai-chat": "bearer",
|
export function authHeaders(c: Pick<Connection, "headers">, key: string | undefined, extra: Record<string, string> = {}): Record<string, string> {
|
||||||
responses: "bearer",
|
|
||||||
anthropic: "x-api-key",
|
|
||||||
gemini: "x-goog-api-key",
|
|
||||||
ollama: "bearer",
|
|
||||||
}
|
|
||||||
|
|
||||||
/** Auth plus the connection's and model's own headers. */
|
|
||||||
export function authHeaders(c: Connection, key: string | undefined, extra: Record<string, string> = {}): Record<string, string> {
|
|
||||||
const h: Record<string, string> = { "content-type": "application/json", ...extra }
|
const h: Record<string, string> = { "content-type": "application/json", ...extra }
|
||||||
const style = c.auth ?? DEFAULT_AUTH[c.dialect]
|
if (key) h.authorization = `Bearer ${key}`
|
||||||
if (key && style === "bearer") h.authorization = `Bearer ${key}`
|
|
||||||
else if (key && style !== "none") h[style] = key
|
|
||||||
return { ...h, ...c.headers }
|
return { ...h, ...c.headers }
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
+28
-46
@@ -1,64 +1,40 @@
|
|||||||
import { findRef, modelRefs } from "./refs.ts"
|
import { findRef, modelRefs } from "./refs.ts"
|
||||||
import type { Loaded } from "../config/load.ts"
|
import type { Loaded } from "../config/load.ts"
|
||||||
import { AnthropicClient } from "./anthropic.ts"
|
|
||||||
import { OpenAIChatClient } from "./openai-chat.ts"
|
import { OpenAIChatClient } from "./openai-chat.ts"
|
||||||
import { ResponsesClient } from "./responses.ts"
|
import type { Client, ResolvedModel } from "./types.ts"
|
||||||
import { GeminiClient } from "./gemini.ts"
|
|
||||||
import { OllamaClient } from "./ollama.ts"
|
|
||||||
import { ProviderError, type Client, type ResolvedModel } from "./types.ts"
|
|
||||||
|
|
||||||
export class ModelError extends Error {}
|
export class ModelError extends Error {}
|
||||||
|
|
||||||
/** `connection/model`. The model part may itself contain slashes (`openrouter/qwen/qwen3-coder`). */
|
/** What a start without a login says: LLeMbas CLI 2.0 has no model of its own, only the server's. */
|
||||||
|
export const NOT_LOGGED_IN = "Not logged in to a LLeMbas server. LLeMbas CLI uses the models of the server it is logged in to: lembas login <address> (e.g. lembas login ai.example.org)."
|
||||||
|
|
||||||
|
/** A model of the server this machine is logged in to, by its served id (`deepseek/deepseek-flash`)
|
||||||
|
* or an older form of its name (refs.ts). Without a ref: config.yaml's `model`, else the server's
|
||||||
|
* default for the account, else its first. */
|
||||||
export function resolveModel(loaded: Loaded, ref: string | undefined): ResolvedModel {
|
export function resolveModel(loaded: Loaded, ref: string | undefined): ResolvedModel {
|
||||||
const refs = loaded.refs ?? modelRefs(loaded.connections)
|
const refs = loaded.refs ?? modelRefs(loaded.connections)
|
||||||
if (!ref) {
|
const [name, connection] = Object.entries(loaded.connections)[0] ?? []
|
||||||
// A LLeMbas instance says which model its account starts on; that, before anything else.
|
if (!name || !connection) {
|
||||||
const instance = Object.entries(loaded.connections).find(([, c]) => c.webui?.default && c.models[c.webui.default])
|
const why = Object.values(loaded.broken ?? {})[0]
|
||||||
if (instance) ref = refs.find((r) => r.connection === instance[0] && r.id === instance[1].webui!.default)?.ref ?? `${instance[0]}/${instance[1].webui!.default}`
|
throw new ModelError(why ? `The LLeMbas login is unusable: ${why}` : NOT_LOGGED_IN)
|
||||||
}
|
}
|
||||||
if (!ref) {
|
if (loaded.broken?.[name]) throw new ModelError(`The LLeMbas login ${name} is unusable: ${loaded.broken[name]}`)
|
||||||
const first = refs[0]
|
if (!ref) ref = connection.webui?.default && connection.models[connection.webui.default] ? connection.webui.default : refs[0]?.ref
|
||||||
if (!first) throw new ModelError("No model configured. Add a connection to ~/.config/lembas/connections.yaml and set `model:` in config.yaml.")
|
if (!ref) throw new ModelError(`${name} serves no model to this account yet. lembas login ${connection.webui?.url ?? ""} again once it does.`.replace(" ", " "))
|
||||||
ref = first.ref
|
|
||||||
}
|
|
||||||
// Any form of the name (`deepseek/x`, or the old `example/x`): the model it names, under
|
|
||||||
// the ref it has now.
|
|
||||||
const known = findRef(loaded.connections, refs, ref)
|
const known = findRef(loaded.connections, refs, ref)
|
||||||
if (known) {
|
if (known) return { ref: known.ref, connectionName: name, connection, id: known.id, spec: connection.models[known.id] ?? {} }
|
||||||
const connection = loaded.connections[known.connection]!
|
// A model the server serves that its list did not have when it was read: the server decides
|
||||||
if (loaded.broken?.[known.connection]) throw new ModelError(`Connection "${known.connection}" is unusable: ${loaded.broken?.[known.connection]}`)
|
// which models exist, so it is asked for rather than refused here. The login's own name in front
|
||||||
return { ref: known.ref, connectionName: known.connection, connection, id: known.id, spec: connection.models[known.id] ?? {} }
|
// of it (an older form) is taken off.
|
||||||
}
|
const id = ref.startsWith(`${name}/`) ? ref.slice(name.length + 1) : ref
|
||||||
const slash = ref.indexOf("/")
|
return { ref: id, connectionName: name, connection, id, spec: {} }
|
||||||
if (slash <= 0) throw new ModelError(`Model "${ref}" must be written connection/model`)
|
|
||||||
const connectionName = ref.slice(0, slash)
|
|
||||||
const id = ref.slice(slash + 1)
|
|
||||||
if (loaded.broken?.[connectionName]) throw new ModelError(`Connection "${connectionName}" is unusable: ${loaded.broken[connectionName]}`)
|
|
||||||
const connection = loaded.connections[connectionName]
|
|
||||||
if (!connection) throw new ModelError(`No connection named "${connectionName}" (have: ${Object.keys(loaded.connections).join(", ") || "none"})`)
|
|
||||||
const spec = connection.models[id]
|
|
||||||
if (!spec && !connection.discover)
|
|
||||||
throw new ModelError(`Connection "${connectionName}" has no model "${id}" (have: ${Object.keys(connection.models).join(", ") || "none"}). Add it, or set discover: true.`)
|
|
||||||
return { ref, connectionName, connection, id, spec: spec ?? {} }
|
|
||||||
}
|
}
|
||||||
|
|
||||||
export function clientFor(m: ResolvedModel): Client {
|
export function clientFor(m: ResolvedModel): Client {
|
||||||
switch (m.connection.dialect) {
|
return new OpenAIChatClient(m)
|
||||||
case "openai-chat":
|
|
||||||
return new OpenAIChatClient(m)
|
|
||||||
case "anthropic":
|
|
||||||
return new AnthropicClient(m)
|
|
||||||
case "responses":
|
|
||||||
return new ResponsesClient(m)
|
|
||||||
case "gemini":
|
|
||||||
return new GeminiClient(m)
|
|
||||||
case "ollama":
|
|
||||||
return new OllamaClient(m)
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Which prompt family a model gets when the config does not say: anthropic, gpt, gemini, or
|
/** Which prompt family a model gets when the server does not say: anthropic, gpt, gemini, or
|
||||||
* local — every open-weight or unknown model, which gets Hermes' act-don't-describe guidance. */
|
* local — every open-weight or unknown model, which gets Hermes' act-don't-describe guidance. */
|
||||||
export function familyOf(m: ResolvedModel): string {
|
export function familyOf(m: ResolvedModel): string {
|
||||||
if (m.spec.family) return m.spec.family
|
if (m.spec.family) return m.spec.family
|
||||||
@@ -68,3 +44,9 @@ export function familyOf(m: ResolvedModel): string {
|
|||||||
if (/\bgpt|^o[134]\b|^o[134]-|codex/.test(id)) return "gpt"
|
if (/\bgpt|^o[134]\b|^o[134]-|codex/.test(id)) return "gpt"
|
||||||
return "local"
|
return "local"
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** The window to plan with, as the server gives it; undefined when it does not say (no percentage,
|
||||||
|
* no automatic compaction). */
|
||||||
|
export function contextWindow(m: ResolvedModel): number | undefined {
|
||||||
|
return m.spec.context || undefined
|
||||||
|
}
|
||||||
+5
-10
@@ -6,14 +6,12 @@ import { paths } from "../config/paths.ts"
|
|||||||
// What endpoints have taught us, kept across runs in ~/.local/state/lembas/learned.json.
|
// What endpoints have taught us, kept across runs in ~/.local/state/lembas/learned.json.
|
||||||
// Only ever narrows behaviour; deleting the file is always safe.
|
// Only ever narrows behaviour; deleting the file is always safe.
|
||||||
interface Learned {
|
interface Learned {
|
||||||
/** `connection/model` → the efforts it actually accepts. */
|
/** A model ref → the efforts it actually accepts. */
|
||||||
efforts: Record<string, Effort[]>
|
efforts: Record<string, Effort[]>
|
||||||
/** base URLs that reject `stream_options`. */
|
/** base URLs that reject `stream_options`. */
|
||||||
noStreamOptions: string[]
|
noStreamOptions: string[]
|
||||||
/** base URLs that reject `return_progress`. */
|
/** base URLs that reject `return_progress`. */
|
||||||
noProgress: string[]
|
noProgress: string[]
|
||||||
/** `connection/model` → context window found by discovery (config always wins). */
|
|
||||||
contexts: Record<string, number>
|
|
||||||
}
|
}
|
||||||
|
|
||||||
let cache: Learned | undefined
|
let cache: Learned | undefined
|
||||||
@@ -25,9 +23,11 @@ function file() {
|
|||||||
export function learned(): Learned {
|
export function learned(): Learned {
|
||||||
if (cache) return cache
|
if (cache) return cache
|
||||||
try {
|
try {
|
||||||
cache = { efforts: {}, noStreamOptions: [], noProgress: [], contexts: {}, ...JSON.parse(readFileSync(file(), "utf8")) }
|
// An older file's `contexts` (what discovery found, before 2.0 took the server's word) is dropped.
|
||||||
|
const { contexts: _, ...kept } = JSON.parse(readFileSync(file(), "utf8")) as Record<string, unknown>
|
||||||
|
cache = { efforts: {}, noStreamOptions: [], noProgress: [], ...kept } as Learned
|
||||||
} catch {
|
} catch {
|
||||||
cache = { efforts: {}, noStreamOptions: [], noProgress: [], contexts: {} }
|
cache = { efforts: {}, noStreamOptions: [], noProgress: [] }
|
||||||
}
|
}
|
||||||
return cache!
|
return cache!
|
||||||
}
|
}
|
||||||
@@ -46,11 +46,6 @@ export function learnEfforts(ref: string, efforts: Effort[]) {
|
|||||||
save()
|
save()
|
||||||
}
|
}
|
||||||
|
|
||||||
export function learnContext(ref: string, n: number) {
|
|
||||||
learned().contexts[ref] = n
|
|
||||||
save()
|
|
||||||
}
|
|
||||||
|
|
||||||
export function learnNoStreamOptions(baseUrl: string) {
|
export function learnNoStreamOptions(baseUrl: string) {
|
||||||
const l = learned()
|
const l = learned()
|
||||||
if (!l.noStreamOptions.includes(baseUrl)) l.noStreamOptions.push(baseUrl)
|
if (!l.noStreamOptions.includes(baseUrl)) l.noStreamOptions.push(baseUrl)
|
||||||
|
|||||||
@@ -1,178 +0,0 @@
|
|||||||
// Ollama, native (`POST {base}/api/chat`, NDJSON), base e.g. http://localhost:11434. Its
|
|
||||||
// OpenAI-compatible /v1 works through openai-chat too; the native API adds num_ctx (without it
|
|
||||||
// Ollama silently cuts every conversation to its small default window) and `think` levels.
|
|
||||||
// UNTESTED: no Ollama server has been available to try it against (see the wiki). Checked against
|
|
||||||
// Hermes Agent's handling (agent/model_metadata.py, agent/reasoning_params.py): /api/show decides
|
|
||||||
// the window to ask for and whether `think` may be sent at all.
|
|
||||||
import { resolveKey } from "../config/load.ts"
|
|
||||||
import { Parts, record, recordFailure, retrying, statusOf } from "./common.ts"
|
|
||||||
import { authHeaders, joinUrl, request, tlsFor } from "./http.ts"
|
|
||||||
import { estimateTokens, estimateValue } from "./tokens.ts"
|
|
||||||
import { ProviderError, type ChatRequest, type Client, type DiscoveredModel, type Message, type ResolvedModel, type StreamEvent } from "./types.ts"
|
|
||||||
|
|
||||||
export function toOllamaMessages(system: string, messages: Message[], vision: boolean): Record<string, unknown>[] {
|
|
||||||
const out: Record<string, unknown>[] = []
|
|
||||||
if (system) out.push({ role: "system", content: system })
|
|
||||||
for (const m of messages) {
|
|
||||||
if (m.role === "user") {
|
|
||||||
const images = vision ? m.parts.filter((p) => p.type === "image").map((p) => (p as { data: string }).data) : []
|
|
||||||
const text = m.parts.map((p) => (p.type === "text" ? p.text : vision ? "" : "[image omitted: this model has no vision]")).filter(Boolean).join("\n")
|
|
||||||
out.push({ role: "user", content: text, ...(images.length ? { images } : {}) })
|
|
||||||
} else if (m.role === "assistant") {
|
|
||||||
const text = m.parts.map((p) => (p.type === "text" ? p.text : "")).join("")
|
|
||||||
const thinking = m.parts.map((p) => (p.type === "reasoning" ? p.text : "")).join("")
|
|
||||||
const calls = m.parts.flatMap((p) => {
|
|
||||||
if (p.type !== "tool_call") return []
|
|
||||||
let args: unknown = {}
|
|
||||||
try {
|
|
||||||
args = p.args.trim() ? JSON.parse(p.args) : {}
|
|
||||||
} catch {}
|
|
||||||
return [{ function: { name: p.name, arguments: args } }]
|
|
||||||
})
|
|
||||||
out.push({ role: "assistant", content: text, ...(thinking ? { thinking } : {}), ...(calls.length ? { tool_calls: calls } : {}) })
|
|
||||||
} else out.push({ role: "tool", content: m.content, tool_name: m.name })
|
|
||||||
}
|
|
||||||
return out
|
|
||||||
}
|
|
||||||
|
|
||||||
/** NDJSON: one JSON object per line. */
|
|
||||||
async function* ndjson(body: ReadableStream<Uint8Array>): AsyncGenerator<unknown> {
|
|
||||||
const dec = new TextDecoder()
|
|
||||||
let buf = ""
|
|
||||||
for await (const chunk of body) {
|
|
||||||
buf += dec.decode(chunk, { stream: true })
|
|
||||||
let nl: number
|
|
||||||
while ((nl = buf.indexOf("\n")) !== -1) {
|
|
||||||
const line = buf.slice(0, nl).trim()
|
|
||||||
buf = buf.slice(nl + 1)
|
|
||||||
if (line)
|
|
||||||
try {
|
|
||||||
yield JSON.parse(line)
|
|
||||||
} catch {}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
const last = (buf + dec.decode()).trim()
|
|
||||||
if (last)
|
|
||||||
try {
|
|
||||||
yield JSON.parse(last)
|
|
||||||
} catch {}
|
|
||||||
}
|
|
||||||
|
|
||||||
/** What /api/show says about a model. */
|
|
||||||
export interface OllamaShow {
|
|
||||||
/** "completion", "tools", "thinking", "vision"…; absent before Ollama 0.6. */
|
|
||||||
capabilities?: string[]
|
|
||||||
/** The runtime window the Modelfile sets (`num_ctx`), which the owner of the server chose. */
|
|
||||||
numCtx?: number
|
|
||||||
/** The trained maximum (`model_info["<arch>.context_length"]`). */
|
|
||||||
trained?: number
|
|
||||||
}
|
|
||||||
|
|
||||||
export function parseShow(j: any): OllamaShow {
|
|
||||||
const out: OllamaShow = {}
|
|
||||||
if (Array.isArray(j?.capabilities)) out.capabilities = j.capabilities.map(String)
|
|
||||||
for (const line of String(j?.parameters ?? "").split("\n")) {
|
|
||||||
const f = line.trim().split(/\s+/)
|
|
||||||
if (f[0] === "num_ctx" && Number(f[1]) > 0) out.numCtx = Number(f[1])
|
|
||||||
}
|
|
||||||
const info = j?.model_info ?? {}
|
|
||||||
const key = Object.keys(info).find((k) => k.endsWith(".context_length"))
|
|
||||||
if (key && Number(info[key]) > 0) out.trained = Number(info[key])
|
|
||||||
return out
|
|
||||||
}
|
|
||||||
|
|
||||||
export class OllamaClient implements Client {
|
|
||||||
private shown?: Promise<OllamaShow | undefined>
|
|
||||||
constructor(private m: ResolvedModel) {}
|
|
||||||
|
|
||||||
/** /api/show for this model, asked once; undefined when the server does not answer it. */
|
|
||||||
show(id = this.m.id): Promise<OllamaShow | undefined> {
|
|
||||||
const ask = async () => {
|
|
||||||
const res = await request(joinUrl(this.m.connection.base_url, "api/show"), { method: "POST", headers: this.headers(), body: JSON.stringify({ model: id, name: id }), timeoutMs: 15_000, tls: tlsFor(this.m.connection) }, this.m.connectionName)
|
|
||||||
return parseShow(await res.json())
|
|
||||||
}
|
|
||||||
if (id !== this.m.id) return ask().catch(() => undefined)
|
|
||||||
return (this.shown ??= ask().catch(() => undefined))
|
|
||||||
}
|
|
||||||
|
|
||||||
private headers() {
|
|
||||||
return authHeaders(this.m.connection, resolveKey(this.m.connectionName, this.m.connection), this.m.spec.headers)
|
|
||||||
}
|
|
||||||
|
|
||||||
buildBody(req: ChatRequest, show?: OllamaShow): Record<string, unknown> {
|
|
||||||
const { m } = this
|
|
||||||
const options: Record<string, unknown> = {}
|
|
||||||
// Without num_ctx Ollama cuts every conversation to its small default window. Configured wins;
|
|
||||||
// else the Modelfile's, else the trained maximum (set `context` to cap it to what memory holds).
|
|
||||||
const ctx = m.spec.context ?? show?.numCtx ?? show?.trained
|
|
||||||
if (ctx) options.num_ctx = ctx
|
|
||||||
if (m.spec.max_output) options.num_predict = m.spec.max_output
|
|
||||||
if (m.spec.temperature !== undefined) options.temperature = m.spec.temperature
|
|
||||||
if (m.spec.top_p !== undefined) options.top_p = m.spec.top_p
|
|
||||||
const body: Record<string, unknown> = { model: m.id, messages: toOllamaMessages(req.system, req.messages, m.spec.vision === true), stream: true, options, ...m.connection.body, ...m.spec.body }
|
|
||||||
if (req.tools.length && m.spec.tools !== false)
|
|
||||||
body.tools = req.tools.map((t) => ({ type: "function", function: { name: t.name, description: t.description, parameters: t.parameters } }))
|
|
||||||
// gpt-oss takes a level; other thinking models take a switch. A model /api/show says cannot
|
|
||||||
// think gets no `think` at all: Ollama refuses the request.
|
|
||||||
const thinks = !show?.capabilities || show.capabilities.includes("thinking")
|
|
||||||
if (thinks && req.effort) body.think = /gpt-oss/.test(m.id) ? (req.effort === "minimal" ? "low" : req.effort === "xhigh" || req.effort === "max" ? "high" : req.effort) : true
|
|
||||||
else if (thinks && m.spec.efforts?.length) body.think = false
|
|
||||||
return body
|
|
||||||
}
|
|
||||||
|
|
||||||
async *stream(req: ChatRequest): AsyncGenerator<StreamEvent> {
|
|
||||||
const body = this.buildBody(req, await this.show())
|
|
||||||
yield* retrying(() => this.attempt(req, body))
|
|
||||||
}
|
|
||||||
|
|
||||||
private async *attempt(req: ChatRequest, body: Record<string, unknown>): AsyncGenerator<StreamEvent> {
|
|
||||||
const { m } = this
|
|
||||||
const res = await request(joinUrl(m.connection.base_url, "api/chat"), { method: "POST", headers: this.headers(), body: JSON.stringify(body), signal: req.signal, timeoutMs: (m.connection.timeout ?? 600) * 1000, tls: tlsFor(m.connection) }, m.ref).catch((e) => {
|
|
||||||
recordFailure(m.ref, body, e)
|
|
||||||
throw e
|
|
||||||
})
|
|
||||||
if (!res.body) throw new ProviderError(`${m.ref}: empty response`)
|
|
||||||
const out = new Parts()
|
|
||||||
let finish = "stop"
|
|
||||||
let usage: StreamEvent | undefined
|
|
||||||
let calls = 0
|
|
||||||
for await (const chunk of ndjson(record(res.body, m.ref, body)) as AsyncGenerator<any>) {
|
|
||||||
if (chunk?.error) throw new ProviderError(`${m.ref}: ${chunk.error}`, statusOf(chunk) ?? 500, undefined, !out.output)
|
|
||||||
const msg = chunk?.message ?? {}
|
|
||||||
if (msg.thinking) {
|
|
||||||
out.add("reasoning", msg.thinking)
|
|
||||||
yield { type: "reasoning", text: msg.thinking }
|
|
||||||
}
|
|
||||||
if (msg.content) {
|
|
||||||
out.add("text", msg.content)
|
|
||||||
yield { type: "text", text: msg.content }
|
|
||||||
}
|
|
||||||
for (const tc of msg.tool_calls ?? []) {
|
|
||||||
const args = JSON.stringify(tc.function?.arguments ?? {})
|
|
||||||
yield { type: "tool_call_delta", index: calls, name: tc.function?.name, argsDelta: args }
|
|
||||||
out.push({ type: "tool_call", id: tc.id ?? `ollama_call_${calls}`, name: tc.function?.name ?? "", args })
|
|
||||||
calls++
|
|
||||||
}
|
|
||||||
if (chunk?.done) {
|
|
||||||
finish = chunk.done_reason === "length" ? "length" : "stop"
|
|
||||||
if (chunk.prompt_eval_count !== undefined || chunk.eval_count !== undefined) usage = { type: "usage", usage: { input: chunk.prompt_eval_count ?? 0, output: chunk.eval_count ?? 0 } }
|
|
||||||
}
|
|
||||||
}
|
|
||||||
const message = out.message()
|
|
||||||
if (calls && finish !== "length") finish = "tool_calls"
|
|
||||||
yield usage ?? { type: "usage", usage: { input: estimateValue(body.messages), output: message.parts.reduce((n, p) => n + estimateTokens(p.type === "tool_call" ? p.args : p.text), 0), estimated: true } }
|
|
||||||
yield { type: "finish", reason: finish, message }
|
|
||||||
}
|
|
||||||
|
|
||||||
async listModels(): Promise<DiscoveredModel[]> {
|
|
||||||
const res = await request(joinUrl(this.m.connection.base_url, "api/tags"), { headers: this.headers(), timeoutMs: 15_000, tls: tlsFor(this.m.connection) }, this.m.connectionName)
|
|
||||||
const j = (await res.json()) as any
|
|
||||||
return (j?.models ?? []).map((x: any) => ({ id: String(x.name ?? x.model ?? "") })).filter((x: DiscoveredModel) => x.id)
|
|
||||||
}
|
|
||||||
|
|
||||||
/** The model's window: the Modelfile's num_ctx, else its trained maximum. */
|
|
||||||
async contextOf(id: string): Promise<number | undefined> {
|
|
||||||
const s = await this.show(id)
|
|
||||||
return s?.numCtx ?? s?.trained
|
|
||||||
}
|
|
||||||
}
|
|
||||||
+18
-59
@@ -1,69 +1,29 @@
|
|||||||
// How a model is named: by its provider, not by the login it came through. A LLeMbas
|
// How a model is named: by the id the LLeMbas server serves it under — `deepseek/deepseek-flash`,
|
||||||
// instance that speaks protocol 2 serves every model as `<provider>/<model>` — `deepseek/deepseek-flash`,
|
// `llama/bonsai` — which is its ref here too. LLeMbas CLI 2.0 talks to one server, the one it is
|
||||||
// `llama/bonsai` — and that id is the model's ref here too: each provider is a connection of its
|
// logged in to, so there is nothing to tell apart: no connection prefix, no second login.
|
||||||
// own as far as the person can see, backed by the instance (requests still go to its /v1, with the
|
|
||||||
// served id). The older form of the ref was `<login connection>/<id>` (`example/deepseek-flash`).
|
|
||||||
//
|
//
|
||||||
// - An instance that does not speak protocol 2 serves bare ids: those stay `<login>/<id>`.
|
// The older forms still resolve, and a config.yaml that names one is rewritten to the id (load.ts):
|
||||||
// - A connection of the user's own with the provider's name (`deepseek:` in connections.yaml)
|
// `<login>/<id>` (`example/deepseek/deepseek-flash`) and, from before providers, `<login>/<model>`
|
||||||
// keeps `deepseek/x`; the instance's is then `<login>/deepseek/x`.
|
// for the one served `<provider>/<model>` of that name (`example/deepseek-flash`).
|
||||||
// - Two instances serving the same provider: the one logged in to first keeps the short form, the
|
|
||||||
// other's are `<login>/<provider>/<model>`.
|
|
||||||
// - The old forms still resolve — `example/deepseek-flash` and `example/deepseek/deepseek-flash`
|
|
||||||
// are `deepseek/deepseek-flash` — and a config.yaml that names one is rewritten to the new form.
|
|
||||||
import type { Connection } from "../config/schema.ts"
|
import type { Connection } from "../config/schema.ts"
|
||||||
|
|
||||||
export interface ModelRef {
|
export interface ModelRef {
|
||||||
/** What the person sees and writes. */
|
/** What the person sees and writes: the served id. */
|
||||||
ref: string
|
ref: string
|
||||||
/** The connection it is spoken to through, and the id sent there. */
|
/** The login it is spoken to through, and the id sent there. */
|
||||||
connection: string
|
connection: string
|
||||||
id: string
|
id: string
|
||||||
/** An instance's model: its provider, when known — what the old `<login>/<model>` is matched by. */
|
|
||||||
provider?: string
|
|
||||||
}
|
}
|
||||||
|
|
||||||
type Conns = Record<string, Connection & { webui?: { providers?: Record<string, string>; v2?: boolean } }>
|
type Conns = Record<string, Connection>
|
||||||
|
|
||||||
/** The provider of a served id: as the instance said it (`provider`), else — from an
|
/** Every model the server serves, in its order, with its ref. */
|
||||||
* instance that speaks protocol 2, whose ids are always `<provider>/<model>` — the part before the
|
export function modelRefs(connections: Conns): ModelRef[] {
|
||||||
* first `/`. An id with a `/` from an older instance is just an id. */
|
return Object.entries(connections).flatMap(([name, c]) => Object.keys(c.models).map((id) => ({ ref: id, connection: name, id })))
|
||||||
export function providerOf(c: Conns[string], id: string): string | undefined {
|
|
||||||
if (!c.webui) return undefined
|
|
||||||
const said = c.webui.providers?.[id]
|
|
||||||
if (said) return said
|
|
||||||
return c.webui.v2 && id.indexOf("/") > 0 ? id.slice(0, id.indexOf("/")) : undefined
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Every model, in config order, with its ref. `logins`: the webui connections in the order they
|
/** A ref as written — the id, or an older form — as the model it names, or undefined. `old`: it
|
||||||
* were logged in to (first keeps a shared provider's short form). */
|
* was an older form, which config.yaml is rewritten from. */
|
||||||
export function modelRefs(connections: Conns, logins: string[] = []): ModelRef[] {
|
|
||||||
const names = new Set(Object.keys(connections))
|
|
||||||
const webui = Object.keys(connections).filter((n) => connections[n]!.webui)
|
|
||||||
const order = [...logins.filter((n) => webui.includes(n)), ...webui.filter((n) => !logins.includes(n))]
|
|
||||||
// Which webui connection a provider's short form belongs to.
|
|
||||||
const owner = new Map<string, string>()
|
|
||||||
for (const w of order)
|
|
||||||
for (const id of Object.keys(connections[w]!.models)) {
|
|
||||||
const provider = providerOf(connections[w]!, id)
|
|
||||||
if (!provider) continue
|
|
||||||
// A connection of that name wins it — unless it is this login itself (`llama/bonsai` on a
|
|
||||||
// login named llama is the same string either way).
|
|
||||||
if ((names.has(provider) && provider !== w) || owner.has(provider)) continue
|
|
||||||
owner.set(provider, w)
|
|
||||||
}
|
|
||||||
const out: ModelRef[] = []
|
|
||||||
for (const [name, c] of Object.entries(connections))
|
|
||||||
for (const id of Object.keys(c.models)) {
|
|
||||||
const provider = providerOf(c, id)
|
|
||||||
const short = provider && owner.get(provider) === name
|
|
||||||
out.push({ ref: short ? id : `${name}/${id}`, connection: name, id, ...(provider ? { provider } : {}) })
|
|
||||||
}
|
|
||||||
return out
|
|
||||||
}
|
|
||||||
|
|
||||||
/** A ref as written — any of the forms above — as the model it names, or undefined. `old`: it was
|
|
||||||
* the form from before providers (`<login>/<model>` for a served `<provider>/<model>`). */
|
|
||||||
export function findRef(connections: Conns, refs: ModelRef[], ref: string): (ModelRef & { old?: boolean }) | undefined {
|
export function findRef(connections: Conns, refs: ModelRef[], ref: string): (ModelRef & { old?: boolean }) | undefined {
|
||||||
const exact = refs.find((r) => r.ref === ref)
|
const exact = refs.find((r) => r.ref === ref)
|
||||||
if (exact) return exact
|
if (exact) return exact
|
||||||
@@ -71,13 +31,12 @@ export function findRef(connections: Conns, refs: ModelRef[], ref: string): (Mod
|
|||||||
if (slash <= 0) return undefined
|
if (slash <= 0) return undefined
|
||||||
const conn = ref.slice(0, slash)
|
const conn = ref.slice(0, slash)
|
||||||
const id = ref.slice(slash + 1)
|
const id = ref.slice(slash + 1)
|
||||||
const c = connections[conn]
|
if (!connections[conn]) return undefined
|
||||||
if (!c) return undefined
|
|
||||||
const direct = refs.find((r) => r.connection === conn && r.id === id)
|
const direct = refs.find((r) => r.connection === conn && r.id === id)
|
||||||
if (direct) return direct
|
if (direct) return { ...direct, old: true }
|
||||||
// `<login>/<model>` from before providers: the one served model of that name, whatever its provider.
|
// `<login>/<model>` from before providers: the one served model of that name, whatever its provider.
|
||||||
if (c.webui && !id.includes("/")) {
|
if (!id.includes("/")) {
|
||||||
const by = refs.filter((r) => r.connection === conn && r.provider && r.id === `${r.provider}/${id}`)
|
const by = refs.filter((r) => r.connection === conn && r.id.endsWith(`/${id}`) && r.id.indexOf("/") === r.id.length - id.length - 1)
|
||||||
if (by.length === 1) return { ...by[0]!, old: true }
|
if (by.length === 1) return { ...by[0]!, old: true }
|
||||||
}
|
}
|
||||||
return undefined
|
return undefined
|
||||||
|
|||||||
@@ -1,174 +0,0 @@
|
|||||||
// OpenAI Responses (`POST {base}/responses`) — api.openai.com, and vLLM. Stateless (store: false):
|
|
||||||
// the whole history is sent as input items, and reasoning items come back opaque (with their
|
|
||||||
// encrypted content when the server provides it) so a reasoning model keeps its thread.
|
|
||||||
import { resolveKey } from "../config/load.ts"
|
|
||||||
import { historyArgs, Parts, record, recordFailure, retrying, statusOf, SwapBanner } from "./common.ts"
|
|
||||||
import { authHeaders, joinUrl, request, tlsFor } from "./http.ts"
|
|
||||||
import { contextFrom } from "./openai-chat.ts"
|
|
||||||
import { sseJson } from "./sse.ts"
|
|
||||||
import { estimateTokens, estimateValue } from "./tokens.ts"
|
|
||||||
import { ProviderError, type ChatRequest, type Client, type DiscoveredModel, type Message, type ResolvedModel, type StreamEvent } from "./types.ts"
|
|
||||||
|
|
||||||
type Item = Record<string, unknown>
|
|
||||||
|
|
||||||
export function toResponsesInput(messages: Message[], vision: boolean): Item[] {
|
|
||||||
const out: Item[] = []
|
|
||||||
for (const m of messages) {
|
|
||||||
if (m.role === "user") {
|
|
||||||
out.push({
|
|
||||||
role: "user",
|
|
||||||
content: m.parts.map((p) =>
|
|
||||||
p.type === "text"
|
|
||||||
? { type: "input_text", text: p.text }
|
|
||||||
: vision
|
|
||||||
? { type: "input_image", image_url: `data:${p.mime};base64,${p.data}` }
|
|
||||||
: { type: "input_text", text: "[image omitted: this model has no vision]" },
|
|
||||||
),
|
|
||||||
})
|
|
||||||
} else if (m.role === "assistant") {
|
|
||||||
let text: string[] = []
|
|
||||||
const flush = () => {
|
|
||||||
if (text.join("").trim()) out.push({ role: "assistant", content: [{ type: "output_text", text: text.join("") }] })
|
|
||||||
text = []
|
|
||||||
}
|
|
||||||
for (const p of m.parts) {
|
|
||||||
if (p.type === "text") text.push(p.text)
|
|
||||||
else if (p.type === "reasoning") {
|
|
||||||
// Only a reasoning item this dialect produced can go back; text from elsewhere cannot.
|
|
||||||
const item = (p.opaque as { responses?: Item } | undefined)?.responses
|
|
||||||
if (item) {
|
|
||||||
flush()
|
|
||||||
out.push(item)
|
|
||||||
}
|
|
||||||
} else {
|
|
||||||
flush()
|
|
||||||
out.push({ type: "function_call", call_id: p.id, name: p.name, arguments: historyArgs(p.args) })
|
|
||||||
}
|
|
||||||
}
|
|
||||||
flush()
|
|
||||||
} else out.push({ type: "function_call_output", call_id: m.callId, output: m.content })
|
|
||||||
}
|
|
||||||
return out
|
|
||||||
}
|
|
||||||
|
|
||||||
export class ResponsesClient implements Client {
|
|
||||||
constructor(private m: ResolvedModel) {}
|
|
||||||
|
|
||||||
private headers() {
|
|
||||||
return authHeaders(this.m.connection, resolveKey(this.m.connectionName, this.m.connection), this.m.spec.headers)
|
|
||||||
}
|
|
||||||
|
|
||||||
buildBody(req: ChatRequest): Record<string, unknown> {
|
|
||||||
const { m } = this
|
|
||||||
const body: Record<string, unknown> = {
|
|
||||||
model: m.id,
|
|
||||||
input: toResponsesInput(req.messages, m.spec.vision === true),
|
|
||||||
stream: true,
|
|
||||||
store: false,
|
|
||||||
...m.connection.body,
|
|
||||||
...m.spec.body,
|
|
||||||
}
|
|
||||||
if (req.system) body.instructions = req.system
|
|
||||||
if (req.tools.length && m.spec.tools !== false)
|
|
||||||
body.tools = req.tools.map((t) => ({ type: "function", name: t.name, description: t.description, parameters: t.parameters, strict: false }))
|
|
||||||
if (m.spec.max_output) body.max_output_tokens = m.spec.max_output
|
|
||||||
if (req.effort) {
|
|
||||||
body.reasoning = { effort: req.effort, summary: "auto" }
|
|
||||||
body.include = ["reasoning.encrypted_content"]
|
|
||||||
} else {
|
|
||||||
if (m.spec.temperature !== undefined) body.temperature = m.spec.temperature
|
|
||||||
if (m.spec.top_p !== undefined) body.top_p = m.spec.top_p
|
|
||||||
}
|
|
||||||
return body
|
|
||||||
}
|
|
||||||
|
|
||||||
async *stream(req: ChatRequest): AsyncGenerator<StreamEvent> {
|
|
||||||
let body = this.buildBody(req)
|
|
||||||
yield* retrying(
|
|
||||||
() => this.attempt(req, body),
|
|
||||||
(e) => {
|
|
||||||
// Some servers do not know `include`; drop it once rather than fail.
|
|
||||||
if (body.include && (e.status === 400 || e.status === 422) && /include/i.test(e.message)) {
|
|
||||||
const { include: _, ...rest } = body
|
|
||||||
body = rest
|
|
||||||
return { retry: true }
|
|
||||||
}
|
|
||||||
return { retry: false }
|
|
||||||
},
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
private async *attempt(req: ChatRequest, body: Record<string, unknown>): AsyncGenerator<StreamEvent> {
|
|
||||||
const { m } = this
|
|
||||||
const res = await request(
|
|
||||||
joinUrl(m.connection.base_url, "responses"),
|
|
||||||
{ method: "POST", headers: this.headers(), body: JSON.stringify(body), signal: req.signal, timeoutMs: (m.connection.timeout ?? 600) * 1000, tls: tlsFor(m.connection) },
|
|
||||||
m.ref,
|
|
||||||
).catch((e) => {
|
|
||||||
recordFailure(m.ref, body, e)
|
|
||||||
throw e
|
|
||||||
})
|
|
||||||
if (!res.body) throw new ProviderError(`${m.ref}: empty response`)
|
|
||||||
|
|
||||||
const out = new Parts()
|
|
||||||
const banner = new SwapBanner()
|
|
||||||
const reasoning = new Map<string, string>()
|
|
||||||
const callIndex = new Map<string, number>()
|
|
||||||
let finish = "stop"
|
|
||||||
let usage: StreamEvent | undefined
|
|
||||||
|
|
||||||
for await (const { data } of sseJson(record(res.body, m.ref, body))) {
|
|
||||||
const ev = data as any
|
|
||||||
const t: string = ev?.type ?? ""
|
|
||||||
if (t === "response.output_text.delta") {
|
|
||||||
out.add("text", ev.delta ?? "")
|
|
||||||
yield { type: "text", text: ev.delta ?? "" }
|
|
||||||
} else if (t === "response.reasoning_summary_text.delta" || t === "response.reasoning_text.delta") {
|
|
||||||
const b = banner.feed(ev.delta ?? "")
|
|
||||||
if (b.notice) yield { type: "notice", message: b.notice }
|
|
||||||
if (!b.text) continue
|
|
||||||
reasoning.set(ev.item_id ?? "", (reasoning.get(ev.item_id ?? "") ?? "") + b.text)
|
|
||||||
if (b.text.trim()) out.output = true
|
|
||||||
yield { type: "reasoning", text: b.text }
|
|
||||||
} else if (t === "response.output_item.added" && ev.item?.type === "function_call") {
|
|
||||||
const index = callIndex.size
|
|
||||||
callIndex.set(ev.item.id ?? ev.item.call_id, index)
|
|
||||||
yield { type: "tool_call_delta", index, name: ev.item.name, argsDelta: "" }
|
|
||||||
} else if (t === "response.function_call_arguments.delta") {
|
|
||||||
out.output = true
|
|
||||||
yield { type: "tool_call_delta", index: callIndex.get(ev.item_id) ?? 0, argsDelta: ev.delta ?? "" }
|
|
||||||
} else if (t === "response.output_item.done") {
|
|
||||||
const item = ev.item ?? {}
|
|
||||||
if (item.type === "function_call") out.push({ type: "tool_call", id: item.call_id ?? item.id, name: item.name, args: item.arguments || "{}" })
|
|
||||||
else if (item.type === "reasoning") {
|
|
||||||
const summary = Array.isArray(item.summary) ? item.summary.map((s: any) => s.text ?? "").join("\n") : ""
|
|
||||||
const text = reasoning.get(item.id ?? "") || summary
|
|
||||||
// vLLM puts the raw reasoning in `content`, which it will not accept back; keep what can return.
|
|
||||||
const { content: _c, ...back } = item
|
|
||||||
if (text || item.encrypted_content) out.push({ type: "reasoning", text, opaque: item.encrypted_content ? { responses: back } : undefined })
|
|
||||||
}
|
|
||||||
} else if (t === "response.completed" || t === "response.incomplete") {
|
|
||||||
const r = ev.response ?? {}
|
|
||||||
if (t === "response.incomplete" || r.status === "incomplete") finish = "length"
|
|
||||||
const u = r.usage
|
|
||||||
if (u) usage = { type: "usage", usage: { input: u.input_tokens ?? 0, output: u.output_tokens ?? 0, reasoning: u.output_tokens_details?.reasoning_tokens, cached: u.input_tokens_details?.cached_tokens || undefined } }
|
|
||||||
} else if (t === "response.failed" || t === "error" || ev?.error) {
|
|
||||||
const err = ev.response?.error ?? ev.error ?? ev
|
|
||||||
throw new ProviderError(`${m.ref}: ${err.message ?? JSON.stringify(err)}`, statusOf(err), undefined, !out.output)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
const message = out.message()
|
|
||||||
if (finish !== "length" && message.parts.some((p) => p.type === "tool_call")) finish = "tool_calls"
|
|
||||||
yield usage ?? {
|
|
||||||
type: "usage",
|
|
||||||
usage: { input: estimateValue(body.input), output: message.parts.reduce((n, p) => n + estimateTokens(p.type === "tool_call" ? p.args : p.text), 0), estimated: true },
|
|
||||||
}
|
|
||||||
yield { type: "finish", reason: finish, message }
|
|
||||||
}
|
|
||||||
|
|
||||||
async listModels(): Promise<DiscoveredModel[]> {
|
|
||||||
const res = await request(joinUrl(this.m.connection.base_url, "models"), { headers: this.headers(), timeoutMs: 15_000, tls: tlsFor(this.m.connection) }, this.m.connectionName)
|
|
||||||
const j = (await res.json()) as any
|
|
||||||
return (Array.isArray(j?.data) ? j.data : []).map((x: any) => ({ id: String(x.id), context: contextFrom(x) })).filter((x: DiscoveredModel) => x.id)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -56,7 +56,7 @@ export interface ChatRequest {
|
|||||||
}
|
}
|
||||||
|
|
||||||
export interface ResolvedModel {
|
export interface ResolvedModel {
|
||||||
/** `connection/model` */
|
/** The served id (`deepseek/deepseek-flash`). */
|
||||||
ref: string
|
ref: string
|
||||||
connectionName: string
|
connectionName: string
|
||||||
connection: Connection
|
connection: Connection
|
||||||
|
|||||||
@@ -19,8 +19,8 @@ export function capacityRefusal(main: ResolvedModel, helper: ResolvedModel): str
|
|||||||
}
|
}
|
||||||
const shared = groupOf(main)
|
const shared = groupOf(main)
|
||||||
if (shared !== undefined && shared === groupOf(helper)) {
|
if (shared !== undefined && shared === groupOf(helper)) {
|
||||||
const where = main.spec.group !== undefined ? `${main.ref}'s server (group ${main.spec.group})` : main.connectionName
|
const where = `${main.ref}'s server (group ${main.spec.group})`
|
||||||
return `${where} holds one model at a time (${main.spec.group !== undefined ? "group" : "one_model_at_a_time"}), so a subagent on ${helper.ref} would unload ${main.ref}, and this session's cached prompt with it, while its turn waits.`
|
return `${where} holds one model at a time, so a subagent on ${helper.ref} would unload ${main.ref}, and this session's cached prompt with it, while its turn waits.`
|
||||||
}
|
}
|
||||||
return ""
|
return ""
|
||||||
}
|
}
|
||||||
@@ -35,15 +35,14 @@ export function capacityBeside(running: ResolvedModel, helper: ResolvedModel): s
|
|||||||
}
|
}
|
||||||
const shared = groupOf(running)
|
const shared = groupOf(running)
|
||||||
if (shared !== undefined && shared === groupOf(helper)) {
|
if (shared !== undefined && shared === groupOf(helper)) {
|
||||||
const where = running.spec.group !== undefined ? `${running.ref}'s server (group ${running.spec.group})` : running.connectionName
|
const where = `${running.ref}'s server (group ${running.spec.group})`
|
||||||
return `${where} holds one model at a time, and a subagent is already running on ${running.ref} there; one on ${helper.ref} would unload it.`
|
return `${where} holds one model at a time, and a subagent is already running on ${running.ref} there; one on ${helper.ref} would unload it.`
|
||||||
}
|
}
|
||||||
return ""
|
return ""
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Which one-model-at-a-time server a model is on: its own `group`, else its connection when the
|
/** Which one-model-at-a-time server a model is on, as the LLeMbas server says (`capacity.group`,
|
||||||
* connection sets one_model_at_a_time, else none. The spec's capacity.group. */
|
* the spec's), or none. */
|
||||||
function groupOf(m: ResolvedModel): string | undefined {
|
function groupOf(m: ResolvedModel): string | undefined {
|
||||||
if (m.spec.group !== undefined) return `group:${m.spec.group}`
|
return m.spec.group !== undefined ? `group:${m.spec.group}` : undefined
|
||||||
return m.connection.one_model_at_a_time ? `connection:${m.connectionName}` : undefined
|
|
||||||
}
|
}
|
||||||
@@ -1,7 +1,7 @@
|
|||||||
// /context: where the window goes. Estimates (≈4 characters a token) per part, scaled so they add
|
// /context: where the window goes. Estimates (≈4 characters a token) per part, scaled so they add
|
||||||
// up to what the server last reported when it reported anything.
|
// up to what the server last reported when it reported anything.
|
||||||
import type { Engine } from "./engine.ts"
|
import type { Engine } from "./engine.ts"
|
||||||
import { contextWindow } from "../provider/discover.ts"
|
import { contextWindow } from "../provider/index.ts"
|
||||||
import { estimateTokens, imageTokens } from "../provider/tokens.ts"
|
import { estimateTokens, imageTokens } from "../provider/tokens.ts"
|
||||||
import { toSpec, type Tool } from "../tool/tool.ts"
|
import { toSpec, type Tool } from "../tool/tool.ts"
|
||||||
|
|
||||||
|
|||||||
+2
-19
@@ -14,7 +14,7 @@ import { fillPrompt } from "../prompt/fill.ts"
|
|||||||
import { splitPurpose, toSpec, ToolError, type Tool, type ToolContext, type ToolResult } from "../tool/tool.ts"
|
import { splitPurpose, toSpec, ToolError, type Tool, type ToolContext, type ToolResult } from "../tool/tool.ts"
|
||||||
import { fill, promptText } from "../prompt/assemble.ts"
|
import { fill, promptText } from "../prompt/assemble.ts"
|
||||||
import { resolveCall } from "../tool/names.ts"
|
import { resolveCall } from "../tool/names.ts"
|
||||||
import { contextWindow } from "../provider/discover.ts"
|
import { contextWindow } from "../provider/index.ts"
|
||||||
import { summaryTurns, type Store } from "./store.ts"
|
import { summaryTurns, type Store } from "./store.ts"
|
||||||
import { estimateTokens, estimateValue } from "../provider/tokens.ts"
|
import { estimateTokens, estimateValue } from "../provider/tokens.ts"
|
||||||
import { duration } from "../duration.ts"
|
import { duration } from "../duration.ts"
|
||||||
@@ -43,9 +43,6 @@ export interface EngineOptions {
|
|||||||
compaction?: { autoAt?: number }
|
compaction?: { autoAt?: number }
|
||||||
/** Starts a subagent for the task tool; absent in a subagent. */
|
/** Starts a subagent for the task tool; absent in a subagent. */
|
||||||
spawn?: ToolContext["spawn"]
|
spawn?: ToolContext["spawn"]
|
||||||
/** The model to switch to when this one's server cannot be reached (its `fallback` list), or
|
|
||||||
* undefined when there is none left to try. */
|
|
||||||
fallback?: (current: ResolvedModel, tried: Set<string>) => { model: ResolvedModel; client: Client; effort: Effort | null } | undefined
|
|
||||||
/** What a message sent mid-turn starts with: the user's words for a session, the main agent's for
|
/** What a message sent mid-turn starts with: the user's words for a session, the main agent's for
|
||||||
* a subagent (loop.json jobs.marks to_subagent). */
|
* a subagent (loop.json jobs.marks to_subagent). */
|
||||||
steerMark?: string
|
steerMark?: string
|
||||||
@@ -264,7 +261,7 @@ export class Engine {
|
|||||||
}
|
}
|
||||||
|
|
||||||
private droppedThinking(): number {
|
private droppedThinking(): number {
|
||||||
if (!this.usage || this.model.connection.dialect !== "openai-chat") return 0
|
if (!this.usage) return 0
|
||||||
const reply = this.messages[this.usageAt - 1]
|
const reply = this.messages[this.usageAt - 1]
|
||||||
if (reply?.role !== "assistant") return 0
|
if (reply?.role !== "assistant") return 0
|
||||||
const estimate = reply.parts.reduce((n, p) => n + (p.type === "reasoning" ? estimateTokens(p.text) : 0), 0)
|
const estimate = reply.parts.reduce((n, p) => n + (p.type === "reasoning" ? estimateTokens(p.text) : 0), 0)
|
||||||
@@ -468,7 +465,6 @@ export class Engine {
|
|||||||
let ranOut: { limit: string; said: string } | undefined
|
let ranOut: { limit: string; said: string } | undefined
|
||||||
let nudges = 0
|
let nudges = 0
|
||||||
let midRetries = 0
|
let midRetries = 0
|
||||||
const tried = new Set<string>()
|
|
||||||
|
|
||||||
for (let step = 1; ; step++) {
|
for (let step = 1; ; step++) {
|
||||||
// Checked between steps, never mid-reply: a reply cut off mid-sentence is worth nothing.
|
// Checked between steps, never mid-reply: a reply cut off mid-sentence is worth nothing.
|
||||||
@@ -519,19 +515,6 @@ export class Engine {
|
|||||||
bus.emit({ type: "done", reason: "cancelled" })
|
bus.emit({ type: "done", reason: "cancelled" })
|
||||||
return "cancelled"
|
return "cancelled"
|
||||||
}
|
}
|
||||||
// The server is not there at all: the model's fallback, if it has one, for the rest of the
|
|
||||||
// session — said, and shown in the status bar.
|
|
||||||
if (e instanceof ProviderError && e.unreachable && this.o.fallback) {
|
|
||||||
tried.add(this.model.ref)
|
|
||||||
const next = this.o.fallback(this.model, tried)
|
|
||||||
if (next) {
|
|
||||||
bus.emit({ type: "notice", message: `${e.message.split("\n")[0]!.slice(0, 160)} — switching to ${next.model.ref}, its fallback` })
|
|
||||||
this.setModel(next.model, next.client, next.effort)
|
|
||||||
bus.emit({ type: "setting", key: "model", value: next.model.ref })
|
|
||||||
step--
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
}
|
|
||||||
// A server failure part-way through a reply (llama.cpp failing to parse gpt-oss's own
|
// A server failure part-way through a reply (llama.cpp failing to parse gpt-oss's own
|
||||||
// output is the one seen) is worth two more goes. What was already shown is discarded.
|
// output is the one seen) is worth two more goes. What was already shown is discarded.
|
||||||
if (e instanceof ProviderError && !e.beforeOutput && (e.status ?? 0) >= 500 && midRetries < 2) {
|
if (e instanceof ProviderError && !e.beforeOutput && (e.status ?? 0) >= 500 && midRetries < 2) {
|
||||||
|
|||||||
+2
-2
@@ -85,7 +85,7 @@ const pendingCodes = new Map<string, (text: string) => void>()
|
|||||||
|
|
||||||
const AGENT_TEMPLATE = (name: string) => `---
|
const AGENT_TEMPLATE = (name: string) => `---
|
||||||
description: what ${name} is for — the model reads this when it picks a subagent
|
description: what ${name} is for — the model reads this when it picks a subagent
|
||||||
# model: connection/model # run it on another model (default: the session's)
|
# model: provider/model # run it on another of the server's models (default: the session's)
|
||||||
# mode: plan # plan: read-only; edit; manual (asks you) — never looser than yours
|
# mode: plan # plan: read-only; edit; manual (asks you) — never looser than yours
|
||||||
# tools: [read, glob, grep, list, bash, web_search, web_fetch] # default: every tool a subagent may have
|
# tools: [read, glob, grep, list, bash, web_search, web_fetch] # default: every tool a subagent may have
|
||||||
# isolation: worktree # work in a git checkout and branch of its own
|
# isolation: worktree # work in a git checkout and branch of its own
|
||||||
@@ -153,7 +153,7 @@ export const COMMANDS: Command[] = [
|
|||||||
},
|
},
|
||||||
{
|
{
|
||||||
name: "model",
|
name: "model",
|
||||||
args: "[connection/model]",
|
args: "[model]",
|
||||||
description: "switch model",
|
description: "switch model",
|
||||||
run(c, arg) {
|
run(c, arg) {
|
||||||
const refs = c.app.modelRefs()
|
const refs = c.app.modelRefs()
|
||||||
|
|||||||
@@ -1,18 +1,9 @@
|
|||||||
import { TextAttributes } from "@opentui/core"
|
import { TextAttributes } from "@opentui/core"
|
||||||
import { useKeyboard } from "@opentui/solid"
|
import { useKeyboard } from "@opentui/solid"
|
||||||
import { join } from "node:path"
|
|
||||||
import { createSignal, Show } from "solid-js"
|
import { createSignal, Show } from "solid-js"
|
||||||
import { paths } from "../../config/paths.ts"
|
import { ModelError, NOT_LOGGED_IN } from "../../provider/index.ts"
|
||||||
import { ModelError } from "../../provider/index.ts"
|
|
||||||
import { useTui } from "../context.ts"
|
import { useTui } from "../context.ts"
|
||||||
|
|
||||||
const EXAMPLE = `connections:
|
|
||||||
local: # any name; llama.cpp, LM Studio, vLLM, Ollama's /v1…
|
|
||||||
dialect: openai-chat
|
|
||||||
base_url: http://127.0.0.1:8080/v1
|
|
||||||
models:
|
|
||||||
qwen3: { context: 131072 }`
|
|
||||||
|
|
||||||
/** Before there is a session: no model to talk to (a fresh install), or a config file that cannot
|
/** Before there is a session: no model to talk to (a fresh install), or a config file that cannot
|
||||||
* be read. Says what to do and where; r tries again once the files are fixed, q or ctrl+c leaves.
|
* be read. Says what to do and where; r tries again once the files are fixed, q or ctrl+c leaves.
|
||||||
* Not an exception: thrown with the terminal still the TUI's, mouse reporting would stay on, and
|
* Not an exception: thrown with the terminal still the TUI's, mouse reporting would stay on, and
|
||||||
@@ -38,7 +29,7 @@ export function Setup(p: { error: Error; onRetry: () => void; onQuit: () => void
|
|||||||
try {
|
try {
|
||||||
const { codeInstructions, login } = await import("../../lembas/login.ts")
|
const { codeInstructions, login } = await import("../../lembas/login.ts")
|
||||||
const r = await login(target, { onCode: (start, d) => setStatus(codeInstructions(start, d)) })
|
const r = await login(target, { onCode: (start, d) => setStatus(codeInstructions(start, d)) })
|
||||||
setStatus(`Signed in: ${r.models.length} model${r.models.length === 1 ? "" : "s"} as ${r.connection}/…${r.instanceDefault ? `, starting on ${r.connection}/${r.instanceDefault}` : ""}.`)
|
setStatus(`Signed in: ${r.models.length} model${r.models.length === 1 ? "" : "s"}${r.instanceDefault ? `, starting on ${r.instanceDefault}` : ""}.`)
|
||||||
p.onRetry()
|
p.onRetry()
|
||||||
} catch (e) {
|
} catch (e) {
|
||||||
setStatus((e as Error).message)
|
setStatus((e as Error).message)
|
||||||
@@ -65,11 +56,11 @@ export function Setup(p: { error: Error; onRetry: () => void; onQuit: () => void
|
|||||||
return (
|
return (
|
||||||
<box flexDirection="column" width="100%" height="100%" paddingLeft={2} paddingTop={1} paddingRight={2} gap={1}>
|
<box flexDirection="column" width="100%" height="100%" paddingLeft={2} paddingTop={1} paddingRight={2} gap={1}>
|
||||||
<text fg={c().title} attributes={TextAttributes.BOLD}>
|
<text fg={c().title} attributes={TextAttributes.BOLD}>
|
||||||
{theme().branding.agentName} — {noModel() ? "no model to talk to yet" : "the configuration has a mistake"}
|
{theme().branding.agentName} — {p.error.message === NOT_LOGGED_IN ? "not logged in to a LLeMbas server" : noModel() ? "no model to talk to" : "the configuration has a mistake"}
|
||||||
</text>
|
</text>
|
||||||
{/* "No model configured" is what the rest of the screen explains; anything else (a model
|
{/* Not being logged in is what the rest of the screen explains; anything else (a model the
|
||||||
that does not exist, a file that does not parse) is shown as it is. */}
|
server does not serve, a file that does not parse) is shown as it is. */}
|
||||||
<Show when={!p.error.message.startsWith("No model configured")}>
|
<Show when={p.error.message !== NOT_LOGGED_IN}>
|
||||||
<text fg={noModel() ? c().text : c().error} wrapMode="word">
|
<text fg={noModel() ? c().text : c().error} wrapMode="word">
|
||||||
{p.error.message}
|
{p.error.message}
|
||||||
</text>
|
</text>
|
||||||
@@ -80,7 +71,7 @@ export function Setup(p: { error: Error; onRetry: () => void; onQuit: () => void
|
|||||||
when={typing()}
|
when={typing()}
|
||||||
fallback={
|
fallback={
|
||||||
<text fg={c().muted} wrapMode="word">
|
<text fg={c().muted} wrapMode="word">
|
||||||
{"Sign in to a LLeMbas instance and use its models: press l (or, from a shell, lembas login <address>)."}
|
{"LLeMbas CLI uses the models of the LLeMbas server it is logged in to. Log in: press l (or, from a shell, lembas login <address>)."}
|
||||||
</text>
|
</text>
|
||||||
}
|
}
|
||||||
>
|
>
|
||||||
@@ -96,24 +87,9 @@ export function Setup(p: { error: Error; onRetry: () => void; onQuit: () => void
|
|||||||
{status()}
|
{status()}
|
||||||
</text>
|
</text>
|
||||||
</Show>
|
</Show>
|
||||||
<text fg={c().muted} wrapMode="word">
|
|
||||||
{`Or, without an instance — nothing is predefined: add the server you use to ${join(paths.config, "connections.yaml")}, for example`}
|
|
||||||
</text>
|
|
||||||
<box paddingLeft={2}>
|
|
||||||
<text fg={c().text}>{EXAMPLE}</text>
|
|
||||||
</box>
|
|
||||||
<text fg={c().muted} wrapMode="word">
|
|
||||||
{`and pick the model in ${join(paths.config, "config.yaml")}:`}
|
|
||||||
</text>
|
|
||||||
<box paddingLeft={2}>
|
|
||||||
<text fg={c().text}>model: local/qwen3</text>
|
|
||||||
</box>
|
|
||||||
<text fg={c().muted} wrapMode="word">
|
|
||||||
A key is never pasted in: api_key: "{"{env:NAME}"}" or "{"{file:~/path}"}". https://llembas.eu/docs/ has the other dialects (Anthropic, OpenAI, Gemini, Ollama).
|
|
||||||
</text>
|
|
||||||
</Show>
|
</Show>
|
||||||
<text fg={c().muted} wrapMode="word">
|
<text fg={c().muted} wrapMode="word">
|
||||||
From a shell, lembas config check reads the files and says what is wrong, and lembas models lists the models it found.
|
From a shell, lembas config check reads the files and says what is wrong, and lembas models lists the server's models.
|
||||||
</text>
|
</text>
|
||||||
<text fg={c().label}>{typing() ? "enter sign in · esc back · ctrl+r try again · ctrl+c quit" : `${noModel() ? "l log in · " : ""}r try again · q quit`}</text>
|
<text fg={c().label}>{typing() ? "enter sign in · esc back · ctrl+r try again · ctrl+c quit" : `${noModel() ? "l log in · " : ""}r try again · q quit`}</text>
|
||||||
</box>
|
</box>
|
||||||
|
|||||||
+7
-10
@@ -34,14 +34,11 @@ function pair(): [Transport, Transport] {
|
|||||||
return [side(a, b), side(b, a)]
|
return [side(a, b), side(b, a)]
|
||||||
}
|
}
|
||||||
|
|
||||||
/** The parent's model on `f`; a second connection `g` for a subagent's model, when given. */
|
/** The parent's model `f/m`; `g/m` for a subagent, when its own script is given (the fake server
|
||||||
function config(url: string, childUrl?: string) {
|
* routes it there). */
|
||||||
|
function config(url: string, child?: boolean) {
|
||||||
mkdirSync(paths.config, { recursive: true })
|
mkdirSync(paths.config, { recursive: true })
|
||||||
writeFileSync(
|
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n type: webui\n url: ${url}\n models: { f/m: {}${child ? ", g/m: {}" : ""} }\n`, { mode: 0o600 })
|
||||||
join(paths.config, "connections.yaml"),
|
|
||||||
`connections:\n f:\n dialect: openai-chat\n base_url: ${url}\n models: { m: {} }\n${childUrl ? ` g:\n dialect: openai-chat\n base_url: ${childUrl}\n models: { m: {} }\n` : ""}`,
|
|
||||||
{ mode: 0o600 },
|
|
||||||
)
|
|
||||||
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\nmode: auto\n")
|
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\nmode: auto\n")
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -145,14 +142,14 @@ test("job/stop stops a command and answers once it has; the -32602 cases: a comm
|
|||||||
}, SLOW)
|
}, SLOW)
|
||||||
|
|
||||||
test("a background subagent: its transcript as history turns; send to a finished one starts it again under the same id", async () => {
|
test("a background subagent: its transcript as history turns; send to a finished one starts it again under the same id", async () => {
|
||||||
|
const child = (other = fakeProvider([{ chunks: [delta({ content: "calc.py:2 divides by len(v) - 1." })] }, { chunks: [delta({ content: "The tests cover it." })] }]))
|
||||||
fake = fakeProvider([
|
fake = fakeProvider([
|
||||||
{ chunks: [toolCall(0, "t1", "task", JSON.stringify({ description: "find the bug", prompt: "Find why average is wrong", model: "g/m", background: true }))] },
|
{ chunks: [toolCall(0, "t1", "task", JSON.stringify({ description: "find the bug", prompt: "Find why average is wrong", model: "g/m", background: true }))] },
|
||||||
{ chunks: [delta({ content: "Started it." })] },
|
{ chunks: [delta({ content: "Started it." })] },
|
||||||
{ chunks: [delta({ content: "Got the report." })] },
|
{ chunks: [delta({ content: "Got the report." })] },
|
||||||
{ chunks: [delta({ content: "Got the second report." })] },
|
{ chunks: [delta({ content: "Got the second report." })] },
|
||||||
])
|
], { route: (model) => (model.startsWith("g/") ? child : undefined) })
|
||||||
other = fakeProvider([{ chunks: [delta({ content: "calc.py:2 divides by len(v) - 1." })] }, { chunks: [delta({ content: "The tests cover it." })] }])
|
config(fake.url, true)
|
||||||
config(fake.url, other.url)
|
|
||||||
const { c, events, sid } = await client()
|
const { c, events, sid } = await client()
|
||||||
await c.request("session/prompt", { sessionId: sid, prompt: [{ type: "text", text: "go" }] })
|
await c.request("session/prompt", { sessionId: sid, prompt: [{ type: "text", text: "go" }] })
|
||||||
expect(await until(() => events.some((e) => e.type === "job" && e.state === "end"))).toBe(true)
|
expect(await until(() => events.some((e) => e.type === "job" && e.state === "end"))).toBe(true)
|
||||||
|
|||||||
@@ -40,7 +40,7 @@ function pair(): [Transport, Transport] {
|
|||||||
|
|
||||||
function config(url: string, extra = "", model = "{}") {
|
function config(url: string, extra = "", model = "{}") {
|
||||||
mkdirSync(paths.config, { recursive: true })
|
mkdirSync(paths.config, { recursive: true })
|
||||||
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n dialect: openai-chat\n base_url: ${url}\n models: { m: ${model} }\n`, { mode: 0o600 })
|
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n type: webui\n url: ${url}\n models: { f/m: ${model} }\n`, { mode: 0o600 })
|
||||||
writeFileSync(join(paths.config, "config.yaml"), `model: f/m\n${extra}`)
|
writeFileSync(join(paths.config, "config.yaml"), `model: f/m\n${extra}`)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -535,7 +535,7 @@ test("answered at the terminal first: the web UI's card is settled", async () =>
|
|||||||
test("the model event: said whenever the model or effort changes, forwarded to LLeMbas", async () => {
|
test("the model event: said whenever the model or effort changes, forwarded to LLeMbas", async () => {
|
||||||
fake = fakeProvider([])
|
fake = fakeProvider([])
|
||||||
mkdirSync(paths.config, { recursive: true })
|
mkdirSync(paths.config, { recursive: true })
|
||||||
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n example:\n dialect: openai-chat\n base_url: ${fake.url}\n models: { a: { efforts: [low, high] }, b: {} }\n`, { mode: 0o600 })
|
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n example:\n type: webui\n url: ${fake.url}\n models: { example/a: { efforts: [low, high] }, example/b: {} }\n`, { mode: 0o600 })
|
||||||
writeFileSync(join(paths.config, "config.yaml"), "model: example/a\n")
|
writeFileSync(join(paths.config, "config.yaml"), "model: example/a\n")
|
||||||
const root = project()
|
const root = project()
|
||||||
const { c, events } = await client(limitsFor(root), { instance: "example" })
|
const { c, events } = await client(limitsFor(root), { instance: "example" })
|
||||||
@@ -771,13 +771,13 @@ test("a command's mode from the web is clamped to remote.max_mode in a terminal
|
|||||||
test("a command naming a model gives the session its model and its effort back afterwards", async () => {
|
test("a command naming a model gives the session its model and its effort back afterwards", async () => {
|
||||||
fake = fakeProvider([{ chunks: [delta({ content: "ok" })] }])
|
fake = fakeProvider([{ chunks: [delta({ content: "ok" })] }])
|
||||||
mkdirSync(paths.config, { recursive: true })
|
mkdirSync(paths.config, { recursive: true })
|
||||||
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n dialect: openai-chat\n base_url: ${fake.url}\n models: { m: { efforts: [low, high], effort: low }, other: {} }\n`, { mode: 0o600 })
|
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n type: webui\n url: ${fake.url}\n models: { f/m: { efforts: [low, high], effort: low }, f/other: {} }\n`, { mode: 0o600 })
|
||||||
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\n")
|
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\n")
|
||||||
const root = project({ ".agent/config.yaml": "", ".agent/commands/quick.md": "---\nmodel: f/other\n---\nQuick." })
|
const root = project({ ".agent/config.yaml": "", ".agent/commands/quick.md": "---\nmodel: f/other\n---\nQuick." })
|
||||||
const { c } = await client(limitsFor(root))
|
const { c } = await client(limitsFor(root))
|
||||||
const s: any = await c.request("session/new", { cwd: root, _meta: { effort: "high" } })
|
const s: any = await c.request("session/new", { cwd: root, _meta: { effort: "high" } })
|
||||||
await c.request("session/prompt", { sessionId: s.sessionId, prompt: [{ type: "text", text: "/quick" }] })
|
await c.request("session/prompt", { sessionId: s.sessionId, prompt: [{ type: "text", text: "/quick" }] })
|
||||||
expect(fake.requests[0].model).toBe("other")
|
expect(fake.requests[0].model).toBe("f/other")
|
||||||
expect(await c.request<any>("_lembas/configure", { sessionId: s.sessionId })).toEqual({ model: "f/m", effort: "high" })
|
expect(await c.request<any>("_lembas/configure", { sessionId: s.sessionId })).toEqual({ model: "f/m", effort: "high" })
|
||||||
})
|
})
|
||||||
|
|
||||||
|
|||||||
+5
-5
@@ -36,7 +36,7 @@ function pair(): [Transport, Transport] {
|
|||||||
|
|
||||||
function config(url: string, extra = "") {
|
function config(url: string, extra = "") {
|
||||||
mkdirSync(paths.config, { recursive: true })
|
mkdirSync(paths.config, { recursive: true })
|
||||||
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n dialect: openai-chat\n base_url: ${url}\n models: { m: {} }\n`, { mode: 0o600 })
|
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n type: webui\n url: ${url}\n models: { f/m: {} }\n`, { mode: 0o600 })
|
||||||
writeFileSync(join(paths.config, "config.yaml"), `model: f/m\n${extra}`)
|
writeFileSync(join(paths.config, "config.yaml"), `model: f/m\n${extra}`)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -210,19 +210,19 @@ test("_lembas/directories is for the link only: an editor's ACP session has no r
|
|||||||
test("the chat's model and effort: named by the instance, at the start and between prompts", async () => {
|
test("the chat's model and effort: named by the instance, at the start and between prompts", async () => {
|
||||||
fake = fakeProvider([{ chunks: [delta({ content: "one" })] }, { chunks: [delta({ content: "two" })] }])
|
fake = fakeProvider([{ chunks: [delta({ content: "one" })] }, { chunks: [delta({ content: "two" })] }])
|
||||||
mkdirSync(paths.config, { recursive: true })
|
mkdirSync(paths.config, { recursive: true })
|
||||||
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n example:\n dialect: openai-chat\n base_url: ${fake.url}\n models: { bonsai: {}, deepseek-flash: { efforts: [low, medium, high] } }\n`, { mode: 0o600 })
|
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n example:\n type: webui\n url: ${fake.url}\n models: { bonsai: {}, deepseek-flash: { efforts: [low, medium, high] } }\n`, { mode: 0o600 })
|
||||||
writeFileSync(join(paths.config, "config.yaml"), "model: example/bonsai\n")
|
writeFileSync(join(paths.config, "config.yaml"), "model: bonsai\n")
|
||||||
const root = project()
|
const root = project()
|
||||||
const { c } = await client({ roots: [root], maxMode: "auto", approvalTimeoutMs: 50, requireTrust: true }, undefined, "example")
|
const { c } = await client({ roots: [root], maxMode: "auto", approvalTimeoutMs: 50, requireTrust: true }, undefined, "example")
|
||||||
// Not the device's own default: the model the chat was started on there.
|
// Not the device's own default: the model the chat was started on there.
|
||||||
const s: any = await c.request("session/new", { cwd: root, _meta: { mode: "auto", lembas_model: "deepseek-flash", effort: "high" } })
|
const s: any = await c.request("session/new", { cwd: root, _meta: { mode: "auto", lembas_model: "deepseek-flash", effort: "high" } })
|
||||||
expect(s._meta.lembas).toMatchObject({ model: "example/deepseek-flash", effort: "high" })
|
expect(s._meta.lembas).toMatchObject({ model: "deepseek-flash", effort: "high" })
|
||||||
await c.request("session/prompt", { sessionId: s.sessionId, prompt: [{ type: "text", text: "hi" }] })
|
await c.request("session/prompt", { sessionId: s.sessionId, prompt: [{ type: "text", text: "hi" }] })
|
||||||
expect(fake.requests[0].model).toBe("deepseek-flash")
|
expect(fake.requests[0].model).toBe("deepseek-flash")
|
||||||
expect(JSON.stringify(fake.requests[0])).toContain('"reasoning_effort":"high"')
|
expect(JSON.stringify(fake.requests[0])).toContain('"reasoning_effort":"high"')
|
||||||
// Changed in the web UI between two prompts.
|
// Changed in the web UI between two prompts.
|
||||||
const changed: any = await c.request("_lembas/configure", { sessionId: s.sessionId, model: "bonsai", effort: "off" })
|
const changed: any = await c.request("_lembas/configure", { sessionId: s.sessionId, model: "bonsai", effort: "off" })
|
||||||
expect(changed).toEqual({ model: "example/bonsai", effort: "off" })
|
expect(changed).toEqual({ model: "bonsai", effort: "off" })
|
||||||
await c.request("session/prompt", { sessionId: s.sessionId, prompt: [{ type: "text", text: "again" }] })
|
await c.request("session/prompt", { sessionId: s.sessionId, prompt: [{ type: "text", text: "again" }] })
|
||||||
expect(fake.requests.at(-1).model).toBe("bonsai")
|
expect(fake.requests.at(-1).model).toBe("bonsai")
|
||||||
expect(JSON.stringify(fake.requests.at(-1))).not.toContain("reasoning_effort")
|
expect(JSON.stringify(fake.requests.at(-1))).not.toContain("reasoning_effort")
|
||||||
|
|||||||
@@ -1,115 +0,0 @@
|
|||||||
import { afterEach, describe, expect, test } from "bun:test"
|
|
||||||
import { readFileSync } from "node:fs"
|
|
||||||
import { join } from "node:path"
|
|
||||||
import { AnthropicClient, toAnthropicMessages } from "../src/provider/anthropic.ts"
|
|
||||||
import type { Message, ResolvedModel, StreamEvent } from "../src/provider/types.ts"
|
|
||||||
import { fakeProvider, type Fake } from "./fake-provider.ts"
|
|
||||||
|
|
||||||
let fake: Fake | undefined
|
|
||||||
afterEach(() => fake?.stop())
|
|
||||||
|
|
||||||
const model = (url: string, spec: ResolvedModel["spec"] = {}): ResolvedModel => ({
|
|
||||||
ref: "a/m",
|
|
||||||
connectionName: "a",
|
|
||||||
id: "m",
|
|
||||||
spec,
|
|
||||||
connection: { dialect: "anthropic", base_url: url, api_key: "k", models: {} },
|
|
||||||
})
|
|
||||||
const sse = (events: Record<string, unknown>[]) => events.map((e) => `event: ${e.type}\ndata: ${JSON.stringify(e)}\n\n`).join("")
|
|
||||||
async function run(c: AnthropicClient, messages: Message[] = [{ role: "user", parts: [{ type: "text", text: "hi" }] }], effort: any = null) {
|
|
||||||
const ev: StreamEvent[] = []
|
|
||||||
for await (const e of c.stream({ system: "sys", messages, tools: [{ name: "read", description: "read", parameters: { type: "object" } }], effort })) ev.push(e)
|
|
||||||
return ev
|
|
||||||
}
|
|
||||||
|
|
||||||
describe("anthropic dialect", () => {
|
|
||||||
test("a real vLLM stream: thinking with its signature, then a tool call", async () => {
|
|
||||||
fake = fakeProvider([{ chunks: [], raw: readFileSync(join(import.meta.dir, "fixtures/sse/vllm-anthropic-thinking-tool.sse"), "utf8") }])
|
|
||||||
const ev = await run(new AnthropicClient(model(fake.url.replace(/\/v1$/, ""))))
|
|
||||||
const fin = ev.find((e) => e.type === "finish")!
|
|
||||||
if (fin.type !== "finish") throw new Error()
|
|
||||||
expect(fin.reason).toBe("tool_calls")
|
|
||||||
const thinking = fin.message.parts.find((p) => p.type === "reasoning")
|
|
||||||
expect(thinking?.type === "reasoning" && thinking.signature).toBeTruthy()
|
|
||||||
expect(fin.message.parts.some((p) => p.type === "tool_call")).toBe(true)
|
|
||||||
expect(ev.some((e) => e.type === "reasoning")).toBe(true)
|
|
||||||
})
|
|
||||||
|
|
||||||
test("request: headers, max_tokens above the thinking budget, no sampling with thinking, cache breakpoints", async () => {
|
|
||||||
fake = fakeProvider([{ chunks: [], raw: sse([{ type: "message_start", message: { usage: { input_tokens: 5 } } }, { type: "message_stop" }]) }])
|
|
||||||
const c = new AnthropicClient(model(fake.url, { max_output: 4000, temperature: 0.5, cache: true, effort_map: { high: 10000 } }))
|
|
||||||
await run(c, undefined, "high")
|
|
||||||
const r = fake.requests[0]
|
|
||||||
expect(r.thinking).toEqual({ type: "enabled", budget_tokens: 10000 })
|
|
||||||
expect(r.max_tokens).toBeGreaterThan(10000)
|
|
||||||
expect(r.temperature).toBeUndefined()
|
|
||||||
expect(r.system[0].cache_control).toEqual({ type: "ephemeral" })
|
|
||||||
expect(r.tools.at(-1).cache_control).toEqual({ type: "ephemeral" })
|
|
||||||
expect(r.messages.at(-1).content.at(-1).cache_control).toEqual({ type: "ephemeral" })
|
|
||||||
})
|
|
||||||
|
|
||||||
test("stream: text, redacted thinking, tool input in pieces, cache usage, stop reasons", async () => {
|
|
||||||
fake = fakeProvider([
|
|
||||||
{
|
|
||||||
chunks: [],
|
|
||||||
raw: sse([
|
|
||||||
{ type: "message_start", message: { usage: { input_tokens: 10, cache_read_input_tokens: 90, output_tokens: 1 } } },
|
|
||||||
{ type: "content_block_start", index: 0, content_block: { type: "redacted_thinking", data: "OPAQUE" } },
|
|
||||||
{ type: "content_block_stop", index: 0 },
|
|
||||||
{ type: "content_block_start", index: 1, content_block: { type: "text", text: "" } },
|
|
||||||
{ type: "content_block_delta", index: 1, delta: { type: "text_delta", text: "Reading." } },
|
|
||||||
{ type: "content_block_stop", index: 1 },
|
|
||||||
{ type: "content_block_start", index: 2, content_block: { type: "tool_use", id: "toolu_1", name: "read", input: {} } },
|
|
||||||
{ type: "content_block_delta", index: 2, delta: { type: "input_json_delta", partial_json: '{"pa' } },
|
|
||||||
{ type: "content_block_delta", index: 2, delta: { type: "input_json_delta", partial_json: 'th":"a"}' } },
|
|
||||||
{ type: "content_block_stop", index: 2 },
|
|
||||||
{ type: "message_delta", delta: { stop_reason: "tool_use" }, usage: { output_tokens: 42 } },
|
|
||||||
{ type: "message_stop" },
|
|
||||||
]),
|
|
||||||
},
|
|
||||||
])
|
|
||||||
const ev = await run(new AnthropicClient(model(fake.url)))
|
|
||||||
const fin = ev.find((e) => e.type === "finish")
|
|
||||||
expect(fin?.type === "finish" && fin.message.parts).toEqual([
|
|
||||||
{ type: "reasoning", text: "", opaque: { redacted: "OPAQUE" } },
|
|
||||||
{ type: "text", text: "Reading." },
|
|
||||||
{ type: "tool_call", id: "toolu_1", name: "read", args: '{"path":"a"}' },
|
|
||||||
])
|
|
||||||
expect(ev.find((e) => e.type === "usage")).toEqual({ type: "usage", usage: { input: 100, output: 42, cached: 90 } })
|
|
||||||
expect(fake.requests[0]).toMatchObject({ model: "m", stream: true })
|
|
||||||
})
|
|
||||||
|
|
||||||
test("overloaded before any output is retried once", async () => {
|
|
||||||
fake = fakeProvider([
|
|
||||||
{ chunks: [], raw: sse([{ type: "message_start", message: { usage: { input_tokens: 1 } } }, { type: "error", error: { type: "overloaded_error", message: "Overloaded" } }]) },
|
|
||||||
{ chunks: [], raw: sse([{ type: "content_block_start", index: 0, content_block: { type: "text", text: "ok" } }, { type: "content_block_stop", index: 0 }, { type: "message_stop" }]) },
|
|
||||||
])
|
|
||||||
const ev = await run(new AnthropicClient(model(fake.url)))
|
|
||||||
expect(ev.some((e) => e.type === "notice" && e.message.includes("Overloaded"))).toBe(true)
|
|
||||||
const fin = ev.find((e) => e.type === "finish")
|
|
||||||
expect(fin?.type === "finish" && fin.message.parts).toEqual([{ type: "text", text: "ok" }])
|
|
||||||
})
|
|
||||||
|
|
||||||
test("history: tool results grouped as a user turn, ids made safe, thinking only with a signature and only while thinking", () => {
|
|
||||||
const msgs: Message[] = [
|
|
||||||
{ role: "user", parts: [{ type: "text", text: "go" }] },
|
|
||||||
{ role: "assistant", parts: [{ type: "reasoning", text: "unsigned (from another provider)" }, { type: "reasoning", text: "signed", signature: "SIG" }, { type: "tool_call", id: "call:1", name: "read", args: '{"path":"a"}' }, { type: "tool_call", id: "call:2", name: "read", args: "{}" }] },
|
|
||||||
{ role: "tool", callId: "call:1", name: "read", content: "A" },
|
|
||||||
{ role: "tool", callId: "call:2", name: "read", content: "", isError: true },
|
|
||||||
{ role: "user", parts: [{ type: "text", text: "and?" }] },
|
|
||||||
]
|
|
||||||
const on = toAnthropicMessages(msgs, false, true)
|
|
||||||
expect(on.map((m) => m.role)).toEqual(["user", "assistant", "user"])
|
|
||||||
expect(on[1]!.content).toEqual([
|
|
||||||
{ type: "thinking", thinking: "signed", signature: "SIG" },
|
|
||||||
{ type: "tool_use", id: "call_1", name: "read", input: { path: "a" } },
|
|
||||||
{ type: "tool_use", id: "call_2", name: "read", input: {} },
|
|
||||||
])
|
|
||||||
expect(on[2]!.content).toEqual([
|
|
||||||
{ type: "tool_result", tool_use_id: "call_1", content: "A" },
|
|
||||||
{ type: "tool_result", tool_use_id: "call_2", content: "(empty)", is_error: true },
|
|
||||||
{ type: "text", text: "and?" },
|
|
||||||
])
|
|
||||||
expect(toAnthropicMessages(msgs, false, false)[1]!.content.some((b) => b.type === "thinking")).toBe(false)
|
|
||||||
})
|
|
||||||
})
|
|
||||||
@@ -16,7 +16,7 @@ afterEach(() => fake?.stop())
|
|||||||
function app(edit: string, config = "") {
|
function app(edit: string, config = "") {
|
||||||
fake = fakeProvider([{ chunks: [toolCall(0, "b1", "bash", '{"command":"echo one"}')] }, { chunks: [delta({ content: "ok" }, "stop")] }])
|
fake = fakeProvider([{ chunks: [toolCall(0, "b1", "bash", '{"command":"echo one"}')] }, { chunks: [delta({ content: "ok" }, "stop")] }])
|
||||||
mkdirSync(paths.config, { recursive: true })
|
mkdirSync(paths.config, { recursive: true })
|
||||||
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n dialect: openai-chat\n base_url: ${fake.url}\n models: { m: {} }\n`, { mode: 0o600 })
|
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n type: webui\n url: ${fake.url}\n models: { f/m: {} }\n`, { mode: 0o600 })
|
||||||
writeFileSync(join(paths.config, "config.yaml"), `model: f/m\ntitles: prompt\n${config}`)
|
writeFileSync(join(paths.config, "config.yaml"), `model: f/m\ntitles: prompt\n${config}`)
|
||||||
return createApp({ cwd: mkdtempSync(join(tmpdir(), "ph-approve-")), mode: "manual", store: false, snapshots: false, asker: { ask: async (): Promise<AskReply> => ({ kind: "once", command: edit }) } })
|
return createApp({ cwd: mkdtempSync(join(tmpdir(), "ph-approve-")), mode: "manual", store: false, snapshots: false, asker: { ask: async (): Promise<AskReply> => ({ kind: "once", command: edit }) } })
|
||||||
}
|
}
|
||||||
|
|||||||
+1
-1
@@ -33,7 +33,7 @@ describe("the project board", () => {
|
|||||||
{ chunks: [delta({ content: "next" })] },
|
{ chunks: [delta({ content: "next" })] },
|
||||||
])
|
])
|
||||||
mkdirSync(paths.config, { recursive: true })
|
mkdirSync(paths.config, { recursive: true })
|
||||||
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n dialect: openai-chat\n base_url: ${fake.url}\n models: { m: {} }\n`, { mode: 0o600 })
|
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n type: webui\n url: ${fake.url}\n models: { f/m: {} }\n`, { mode: 0o600 })
|
||||||
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\n")
|
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\n")
|
||||||
const cwd = mkdtempSync(join(tmpdir(), "ph-board-"))
|
const cwd = mkdtempSync(join(tmpdir(), "ph-board-"))
|
||||||
mkdirSync(join(cwd, ".agent"))
|
mkdirSync(join(cwd, ".agent"))
|
||||||
|
|||||||
@@ -16,7 +16,7 @@ afterEach(() => fake?.stop())
|
|||||||
function app(limits: string, script: Parameters<typeof fakeProvider>[0], ask: () => Promise<AskReply> = async () => ({ kind: "once" })) {
|
function app(limits: string, script: Parameters<typeof fakeProvider>[0], ask: () => Promise<AskReply> = async () => ({ kind: "once" })) {
|
||||||
fake = fakeProvider(script)
|
fake = fakeProvider(script)
|
||||||
mkdirSync(paths.config, { recursive: true })
|
mkdirSync(paths.config, { recursive: true })
|
||||||
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n dialect: openai-chat\n base_url: ${fake.url}\n models: { m: {} }\n`, { mode: 0o600 })
|
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n type: webui\n url: ${fake.url}\n models: { f/m: {} }\n`, { mode: 0o600 })
|
||||||
writeFileSync(join(paths.config, "config.yaml"), `model: f/m\ntitles: prompt\nlimits: { ${limits} }\n`)
|
writeFileSync(join(paths.config, "config.yaml"), `model: f/m\ntitles: prompt\nlimits: { ${limits} }\n`)
|
||||||
const cwd = mkdtempSync(join(tmpdir(), "ph-budget-"))
|
const cwd = mkdtempSync(join(tmpdir(), "ph-budget-"))
|
||||||
writeFileSync(join(cwd, "big.txt"), "x".repeat(5000) + "\n")
|
writeFileSync(join(cwd, "big.txt"), "x".repeat(5000) + "\n")
|
||||||
|
|||||||
+7
-16
@@ -14,24 +14,15 @@ import { delta, fakeProvider, toolCall, type Fake } from "./fake-provider.ts"
|
|||||||
let fake: Fake | undefined
|
let fake: Fake | undefined
|
||||||
afterEach(() => fake?.stop())
|
afterEach(() => fake?.stop())
|
||||||
|
|
||||||
const model = (conn: string, id: string, o: { one?: boolean; single?: boolean } = {}): ResolvedModel => ({
|
const model = (conn: string, id: string, o: { single?: boolean } = {}): ResolvedModel => ({
|
||||||
ref: `${conn}/${id}`,
|
ref: `${conn}/${id}`,
|
||||||
connectionName: conn,
|
connectionName: "ai",
|
||||||
connection: { dialect: "openai-chat", base_url: "http://x/v1", models: {}, ...(o.one ? { one_model_at_a_time: true } : {}) } as ResolvedModel["connection"],
|
connection: { base_url: "http://x/v1", models: {} },
|
||||||
id,
|
id: `${conn}/${id}`,
|
||||||
spec: o.single ? { single_session: true } : {},
|
spec: o.single ? { single_session: true } : {},
|
||||||
})
|
})
|
||||||
|
|
||||||
test("one model at a time: another model of the same server is refused, the session's own is not", () => {
|
test("a model's group (the server's capacity.group): another model of the same GPU is refused, the session's own is not", () => {
|
||||||
const main = model("swap", "qwen", { one: true })
|
|
||||||
expect(capacityRefusal(main, model("swap", "gemma", { one: true }))).toContain("holds one model at a time")
|
|
||||||
expect(capacityRefusal(main, main)).toBe("")
|
|
||||||
expect(capacityRefusal(main, model("cloud", "big"))).toBe("")
|
|
||||||
// Without the flag, as before.
|
|
||||||
expect(capacityRefusal(model("swap", "qwen"), model("swap", "gemma"))).toBe("")
|
|
||||||
})
|
|
||||||
|
|
||||||
test("a model's group: models one connection serves from several servers are told apart", () => {
|
|
||||||
const grouped = (id: string, group?: string): ResolvedModel => ({ ...model("ai", id), spec: group ? { group } : {} })
|
const grouped = (id: string, group?: string): ResolvedModel => ({ ...model("ai", id), spec: group ? { group } : {} })
|
||||||
const main = grouped("qwen", "gpu")
|
const main = grouped("qwen", "gpu")
|
||||||
expect(capacityRefusal(main, grouped("gemma", "gpu"))).toContain("holds one model at a time")
|
expect(capacityRefusal(main, grouped("gemma", "gpu"))).toContain("holds one model at a time")
|
||||||
@@ -55,14 +46,14 @@ test("task on a model the server cannot hold beside the session's is refused, an
|
|||||||
mkdirSync(paths.config, { recursive: true })
|
mkdirSync(paths.config, { recursive: true })
|
||||||
writeFileSync(
|
writeFileSync(
|
||||||
join(paths.config, "connections.yaml"),
|
join(paths.config, "connections.yaml"),
|
||||||
`connections:\n f:\n dialect: openai-chat\n base_url: ${fake.url}\n one_model_at_a_time: true\n models: { m: {}, other: {} }\n`,
|
`connections:\n f:\n type: webui\n url: ${fake.url}\n models: { f/m: { group: gpu }, f/other: { group: gpu } }\n`,
|
||||||
{ mode: 0o600 },
|
{ mode: 0o600 },
|
||||||
)
|
)
|
||||||
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\n")
|
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\n")
|
||||||
const a = createApp({ cwd: mkdtempSync(join(tmpdir(), "ph-cap-")), mode: "edit", store: false, asker: { ask: async (): Promise<AskReply> => ({ kind: "once" }) } })
|
const a = createApp({ cwd: mkdtempSync(join(tmpdir(), "ph-cap-")), mode: "edit", store: false, asker: { ask: async (): Promise<AskReply> => ({ kind: "once" }) } })
|
||||||
await a.engine.prompt("get a second opinion")
|
await a.engine.prompt("get a second opinion")
|
||||||
// Two requests: the session's, then its next step — never one for f/other.
|
// Two requests: the session's, then its next step — never one for f/other.
|
||||||
expect(fake.requests.map((r: any) => r.model)).toEqual(["m", "m"])
|
expect(fake.requests.map((r: any) => r.model)).toEqual(["f/m", "f/m"])
|
||||||
const result = fake.requests[1].messages.at(-1).content as string
|
const result = fake.requests[1].messages.at(-1).content as string
|
||||||
expect(result).toContain("holds one model at a time")
|
expect(result).toContain("holds one model at a time")
|
||||||
expect(result).toContain("Do this part yourself")
|
expect(result).toContain("Do this part yourself")
|
||||||
|
|||||||
+29
-42
@@ -4,7 +4,7 @@ import { tmpdir } from "node:os"
|
|||||||
import { join } from "node:path"
|
import { join } from "node:path"
|
||||||
import { paths } from "../src/config/paths.ts"
|
import { paths } from "../src/config/paths.ts"
|
||||||
import { loadConfig, resolveKey } from "../src/config/load.ts"
|
import { loadConfig, resolveKey } from "../src/config/load.ts"
|
||||||
import { resolveModel } from "../src/provider/index.ts"
|
import { NOT_LOGGED_IN, resolveModel } from "../src/provider/index.ts"
|
||||||
|
|
||||||
const write = (file: string, text: string) => {
|
const write = (file: string, text: string) => {
|
||||||
mkdirSync(join(file, ".."), { recursive: true })
|
mkdirSync(join(file, ".."), { recursive: true })
|
||||||
@@ -17,7 +17,7 @@ beforeEach(() => {
|
|||||||
})
|
})
|
||||||
|
|
||||||
describe("config", () => {
|
describe("config", () => {
|
||||||
test("connections: substitution, a broken one does not break the others, mode warning", () => {
|
test("connections: the login, its key from a file; a custom connection is read past with a warning (2.0)", () => {
|
||||||
write(join(paths.config, "keyfile"), "sk-from-file\n")
|
write(join(paths.config, "keyfile"), "sk-from-file\n")
|
||||||
write(
|
write(
|
||||||
join(paths.config, "connections.yaml"),
|
join(paths.config, "connections.yaml"),
|
||||||
@@ -27,32 +27,40 @@ describe("config", () => {
|
|||||||
base_url: http://llm.example/v1
|
base_url: http://llm.example/v1
|
||||||
api_key: "{env:PH_TEST_KEY}"
|
api_key: "{env:PH_TEST_KEY}"
|
||||||
models:
|
models:
|
||||||
qwen: { context: 131072, max_output: 8192, efforts: [low, high], effort: high }
|
qwen: { context: 131072 }
|
||||||
file:
|
example:
|
||||||
dialect: openai-chat
|
type: webui
|
||||||
base_url: http://x/v1
|
url: https://ai.example.org
|
||||||
api_key: "{file:${join(paths.config, "keyfile")}}"
|
api_key: "{file:${join(paths.config, "keyfile")}}"
|
||||||
models: { a: {} }
|
models:
|
||||||
broken:
|
llama/qwen: { context: 131072, max_output: 8192, efforts: [low, high], effort: high }
|
||||||
dialect: openai-chat
|
|
||||||
base_url: http://y/v1
|
|
||||||
api_key: "{env:PH_MISSING}"
|
|
||||||
models: { b: {} }
|
|
||||||
`,
|
`,
|
||||||
)
|
)
|
||||||
chmodSync(join(paths.config, "connections.yaml"), 0o644)
|
chmodSync(join(paths.config, "connections.yaml"), 0o644)
|
||||||
const l = loadConfig()
|
const l = loadConfig()
|
||||||
expect(l.connections.swap!.api_key).toBe("sk-test")
|
expect(Object.keys(l.connections)).toEqual(["example"])
|
||||||
expect(resolveKey("file", l.connections.file!)).toBe("sk-from-file")
|
expect(l.connections.example!.base_url).toBe("https://ai.example.org/v1")
|
||||||
expect(l.broken.broken).toContain("PH_MISSING")
|
expect(resolveKey("example", l.connections.example!)).toBe("sk-from-file")
|
||||||
|
expect(l.warnings.some((w) => w.includes("swap is a custom connection") && w.includes("lembas login"))).toBe(true)
|
||||||
expect(l.warnings.some((w) => w.includes("chmod 600"))).toBe(true)
|
expect(l.warnings.some((w) => w.includes("chmod 600"))).toBe(true)
|
||||||
expect(resolveModel(l, "swap/qwen").spec.context).toBe(131072)
|
expect(resolveModel(l, "llama/qwen").spec.context).toBe(131072)
|
||||||
expect(() => resolveModel(l, "broken/b")).toThrow("unusable")
|
// The older form, the login's name in front, names the same model.
|
||||||
expect(() => resolveModel(l, "swap/nope")).toThrow('no model "nope"')
|
expect(resolveModel(l, "example/llama/qwen").ref).toBe("llama/qwen")
|
||||||
|
// One the server's list does not have yet is asked for, not refused here: the server decides.
|
||||||
|
expect(resolveModel(l, "llama/new")).toMatchObject({ ref: "llama/new", id: "llama/new", spec: {} })
|
||||||
})
|
})
|
||||||
|
|
||||||
test("key_cmd", () => {
|
test("no login: no model, and the error says how to log in", () => {
|
||||||
expect(resolveKey("k", { dialect: "openai-chat", base_url: "http://x", key_cmd: "echo sk-cmd", models: {} })).toBe("sk-cmd")
|
write(join(paths.config, "connections.yaml"), "connections: {}\n")
|
||||||
|
expect(() => resolveModel(loadConfig(), undefined)).toThrow(NOT_LOGGED_IN)
|
||||||
|
expect(NOT_LOGGED_IN).toContain("lembas login <address>")
|
||||||
|
})
|
||||||
|
|
||||||
|
test("a login whose key file is gone is unusable, said as such", () => {
|
||||||
|
write(join(paths.config, "connections.yaml"), `connections:\n example:\n type: webui\n url: https://ai.example.org\n api_key: "{env:PH_MISSING}"\n`)
|
||||||
|
const l = loadConfig()
|
||||||
|
expect(l.broken.example).toContain("PH_MISSING")
|
||||||
|
expect(() => resolveModel(l, undefined)).toThrow("unusable")
|
||||||
})
|
})
|
||||||
|
|
||||||
test("project config: merged when trusted, ignored when not, global-only keys refused", () => {
|
test("project config: merged when trusted, ignored when not, global-only keys refused", () => {
|
||||||
@@ -76,34 +84,13 @@ describe("config", () => {
|
|||||||
})
|
})
|
||||||
})
|
})
|
||||||
|
|
||||||
test("effort_map may name only some efforts (zod 4 records keyed by an enum are exhaustive)", async () => {
|
|
||||||
const { ModelSpec } = await import("../src/config/schema.ts")
|
|
||||||
expect(ModelSpec.safeParse({ effort_map: { high: 10000 } }).success).toBe(true)
|
|
||||||
expect(ModelSpec.safeParse({ effort_map: { huge: 1 } }).success).toBe(false)
|
|
||||||
})
|
|
||||||
|
|
||||||
test("a URL may be {env:}/{file:}: accepted as written, and checked once filled in", () => {
|
test("a URL may be {env:}/{file:}: accepted as written, and checked once filled in", () => {
|
||||||
process.env.PH_URL = "http://llm.example/v1"
|
process.env.PH_URL = "http://llm.example/v1"
|
||||||
process.env.PH_NOT_URL = "not a url"
|
process.env.PH_NOT_URL = "not a url"
|
||||||
write(
|
write(join(paths.config, "connections.yaml"), "connections: {}\n")
|
||||||
join(paths.config, "connections.yaml"),
|
|
||||||
`connections:
|
|
||||||
ok: { dialect: openai-chat, base_url: "{env:PH_URL}", models: { m: {} } }
|
|
||||||
bad: { dialect: openai-chat, base_url: "{env:PH_NOT_URL}", models: { m: {} } }
|
|
||||||
typo: { dialect: openai-chat, base_url: "{env:PH_URL", models: { m: {} } }
|
|
||||||
`,
|
|
||||||
)
|
|
||||||
chmodSync(join(paths.config, "connections.yaml"), 0o600)
|
chmodSync(join(paths.config, "connections.yaml"), 0o600)
|
||||||
write(join(paths.config, "config.yaml"), 'search:\n searxng: { base_url: "{env:PH_URL}" }\n firecrawl: { base_url: "{env:PH_NOT_URL}" }\n')
|
write(join(paths.config, "config.yaml"), 'search:\n searxng: { base_url: "{env:PH_URL}" }\n firecrawl: { base_url: "{env:PH_NOT_URL}" }\n')
|
||||||
expect(() => loadConfig()).toThrow(/typo\.base_url: Invalid URL/)
|
|
||||||
write(join(paths.config, "connections.yaml"), `connections:
|
|
||||||
ok: { dialect: openai-chat, base_url: "{env:PH_URL}", models: { m: {} } }
|
|
||||||
bad: { dialect: openai-chat, base_url: "{env:PH_NOT_URL}", models: { m: {} } }
|
|
||||||
`)
|
|
||||||
chmodSync(join(paths.config, "connections.yaml"), 0o600)
|
|
||||||
const l = loadConfig()
|
const l = loadConfig()
|
||||||
expect(l.connections.ok!.base_url).toBe("http://llm.example/v1")
|
|
||||||
expect(l.broken.bad).toContain("base_url: Invalid URL")
|
|
||||||
expect(l.config.search?.searxng?.base_url).toBe("http://llm.example/v1")
|
expect(l.config.search?.searxng?.base_url).toBe("http://llm.example/v1")
|
||||||
expect(l.config.search?.firecrawl).toBeUndefined()
|
expect(l.config.search?.firecrawl).toBeUndefined()
|
||||||
expect(l.warnings.some((w) => w.includes("firecrawl is off"))).toBe(true)
|
expect(l.warnings.some((w) => w.includes("firecrawl is off"))).toBe(true)
|
||||||
|
|||||||
+3
-21
@@ -5,10 +5,7 @@ import { join } from "node:path"
|
|||||||
import { createApp } from "../src/app.ts"
|
import { createApp } from "../src/app.ts"
|
||||||
import type { AskReply, Event } from "../src/bus/index.ts"
|
import type { AskReply, Event } from "../src/bus/index.ts"
|
||||||
import { paths } from "../src/config/paths.ts"
|
import { paths } from "../src/config/paths.ts"
|
||||||
import { discoverContext } from "../src/provider/discover.ts"
|
|
||||||
import { resetLearned } from "../src/provider/learned.ts"
|
import { resetLearned } from "../src/provider/learned.ts"
|
||||||
import { OpenAIChatClient } from "../src/provider/openai-chat.ts"
|
|
||||||
import type { ResolvedModel } from "../src/provider/types.ts"
|
|
||||||
import { contextBreakdown, renderBreakdown } from "../src/session/context.ts"
|
import { contextBreakdown, renderBreakdown } from "../src/session/context.ts"
|
||||||
import { delta, fakeProvider, toolCall, usage, type Fake } from "./fake-provider.ts"
|
import { delta, fakeProvider, toolCall, usage, type Fake } from "./fake-provider.ts"
|
||||||
|
|
||||||
@@ -18,7 +15,6 @@ afterEach(() => {
|
|||||||
resetLearned()
|
resetLearned()
|
||||||
})
|
})
|
||||||
|
|
||||||
const rm = (url: string, id: string): ResolvedModel => ({ ref: `c/${id}`, connectionName: "c", id, spec: {}, connection: { dialect: "openai-chat", base_url: url, models: {} } })
|
|
||||||
|
|
||||||
function app(yaml: string, script: Parameters<typeof fakeProvider>[0], config = "") {
|
function app(yaml: string, script: Parameters<typeof fakeProvider>[0], config = "") {
|
||||||
fake = fakeProvider(script)
|
fake = fakeProvider(script)
|
||||||
@@ -31,24 +27,10 @@ function app(yaml: string, script: Parameters<typeof fakeProvider>[0], config =
|
|||||||
}
|
}
|
||||||
|
|
||||||
describe("context", () => {
|
describe("context", () => {
|
||||||
test("discovery: the model list first, then llama-server /props through llama-swap's passthrough", async () => {
|
|
||||||
fake = fakeProvider([])
|
|
||||||
expect(await discoverContext(rm(fake.url, "m1"), new OpenAIChatClient(rm(fake.url, "m1")))).toBe(32768)
|
|
||||||
expect(await discoverContext(rm(fake.url, "swapped"), new OpenAIChatClient(rm(fake.url, "swapped")))).toBe(16384)
|
|
||||||
expect(fake.calls.map((c) => c.path)).toContain("/upstream/swapped/props")
|
|
||||||
})
|
|
||||||
|
|
||||||
test("switching to another connection calls the old one's unload_url", async () => {
|
|
||||||
const a = app(`connections:\n f:\n dialect: openai-chat\n base_url: URL\n unload_url: URL/../unload\n unload_method: GET\n models: { m: {} }\n g:\n dialect: openai-chat\n base_url: URL\n models: { n: {} }\n`, [])
|
|
||||||
a.switchModel("g/n")
|
|
||||||
await Bun.sleep(100)
|
|
||||||
expect(fake!.calls.some((c) => c.path.endsWith("/unload"))).toBe(true)
|
|
||||||
})
|
|
||||||
|
|
||||||
test("past auto_at: nothing already sent is cut; the conversation is compacted, with a table of the reads", async () => {
|
test("past auto_at: nothing already sent is cut; the conversation is compacted, with a table of the reads", async () => {
|
||||||
// window 16000, auto_at 0.6 → limit 9600. Harness spec 3.0.0 (context.md): old tool outputs are
|
// window 16000, auto_at 0.6 → limit 9600. Harness spec 3.0.0 (context.md): old tool outputs are
|
||||||
// never pruned in place — that would make the server read everything after them again.
|
// never pruned in place — that would make the server read everything after them again.
|
||||||
const a = app(`connections:\n f:\n dialect: openai-chat\n base_url: URL\n models: { m: { context: 16000 } }\n`, [
|
const a = app(`connections:\n f:\n type: webui\n url: URL\n models: { f/m: { context: 16000 } }\n`, [
|
||||||
{ chunks: [toolCall(0, "c1", "read", '{"path":"big.txt"}'), usage(1000, 10)] },
|
{ chunks: [toolCall(0, "c1", "read", '{"path":"big.txt"}'), usage(1000, 10)] },
|
||||||
{ chunks: [toolCall(0, "c2", "read", '{"path":"big2.txt"}'), usage(6600, 10)] },
|
{ chunks: [toolCall(0, "c2", "read", '{"path":"big2.txt"}'), usage(6600, 10)] },
|
||||||
{ chunks: [delta({ content: "## Summary\nRead two files." })] },
|
{ chunks: [delta({ content: "## Summary\nRead two files." })] },
|
||||||
@@ -75,7 +57,7 @@ describe("context", () => {
|
|||||||
})
|
})
|
||||||
|
|
||||||
test("too full: compacted mid-task, then told to carry on", async () => {
|
test("too full: compacted mid-task, then told to carry on", async () => {
|
||||||
const a = app(`connections:\n f:\n dialect: openai-chat\n base_url: URL\n models: { m: { context: 4000 } }\n`, [
|
const a = app(`connections:\n f:\n type: webui\n url: URL\n models: { f/m: { context: 4000 } }\n`, [
|
||||||
{ chunks: [delta({ content: "y".repeat(8000) }), toolCall(0, "c1", "list", "{}"), usage(3900, 2000)] },
|
{ chunks: [delta({ content: "y".repeat(8000) }), toolCall(0, "c1", "list", "{}"), usage(3900, 2000)] },
|
||||||
{ chunks: [delta({ content: "## What we are doing\nA long task." })] },
|
{ chunks: [delta({ content: "## What we are doing\nA long task." })] },
|
||||||
{ chunks: [delta({ content: "carried on" }), usage(500, 5)] },
|
{ chunks: [delta({ content: "carried on" }), usage(500, 5)] },
|
||||||
@@ -93,7 +75,7 @@ describe("context", () => {
|
|||||||
})
|
})
|
||||||
|
|
||||||
test("/context: parts of the system prompt, tools, the conversation, against the window", async () => {
|
test("/context: parts of the system prompt, tools, the conversation, against the window", async () => {
|
||||||
const a = app(`connections:\n f:\n dialect: openai-chat\n base_url: URL\n models: { m: { context: 32768 } }\n`, [{ chunks: [toolCall(0, "c1", "read", '{"path":"big.txt"}'), usage(1500, 20)] }, { chunks: [delta({ content: "ok" }), usage(3100, 5)] }])
|
const a = app(`connections:\n f:\n type: webui\n url: URL\n models: { f/m: { context: 32768 } }\n`, [{ chunks: [toolCall(0, "c1", "read", '{"path":"big.txt"}'), usage(1500, 20)] }, { chunks: [delta({ content: "ok" }), usage(3100, 5)] }])
|
||||||
await a.engine.prompt("read it")
|
await a.engine.prompt("read it")
|
||||||
const system = a.engine.o.system(a.engine.mode, a.engine.model)
|
const system = a.engine.o.system(a.engine.mode, a.engine.model)
|
||||||
const text = renderBreakdown(contextBreakdown(a.engine, system, [], undefined, a.engine.o.tools))
|
const text = renderBreakdown(contextBreakdown(a.engine, system, [], undefined, a.engine.o.tools))
|
||||||
|
|||||||
@@ -19,7 +19,7 @@ function setup(script: Parameters<typeof fakeProvider>[0]) {
|
|||||||
mkdirSync(paths.config, { recursive: true })
|
mkdirSync(paths.config, { recursive: true })
|
||||||
writeFileSync(
|
writeFileSync(
|
||||||
join(paths.config, "connections.yaml"),
|
join(paths.config, "connections.yaml"),
|
||||||
`connections:\n fake:\n dialect: openai-chat\n base_url: ${fake.url}\n models:\n m: { context: 32768, max_output: 1000 }\n`,
|
`connections:\n fake:\n type: webui\n url: ${fake.url}\n models:\n fake/m: { context: 32768, max_output: 1000 }\n`,
|
||||||
{ mode: 0o600 },
|
{ mode: 0o600 },
|
||||||
)
|
)
|
||||||
writeFileSync(join(paths.config, "config.yaml"), "model: fake/m\n")
|
writeFileSync(join(paths.config, "config.yaml"), "model: fake/m\n")
|
||||||
@@ -113,7 +113,7 @@ describe("the connection timeout", () => {
|
|||||||
const dir = setup(script)
|
const dir = setup(script)
|
||||||
writeFileSync(
|
writeFileSync(
|
||||||
join(paths.config, "connections.yaml"),
|
join(paths.config, "connections.yaml"),
|
||||||
`connections:\n fake:\n dialect: openai-chat\n base_url: ${fake!.url}\n timeout: 0.4\n models:\n m: { context: 32768 }\n`,
|
`connections:\n fake:\n type: webui\n url: ${fake!.url}\n timeout: 0.4\n models:\n fake/m: { context: 32768 }\n`,
|
||||||
{ mode: 0o600 },
|
{ mode: 0o600 },
|
||||||
)
|
)
|
||||||
return dir
|
return dir
|
||||||
|
|||||||
+1
-1
@@ -17,7 +17,7 @@ function setup(script: Parameters<typeof fakeProvider>[0]) {
|
|||||||
mkdirSync(paths.config, { recursive: true })
|
mkdirSync(paths.config, { recursive: true })
|
||||||
writeFileSync(
|
writeFileSync(
|
||||||
join(paths.config, "connections.yaml"),
|
join(paths.config, "connections.yaml"),
|
||||||
`connections:\n fake:\n dialect: openai-chat\n base_url: ${fake.url}\n models:\n m: { context: 32768 }\n`,
|
`connections:\n fake:\n type: webui\n url: ${fake.url}\n models:\n fake/m: { context: 32768 }\n`,
|
||||||
{ mode: 0o600 },
|
{ mode: 0o600 },
|
||||||
)
|
)
|
||||||
writeFileSync(join(paths.config, "config.yaml"), "model: fake/m\n")
|
writeFileSync(join(paths.config, "config.yaml"), "model: fake/m\n")
|
||||||
|
|||||||
+11
-5
@@ -11,8 +11,12 @@ export interface Fake {
|
|||||||
stop(): void
|
stop(): void
|
||||||
}
|
}
|
||||||
|
|
||||||
/** `show`: what Ollama's /api/show answers (404 when not given); it takes no scripted response. */
|
/** `route`: another fake that answers for a model (by the id in the request), as one LLeMbas server
|
||||||
export function fakeProvider(script: Scripted[], opts: { show?: unknown } = {}): Fake {
|
* serves models from several backends — the request is passed to it whole, and recorded there. */
|
||||||
|
/** `served`: what GET /v1/models lists, as a LLeMbas server lists its models (default two
|
||||||
|
* placeholders). Any other GET — the instance's own description, its personalization — is a 404,
|
||||||
|
* which a client reads as "this server does not have it". */
|
||||||
|
export function fakeProvider(script: Scripted[], opts: { route?: (model: string) => Fake | undefined; served?: Record<string, unknown>[] } = {}): Fake {
|
||||||
const requests: any[] = []
|
const requests: any[] = []
|
||||||
const calls: { path: string; headers: Record<string, string> }[] = []
|
const calls: { path: string; headers: Record<string, string> }[] = []
|
||||||
let i = 0
|
let i = 0
|
||||||
@@ -21,11 +25,13 @@ export function fakeProvider(script: Scripted[], opts: { show?: unknown } = {}):
|
|||||||
async fetch(req) {
|
async fetch(req) {
|
||||||
const url = new URL(req.url)
|
const url = new URL(req.url)
|
||||||
if (req.method !== "POST" || url.pathname.endsWith("/unload")) calls.push({ path: url.pathname + url.search, headers: Object.fromEntries(req.headers.entries()) })
|
if (req.method !== "POST" || url.pathname.endsWith("/unload")) calls.push({ path: url.pathname + url.search, headers: Object.fromEntries(req.headers.entries()) })
|
||||||
if (url.pathname.endsWith("/models")) return Response.json({ data: [{ id: "m1", meta: { n_ctx: 32768 } }, { id: "m2", max_model_len: 8192 }] })
|
if (url.pathname.endsWith("/models")) return Response.json({ data: opts.served ?? [{ id: "m1", meta: { n_ctx: 32768 } }, { id: "m2", max_model_len: 8192 }] })
|
||||||
if (url.pathname.endsWith("/props")) return url.pathname.includes("/upstream/swapped/") ? Response.json({ default_generation_settings: { n_ctx: 16384 } }) : new Response("no", { status: 404 })
|
if (url.pathname.endsWith("/props")) return url.pathname.includes("/upstream/swapped/") ? Response.json({ default_generation_settings: { n_ctx: 16384 } }) : new Response("no", { status: 404 })
|
||||||
if (url.pathname.endsWith("/unload")) return new Response("ok")
|
if (url.pathname.endsWith("/unload")) return new Response("ok")
|
||||||
if (url.pathname.endsWith("/api/show")) return opts.show ? Response.json(opts.show) : new Response("not found", { status: 404 })
|
if (req.method === "GET") return new Response("not found", { status: 404 })
|
||||||
const body = await req.json()
|
const body = (await req.json()) as any
|
||||||
|
const to = typeof body?.model === "string" ? opts.route?.(body.model) : undefined
|
||||||
|
if (to) return fetch(to.url.replace(/\/v1$/, "") + url.pathname + url.search, { method: "POST", headers: req.headers, body: JSON.stringify(body) })
|
||||||
requests.push(body)
|
requests.push(body)
|
||||||
calls.push({ path: url.pathname + url.search, headers: Object.fromEntries(req.headers.entries()) })
|
calls.push({ path: url.pathname + url.search, headers: Object.fromEntries(req.headers.entries()) })
|
||||||
const r = script[i++]
|
const r = script[i++]
|
||||||
|
|||||||
@@ -1,225 +0,0 @@
|
|||||||
// Gemini and Ollama are UNTESTED against real servers (the wiki says so). These fixtures follow the
|
|
||||||
// documented shapes and what Hermes Agent and OpenCode handle.
|
|
||||||
import { afterEach, describe, expect, test } from "bun:test"
|
|
||||||
import { DEFAULT_MAX_OUTPUT, GeminiClient, geminiJsonSchema, geminiSchema, SKIP_SIGNATURE, thinkingConfig, toGeminiContents } from "../src/provider/gemini.ts"
|
|
||||||
import { OllamaClient, parseShow } from "../src/provider/ollama.ts"
|
|
||||||
import type { Message, ResolvedModel, StreamEvent } from "../src/provider/types.ts"
|
|
||||||
import { BUILTIN_TOOLS } from "../src/tool/registry.ts"
|
|
||||||
import { toSpec } from "../src/tool/tool.ts"
|
|
||||||
import { fakeProvider, type Fake } from "./fake-provider.ts"
|
|
||||||
|
|
||||||
let fake: Fake | undefined
|
|
||||||
afterEach(() => fake?.stop())
|
|
||||||
|
|
||||||
const m = (dialect: "gemini" | "ollama", url: string, id: string, spec: ResolvedModel["spec"] = {}): ResolvedModel => ({
|
|
||||||
ref: `x/${id}`,
|
|
||||||
connectionName: "x",
|
|
||||||
id,
|
|
||||||
spec,
|
|
||||||
connection: { dialect, base_url: url, api_key: "KEY", models: {} },
|
|
||||||
})
|
|
||||||
async function run(c: GeminiClient | OllamaClient, messages: Message[] = [{ role: "user", parts: [{ type: "text", text: "hi" }] }], effort: any = null) {
|
|
||||||
const ev: StreamEvent[] = []
|
|
||||||
for await (const e of c.stream({ system: "sys", messages, tools: BUILTIN_TOOLS.map(toSpec).slice(0, 2), effort })) ev.push(e)
|
|
||||||
return ev
|
|
||||||
}
|
|
||||||
const finishOf = (ev: StreamEvent[]) => {
|
|
||||||
const f = ev.find((e) => e.type === "finish")
|
|
||||||
if (!f || f.type !== "finish") throw new Error("no finish")
|
|
||||||
return f
|
|
||||||
}
|
|
||||||
|
|
||||||
describe("gemini", () => {
|
|
||||||
test("every built-in tool's schema is reduced to what Gemini accepts", () => {
|
|
||||||
const allowed = new Set(["type", "format", "title", "description", "nullable", "enum", "maxItems", "minItems", "properties", "required", "minProperties", "maxProperties", "minLength", "maxLength", "pattern", "example", "anyOf", "propertyOrdering", "default", "items", "minimum", "maximum"])
|
|
||||||
const walk = (s: any, path: string) => {
|
|
||||||
if (!s || typeof s !== "object") return
|
|
||||||
for (const [k, v] of Object.entries(s)) {
|
|
||||||
if (k === "properties") for (const [p, sub] of Object.entries(v as object)) walk(sub, `${path}.${p}`)
|
|
||||||
else {
|
|
||||||
expect(allowed.has(k) ? k : `${path}: ${k}`).toBe(k)
|
|
||||||
if (k === "items" || k === "anyOf") walk(v, path)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
for (const t of BUILTIN_TOOLS.map(toSpec)) walk(geminiSchema(t.parameters), t.name)
|
|
||||||
expect(geminiSchema({ type: ["string", "null"], additionalProperties: false })).toEqual({ type: "string", nullable: true })
|
|
||||||
})
|
|
||||||
|
|
||||||
test("subset schema: unions keep every branch, enums become strings, required only names what exists", () => {
|
|
||||||
expect(geminiSchema({ type: ["array", "string", "null"], items: { type: "string", $comment: "x" }, minItems: 1 })).toEqual({
|
|
||||||
anyOf: [{ type: "array", items: { type: "string" }, minItems: 1 }, { type: "string" }],
|
|
||||||
nullable: true,
|
|
||||||
})
|
|
||||||
expect(geminiSchema({ type: "integer", enum: [1, 2, 2, null] })).toEqual({ type: "integer", enum: ["1", "2"] })
|
|
||||||
expect(geminiSchema({ type: "object", properties: { a: { type: "string" } }, required: ["a", "b"] })).toEqual({ type: "object", properties: { a: { type: "string" } }, required: ["a"] })
|
|
||||||
expect(geminiSchema({ type: "object", required: ["b"] })).toEqual({ type: "object" })
|
|
||||||
})
|
|
||||||
|
|
||||||
test("JSON Schema (v1beta): local $refs inlined with their siblings, $defs and $schema gone; a circular one goes as it is", () => {
|
|
||||||
const s = { $schema: "x", type: "object", $defs: { P: { type: "string", description: "p" } }, properties: { a: { $ref: "#/$defs/P", description: "mine" } } }
|
|
||||||
expect(geminiJsonSchema(s)).toEqual({ type: "object", properties: { a: { type: "string", description: "mine" } } })
|
|
||||||
const loop = { type: "object", $defs: { N: { type: "object", properties: { next: { $ref: "#/$defs/N" } } } }, properties: { n: { $ref: "#/$defs/N" } } }
|
|
||||||
expect(geminiJsonSchema(loop)).toEqual(loop)
|
|
||||||
expect(geminiJsonSchema({ type: "object" })).toEqual({ type: "object", properties: {} })
|
|
||||||
expect(geminiJsonSchema(undefined)).toEqual({ type: "object", properties: {} })
|
|
||||||
})
|
|
||||||
|
|
||||||
test("thinking: budgets for 2.x (2.5 Pro to 32k), levels for 3 fitted to what each model has", () => {
|
|
||||||
expect(thinkingConfig("gemini-2.5-pro", "max")).toEqual({ thinkingBudget: 32768, includeThoughts: true })
|
|
||||||
expect(thinkingConfig("models/gemini-2.5-flash", "max")).toEqual({ thinkingBudget: 24576, includeThoughts: true })
|
|
||||||
expect(thinkingConfig("gemini-2.5-flash", "low", { low: 500 })).toEqual({ thinkingBudget: 500, includeThoughts: true })
|
|
||||||
const level = (id: string, e: any) => thinkingConfig(id, e).thinkingLevel
|
|
||||||
expect(level("gemini-3-pro", "minimal")).toBe("low")
|
|
||||||
expect(level("gemini-3-pro", "medium")).toBe("medium")
|
|
||||||
expect(level("gemini-3-flash", "minimal")).toBe("minimal")
|
|
||||||
expect(level("gemini-3.1-pro", "xhigh")).toBe("high")
|
|
||||||
expect(level("gemma-4-31b-it", "medium")).toBe("high")
|
|
||||||
})
|
|
||||||
|
|
||||||
test("request: URL, key header, system, thinking budget (2.5) or level (3); stream: thoughts, text, a call with its signature", async () => {
|
|
||||||
const frames = [
|
|
||||||
{ candidates: [{ content: { role: "model", parts: [{ text: "Let me look.", thought: true }] } }] },
|
|
||||||
{ candidates: [{ content: { role: "model", parts: [{ text: "Reading the file." }] } }] },
|
|
||||||
{ candidates: [{ content: { role: "model", parts: [{ functionCall: { name: "read", args: { path: "a.ts" } }, thoughtSignature: "SIG" }] }, finishReason: "STOP" }], usageMetadata: { promptTokenCount: 50, candidatesTokenCount: 10, thoughtsTokenCount: 5, cachedContentTokenCount: 20 } },
|
|
||||||
]
|
|
||||||
fake = fakeProvider([{ chunks: frames }, { chunks: frames }])
|
|
||||||
const base = fake.url.replace(/\/v1$/, "/v1beta")
|
|
||||||
const ev = await run(new GeminiClient(m("gemini", base, "gemini-2.5-pro", { max_output: 1000 })), undefined, "medium")
|
|
||||||
expect(fake.calls[0]!.path).toBe("/v1beta/models/gemini-2.5-pro:streamGenerateContent?alt=sse")
|
|
||||||
expect(fake.calls[0]!.headers["x-goog-api-key"]).toBe("KEY")
|
|
||||||
expect(fake.calls[0]!.headers.authorization).toBeUndefined()
|
|
||||||
const r = fake.requests[0]
|
|
||||||
expect(r.systemInstruction).toEqual({ parts: [{ text: "sys" }] })
|
|
||||||
expect(r.generationConfig).toEqual({ maxOutputTokens: 1000, thinkingConfig: { thinkingBudget: 8192, includeThoughts: true } })
|
|
||||||
expect(r.tools[0].functionDeclarations[0].name).toBe("read")
|
|
||||||
expect(r.tools[0].functionDeclarations[0].parametersJsonSchema.type).toBe("object")
|
|
||||||
expect(r.tools[0].functionDeclarations[0].parameters).toBeUndefined()
|
|
||||||
const fin = finishOf(ev)
|
|
||||||
expect(fin.reason).toBe("tool_calls")
|
|
||||||
expect(fin.message.parts).toEqual([
|
|
||||||
{ type: "reasoning", text: "Let me look." },
|
|
||||||
{ type: "text", text: "Reading the file." },
|
|
||||||
{ type: "tool_call", id: "gemini_call_0", name: "read", args: '{"path":"a.ts"}', signature: "SIG" },
|
|
||||||
])
|
|
||||||
expect(ev.find((e) => e.type === "usage")).toEqual({ type: "usage", usage: { input: 50, output: 15, reasoning: 5, cached: 20 } })
|
|
||||||
await run(new GeminiClient(m("gemini", base, "gemini-3-pro")), undefined, "high")
|
|
||||||
expect(fake.requests[1].generationConfig).toEqual({ maxOutputTokens: DEFAULT_MAX_OUTPUT, thinkingConfig: { thinkingLevel: "high", includeThoughts: true } })
|
|
||||||
})
|
|
||||||
|
|
||||||
test("another API version gets the OpenAPI subset as `parameters`", async () => {
|
|
||||||
fake = fakeProvider([{ chunks: [{ candidates: [{ content: { parts: [{ text: "ok" }] }, finishReason: "STOP" }] }] }])
|
|
||||||
await run(new GeminiClient(m("gemini", fake.url, "gemini-2.0-flash")))
|
|
||||||
const d = fake.requests[0].tools[0].functionDeclarations[0]
|
|
||||||
expect(d.parametersJsonSchema).toBeUndefined()
|
|
||||||
expect(d.parameters.type).toBe("object")
|
|
||||||
})
|
|
||||||
|
|
||||||
test("stream: a call sent again stays one call; a different one is a new call; Gemini 3 ids are kept", async () => {
|
|
||||||
const call = (args: unknown, extra: Record<string, unknown> = {}) => ({ candidates: [{ content: { parts: [{ functionCall: { name: "read", args, ...extra } }] } }] })
|
|
||||||
fake = fakeProvider([
|
|
||||||
{ chunks: [call({ path: "a", limit: 5 }), call({ limit: 5, path: "a" }), call({ path: "b" }), { candidates: [{ finishReason: "STOP" }] }] },
|
|
||||||
{ chunks: [call({ path: "c" }, { id: "fc_1" }), { candidates: [{ finishReason: "SAFETY" }] }] },
|
|
||||||
])
|
|
||||||
const base = fake.url.replace(/\/v1$/, "/v1beta")
|
|
||||||
const fin = finishOf(await run(new GeminiClient(m("gemini", base, "gemini-2.5-flash"))))
|
|
||||||
expect(fin.message.parts).toEqual([
|
|
||||||
{ type: "tool_call", id: "gemini_call_0", name: "read", args: '{"limit":5,"path":"a"}' },
|
|
||||||
{ type: "tool_call", id: "gemini_call_1", name: "read", args: '{"path":"b"}' },
|
|
||||||
])
|
|
||||||
const fin3 = finishOf(await run(new GeminiClient(m("gemini", base, "gemini-3-pro"))))
|
|
||||||
expect(fin3.message.parts).toEqual([{ type: "tool_call", id: "fc_1", name: "read", args: '{"path":"c"}' }])
|
|
||||||
expect(fin3.reason).toBe("tool_calls")
|
|
||||||
})
|
|
||||||
|
|
||||||
test("history: the signature rides back on its call; results answer by name; roles merge", () => {
|
|
||||||
const c = toGeminiContents(
|
|
||||||
[
|
|
||||||
{ role: "user", parts: [{ type: "text", text: "go" }] },
|
|
||||||
{ role: "assistant", parts: [{ type: "reasoning", text: "x" }, { type: "tool_call", id: "gemini_call_0", name: "read", args: '{"path":"a"}', signature: "SIG" }] },
|
|
||||||
{ role: "tool", callId: "gemini_call_0", name: "read", content: "A" },
|
|
||||||
{ role: "user", parts: [{ type: "text", text: "and?" }] },
|
|
||||||
],
|
|
||||||
false,
|
|
||||||
)
|
|
||||||
expect(c).toEqual([
|
|
||||||
{ role: "user", parts: [{ text: "go" }] },
|
|
||||||
{ role: "model", parts: [{ functionCall: { name: "read", args: { path: "a" } }, thoughtSignature: "SIG" }] },
|
|
||||||
{ role: "user", parts: [{ functionResponse: { name: "read", response: { output: "A" } } }] },
|
|
||||||
// never folded into the tool result: Gemini 3 would read it as part of it
|
|
||||||
{ role: "model", parts: [{ text: "[The previous response was interrupted before it completed.]" }] },
|
|
||||||
{ role: "user", parts: [{ text: "and?" }] },
|
|
||||||
])
|
|
||||||
})
|
|
||||||
|
|
||||||
test("history for Gemini 3: ids on both sides, the sentinel for an unsigned call, JSON results structured unless they hold a $ref", () => {
|
|
||||||
const c = toGeminiContents(
|
|
||||||
[
|
|
||||||
{ role: "user", parts: [{ type: "text", text: "go" }] },
|
|
||||||
{ role: "assistant", parts: [{ type: "tool_call", id: "c1", name: "read", args: "{}" }, { type: "tool_call", id: "c2", name: "grep", args: "{}" }] },
|
|
||||||
{ role: "tool", callId: "c1", name: "read", content: '{"lines": 3}' },
|
|
||||||
{ role: "tool", callId: "c2", name: "grep", content: '{"$ref": "#/$defs/X"}', isError: false },
|
|
||||||
],
|
|
||||||
false,
|
|
||||||
true,
|
|
||||||
)
|
|
||||||
expect(c[1]!.parts[0]).toEqual({ functionCall: { name: "read", args: {}, id: "c1" }, thoughtSignature: SKIP_SIGNATURE })
|
|
||||||
expect(c[2]).toEqual({
|
|
||||||
role: "user",
|
|
||||||
parts: [{ functionResponse: { name: "read", response: { lines: 3 }, id: "c1" } }, { functionResponse: { name: "grep", response: { output: '{"$ref": "#/$defs/X"}' }, id: "c2" } }],
|
|
||||||
})
|
|
||||||
})
|
|
||||||
})
|
|
||||||
|
|
||||||
describe("ollama", () => {
|
|
||||||
const nd = (lines: unknown[]) => lines.map((l) => JSON.stringify(l)).join("\n") + "\n"
|
|
||||||
|
|
||||||
test("NDJSON: thinking, text, whole tool calls, counts; num_ctx and think in the request", async () => {
|
|
||||||
fake = fakeProvider([
|
|
||||||
{
|
|
||||||
chunks: [],
|
|
||||||
raw: nd([
|
|
||||||
{ message: { role: "assistant", content: "", thinking: "Hmm." }, done: false },
|
|
||||||
{ message: { role: "assistant", content: "Reading." }, done: false },
|
|
||||||
{ message: { role: "assistant", content: "", tool_calls: [{ function: { name: "read", arguments: { path: "a.ts" } } }] }, done: false },
|
|
||||||
{ message: { role: "assistant", content: "" }, done: true, done_reason: "stop", prompt_eval_count: 30, eval_count: 7 },
|
|
||||||
]),
|
|
||||||
},
|
|
||||||
])
|
|
||||||
const ev = await run(new OllamaClient(m("ollama", fake.url.replace(/\/v1$/, ""), "gpt-oss:20b", { context: 32768, max_output: 2000 })), undefined, "high")
|
|
||||||
expect(fake.calls[0]!.path).toBe("/api/chat")
|
|
||||||
expect(fake.requests[0]).toMatchObject({ model: "gpt-oss:20b", stream: true, think: "high", options: { num_ctx: 32768, num_predict: 2000 } })
|
|
||||||
expect(fake.requests[0].messages[0]).toEqual({ role: "system", content: "sys" })
|
|
||||||
const fin = finishOf(ev)
|
|
||||||
expect(fin.message.parts).toEqual([
|
|
||||||
{ type: "reasoning", text: "Hmm." },
|
|
||||||
{ type: "text", text: "Reading." },
|
|
||||||
{ type: "tool_call", id: "ollama_call_0", name: "read", args: '{"path":"a.ts"}' },
|
|
||||||
])
|
|
||||||
expect(fin.reason).toBe("tool_calls")
|
|
||||||
expect(ev.find((e) => e.type === "usage")).toEqual({ type: "usage", usage: { input: 30, output: 7 } })
|
|
||||||
})
|
|
||||||
|
|
||||||
test("think: a switch for other models, off when efforts exist but none is chosen; an error line is retried once", async () => {
|
|
||||||
fake = fakeProvider([{ chunks: [], raw: nd([{ error: "model runner has unexpectedly stopped" }]) }, { chunks: [], raw: nd([{ message: { content: "ok" }, done: true }]) }, { chunks: [], raw: nd([{ message: { content: "ok" }, done: true }]) }])
|
|
||||||
const base = fake.url.replace(/\/v1$/, "")
|
|
||||||
const ev = await run(new OllamaClient(m("ollama", base, "qwen3:8b", { efforts: ["low", "high"] })), undefined, "low")
|
|
||||||
expect(fake.requests[0].think).toBe(true)
|
|
||||||
expect(ev.some((e) => e.type === "notice" && e.message.includes("unexpectedly stopped"))).toBe(true)
|
|
||||||
await run(new OllamaClient(m("ollama", base, "qwen3:8b", { efforts: ["low", "high"] })))
|
|
||||||
expect(fake.requests[2].think).toBe(false)
|
|
||||||
})
|
|
||||||
|
|
||||||
test("/api/show: no `think` for a model that cannot think; num_ctx from the Modelfile, else the trained maximum", async () => {
|
|
||||||
const show = { capabilities: ["completion", "tools"], parameters: "stop \"<|im_end|>\"\nnum_ctx 16384", model_info: { "qwen3.context_length": 40960 } }
|
|
||||||
expect(parseShow(show)).toEqual({ capabilities: ["completion", "tools"], numCtx: 16384, trained: 40960 })
|
|
||||||
fake = fakeProvider([{ chunks: [], raw: nd([{ message: { content: "ok" }, done: true }]) }], { show })
|
|
||||||
const base = fake.url.replace(/\/v1$/, "")
|
|
||||||
const c = new OllamaClient(m("ollama", base, "llama3:8b", { efforts: ["low", "high"] }))
|
|
||||||
await run(c, undefined, "high")
|
|
||||||
expect(fake.requests[0].think).toBeUndefined()
|
|
||||||
expect(fake.requests[0].options.num_ctx).toBe(16384)
|
|
||||||
expect(await c.contextOf("llama3:8b")).toBe(16384)
|
|
||||||
expect(parseShow({ model_info: { "llama.context_length": 8192 } })).toEqual({ trained: 8192 })
|
|
||||||
})
|
|
||||||
})
|
|
||||||
@@ -190,7 +190,7 @@ test("every {{variable}} in a description is declared, and none reaches a model"
|
|||||||
expect(used.sort()).toEqual(Object.keys(s.variables ?? {}).sort())
|
expect(used.sort()).toEqual(Object.keys(s.variables ?? {}).sort())
|
||||||
}
|
}
|
||||||
mkdirSync(paths.config, { recursive: true })
|
mkdirSync(paths.config, { recursive: true })
|
||||||
writeFileSync(join(paths.config, "connections.yaml"), "connections:\n f:\n dialect: openai-chat\n base_url: http://127.0.0.1:9/v1\n models: { m: {} }\n", { mode: 0o600 })
|
writeFileSync(join(paths.config, "connections.yaml"), "connections:\n f:\n type: webui\n url: http://127.0.0.1:9/v1\n models: { f/m: {} }\n", { mode: 0o600 })
|
||||||
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\nlimits: { bash_timeout: 90 }\n")
|
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\nlimits: { bash_timeout: 90 }\n")
|
||||||
const app = createApp({ cwd: mkdtempSync(join(tmpdir(), "ph-vars-")), store: false, snapshots: false, asker: { ask: async () => ({ kind: "once" }) } })
|
const app = createApp({ cwd: mkdtempSync(join(tmpdir(), "ph-vars-")), store: false, snapshots: false, asker: { ask: async () => ({ kind: "once" }) } })
|
||||||
for (const t of app.engine.o.tools) {
|
for (const t of app.engine.o.tools) {
|
||||||
|
|||||||
@@ -18,7 +18,7 @@ afterEach(() => fake?.stop())
|
|||||||
function app(script: Parameters<typeof fakeProvider>[0]) {
|
function app(script: Parameters<typeof fakeProvider>[0]) {
|
||||||
fake = fakeProvider(script)
|
fake = fakeProvider(script)
|
||||||
mkdirSync(paths.config, { recursive: true })
|
mkdirSync(paths.config, { recursive: true })
|
||||||
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n dialect: openai-chat\n base_url: ${fake.url}\n models: { m: {} }\n`, { mode: 0o600 })
|
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n type: webui\n url: ${fake.url}\n models: { f/m: {} }\n`, { mode: 0o600 })
|
||||||
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\n")
|
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\n")
|
||||||
return createApp({ cwd: mkdtempSync(join(tmpdir(), "ph-hl-")), mode: "auto", store: false, snapshots: false, asker: { ask: async (): Promise<AskReply> => ({ kind: "once" }) } })
|
return createApp({ cwd: mkdtempSync(join(tmpdir(), "ph-hl-")), mode: "auto", store: false, snapshots: false, asker: { ask: async (): Promise<AskReply> => ({ kind: "once" }) } })
|
||||||
}
|
}
|
||||||
|
|||||||
+1
-1
@@ -38,7 +38,7 @@ function pair(): [Transport, Transport] {
|
|||||||
|
|
||||||
function config(url: string, extra = "") {
|
function config(url: string, extra = "") {
|
||||||
mkdirSync(paths.config, { recursive: true })
|
mkdirSync(paths.config, { recursive: true })
|
||||||
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n dialect: openai-chat\n base_url: ${url}\n models: { m: {} }\n`, { mode: 0o600 })
|
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n type: webui\n url: ${url}\n models: { f/m: {} }\n`, { mode: 0o600 })
|
||||||
writeFileSync(join(paths.config, "config.yaml"), `model: f/m\n${extra}`)
|
writeFileSync(join(paths.config, "config.yaml"), `model: f/m\n${extra}`)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -51,7 +51,7 @@ describe("an image counts by its dimensions, never its encoding", () => {
|
|||||||
{ chunks: [delta({ content: "a screenshot" }), usage(56_000, 5)] },
|
{ chunks: [delta({ content: "a screenshot" }), usage(56_000, 5)] },
|
||||||
])
|
])
|
||||||
mkdirSync(paths.config, { recursive: true })
|
mkdirSync(paths.config, { recursive: true })
|
||||||
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n dialect: openai-chat\n base_url: ${fake.url}\n models:\n m: { vision: true, context: 262144 }\n`, { mode: 0o600 })
|
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n type: webui\n url: ${fake.url}\n models:\n f/m: { vision: true, context: 262144 }\n`, { mode: 0o600 })
|
||||||
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\n")
|
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\n")
|
||||||
const root = mkdtempSync(join(tmpdir(), "ph-img-"))
|
const root = mkdtempSync(join(tmpdir(), "ph-img-"))
|
||||||
writeFileSync(join(root, "shot.png"), png(2560, 1440, 1_500_000))
|
writeFileSync(join(root, "shot.png"), png(2560, 1440, 1_500_000))
|
||||||
|
|||||||
@@ -33,7 +33,7 @@ describe("images", () => {
|
|||||||
function setup(vision: boolean, script: Parameters<typeof fakeProvider>[0]) {
|
function setup(vision: boolean, script: Parameters<typeof fakeProvider>[0]) {
|
||||||
fake = fakeProvider(script)
|
fake = fakeProvider(script)
|
||||||
mkdirSync(paths.config, { recursive: true })
|
mkdirSync(paths.config, { recursive: true })
|
||||||
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n dialect: openai-chat\n base_url: ${fake.url}\n models:\n m: { vision: ${vision} }\n`, { mode: 0o600 })
|
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n type: webui\n url: ${fake.url}\n models:\n f/m: { vision: ${vision} }\n`, { mode: 0o600 })
|
||||||
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\n")
|
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\n")
|
||||||
return createApp({ cwd: project(), mode: "edit", store: false, asker: { ask: async (): Promise<AskReply> => ({ kind: "once" }) } })
|
return createApp({ cwd: project(), mode: "edit", store: false, asker: { ask: async (): Promise<AskReply> => ({ kind: "once" }) } })
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -17,7 +17,7 @@ afterEach(() => fake?.stop())
|
|||||||
|
|
||||||
function app(url: string) {
|
function app(url: string) {
|
||||||
mkdirSync(paths.config, { recursive: true })
|
mkdirSync(paths.config, { recursive: true })
|
||||||
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n dialect: openai-chat\n base_url: ${url}\n models: { m: {} }\n`, { mode: 0o600 })
|
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n type: webui\n url: ${url}\n models: { f/m: {} }\n`, { mode: 0o600 })
|
||||||
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\n")
|
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\n")
|
||||||
return createApp({ cwd: mkdtempSync(join(tmpdir(), "ph-wake-")), mode: "auto", store: false, asker: { ask: async (): Promise<AskReply> => ({ kind: "once" }) } })
|
return createApp({ cwd: mkdtempSync(join(tmpdir(), "ph-wake-")), mode: "auto", store: false, asker: { ask: async (): Promise<AskReply> => ({ kind: "once" }) } })
|
||||||
}
|
}
|
||||||
|
|||||||
+1
-1
@@ -155,7 +155,7 @@ describe("in an app", () => {
|
|||||||
function app(script: Parameters<typeof fakeProvider>[0], store?: Store) {
|
function app(script: Parameters<typeof fakeProvider>[0], store?: Store) {
|
||||||
fake = fakeProvider(script)
|
fake = fakeProvider(script)
|
||||||
mkdirSync(paths.config, { recursive: true })
|
mkdirSync(paths.config, { recursive: true })
|
||||||
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n dialect: openai-chat\n base_url: ${fake.url}\n models: { m: {} }\n`, { mode: 0o600 })
|
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n type: webui\n url: ${fake.url}\n models: { f/m: {} }\n`, { mode: 0o600 })
|
||||||
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\ntitles: prompt\n")
|
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\ntitles: prompt\n")
|
||||||
return createApp({ cwd: dir(), mode: "auto", store: store ?? false, snapshots: false, asker: { ask: async (): Promise<AskReply> => ({ kind: "once" }) } })
|
return createApp({ cwd: dir(), mode: "auto", store: store ?? false, snapshots: false, asker: { ask: async (): Promise<AskReply> => ({ kind: "once" }) } })
|
||||||
}
|
}
|
||||||
|
|||||||
+14
-16
@@ -25,14 +25,16 @@ test("draft-07, described, and no string held to a format ({env:} stands in for
|
|||||||
const props = (schemas.config as any).properties
|
const props = (schemas.config as any).properties
|
||||||
expect(props.mode.enum).toEqual(["manual", "edit", "auto", "plan"])
|
expect(props.mode.enum).toEqual(["manual", "edit", "auto", "plan"])
|
||||||
expect(props.small_model.description).toContain("session titles")
|
expect(props.small_model.description).toContain("session titles")
|
||||||
const kinds = (schemas.connections as any).properties.connections.additionalProperties.anyOf
|
// 2.0: the login is the only kind of entry.
|
||||||
expect(kinds[0].properties.type.const ?? kinds[0].properties.type.enum[0]).toBe("webui")
|
const login = (schemas.connections as any).properties.connections.additionalProperties
|
||||||
expect(kinds[1].properties.base_url.description).toContain("Where the API is")
|
expect(login.properties.type.const ?? login.properties.type.enum[0]).toBe("webui")
|
||||||
|
expect(login.properties.url.description).toContain("The instance's address")
|
||||||
|
expect(login.properties.dialect).toBeUndefined()
|
||||||
})
|
})
|
||||||
|
|
||||||
const goodConfig = `
|
const goodConfig = `
|
||||||
model: local/qwen3
|
model: llama/qwen3
|
||||||
small_model: local/tiny
|
small_model: llama/tiny
|
||||||
titles: prompt
|
titles: prompt
|
||||||
mode: edit
|
mode: edit
|
||||||
icons: plain
|
icons: plain
|
||||||
@@ -59,19 +61,14 @@ mcp:
|
|||||||
`
|
`
|
||||||
const goodConnections = `
|
const goodConnections = `
|
||||||
connections:
|
connections:
|
||||||
local:
|
example:
|
||||||
dialect: openai-chat
|
type: webui
|
||||||
base_url: "{env:LLM_URL}"
|
url: https://ai.example.org
|
||||||
|
api_key: "{file:~/.config/lembas/lembas/example.key}"
|
||||||
tls: { ca: ~/ca.crt }
|
tls: { ca: ~/ca.crt }
|
||||||
quirks: { stream_usage: off }
|
quirks: { stream_usage: off }
|
||||||
models:
|
models:
|
||||||
qwen3: { context: 131072, max_output: 65536, efforts: [low, high], effort: low, vision: true }
|
llama/qwen3: { context: 131072, max_output: 65536, efforts: [low, high], effort: low, vision: true }
|
||||||
anthropic:
|
|
||||||
dialect: anthropic
|
|
||||||
base_url: https://api.anthropic.com
|
|
||||||
api_key: "{env:ANTHROPIC_API_KEY}"
|
|
||||||
models:
|
|
||||||
claude: { effort_map: { low: 4000, high: 32000 }, cache: true }
|
|
||||||
`
|
`
|
||||||
|
|
||||||
test("what the loader accepts, the schema accepts", () => {
|
test("what the loader accepts, the schema accepts", () => {
|
||||||
@@ -94,7 +91,8 @@ test("…and what it refuses, the schema marks", () => {
|
|||||||
expect(Config.safeParse(bad).success).toBe(false)
|
expect(Config.safeParse(bad).success).toBe(false)
|
||||||
expect(checkConfig(bad)).toBe(false)
|
expect(checkConfig(bad)).toBe(false)
|
||||||
}
|
}
|
||||||
for (const bad of [{ connections: { x: { dialect: "openai", base_url: "http://h/v1" } } }, { connections: { x: { dialect: "anthropic" } } }, { connections: { x: { dialect: "anthropic", base_url: "u", models: { m: { context: -1 } } } } }]) {
|
// A custom connection (1.x): the loader reads past it with a warning, and the schema marks it.
|
||||||
|
for (const bad of [{ connections: { x: { dialect: "openai-chat", base_url: "http://h/v1" } } }, { connections: { x: { type: "webui" } } }, { connections: { x: { type: "webui", url: "https://h", models: { m: { context: -1 } } } } }, { connections: { x: { type: "webui", url: "https://h", key_cmd: "pass x" } } }]) {
|
||||||
expect(checkConnections(bad)).toBe(false)
|
expect(checkConnections(bad)).toBe(false)
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
|
|||||||
@@ -51,7 +51,7 @@ test("the instance's library replaces this machine's, under the tools' own names
|
|||||||
mkdirSync(join(paths.config, "lembas"), { recursive: true })
|
mkdirSync(join(paths.config, "lembas"), { recursive: true })
|
||||||
writeFileSync(join(paths.config, "lembas.json"), JSON.stringify({ instances: { ai: { base_url: lembas.base, connection: "ai", logged_in_at: "" } } }))
|
writeFileSync(join(paths.config, "lembas.json"), JSON.stringify({ instances: { ai: { base_url: lembas.base, connection: "ai", logged_in_at: "" } } }))
|
||||||
writeFileSync(join(paths.config, "lembas", "ai.key"), "lmb_lib\n", { mode: 0o600 })
|
writeFileSync(join(paths.config, "lembas", "ai.key"), "lmb_lib\n", { mode: 0o600 })
|
||||||
writeFileSync(join(paths.config, "connections.yaml"), "connections:\n f:\n dialect: openai-chat\n base_url: http://127.0.0.1:1/v1\n models: { m: {} }\n", { mode: 0o600 })
|
writeFileSync(join(paths.config, "connections.yaml"), "connections:\n f:\n type: webui\n url: http://127.0.0.1:1/v1\n models: { f/m: {} }\n", { mode: 0o600 })
|
||||||
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\nlibrary: lembas\n")
|
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\nlibrary: lembas\n")
|
||||||
const app = createApp({ cwd: mkdtempSync(join(tmpdir(), "ph-lib-")), store: false, asker: { ask: async (): Promise<AskReply> => ({ kind: "once" }) } })
|
const app = createApp({ cwd: mkdtempSync(join(tmpdir(), "ph-lib-")), store: false, asker: { ask: async (): Promise<AskReply> => ({ kind: "once" }) } })
|
||||||
await app.mcpReady
|
await app.mcpReady
|
||||||
@@ -74,7 +74,7 @@ test("a session the client gave a library scope sends its id on every call to th
|
|||||||
mkdirSync(join(paths.config, "lembas"), { recursive: true })
|
mkdirSync(join(paths.config, "lembas"), { recursive: true })
|
||||||
writeFileSync(join(paths.config, "lembas.json"), JSON.stringify({ instances: { ai: { base_url: lembas.base, connection: "ai", logged_in_at: "" } } }))
|
writeFileSync(join(paths.config, "lembas.json"), JSON.stringify({ instances: { ai: { base_url: lembas.base, connection: "ai", logged_in_at: "" } } }))
|
||||||
writeFileSync(join(paths.config, "lembas", "ai.key"), "lmb_lib\n", { mode: 0o600 })
|
writeFileSync(join(paths.config, "lembas", "ai.key"), "lmb_lib\n", { mode: 0o600 })
|
||||||
writeFileSync(join(paths.config, "connections.yaml"), "connections:\n f:\n dialect: openai-chat\n base_url: http://127.0.0.1:1/v1\n models: { m: {} }\n", { mode: 0o600 })
|
writeFileSync(join(paths.config, "connections.yaml"), "connections:\n f:\n type: webui\n url: http://127.0.0.1:1/v1\n models: { f/m: {} }\n", { mode: 0o600 })
|
||||||
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\nlibrary: lembas\n")
|
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\nlibrary: lembas\n")
|
||||||
const app = createApp({ cwd: mkdtempSync(join(tmpdir(), "ph-lib-")), asker: { ask: async (): Promise<AskReply> => ({ kind: "once" }) } })
|
const app = createApp({ cwd: mkdtempSync(join(tmpdir(), "ph-lib-")), asker: { ask: async (): Promise<AskReply> => ({ kind: "once" }) } })
|
||||||
stops.push(() => void app.close())
|
stops.push(() => void app.close())
|
||||||
@@ -104,7 +104,7 @@ test("a session the client gave a library scope sends its id on every call to th
|
|||||||
|
|
||||||
test("without a login it says so and keeps this machine's library", () => {
|
test("without a login it says so and keeps this machine's library", () => {
|
||||||
writeFileSync(join(paths.config, "lembas.json"), JSON.stringify({ instances: {} }))
|
writeFileSync(join(paths.config, "lembas.json"), JSON.stringify({ instances: {} }))
|
||||||
writeFileSync(join(paths.config, "connections.yaml"), "connections:\n f:\n dialect: openai-chat\n base_url: http://127.0.0.1:1/v1\n models: { m: {} }\n", { mode: 0o600 })
|
writeFileSync(join(paths.config, "connections.yaml"), "connections:\n f:\n type: webui\n url: http://127.0.0.1:1/v1\n models: { f/m: {} }\n", { mode: 0o600 })
|
||||||
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\nlibrary: lembas\n")
|
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\nlibrary: lembas\n")
|
||||||
const app = createApp({ cwd: mkdtempSync(join(tmpdir(), "ph-lib-")), store: false, asker: { ask: async (): Promise<AskReply> => ({ kind: "once" }) } })
|
const app = createApp({ cwd: mkdtempSync(join(tmpdir(), "ph-lib-")), store: false, asker: { ask: async (): Promise<AskReply> => ({ kind: "once" }) } })
|
||||||
expect(app.loaded.warnings.join(" ")).toContain("not logged in")
|
expect(app.loaded.warnings.join(" ")).toContain("not logged in")
|
||||||
|
|||||||
@@ -187,14 +187,14 @@ test("login writes one webui connection named after the instance, and no models"
|
|||||||
|
|
||||||
const loaded = loadConfig()
|
const loaded = loadConfig()
|
||||||
const c = loaded.connections.example!
|
const c = loaded.connections.example!
|
||||||
expect(c.dialect).toBe("openai-chat")
|
expect(c.webui?.url).toBe(fake.url)
|
||||||
expect(c.base_url).toBe(`${fake.url}/v1`)
|
expect(c.base_url).toBe(`${fake.url}/v1`)
|
||||||
expect(c.api_key).toBe("lmb_secret")
|
expect(c.api_key).toBe("lmb_secret")
|
||||||
// The instance's models with its settings, the default first; the group named after the connection.
|
// The instance's models with its settings, the default first; the group named after the connection.
|
||||||
expect(Object.keys(c.models)).toEqual(["qwen", "gemma"])
|
expect(Object.keys(c.models)).toEqual(["qwen", "gemma"])
|
||||||
expect(c.models.qwen).toEqual({ name: "Qwen", context: 131072, temperature: 0.6, top_p: 0.95, efforts: ["low", "high"], tools: true, group: "example:gpu" })
|
expect(c.models.qwen).toEqual({ name: "Qwen", context: 131072, temperature: 0.6, top_p: 0.95, efforts: ["low", "high"], tools: true, group: "example:gpu" })
|
||||||
expect(c.models.gemma).toEqual({ context: 32768, vision: true, single_session: true, group: "example:gpu" })
|
expect(c.models.gemma).toEqual({ context: 32768, vision: true, single_session: true, group: "example:gpu" })
|
||||||
expect(resolveModel(loaded, undefined).ref).toBe("example/qwen")
|
expect(resolveModel(loaded, undefined).ref).toBe("qwen")
|
||||||
})
|
})
|
||||||
|
|
||||||
test("a model changed on the instance is here at the next start, without logging in again", async () => {
|
test("a model changed on the instance is here at the next start, without logging in again", async () => {
|
||||||
@@ -205,7 +205,7 @@ test("a model changed on the instance is here at the next start, without logging
|
|||||||
expect(r).toEqual([{ name: "example", ok: true, models: 1 }])
|
expect(r).toEqual([{ name: "example", ok: true, models: 1 }])
|
||||||
const loaded = loadConfig()
|
const loaded = loadConfig()
|
||||||
expect(Object.keys(loaded.connections.example!.models)).toEqual(["llama"])
|
expect(Object.keys(loaded.connections.example!.models)).toEqual(["llama"])
|
||||||
expect(resolveModel(loaded, undefined).ref).toBe("example/llama")
|
expect(resolveModel(loaded, undefined).ref).toBe("llama")
|
||||||
// A model the instance does not list yet is still asked for, not refused here.
|
// A model the instance does not list yet is still asked for, not refused here.
|
||||||
expect(resolveModel(loaded, "example/brand-new").id).toBe("brand-new")
|
expect(resolveModel(loaded, "example/brand-new").id).toBe("brand-new")
|
||||||
})
|
})
|
||||||
@@ -266,7 +266,7 @@ test("an older kind of login is replaced: the old connection, its key and its pi
|
|||||||
const c = cfg()
|
const c = cfg()
|
||||||
// It was the instance's default, so the instance decides it now; the rest follow the new name.
|
// It was the instance's default, so the instance decides it now; the rest follow the new name.
|
||||||
expect(c.model).toBeUndefined()
|
expect(c.model).toBeUndefined()
|
||||||
expect(c.small_model).toBe("example/gemma")
|
expect(c.small_model).toBe("gemma")
|
||||||
expect(r.changes.join("\n")).toContain("the connection ai from an earlier login is now example")
|
expect(r.changes.join("\n")).toContain("the connection ai from an earlier login is now example")
|
||||||
})
|
})
|
||||||
|
|
||||||
|
|||||||
@@ -96,10 +96,13 @@ test("an OpenAI-shaped /embeddings: in order, unit length", async () => {
|
|||||||
return Response.json({ data: input.map((_, i) => ({ index: i, embedding: [3, 4] })).reverse() })
|
return Response.json({ data: input.map((_, i) => ({ index: i, embedding: [3, 4] })).reverse() })
|
||||||
},
|
},
|
||||||
})
|
})
|
||||||
const e = embedderFor({ connections: { emb: { dialect: "openai-chat", base_url: `http://127.0.0.1:${server.port}/v1`, models: {} } } } as never, "emb/nomic")!
|
// The server's embedding model, by its served id.
|
||||||
|
const loaded = { connections: { emb: { base_url: `http://127.0.0.1:${server.port}/v1`, models: {}, webui: { url: `http://127.0.0.1:${server.port}`, embeddings: ["llama/nomic"] } } }, refs: [] } as never
|
||||||
|
const e = embedderFor(loaded, "llama/nomic")!
|
||||||
const [v] = await e.embed(["x", "y"])
|
const [v] = await e.embed(["x", "y"])
|
||||||
expect(Array.from(v!)).toEqual([0.6000000238418579, 0.800000011920929])
|
expect(Array.from(v!)).toEqual([0.6000000238418579, 0.800000011920929])
|
||||||
expect(() => embedderFor({ connections: { a: { dialect: "anthropic", base_url: "http://x", models: {} } } } as never, "a/b")).toThrow("no embeddings")
|
// Not logged in: said.
|
||||||
|
expect(() => embedderFor({ connections: {}, refs: [] } as never, "llama/nomic")).toThrow("not logged in")
|
||||||
})
|
})
|
||||||
|
|
||||||
test("ingest: text and code; HTML as markdown; binaries skipped; long text cut and said; links out of the directory ignored", async () => {
|
test("ingest: text and code; HTML as markdown; binaries skipped; long text cut and said; links out of the directory ignored", async () => {
|
||||||
@@ -135,7 +138,7 @@ test("the tools: knowledge_search is offered only with something to search; a pr
|
|||||||
{ chunks: [delta({ content: "ok" }, "stop")] },
|
{ chunks: [delta({ content: "ok" }, "stop")] },
|
||||||
])
|
])
|
||||||
mkdirSync(paths.config, { recursive: true })
|
mkdirSync(paths.config, { recursive: true })
|
||||||
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n dialect: openai-chat\n base_url: ${fake.url}\n models: { m: {} }\n`, { mode: 0o600 })
|
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n type: webui\n url: ${fake.url}\n models: { f/m: {} }\n`, { mode: 0o600 })
|
||||||
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\ntitles: prompt\n")
|
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\ntitles: prompt\n")
|
||||||
const cwd = mkdtempSync(join(tmpdir(), "ph-libtools-"))
|
const cwd = mkdtempSync(join(tmpdir(), "ph-libtools-"))
|
||||||
mkdirSync(join(cwd, ".agent"))
|
mkdirSync(join(cwd, ".agent"))
|
||||||
|
|||||||
+5
-5
@@ -34,7 +34,7 @@ function repo(): string {
|
|||||||
/** The user's own files: a connection and a config.yaml local mode must not read. */
|
/** The user's own files: a connection and a config.yaml local mode must not read. */
|
||||||
function userFiles(extraConfig = "") {
|
function userFiles(extraConfig = "") {
|
||||||
mkdirSync(globalPaths.config, { recursive: true })
|
mkdirSync(globalPaths.config, { recursive: true })
|
||||||
writeFileSync(join(globalPaths.config, "connections.yaml"), "connections:\n mine:\n dialect: openai-chat\n base_url: http://127.0.0.1:9/v1\n models: { m: {} }\n", { mode: 0o600 })
|
writeFileSync(join(globalPaths.config, "connections.yaml"), "connections:\n mine:\n type: webui\n url: http://127.0.0.1:9/v1\n models: { mine/m: {} }\n", { mode: 0o600 })
|
||||||
writeFileSync(join(globalPaths.config, "config.yaml"), `model: mine/m\ntheme: forest\n${extraConfig}`)
|
writeFileSync(join(globalPaths.config, "config.yaml"), `model: mine/m\ntheme: forest\n${extraConfig}`)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -67,7 +67,7 @@ test("a local directory that came with the repository is refused", () => {
|
|||||||
expect(isLocal()).toBe(false)
|
expect(isLocal()).toBe(false)
|
||||||
})
|
})
|
||||||
|
|
||||||
test("the user's config.yaml is not read; their connections are, and a local one wins by name", () => {
|
test("the user's config.yaml is not read; their login is, and a local one wins by name", () => {
|
||||||
userFiles()
|
userFiles()
|
||||||
const dir = repo()
|
const dir = repo()
|
||||||
enterLocal(dir)
|
enterLocal(dir)
|
||||||
@@ -76,10 +76,10 @@ test("the user's config.yaml is not read; their connections are, and a local one
|
|||||||
expect(loaded.config.theme).toBeUndefined()
|
expect(loaded.config.theme).toBeUndefined()
|
||||||
expect(Object.keys(loaded.connections)).toEqual(["mine"])
|
expect(Object.keys(loaded.connections)).toEqual(["mine"])
|
||||||
writeFileSync(join(paths.config, "config.yaml"), "model: mine/m\n")
|
writeFileSync(join(paths.config, "config.yaml"), "model: mine/m\n")
|
||||||
writeFileSync(join(paths.config, "connections.yaml"), "connections:\n mine:\n dialect: openai-chat\n base_url: http://127.0.0.1:8/v1\n models: { n: {} }\n", { mode: 0o600 })
|
writeFileSync(join(paths.config, "connections.yaml"), "connections:\n mine:\n type: webui\n url: http://127.0.0.1:8/v1\n models: { mine/n: {} }\n", { mode: 0o600 })
|
||||||
loaded = loadConfig()
|
loaded = loadConfig()
|
||||||
expect(loaded.config.model).toBe("mine/m")
|
expect(loaded.config.model).toBe("mine/m")
|
||||||
expect(Object.keys(loaded.connections.mine!.models)).toEqual(["n"])
|
expect(Object.keys(loaded.connections.mine!.models)).toEqual(["mine/n"])
|
||||||
})
|
})
|
||||||
|
|
||||||
test("a login made outside local mode is read, never written or signed out", async () => {
|
test("a login made outside local mode is read, never written or signed out", async () => {
|
||||||
@@ -152,7 +152,7 @@ test("the CLI: -l refuses what it turns off, and a run writes nothing outside th
|
|||||||
const home = realpathSync(mkdtempSync(join(tmpdir(), "lembas-local-home-")))
|
const home = realpathSync(mkdtempSync(join(tmpdir(), "lembas-local-home-")))
|
||||||
const tmp = realpathSync(mkdtempSync(join(tmpdir(), "lembas-local-tmp-")))
|
const tmp = realpathSync(mkdtempSync(join(tmpdir(), "lembas-local-tmp-")))
|
||||||
mkdirSync(join(home, ".config/lembas"), { recursive: true })
|
mkdirSync(join(home, ".config/lembas"), { recursive: true })
|
||||||
writeFileSync(join(home, ".config/lembas/connections.yaml"), `connections:\n fake:\n dialect: openai-chat\n base_url: ${fake.url}\n models: { m: {} }\n`, { mode: 0o600 })
|
writeFileSync(join(home, ".config/lembas/connections.yaml"), `connections:\n fake:\n type: webui\n url: ${fake.url}\n models: { fake/m: {} }\n`, { mode: 0o600 })
|
||||||
writeFileSync(join(home, ".config/lembas/config.yaml"), "model: nowhere/x\n")
|
writeFileSync(join(home, ".config/lembas/config.yaml"), "model: nowhere/x\n")
|
||||||
const dir = repo()
|
const dir = repo()
|
||||||
const env = { ...process.env, LEMBAS_HOME: home, HOME: home, TMPDIR: tmp, XDG_DATA_HOME: join(home, ".local/share"), XDG_STATE_HOME: join(home, ".local/state") }
|
const env = { ...process.env, LEMBAS_HOME: home, HOME: home, TMPDIR: tmp, XDG_DATA_HOME: join(home, ".local/share"), XDG_STATE_HOME: join(home, ".local/state") }
|
||||||
|
|||||||
+1
-1
@@ -181,7 +181,7 @@ describe("through the engine", () => {
|
|||||||
{ chunks: [delta({ content: "It said it." }, "stop"), usage(20, 5)] },
|
{ chunks: [delta({ content: "It said it." }, "stop"), usage(20, 5)] },
|
||||||
])
|
])
|
||||||
mkdirSync(paths.config, { recursive: true })
|
mkdirSync(paths.config, { recursive: true })
|
||||||
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n fake:\n dialect: openai-chat\n base_url: ${fake.url}\n models:\n m: { context: 32768 }\n`, { mode: 0o600 })
|
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n fake:\n type: webui\n url: ${fake.url}\n models:\n fake/m: { context: 32768 }\n`, { mode: 0o600 })
|
||||||
writeFileSync(join(paths.config, "config.yaml"), `model: fake/m\nmcp:\n fix:\n command: ["${BUN}", "${SERVER}"]\n`)
|
writeFileSync(join(paths.config, "config.yaml"), `model: fake/m\nmcp:\n fix:\n command: ["${BUN}", "${SERVER}"]\n`)
|
||||||
const dir = mkdtempSync(join(tmpdir(), "lembas-mcp-"))
|
const dir = mkdtempSync(join(tmpdir(), "lembas-mcp-"))
|
||||||
const asked: string[] = []
|
const asked: string[] = []
|
||||||
|
|||||||
+2
-32
@@ -24,7 +24,7 @@ function setup(script: Parameters<typeof fakeProvider>[0], connections: (url: st
|
|||||||
}
|
}
|
||||||
|
|
||||||
const two = (url: string) =>
|
const two = (url: string) =>
|
||||||
`connections:\n f:\n dialect: openai-chat\n base_url: ${url}\n models:\n main: {}\n big: { name: Big One, notes: slow but careful; best at reviews }\n`
|
`connections:\n f:\n type: webui\n url: ${url}\n models:\n f/main: {}\n f/big: { name: Big One, notes: slow but careful; best at reviews }\n`
|
||||||
|
|
||||||
test("the model is told which other models there are, with their notes", async () => {
|
test("the model is told which other models there are, with their notes", async () => {
|
||||||
const { app } = setup([{ chunks: [delta({ content: "ok" }, "stop")] }], two)
|
const { app } = setup([{ chunks: [delta({ content: "ok" }, "stop")] }], two)
|
||||||
@@ -44,38 +44,8 @@ test("task runs a subagent on the model asked for", async () => {
|
|||||||
two,
|
two,
|
||||||
)
|
)
|
||||||
expect(await app.engine.prompt("ask the big one")).toBe("stop")
|
expect(await app.engine.prompt("ask the big one")).toBe("stop")
|
||||||
expect(fake!.requests.map((r) => r.model)).toEqual(["main", "big", "main"])
|
expect(fake!.requests.map((r) => r.model)).toEqual(["f/main", "f/big", "f/main"])
|
||||||
const result = app.engine.messages.find((m) => m.role === "tool")
|
const result = app.engine.messages.find((m) => m.role === "tool")
|
||||||
expect(result && result.role === "tool" && result.content).toContain("Yes.")
|
expect(result && result.role === "tool" && result.content).toContain("Yes.")
|
||||||
})
|
})
|
||||||
|
|
||||||
test("a server that cannot be reached: the model's fallback takes over, said and kept", async () => {
|
|
||||||
const { app, events } = setup([{ chunks: [delta({ content: "from the fallback" }, "stop")] }], (url) =>
|
|
||||||
[
|
|
||||||
"connections:",
|
|
||||||
" gone:",
|
|
||||||
" dialect: openai-chat",
|
|
||||||
" base_url: http://127.0.0.1:9/v1",
|
|
||||||
" timeout: 5",
|
|
||||||
" models:",
|
|
||||||
" main: { fallback: [f/main] }",
|
|
||||||
" f:",
|
|
||||||
" dialect: openai-chat",
|
|
||||||
` base_url: ${url}`,
|
|
||||||
" models:",
|
|
||||||
" main: {}",
|
|
||||||
"",
|
|
||||||
].join("\n"),
|
|
||||||
)
|
|
||||||
app.settings.set("model", "gone/main")
|
|
||||||
expect(await app.engine.prompt("hi")).toBe("stop")
|
|
||||||
expect(app.engine.model.ref).toBe("f/main")
|
|
||||||
expect(events.some((e) => e.type === "notice" && e.message.includes("switching to f/main, its fallback"))).toBe(true)
|
|
||||||
expect(events.some((e) => e.type === "setting" && e.key === "model")).toBe(true)
|
|
||||||
})
|
|
||||||
|
|
||||||
test("no fallback: the error stands", async () => {
|
|
||||||
const { app } = setup([], (url) => `connections:\n gone:\n dialect: openai-chat\n base_url: http://127.0.0.1:9/v1\n models:\n main: {}\n f:\n dialect: openai-chat\n base_url: ${url}\n models:\n main: {}\n`)
|
|
||||||
app.settings.set("model", "gone/main")
|
|
||||||
expect(await app.engine.prompt("hi")).toBe("error")
|
|
||||||
})
|
|
||||||
+1
-1
@@ -85,7 +85,7 @@ test("a model calling notes with an action still gets its note written, and read
|
|||||||
{ chunks: [delta({ content: "ok" }, "stop")] },
|
{ chunks: [delta({ content: "ok" }, "stop")] },
|
||||||
])
|
])
|
||||||
mkdirSync(paths.config, { recursive: true })
|
mkdirSync(paths.config, { recursive: true })
|
||||||
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n dialect: openai-chat\n base_url: ${fake.url}\n models: { m: {} }\n`, { mode: 0o600 })
|
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n type: webui\n url: ${fake.url}\n models: { f/m: {} }\n`, { mode: 0o600 })
|
||||||
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\ntitles: prompt\n")
|
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\ntitles: prompt\n")
|
||||||
const app = createApp({ cwd: mkdtempSync(join(tmpdir(), "ph-names-")), store: false, snapshots: false, asker: { ask: async () => ({ kind: "once" }) } })
|
const app = createApp({ cwd: mkdtempSync(join(tmpdir(), "ph-names-")), store: false, snapshots: false, asker: { ask: async () => ({ kind: "once" }) } })
|
||||||
for (const n of app.library.notes(["global"])) app.library.deleteNote(n.id)
|
for (const n of app.library.notes(["global"])) app.library.deleteNote(n.id)
|
||||||
|
|||||||
@@ -137,7 +137,7 @@ test("logged in, the account's personalization is used instead of the local keys
|
|||||||
test("/settings writes the personalization to the account, not to config.yaml", async () => {
|
test("/settings writes the personalization to the account, not to config.yaml", async () => {
|
||||||
inst = instance()
|
inst = instance()
|
||||||
model = fakeProvider([])
|
model = fakeProvider([])
|
||||||
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n dialect: openai-chat\n base_url: ${model.url}\n models: { m: {} }\n`, { mode: 0o600 })
|
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n type: webui\n url: ${model.url}\n models: { f/m: {} }\n`, { mode: 0o600 })
|
||||||
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\nlibrary: local\n")
|
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\nlibrary: local\n")
|
||||||
loggedIn(inst)
|
loggedIn(inst)
|
||||||
await fetchWebui("example", inst.url, "lmb_key", undefined)
|
await fetchWebui("example", inst.url, "lmb_key", undefined)
|
||||||
@@ -188,7 +188,7 @@ test("the library: the account's by default when the login may read it; an expli
|
|||||||
test("opted out on the web, or a preset the CLI does not know: setting one key here wipes nothing there", async () => {
|
test("opted out on the web, or a preset the CLI does not know: setting one key here wipes nothing there", async () => {
|
||||||
inst = instance({ personal: { enabled: false, available: true, personality: "pirate", personality_custom: "Arr.", instructions: "Answer in Slovak." } })
|
inst = instance({ personal: { enabled: false, available: true, personality: "pirate", personality_custom: "Arr.", instructions: "Answer in Slovak." } })
|
||||||
model = fakeProvider([])
|
model = fakeProvider([])
|
||||||
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n dialect: openai-chat\n base_url: ${model.url}\n models: { m: {} }\n`, { mode: 0o600 })
|
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n type: webui\n url: ${model.url}\n models: { f/m: {} }\n`, { mode: 0o600 })
|
||||||
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\nlibrary: local\n")
|
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\nlibrary: local\n")
|
||||||
loggedIn(inst)
|
loggedIn(inst)
|
||||||
await fetchWebui("example", inst.url, "lmb_key", undefined)
|
await fetchWebui("example", inst.url, "lmb_key", undefined)
|
||||||
|
|||||||
@@ -68,7 +68,7 @@ test("the engine keeps the plan with the session: said on the bus, stored, back
|
|||||||
{ chunks: [delta({ content: "ok" })] },
|
{ chunks: [delta({ content: "ok" })] },
|
||||||
])
|
])
|
||||||
mkdirSync(paths.config, { recursive: true })
|
mkdirSync(paths.config, { recursive: true })
|
||||||
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n dialect: openai-chat\n base_url: ${fake.url}\n models: { m: {} }\n`, { mode: 0o600 })
|
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n type: webui\n url: ${fake.url}\n models: { f/m: {} }\n`, { mode: 0o600 })
|
||||||
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\ntitles: prompt\n")
|
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\ntitles: prompt\n")
|
||||||
const cwd = mkdtempSync(join(tmpdir(), "ph-planstore-"))
|
const cwd = mkdtempSync(join(tmpdir(), "ph-planstore-"))
|
||||||
const store = new Store(join(cwd, "sessions.db"))
|
const store = new Store(join(cwd, "sessions.db"))
|
||||||
|
|||||||
+1
-1
@@ -16,7 +16,7 @@ const RENDERED = "# Fix the average\nObjectives: [o1] Tests pass (open)\n## p1 C
|
|||||||
function app(script: Parameters<typeof fakeProvider>[0], plan?: (r: PlanRequest) => Promise<PlanReply>) {
|
function app(script: Parameters<typeof fakeProvider>[0], plan?: (r: PlanRequest) => Promise<PlanReply>) {
|
||||||
fake = fakeProvider(script)
|
fake = fakeProvider(script)
|
||||||
mkdirSync(paths.config, { recursive: true })
|
mkdirSync(paths.config, { recursive: true })
|
||||||
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n dialect: openai-chat\n base_url: ${fake.url}\n models: { m: {} }\n`, { mode: 0o600 })
|
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n type: webui\n url: ${fake.url}\n models: { f/m: {} }\n`, { mode: 0o600 })
|
||||||
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\n")
|
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\n")
|
||||||
const cwd = mkdtempSync(join(tmpdir(), "ph-plan-"))
|
const cwd = mkdtempSync(join(tmpdir(), "ph-plan-"))
|
||||||
Bun.spawnSync(["git", "init", "-q", cwd])
|
Bun.spawnSync(["git", "init", "-q", cwd])
|
||||||
|
|||||||
+26
-57
@@ -1,8 +1,8 @@
|
|||||||
// Models from a LLeMbas instance named by their provider: `deepseek/deepseek-flash`, not
|
// Models of the LLeMbas server named by the id it serves them under: `deepseek/deepseek-flash`, not
|
||||||
// `example/deepseek-flash`. The old forms still resolve, a config naming one is rewritten, a
|
// `example/deepseek-flash`. LLeMbas CLI 2.0 talks to one server, so the id is the ref; the older
|
||||||
// connection of the user's own wins its name, two instances share a provider by login order, and an
|
// forms still resolve and a config naming one is rewritten; a custom connection beside the login is
|
||||||
// instance that does not speak protocol 2 (bare ids) keeps `<login>/<id>`. On the link, the model event and the
|
// read past. On the link, the model event and the announcement say whether the session's model is
|
||||||
// announcement say whether the session's model is this link's instance's.
|
// this link's instance's.
|
||||||
import { afterEach, beforeEach, expect, test } from "bun:test"
|
import { afterEach, beforeEach, expect, test } from "bun:test"
|
||||||
import { existsSync, mkdirSync, mkdtempSync, readFileSync, realpathSync, rmSync, writeFileSync } from "node:fs"
|
import { existsSync, mkdirSync, mkdtempSync, readFileSync, realpathSync, rmSync, writeFileSync } from "node:fs"
|
||||||
import { tmpdir } from "node:os"
|
import { tmpdir } from "node:os"
|
||||||
@@ -35,27 +35,19 @@ function clean() {
|
|||||||
rmSync(join(paths.state, "webui"), { recursive: true, force: true })
|
rmSync(join(paths.state, "webui"), { recursive: true, force: true })
|
||||||
}
|
}
|
||||||
|
|
||||||
const conns = (o: Record<string, { webui?: boolean; models: string[] }>) =>
|
const conns = (o: Record<string, { models: string[] }>) =>
|
||||||
Object.fromEntries(Object.entries(o).map(([n, c]) => [n, { dialect: "openai-chat", base_url: "http://x/v1", models: Object.fromEntries(c.models.map((m) => [m, {}])), ...(c.webui ? { webui: { url: "http://x", v2: true } } : {}) }])) as never
|
Object.fromEntries(Object.entries(o).map(([n, c]) => [n, { base_url: "http://x/v1", models: Object.fromEntries(c.models.map((m) => [m, {}])), webui: { url: "http://x" } }])) as never
|
||||||
|
|
||||||
test("refs: by provider; a connection of the user's own wins its name; a bare id keeps the login", () => {
|
test("refs: the served id; the older forms name the same model", () => {
|
||||||
const c = conns({ example: { webui: true, models: ["deepseek/deepseek-flash", "llama/bonsai", "old"] }, llama: { models: ["mine"] } })
|
const c = conns({ example: { models: ["deepseek/deepseek-flash", "llama/bonsai", "old"] } })
|
||||||
const refs = modelRefs(c, ["example"])
|
const refs = modelRefs(c)
|
||||||
expect(refs.map((r) => r.ref)).toEqual(["deepseek/deepseek-flash", "example/llama/bonsai", "example/old", "llama/mine"])
|
expect(refs.map((r) => r.ref)).toEqual(["deepseek/deepseek-flash", "llama/bonsai", "old"])
|
||||||
// Every old way of writing them.
|
expect(findRef(c, refs, "example/deepseek-flash")).toMatchObject({ ref: "deepseek/deepseek-flash", old: true })
|
||||||
expect(findRef(c, refs, "example/deepseek-flash")?.ref).toBe("deepseek/deepseek-flash")
|
expect(findRef(c, refs, "example/deepseek/deepseek-flash")).toMatchObject({ ref: "deepseek/deepseek-flash", old: true })
|
||||||
expect(findRef(c, refs, "example/deepseek/deepseek-flash")?.ref).toBe("deepseek/deepseek-flash")
|
expect(findRef(c, refs, "example/bonsai")?.ref).toBe("llama/bonsai")
|
||||||
expect(findRef(c, refs, "example/bonsai")?.ref).toBe("example/llama/bonsai")
|
expect(findRef(c, refs, "example/old")?.ref).toBe("old")
|
||||||
expect(findRef(c, refs, "llama/mine")).toMatchObject({ connection: "llama", id: "mine" })
|
expect(findRef(c, refs, "llama/bonsai")?.old).toBeUndefined()
|
||||||
expect(findRef(c, refs, "llama/bonsai")).toBeUndefined()
|
expect(findRef(c, refs, "other/bonsai")).toBeUndefined()
|
||||||
})
|
|
||||||
|
|
||||||
test("refs: two instances with one provider — the first logged in to keeps the short form", () => {
|
|
||||||
const c = conns({ example: { webui: true, models: ["deepseek/a"] }, work: { webui: true, models: ["deepseek/a", "openai/b"] } })
|
|
||||||
expect(modelRefs(c, ["example", "work"]).map((r) => r.ref)).toEqual(["deepseek/a", "work/deepseek/a", "openai/b"])
|
|
||||||
expect(modelRefs(c, ["work", "example"]).map((r) => r.ref)).toEqual(["example/deepseek/a", "deepseek/a", "openai/b"])
|
|
||||||
// A login named after its own provider: the same string either way.
|
|
||||||
expect(modelRefs(conns({ llama: { webui: true, models: ["llama/bonsai"] } }), ["llama"]).map((r) => r.ref)).toEqual(["llama/bonsai"])
|
|
||||||
})
|
})
|
||||||
|
|
||||||
/** Logged in to an instance served by `fake`, whose models are `models` (as /v1/models said them;
|
/** Logged in to an instance served by `fake`, whose models are `models` (as /v1/models said them;
|
||||||
@@ -94,23 +86,13 @@ test("logged in: the model is deepseek/deepseek-flash everywhere, requests go to
|
|||||||
await app.close()
|
await app.close()
|
||||||
})
|
})
|
||||||
|
|
||||||
test("an instance that does not speak protocol 2 (bare ids): <login>/<id>", () => {
|
test("a custom connection beside the login is read past, and said (2.0 has none)", () => {
|
||||||
fake = fakeProvider([])
|
|
||||||
loggedIn(["deepseek-flash"])
|
|
||||||
writeFileSync(join(paths.config, "config.yaml"), "model: example/deepseek-flash\n")
|
|
||||||
const loaded = loadConfig()
|
|
||||||
expect(loaded.refs.map((r) => r.ref)).toEqual(["example/deepseek-flash"])
|
|
||||||
expect(resolveModel(loaded, undefined).ref).toBe("example/deepseek-flash")
|
|
||||||
expect(readFileSync(join(paths.config, "config.yaml"), "utf8")).toBe("model: example/deepseek-flash\n")
|
|
||||||
})
|
|
||||||
|
|
||||||
test("a connection of the user's own named like a provider keeps it; the instance's is <login>/<provider>/<model>", () => {
|
|
||||||
fake = fakeProvider([])
|
fake = fakeProvider([])
|
||||||
loggedIn(["deepseek/deepseek-flash"], ` deepseek:\n dialect: openai-chat\n base_url: ${"http://127.0.0.1:9/v1"}\n models: { deepseek-flash: {} }\n`)
|
loggedIn(["deepseek/deepseek-flash"], ` deepseek:\n dialect: openai-chat\n base_url: ${"http://127.0.0.1:9/v1"}\n models: { deepseek-flash: {} }\n`)
|
||||||
const loaded = loadConfig()
|
const loaded = loadConfig()
|
||||||
expect(loaded.refs.map((r) => r.ref)).toEqual(["example/deepseek/deepseek-flash", "deepseek/deepseek-flash"])
|
expect(loaded.refs.map((r) => r.ref)).toEqual(["deepseek/deepseek-flash"])
|
||||||
expect(resolveModel(loaded, "deepseek/deepseek-flash").connectionName).toBe("deepseek")
|
expect(resolveModel(loaded, "deepseek/deepseek-flash").connectionName).toBe("example")
|
||||||
expect(resolveModel(loaded, "example/deepseek/deepseek-flash").connectionName).toBe("example")
|
expect(loaded.warnings.some((w) => w.includes("deepseek is a custom connection"))).toBe(true)
|
||||||
})
|
})
|
||||||
|
|
||||||
function pair(): [Transport, Transport] {
|
function pair(): [Transport, Transport] {
|
||||||
@@ -145,13 +127,13 @@ test("on the link: the instance names its model as served, the session reports t
|
|||||||
|
|
||||||
// The audit of 568923c.
|
// The audit of 568923c.
|
||||||
|
|
||||||
test("only the old <login>/<model> is rewritten, once, in place: a pinned long form, anchors and comments stay", () => {
|
test("an older form is rewritten, once, in place: anchors and comments stay", () => {
|
||||||
fake = fakeProvider([])
|
fake = fakeProvider([])
|
||||||
loggedIn(["deepseek/deepseek-flash", "llama/nomic-embed"])
|
loggedIn(["deepseek/deepseek-flash", "llama/nomic-embed"])
|
||||||
// Pinned on purpose: means what it says, and is left alone.
|
// The login's name in front: one server, so it is the same model, written the old way.
|
||||||
writeFileSync(join(paths.config, "config.yaml"), "model: example/deepseek/deepseek-flash\n")
|
writeFileSync(join(paths.config, "config.yaml"), "model: example/deepseek/deepseek-flash\n")
|
||||||
expect(loadConfig().config.model).toBe("example/deepseek/deepseek-flash")
|
expect(loadConfig().config.model).toBe("deepseek/deepseek-flash")
|
||||||
expect(readFileSync(join(paths.config, "config.yaml"), "utf8")).toBe("model: example/deepseek/deepseek-flash\n")
|
expect(readFileSync(join(paths.config, "config.yaml"), "utf8")).toBe("model: deepseek/deepseek-flash\n")
|
||||||
// The old form behind an anchor, with a comment: the scalar changes, nothing else.
|
// The old form behind an anchor, with a comment: the scalar changes, nothing else.
|
||||||
writeFileSync(join(paths.config, "config.yaml"), "model: &m example/deepseek-flash # my pick\nsmall_model: *m\nembedding: example/nomic-embed\n")
|
writeFileSync(join(paths.config, "config.yaml"), "model: &m example/deepseek-flash # my pick\nsmall_model: *m\nembedding: example/nomic-embed\n")
|
||||||
const l = loadConfig()
|
const l = loadConfig()
|
||||||
@@ -168,25 +150,12 @@ test("only the old <login>/<model> is rewritten, once, in place: a pinned long f
|
|||||||
expect(existsSync(join(paths.state, "migrations.json"))).toBe(true)
|
expect(existsSync(join(paths.state, "migrations.json"))).toBe(true)
|
||||||
})
|
})
|
||||||
|
|
||||||
test("the provider: as the instance says it; from a protocol-2 instance, before the /; else the id is just an id", () => {
|
|
||||||
fake = fakeProvider([])
|
|
||||||
// Not protocol 2: an id with a / and no provider is `<login>/<id>`.
|
|
||||||
loggedIn(["org/model-a"], "", { noProvider: true })
|
|
||||||
expect(loadConfig().refs.map((r) => r.ref)).toEqual(["example/org/model-a"])
|
|
||||||
// The instance speaks protocol 2: its ids are `<provider>/<model>`.
|
|
||||||
loggedIn(["org/model-a"], "", { noProvider: true, protocols: [1, 2] })
|
|
||||||
expect(loadConfig().refs.map((r) => r.ref)).toEqual(["org/model-a"])
|
|
||||||
// `provider` said: used, whatever the protocol.
|
|
||||||
loggedIn(["org/model-a"])
|
|
||||||
expect(loadConfig().refs.map((r) => r.ref)).toEqual(["org/model-a"])
|
|
||||||
})
|
|
||||||
|
|
||||||
test("a models: override keyed by the old bare name lands on the served model; no phantom appears", () => {
|
test("a models: override keyed by the old bare name lands on the served model; no phantom appears", () => {
|
||||||
fake = fakeProvider([])
|
fake = fakeProvider([])
|
||||||
loggedIn(["deepseek/deepseek-flash"], " models: { deepseek-flash: { context: 1000 }, upcoming: { context: 5 } }\n")
|
loggedIn(["deepseek/deepseek-flash"], " models: { deepseek-flash: { context: 1000 }, upcoming: { context: 5 } }\n")
|
||||||
writeFileSync(join(paths.config, "config.yaml"), "model: example/deepseek-flash\n")
|
writeFileSync(join(paths.config, "config.yaml"), "model: example/deepseek-flash\n")
|
||||||
const l = loadConfig()
|
const l = loadConfig()
|
||||||
expect(l.refs.map((r) => r.ref)).toEqual(["deepseek/deepseek-flash", "example/upcoming"])
|
expect(l.refs.map((r) => r.ref)).toEqual(["deepseek/deepseek-flash", "upcoming"])
|
||||||
const m = resolveModel(l, l.config.model)
|
const m = resolveModel(l, l.config.model)
|
||||||
expect(m).toMatchObject({ id: "deepseek/deepseek-flash", spec: { context: 1000 } })
|
expect(m).toMatchObject({ id: "deepseek/deepseek-flash", spec: { context: 1000 } })
|
||||||
})
|
})
|
||||||
|
|||||||
@@ -13,7 +13,7 @@ afterEach(() => {
|
|||||||
})
|
})
|
||||||
|
|
||||||
function model(url: string, spec: ResolvedModel["spec"] = {}, conn: Partial<Connection> = {}): ResolvedModel {
|
function model(url: string, spec: ResolvedModel["spec"] = {}, conn: Partial<Connection> = {}): ResolvedModel {
|
||||||
const connection: Connection = { dialect: "openai-chat", base_url: url, models: { m: spec }, ...conn }
|
const connection: Connection = { base_url: url, models: { m: spec }, ...conn }
|
||||||
return { ref: `c${Math.random().toString(36).slice(2, 7)}/m`, connectionName: "c", connection, id: "m", spec }
|
return { ref: `c${Math.random().toString(36).slice(2, 7)}/m`, connectionName: "c", connection, id: "m", spec }
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -51,7 +51,7 @@ test("the asker is told the purpose, and the tool runs without it", async () =>
|
|||||||
{ chunks: [delta({ content: "ok" }, "stop")] },
|
{ chunks: [delta({ content: "ok" }, "stop")] },
|
||||||
])
|
])
|
||||||
mkdirSync(paths.config, { recursive: true })
|
mkdirSync(paths.config, { recursive: true })
|
||||||
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n dialect: openai-chat\n base_url: ${fake.url}\n models: { m: {} }\n`, { mode: 0o600 })
|
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n type: webui\n url: ${fake.url}\n models: { f/m: {} }\n`, { mode: 0o600 })
|
||||||
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\n")
|
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\n")
|
||||||
const asked: (string | undefined)[] = []
|
const asked: (string | undefined)[] = []
|
||||||
const app = createApp({ cwd: mkdtempSync(join(tmpdir(), "ph-purpose-")), store: false, snapshots: false, asker: { ask: async (r): Promise<AskReply> => (asked.push(r.purpose), { kind: "once" }) } })
|
const app = createApp({ cwd: mkdtempSync(join(tmpdir(), "ph-purpose-")), store: false, snapshots: false, asker: { ask: async (r): Promise<AskReply> => (asked.push(r.purpose), { kind: "once" }) } })
|
||||||
|
|||||||
@@ -18,7 +18,7 @@ afterEach(() => fake?.stop())
|
|||||||
|
|
||||||
async function replay(name: string) {
|
async function replay(name: string) {
|
||||||
fake = fakeProvider([{ chunks: [], raw: fixture(name) }])
|
fake = fakeProvider([{ chunks: [], raw: fixture(name) }])
|
||||||
const m: ResolvedModel = { ref: "swap/x", connectionName: "swap", id: "x", spec: {}, connection: { dialect: "openai-chat", base_url: fake.url, models: {} } }
|
const m: ResolvedModel = { ref: "swap/x", connectionName: "swap", id: "x", spec: {}, connection: { base_url: fake.url, models: {} } }
|
||||||
const events: StreamEvent[] = []
|
const events: StreamEvent[] = []
|
||||||
for await (const e of new OpenAIChatClient(m).stream({ system: "", messages: [{ role: "user", parts: [{ type: "text", text: "x" }] }], tools: [] }))
|
for await (const e of new OpenAIChatClient(m).stream({ system: "", messages: [{ role: "user", parts: [{ type: "text", text: "x" }] }], tools: [] }))
|
||||||
events.push(e)
|
events.push(e)
|
||||||
@@ -48,7 +48,7 @@ describe("recorded llama-swap streams", () => {
|
|||||||
test("qwen35: an answer given only inside reasoning is followed by one request for the answer", async () => {
|
test("qwen35: an answer given only inside reasoning is followed by one request for the answer", async () => {
|
||||||
fake = fakeProvider([{ chunks: [], raw: fixture("qwen35-answer-only-in-reasoning.sse") }, { chunks: [delta({ content: "The divisor was off by one." })] }])
|
fake = fakeProvider([{ chunks: [], raw: fixture("qwen35-answer-only-in-reasoning.sse") }, { chunks: [delta({ content: "The divisor was off by one." })] }])
|
||||||
mkdirSync(paths.config, { recursive: true })
|
mkdirSync(paths.config, { recursive: true })
|
||||||
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n dialect: openai-chat\n base_url: ${fake.url}\n models: { m: {} }\n`, { mode: 0o600 })
|
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n type: webui\n url: ${fake.url}\n models: { f/m: {} }\n`, { mode: 0o600 })
|
||||||
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\n")
|
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\n")
|
||||||
const dir = mkdtempSync(join(tmpdir(), "lembas-replay-"))
|
const dir = mkdtempSync(join(tmpdir(), "lembas-replay-"))
|
||||||
const app = createApp({ cwd: dir, mode: "edit", store: false, asker: { ask: async (): Promise<AskReply> => ({ kind: "deny" }) } })
|
const app = createApp({ cwd: dir, mode: "edit", store: false, asker: { ask: async (): Promise<AskReply> => ({ kind: "deny" }) } })
|
||||||
@@ -95,7 +95,7 @@ describe("effort refused inside a 200 stream (llama-swap loading, then the templ
|
|||||||
connectionName: "swap",
|
connectionName: "swap",
|
||||||
id: "bonsai",
|
id: "bonsai",
|
||||||
spec: { efforts: ["low", "medium", "high"] },
|
spec: { efforts: ["low", "medium", "high"] },
|
||||||
connection: { dialect: "openai-chat", base_url: fake.url, models: {} },
|
connection: { base_url: fake.url, models: {} },
|
||||||
}
|
}
|
||||||
const events: StreamEvent[] = []
|
const events: StreamEvent[] = []
|
||||||
for await (const e of new OpenAIChatClient(m).stream({ system: "", messages: [{ role: "user", parts: [{ type: "text", text: "x" }] }], tools: [], effort: "high" }))
|
for await (const e of new OpenAIChatClient(m).stream({ system: "", messages: [{ role: "user", parts: [{ type: "text", text: "x" }] }], tools: [], effort: "high" }))
|
||||||
@@ -111,7 +111,7 @@ describe("effort refused inside a 200 stream (llama-swap loading, then the templ
|
|||||||
|
|
||||||
test("an error after output has started is not retried", async () => {
|
test("an error after output has started is not retried", async () => {
|
||||||
fake = fakeProvider([{ chunks: [], raw: 'data: {"choices":[{"delta":{"content":"partial"}}]}\n\nerror: {"error":{"code":500,"message":"Unexpected reasoning effort high"}}\n\n' }])
|
fake = fakeProvider([{ chunks: [], raw: 'data: {"choices":[{"delta":{"content":"partial"}}]}\n\nerror: {"error":{"code":500,"message":"Unexpected reasoning effort high"}}\n\n' }])
|
||||||
const m: ResolvedModel = { ref: "swap/late", connectionName: "swap", id: "x", spec: { efforts: ["high"] }, connection: { dialect: "openai-chat", base_url: fake.url, models: {} } }
|
const m: ResolvedModel = { ref: "swap/late", connectionName: "swap", id: "x", spec: { efforts: ["high"] }, connection: { base_url: fake.url, models: {} } }
|
||||||
const run = async () => {
|
const run = async () => {
|
||||||
for await (const _ of new OpenAIChatClient(m).stream({ system: "", messages: [], tools: [], effort: "high" })) void _
|
for await (const _ of new OpenAIChatClient(m).stream({ system: "", messages: [], tools: [], effort: "high" })) void _
|
||||||
}
|
}
|
||||||
@@ -123,7 +123,7 @@ describe("effort refused inside a 200 stream (llama-swap loading, then the templ
|
|||||||
describe("llama-swap unloads the model under a running request", () => {
|
describe("llama-swap unloads the model under a running request", () => {
|
||||||
test("'group: model unloaded' (a string code) is a server error: retried once, and the reply arrives", async () => {
|
test("'group: model unloaded' (a string code) is a server error: retried once, and the reply arrives", async () => {
|
||||||
fake = fakeProvider([{ chunks: [], raw: fixture("llama-swap-model-unloaded.sse") }, { chunks: [delta({ content: "second try" })] }])
|
fake = fakeProvider([{ chunks: [], raw: fixture("llama-swap-model-unloaded.sse") }, { chunks: [delta({ content: "second try" })] }])
|
||||||
const m: ResolvedModel = { ref: "swap/x", connectionName: "swap", id: "x", spec: {}, connection: { dialect: "openai-chat", base_url: fake.url, models: {} } }
|
const m: ResolvedModel = { ref: "swap/x", connectionName: "swap", id: "x", spec: {}, connection: { base_url: fake.url, models: {} } }
|
||||||
const events: StreamEvent[] = []
|
const events: StreamEvent[] = []
|
||||||
for await (const e of new OpenAIChatClient(m).stream({ system: "", messages: [{ role: "user", parts: [{ type: "text", text: "x" }] }], tools: [] })) events.push(e)
|
for await (const e of new OpenAIChatClient(m).stream({ system: "", messages: [{ role: "user", parts: [{ type: "text", text: "x" }] }], tools: [] })) events.push(e)
|
||||||
expect(events.some((e) => e.type === "notice" && e.message.includes("group: model unloaded"))).toBe(true)
|
expect(events.some((e) => e.type === "notice" && e.message.includes("group: model unloaded"))).toBe(true)
|
||||||
|
|||||||
@@ -1,83 +0,0 @@
|
|||||||
import { afterEach, describe, expect, test } from "bun:test"
|
|
||||||
import { readFileSync } from "node:fs"
|
|
||||||
import { join } from "node:path"
|
|
||||||
import { ResponsesClient, toResponsesInput } from "../src/provider/responses.ts"
|
|
||||||
import type { Message, ResolvedModel, StreamEvent } from "../src/provider/types.ts"
|
|
||||||
import { fakeProvider, type Fake } from "./fake-provider.ts"
|
|
||||||
|
|
||||||
let fake: Fake | undefined
|
|
||||||
afterEach(() => fake?.stop())
|
|
||||||
|
|
||||||
const model = (url: string, spec: ResolvedModel["spec"] = {}): ResolvedModel => ({ ref: "r/m", connectionName: "r", id: "m", spec, connection: { dialect: "responses", base_url: url, models: {} } })
|
|
||||||
const sse = (events: Record<string, unknown>[]) => events.map((e) => `data: ${JSON.stringify(e)}\n\n`).join("")
|
|
||||||
async function run(c: ResponsesClient, messages: Message[] = [{ role: "user", parts: [{ type: "text", text: "hi" }] }], effort: any = null) {
|
|
||||||
const ev: StreamEvent[] = []
|
|
||||||
for await (const e of c.stream({ system: "sys", messages, tools: [{ name: "list", description: "d", parameters: { type: "object" } }], effort })) ev.push(e)
|
|
||||||
return ev
|
|
||||||
}
|
|
||||||
|
|
||||||
describe("responses dialect", () => {
|
|
||||||
test("a real vLLM stream: reasoning text, then a function call, with usage", async () => {
|
|
||||||
fake = fakeProvider([{ chunks: [], raw: readFileSync(join(import.meta.dir, "fixtures/sse/vllm-responses-reasoning-tool.sse"), "utf8") }])
|
|
||||||
const ev = await run(new ResponsesClient(model(fake.url)))
|
|
||||||
const fin = ev.find((e) => e.type === "finish")!
|
|
||||||
if (fin.type !== "finish") throw new Error()
|
|
||||||
expect(fin.reason).toBe("tool_calls")
|
|
||||||
expect(fin.message.parts[0]).toMatchObject({ type: "reasoning" })
|
|
||||||
expect(fin.message.parts[1]).toMatchObject({ type: "tool_call", name: "list" })
|
|
||||||
expect(ev.find((e) => e.type === "usage")).toMatchObject({ usage: { input: 4788, output: 111, reasoning: 21 } })
|
|
||||||
})
|
|
||||||
|
|
||||||
test("request shape; an encrypted reasoning item goes back verbatim on the next turn", async () => {
|
|
||||||
const reasoningItem = { id: "rs_1", type: "reasoning", summary: [{ type: "summary_text", text: "plan" }], encrypted_content: "ENC" }
|
|
||||||
fake = fakeProvider([
|
|
||||||
{
|
|
||||||
chunks: [],
|
|
||||||
raw: sse([
|
|
||||||
{ type: "response.reasoning_summary_text.delta", item_id: "rs_1", delta: "plan" },
|
|
||||||
{ type: "response.output_item.done", item: reasoningItem },
|
|
||||||
{ type: "response.output_item.added", item: { type: "function_call", id: "fc_1", call_id: "call_1", name: "list" } },
|
|
||||||
{ type: "response.function_call_arguments.delta", item_id: "fc_1", delta: "{}" },
|
|
||||||
{ type: "response.output_item.done", item: { type: "function_call", id: "fc_1", call_id: "call_1", name: "list", arguments: "{}" } },
|
|
||||||
{ type: "response.completed", response: { status: "completed", usage: { input_tokens: 10, output_tokens: 5, input_tokens_details: { cached_tokens: 4 } } } },
|
|
||||||
]),
|
|
||||||
},
|
|
||||||
{ chunks: [], raw: sse([{ type: "response.output_text.delta", delta: "done" }, { type: "response.completed", response: { status: "completed" } }]) },
|
|
||||||
])
|
|
||||||
const c = new ResponsesClient(model(fake.url, { max_output: 999 }))
|
|
||||||
const first = await run(c, undefined, "high")
|
|
||||||
const fin = first.find((e) => e.type === "finish")!
|
|
||||||
if (fin.type !== "finish") throw new Error()
|
|
||||||
expect(first.find((e) => e.type === "usage")).toEqual({ type: "usage", usage: { input: 10, output: 5, reasoning: undefined, cached: 4 } })
|
|
||||||
expect(fake.requests[0]).toMatchObject({ model: "m", store: false, instructions: "sys", max_output_tokens: 999, reasoning: { effort: "high", summary: "auto" }, include: ["reasoning.encrypted_content"] })
|
|
||||||
expect(fake.requests[0].tools[0]).toMatchObject({ type: "function", name: "list" })
|
|
||||||
await run(c, [{ role: "user", parts: [{ type: "text", text: "hi" }] }, fin.message, { role: "tool", callId: "call_1", name: "list", content: "a.txt" }], "high")
|
|
||||||
expect(fake.requests[1].input).toEqual([
|
|
||||||
{ role: "user", content: [{ type: "input_text", text: "hi" }] },
|
|
||||||
reasoningItem,
|
|
||||||
{ type: "function_call", call_id: "call_1", name: "list", arguments: "{}" },
|
|
||||||
{ type: "function_call_output", call_id: "call_1", output: "a.txt" },
|
|
||||||
])
|
|
||||||
})
|
|
||||||
|
|
||||||
test("a server that does not know `include` gets one retry without it", async () => {
|
|
||||||
fake = fakeProvider([{ status: 400, body: '{"error":{"message":"Unknown parameter: include"}}' }, { chunks: [], raw: sse([{ type: "response.output_text.delta", delta: "ok" }]) }])
|
|
||||||
await run(new ResponsesClient(model(fake.url)), undefined, "low")
|
|
||||||
expect(fake.requests[1].include).toBeUndefined()
|
|
||||||
expect(fake.requests[1].reasoning).toEqual({ effort: "low", summary: "auto" })
|
|
||||||
})
|
|
||||||
|
|
||||||
test("history from another dialect: reasoning without an item is not sent; images only with vision", () => {
|
|
||||||
const input = toResponsesInput(
|
|
||||||
[
|
|
||||||
{ role: "user", parts: [{ type: "text", text: "see" }, { type: "image", mime: "image/png", data: "AAA" }] },
|
|
||||||
{ role: "assistant", parts: [{ type: "reasoning", text: "thought elsewhere" }, { type: "text", text: "ok" }] },
|
|
||||||
],
|
|
||||||
true,
|
|
||||||
)
|
|
||||||
expect(input).toEqual([
|
|
||||||
{ role: "user", content: [{ type: "input_text", text: "see" }, { type: "input_image", image_url: "data:image/png;base64,AAA" }] },
|
|
||||||
{ role: "assistant", content: [{ type: "output_text", text: "ok" }] },
|
|
||||||
])
|
|
||||||
})
|
|
||||||
})
|
|
||||||
@@ -19,7 +19,7 @@ const yes = { ask: async (): Promise<AskReply> => ({ kind: "once" }) }
|
|||||||
function setup(script: Parameters<typeof fakeProvider>[0]) {
|
function setup(script: Parameters<typeof fakeProvider>[0]) {
|
||||||
fake = fakeProvider(script)
|
fake = fakeProvider(script)
|
||||||
mkdirSync(paths.config, { recursive: true })
|
mkdirSync(paths.config, { recursive: true })
|
||||||
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n fake:\n dialect: openai-chat\n base_url: ${fake.url}\n models:\n m: { context: 32768 }\n`, { mode: 0o600 })
|
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n fake:\n type: webui\n url: ${fake.url}\n models:\n fake/m: { context: 32768 }\n`, { mode: 0o600 })
|
||||||
writeFileSync(join(paths.config, "config.yaml"), "model: fake/m\n")
|
writeFileSync(join(paths.config, "config.yaml"), "model: fake/m\n")
|
||||||
const dir = mkdtempSync(join(tmpdir(), "lembas-results-"))
|
const dir = mkdtempSync(join(tmpdir(), "lembas-results-"))
|
||||||
writeFileSync(join(dir, "a.txt"), "one\ntwo\nthree\n")
|
writeFileSync(join(dir, "a.txt"), "one\ntwo\nthree\n")
|
||||||
|
|||||||
@@ -41,7 +41,7 @@ function pair(): [Transport, Transport] {
|
|||||||
|
|
||||||
function config(url: string) {
|
function config(url: string) {
|
||||||
mkdirSync(paths.config, { recursive: true })
|
mkdirSync(paths.config, { recursive: true })
|
||||||
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n dialect: openai-chat\n base_url: ${url}\n models: { m: {} }\n`, { mode: 0o600 })
|
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n type: webui\n url: ${url}\n models: { f/m: {} }\n`, { mode: 0o600 })
|
||||||
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\n")
|
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\n")
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -21,7 +21,7 @@ function home(config = "model: f/m\n", script: Parameters<typeof fakeProvider>[0
|
|||||||
mkdirSync(paths.config, { recursive: true })
|
mkdirSync(paths.config, { recursive: true })
|
||||||
writeFileSync(
|
writeFileSync(
|
||||||
join(paths.config, "connections.yaml"),
|
join(paths.config, "connections.yaml"),
|
||||||
`connections:\n f:\n dialect: openai-chat\n base_url: ${fake.url}\n models:\n m: { efforts: [low, medium, high], effort: medium }\n big: {}\n`,
|
`connections:\n f:\n type: webui\n url: ${fake.url}\n models:\n f/m: { efforts: [low, medium, high], effort: medium }\n f/big: {}\n`,
|
||||||
{ mode: 0o600 },
|
{ mode: 0o600 },
|
||||||
)
|
)
|
||||||
writeFileSync(join(paths.config, "config.yaml"), config)
|
writeFileSync(join(paths.config, "config.yaml"), config)
|
||||||
@@ -68,7 +68,6 @@ test("session, global and project scope: what is written where, and where a valu
|
|||||||
expect(readFileSync(join(cwd, ".agent/config.yaml"), "utf8")).toContain("auto_at: 0.6")
|
expect(readFileSync(join(cwd, ".agent/config.yaml"), "utf8")).toContain("auto_at: 0.6")
|
||||||
expect(s.get("compaction.auto_at")).toMatchObject({ value: 0.6, source: "project" })
|
expect(s.get("compaction.auto_at")).toMatchObject({ value: 0.6, source: "project" })
|
||||||
|
|
||||||
expect(() => s.set("model", "f/nope")).toThrow()
|
|
||||||
expect(() => s.set("effort", "max")).toThrow("takes")
|
expect(() => s.set("effort", "max")).toThrow("takes")
|
||||||
expect(() => s.set("update.channel", "beta", "project")).toThrow("can be set for: global")
|
expect(() => s.set("update.channel", "beta", "project")).toThrow("can be set for: global")
|
||||||
})
|
})
|
||||||
@@ -92,7 +91,7 @@ test("the agent changes a setting through the tool, with approval; a model chang
|
|||||||
expect(asked[0]!.always).toEqual(["session effort=*"])
|
expect(asked[0]!.always).toEqual(["session effort=*"])
|
||||||
expect(app.engine.model.ref).toBe("f/big")
|
expect(app.engine.model.ref).toBe("f/big")
|
||||||
// The third request went to the new model.
|
// The third request went to the new model.
|
||||||
expect(fake!.requests[2]!.model).toBe("big")
|
expect(fake!.requests[2]!.model).toBe("f/big")
|
||||||
})
|
})
|
||||||
|
|
||||||
test("settings_tool: allow — no asking; but a less strict mode is asked every time, with no 'always'", async () => {
|
test("settings_tool: allow — no asking; but a less strict mode is asked every time, with no 'always'", async () => {
|
||||||
|
|||||||
+1
-1
@@ -22,7 +22,7 @@ function setup(script: Parameters<typeof fakeProvider>[0], config = "") {
|
|||||||
mkdirSync(paths.config, { recursive: true })
|
mkdirSync(paths.config, { recursive: true })
|
||||||
writeFileSync(
|
writeFileSync(
|
||||||
join(paths.config, "connections.yaml"),
|
join(paths.config, "connections.yaml"),
|
||||||
`connections:\n f:\n dialect: openai-chat\n base_url: ${fake.url}\n models:\n main: { context: 32768 }\n a: {}\n`,
|
`connections:\n f:\n type: webui\n url: ${fake.url}\n models:\n f/main: { context: 32768 }\n f/a: {}\n`,
|
||||||
{ mode: 0o600 },
|
{ mode: 0o600 },
|
||||||
)
|
)
|
||||||
writeFileSync(join(paths.config, "config.yaml"), `model: f/main\ntitles: prompt\n${config}`)
|
writeFileSync(join(paths.config, "config.yaml"), `model: f/main\ntitles: prompt\n${config}`)
|
||||||
|
|||||||
+11
-12
@@ -13,17 +13,15 @@ let fake: Fake | undefined
|
|||||||
let other: Fake | undefined
|
let other: Fake | undefined
|
||||||
afterEach(() => (fake?.stop(), other?.stop(), (other = undefined)))
|
afterEach(() => (fake?.stop(), other?.stop(), (other = undefined)))
|
||||||
|
|
||||||
/** Two scripted servers: the parent's model on `f`, the child's on `g` (task's `model: "g/m"`) —
|
/** Two scripts: the parent's model `f/m`, the child's `g/…` (task's `model: "g/m"`), routed by the
|
||||||
* a shared script would hand a reply to whichever engine asked first. */
|
* fake server — a shared script would hand a reply to whichever engine asked first. */
|
||||||
function pair(parent: Parameters<typeof fakeProvider>[0], child: Parameters<typeof fakeProvider>[0], o: { oneAtATime?: boolean; config?: string; asker?: Asker } = {}) {
|
function pair(parent: Parameters<typeof fakeProvider>[0], child: Parameters<typeof fakeProvider>[0], o: { oneAtATime?: boolean; config?: string; asker?: Asker } = {}) {
|
||||||
fake = fakeProvider(parent)
|
const child$ = (other = fakeProvider(child))
|
||||||
other = fakeProvider(child)
|
fake = fakeProvider(parent, { route: (model) => (model.startsWith("g/") ? child$ : undefined) })
|
||||||
mkdirSync(paths.config, { recursive: true })
|
mkdirSync(paths.config, { recursive: true })
|
||||||
writeFileSync(
|
// One server; its g models answered by the child's script, sharing one GPU when asked to.
|
||||||
join(paths.config, "connections.yaml"),
|
const g = o.oneAtATime ? "{ group: gpu }" : "{}"
|
||||||
`connections:\n f:\n dialect: openai-chat\n base_url: ${fake.url}\n models: { m: {} }\n g:\n dialect: openai-chat\n base_url: ${other.url}\n${o.oneAtATime ? " one_model_at_a_time: true\n" : ""} models: { m: {}, n: {} }\n`,
|
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n type: webui\n url: ${fake.url}\n models: { f/m: {}, g/m: ${g}, g/n: ${g} }\n`, { mode: 0o600 })
|
||||||
{ mode: 0o600 },
|
|
||||||
)
|
|
||||||
writeFileSync(join(paths.config, "config.yaml"), `model: f/m\n${o.config ?? ""}`)
|
writeFileSync(join(paths.config, "config.yaml"), `model: f/m\n${o.config ?? ""}`)
|
||||||
const cwd = mkdtempSync(join(tmpdir(), "ph-sub-"))
|
const cwd = mkdtempSync(join(tmpdir(), "ph-sub-"))
|
||||||
writeFileSync(join(cwd, "calc.py"), "def average(v):\n return sum(v) / (len(v) - 1)\n")
|
writeFileSync(join(cwd, "calc.py"), "def average(v):\n return sum(v) / (len(v) - 1)\n")
|
||||||
@@ -40,7 +38,7 @@ const settled = async (a: ReturnType<typeof createApp>, n: number) => {
|
|||||||
function app(script: Parameters<typeof fakeProvider>[0]) {
|
function app(script: Parameters<typeof fakeProvider>[0]) {
|
||||||
fake = fakeProvider(script)
|
fake = fakeProvider(script)
|
||||||
mkdirSync(paths.config, { recursive: true })
|
mkdirSync(paths.config, { recursive: true })
|
||||||
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n dialect: openai-chat\n base_url: ${fake.url}\n models: { m: {} }\n`, { mode: 0o600 })
|
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n type: webui\n url: ${fake.url}\n models: { f/m: {} }\n`, { mode: 0o600 })
|
||||||
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\n")
|
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\n")
|
||||||
const cwd = mkdtempSync(join(tmpdir(), "ph-sub-"))
|
const cwd = mkdtempSync(join(tmpdir(), "ph-sub-"))
|
||||||
writeFileSync(join(cwd, "calc.py"), "def average(v):\n return sum(v) / (len(v) - 1)\n")
|
writeFileSync(join(cwd, "calc.py"), "def average(v):\n return sum(v) / (len(v) - 1)\n")
|
||||||
@@ -176,7 +174,7 @@ test("a worktree subagent cannot touch the main checkout, even unrestricted", as
|
|||||||
{ chunks: [delta({ content: "tried" })] },
|
{ chunks: [delta({ content: "tried" })] },
|
||||||
{ chunks: [delta({ content: "ok" })] },
|
{ chunks: [delta({ content: "ok" })] },
|
||||||
])
|
])
|
||||||
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n dialect: openai-chat\n base_url: ${fake.url}\n models: { m: {} }\n`, { mode: 0o600 })
|
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n f:\n type: webui\n url: ${fake.url}\n models: { f/m: {} }\n`, { mode: 0o600 })
|
||||||
for (const args of [["init", "-q", "-b", "main"], ["config", "user.email", "t@e"], ["config", "user.name", "T"], ["config", "commit.gpgsign", "false"], ["add", "calc.py"], ["commit", "-q", "-m", "init"]]) git(root, args)
|
for (const args of [["init", "-q", "-b", "main"], ["config", "user.email", "t@e"], ["config", "user.name", "T"], ["config", "commit.gpgsign", "false"], ["add", "calc.py"], ["commit", "-q", "-m", "init"]]) git(root, args)
|
||||||
const b = createApp({ cwd: root, mode: "auto", store: false, asker: { ask: async (): Promise<AskReply> => ({ kind: "once" }) } })
|
const b = createApp({ cwd: root, mode: "auto", store: false, asker: { ask: async (): Promise<AskReply> => ({ kind: "once" }) } })
|
||||||
await b.engine.prompt("go")
|
await b.engine.prompt("go")
|
||||||
@@ -300,8 +298,9 @@ describe("background subagents (spec 2.0.0)", () => {
|
|||||||
const results = fake!.requests[1].messages.filter((m: any) => m.role === "tool").map((m: any) => m.content as string)
|
const results = fake!.requests[1].messages.filter((m: any) => m.role === "tool").map((m: any) => m.content as string)
|
||||||
expect(results[0]).toStartWith("[#0001]\nStarted subagent explore as job1")
|
expect(results[0]).toStartWith("[#0001]\nStarted subagent explore as job1")
|
||||||
expect(results[1]).toContain("holds one model at a time, and a subagent is already running on g/m there")
|
expect(results[1]).toContain("holds one model at a time, and a subagent is already running on g/m there")
|
||||||
expect(other!.requests.map((r: any) => r.model)).toEqual(["m"])
|
|
||||||
await settled(a, 3)
|
await settled(a, 3)
|
||||||
|
for (let i = 0; i < 100 && !other!.requests.length; i++) await Bun.sleep(20)
|
||||||
|
expect(other!.requests.map((r: any) => r.model)).toEqual(["g/m"])
|
||||||
})
|
})
|
||||||
|
|
||||||
test("a broken agent file is said at start, and the agent is still offered with its body", () => {
|
test("a broken agent file is said at start, and the agent is still offered with its body", () => {
|
||||||
|
|||||||
@@ -18,7 +18,7 @@ function setup(script: Parameters<typeof fakeProvider>[0], config = "") {
|
|||||||
mkdirSync(paths.config, { recursive: true })
|
mkdirSync(paths.config, { recursive: true })
|
||||||
writeFileSync(
|
writeFileSync(
|
||||||
join(paths.config, "connections.yaml"),
|
join(paths.config, "connections.yaml"),
|
||||||
`connections:\n fake:\n dialect: openai-chat\n base_url: ${fake.url}\n models:\n m: { context: 32768 }\n tiny: { context: 8192 }\n`,
|
`connections:\n fake:\n type: webui\n url: ${fake.url}\n models:\n fake/m: { context: 32768 }\n fake/tiny: { context: 8192 }\n`,
|
||||||
{ mode: 0o600 },
|
{ mode: 0o600 },
|
||||||
)
|
)
|
||||||
writeFileSync(join(paths.config, "config.yaml"), `model: fake/m\n${config}`)
|
writeFileSync(join(paths.config, "config.yaml"), `model: fake/m\n${config}`)
|
||||||
@@ -111,8 +111,8 @@ describe("a session gets a title", () => {
|
|||||||
const app = createApp({ cwd: dir, asker: noAsk, store, snapshots: false })
|
const app = createApp({ cwd: dir, asker: noAsk, store, snapshots: false })
|
||||||
await app.engine.prompt("something")
|
await app.engine.prompt("something")
|
||||||
await until(() => store.session(app.engine.sessionId!)!.title === "Tiny title")
|
await until(() => store.session(app.engine.sessionId!)!.title === "Tiny title")
|
||||||
expect(fake!.requests[0].model).toBe("m")
|
expect(fake!.requests[0].model).toBe("fake/m")
|
||||||
expect(fake!.requests[1].model).toBe("tiny")
|
expect(fake!.requests[1].model).toBe("fake/tiny")
|
||||||
})
|
})
|
||||||
|
|
||||||
test("a failed title request leaves the prompt's title", async () => {
|
test("a failed title request leaves the prompt's title", async () => {
|
||||||
|
|||||||
+1
-1
@@ -31,7 +31,7 @@ const model = (tls?: { ca?: string; insecure?: boolean }): ResolvedModel => ({
|
|||||||
connectionName: "t",
|
connectionName: "t",
|
||||||
id: "m",
|
id: "m",
|
||||||
spec: {},
|
spec: {},
|
||||||
connection: { dialect: "openai-chat", base_url: `https://127.0.0.1:${server.port}/v1`, tls, models: {} },
|
connection: { base_url: `https://127.0.0.1:${server.port}/v1`, tls, models: {} },
|
||||||
})
|
})
|
||||||
|
|
||||||
test("a private CA is refused by default, with advice", async () => {
|
test("a private CA is refused by default, with advice", async () => {
|
||||||
|
|||||||
@@ -39,7 +39,7 @@ test("The CLI's own files under .agent that are links are refused, not written t
|
|||||||
|
|
||||||
test("an untrusted project's task board is neither in the prompt nor writable", async () => {
|
test("an untrusted project's task board is neither in the prompt nor writable", async () => {
|
||||||
mkdirSync(paths.config, { recursive: true })
|
mkdirSync(paths.config, { recursive: true })
|
||||||
writeFileSync(join(paths.config, "connections.yaml"), "connections:\n f:\n dialect: openai-chat\n base_url: http://127.0.0.1:9/v1\n models: { m: {} }\n", { mode: 0o600 })
|
writeFileSync(join(paths.config, "connections.yaml"), "connections:\n f:\n type: webui\n url: http://127.0.0.1:9/v1\n models: { f/m: {} }\n", { mode: 0o600 })
|
||||||
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\n")
|
writeFileSync(join(paths.config, "config.yaml"), "model: f/m\n")
|
||||||
const root = mkdtempSync(join(tmpdir(), "ph-untrusted-"))
|
const root = mkdtempSync(join(tmpdir(), "ph-untrusted-"))
|
||||||
mkdirSync(join(root, ".agent"))
|
mkdirSync(join(root, ".agent"))
|
||||||
|
|||||||
@@ -14,7 +14,7 @@ afterEach(() => setup?.renderer.destroy())
|
|||||||
async function boot(cwd: string) {
|
async function boot(cwd: string) {
|
||||||
const fake = fakeProvider([])
|
const fake = fakeProvider([])
|
||||||
mkdirSync(paths.config, { recursive: true })
|
mkdirSync(paths.config, { recursive: true })
|
||||||
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n fake:\n dialect: openai-chat\n base_url: ${fake.url}\n models: { m: {} }\n`, { mode: 0o600 })
|
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n fake:\n type: webui\n url: ${fake.url}\n models: { fake/m: {} }\n`, { mode: 0o600 })
|
||||||
writeFileSync(join(paths.config, "config.yaml"), "model: fake/m\n")
|
writeFileSync(join(paths.config, "config.yaml"), "model: fake/m\n")
|
||||||
let quit = 0
|
let quit = 0
|
||||||
setup = await testRender(() => <Boot options={{ cwd, store: false }} onQuit={() => quit++} />, { width: 100, height: 30 })
|
setup = await testRender(() => <Boot options={{ cwd, store: false }} onQuit={() => quit++} />, { width: 100, height: 30 })
|
||||||
|
|||||||
@@ -16,7 +16,7 @@ export async function tui(script: Scripted[], size = { width: 100, height: 30 },
|
|||||||
mkdirSync(paths.config, { recursive: true })
|
mkdirSync(paths.config, { recursive: true })
|
||||||
writeFileSync(
|
writeFileSync(
|
||||||
join(paths.config, "connections.yaml"),
|
join(paths.config, "connections.yaml"),
|
||||||
`connections:\n fake:\n dialect: openai-chat\n base_url: ${fake.url}\n models:\n coder: { context: 32768, efforts: [low, high], effort: low }\n other: { context: 8192 }\n`,
|
`connections:\n fake:\n type: webui\n url: ${fake.url}\n models:\n fake/coder: { context: 32768, efforts: [low, high], effort: low }\n fake/other: { context: 8192 }\n`,
|
||||||
{ mode: 0o600 },
|
{ mode: 0o600 },
|
||||||
)
|
)
|
||||||
writeFileSync(join(paths.config, "config.yaml"), `model: fake/coder\n${extraConfig}`)
|
writeFileSync(join(paths.config, "config.yaml"), `model: fake/coder\n${extraConfig}`)
|
||||||
|
|||||||
@@ -31,16 +31,16 @@ test("no model yet: the sign-in is first and focused; ctrl+r opens the session o
|
|||||||
return setup!.captureCharFrame()
|
return setup!.captureCharFrame()
|
||||||
}
|
}
|
||||||
let f = await frame()
|
let f = await frame()
|
||||||
expect(f).toContain("no model to talk to yet")
|
// 2.0: the server's models or none — the screen is the sign-in, and nothing else to fill in.
|
||||||
|
expect(f).toContain("not logged in to a LLeMbas server")
|
||||||
expect(f).toContain("Log in to a LLeMbas instance")
|
expect(f).toContain("Log in to a LLeMbas instance")
|
||||||
// Before the file instructions: the screen opens on the sign-in.
|
expect(f).not.toContain("connections.yaml")
|
||||||
expect(f.indexOf("Log in to a LLeMbas instance")).toBeLessThan(f.indexOf("connections.yaml"))
|
|
||||||
// Focused: what is typed is the address, without pressing l first.
|
// Focused: what is typed is the address, without pressing l first.
|
||||||
await setup.mockInput.typeText("ai.example")
|
await setup.mockInput.typeText("ai.example")
|
||||||
f = await frame()
|
f = await frame()
|
||||||
expect(f).toContain("Address: ai.example")
|
expect(f).toContain("Address: ai.example")
|
||||||
// A model added meanwhile: ctrl+r, a letter-free key, opens the session.
|
// A model added meanwhile: ctrl+r, a letter-free key, opens the session.
|
||||||
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n fake:\n dialect: openai-chat\n base_url: ${fake.url}\n models: { m: {} }\n`, { mode: 0o600 })
|
writeFileSync(join(paths.config, "connections.yaml"), `connections:\n fake:\n type: webui\n url: ${fake.url}\n models: { fake/m: {} }\n`, { mode: 0o600 })
|
||||||
writeFileSync(join(paths.config, "config.yaml"), "model: fake/m\n")
|
writeFileSync(join(paths.config, "config.yaml"), "model: fake/m\n")
|
||||||
setup.mockInput.pressKey("r", { ctrl: true })
|
setup.mockInput.pressKey("r", { ctrl: true })
|
||||||
f = await frame()
|
f = await frame()
|
||||||
|
|||||||
Reference in new issue
Block a user