Custom HTTP tools an administrator defines

A row in custom_tools becomes a ToolDef like any built-in, offered beside
the thirteen. The registry had to stop being an import-time constant for
that: `resolve_tools` now returns the schemas *and* the runners together,
carried to the loop on the ToolContext.

That closes a hole on the way. `run_tool` looked names up in the global
REGISTRY with no reference to what had been offered, so a model naming a
tool its chat was gated out of -- a family switched off, a permission the
reader lacks -- had it run anyway. The resolved set is now authoritative.

Arguments come from a model, so an argument may fill a hole but never move
the target: the scheme and host of a URL template are literal, values are
escaped for where they land, and the origin is pinned afterwards. Every
redirect hop is checked the way services/fetch.py checks one, and the
secret is dropped if a hop leaves the origin it was issued for.

Also fixes the tool-activity block claiming every library tool had
"searched the web", which it has done since the second family landed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Jaroslav Beneš
2026-08-01 16:26:47 +02:00
parent d9f274ec1a
commit bc84fec21d
26 changed files with 2771 additions and 60 deletions
+4 -4
View File
@@ -43,6 +43,10 @@
{{ icon("globe", "icon--sm") }}
<span class="nav-item__label">Web search</span>
</a>
<a class="nav-item {{ 'is-active' if section == 'tools' }}" href="/admin/tools">
{{ icon("link", "icon--sm") }}
<span class="nav-item__label">Tools</span>
</a>
<a class="nav-item {{ 'is-active' if section == 'prompts' }}" href="/admin/prompts">
{{ icon("sparkle", "icon--sm") }}
<span class="nav-item__label">Prompts</span>
@@ -64,10 +68,6 @@
<div class="nav-group">
<div class="nav-group__label">Not yet built</div>
<span class="nav-item is-disabled">
{{ icon("gear", "icon--sm") }}
<span class="nav-item__label">Tools</span>
</span>
<span class="nav-item is-disabled">
{{ icon("server", "icon--sm") }}
<span class="nav-item__label">Agents</span>
@@ -0,0 +1,21 @@
{% from "_macros.html" import icon %}
{#
The result of calling one tool by hand.
What comes back is whatever the endpoint chose to send, which makes it exactly
as untrusted as a search result or model output. It is shown escaped, inside a
<pre>, and never rendered as Markdown.
#}
{% if error %}
<div class="alert alert--error">{{ icon("warning", "icon--sm") }} <span>{{ error }}</span></div>
{% else %}
<div class="alert alert--success">
{{ icon("check", "icon--sm") }}
<span>{{ detail or tool.slug }} answered.</span>
</div>
{% endif %}
{% if outcome %}
<p class="field__hint">This is what the model would read back:</p>
<pre class="tool-result__text">{{ outcome.content }}</pre>
{% endif %}
@@ -0,0 +1,308 @@
{% extends "admin/_layout.html" %}
{% from "_macros.html" import icon %}
{% set section = "tools" %}
{% block title %}{{ "New tool" if is_new else tool.name }} - LLeMbas{% endblock %}
{% block heading %}{{ "New tool" if is_new else tool.name }}{% endblock %}
{% block admin_content %}
<nav class="crumbs">
<a class="crumbs__back" href="/admin/tools">
{{ icon("chevron-right", "icon--sm crumbs__icon") }} All tools
</a>
</nav>
{% if error %}
<div class="alert alert--error">{{ icon("warning", "icon--sm") }} <span>{{ error }}</span></div>
{% endif %}
<form method="post" action="{{ '/admin/tools' if is_new else '/admin/tools/' ~ tool.id }}"
class="form-grid">
<section class="card">
<h2 class="card__title">What it is</h2>
<div class="field">
<label class="field__label" for="name">Name</label>
<input class="input" id="name" name="name" value="{{ tool.name }}" required
maxlength="120" placeholder="Weather">
<p class="field__hint">Shown in the transcript when the model uses it.</p>
</div>
<div class="field">
<label class="field__label" for="slug">Identifier</label>
<input class="input input--mono" id="slug" name="slug" value="{{ tool.slug }}" required
maxlength="48" placeholder="weather" pattern="[a-z0-9][a-z0-9_\-]*">
<p class="field__hint">
The name the model calls, and the key its guidance is stored under.
Lowercase letters, digits, hyphens and underscores.
{% if not is_new %}Changing it starts its guidance afresh.{% endif %}
</p>
</div>
<div class="field">
<label class="field__label" for="description">Description</label>
<textarea class="textarea" id="description" name="description" rows="3"
placeholder="Look up the current weather for a city."
>{{ tool.description }}</textarea>
<p class="field__hint">
Sent to the model verbatim. This is the whole basis on which it decides
whether to call this tool, so say what it does and when it is the right
thing to use.
</p>
</div>
<div class="field">
<label class="field__label" for="parameters">Parameters</label>
<textarea class="textarea input--mono" id="parameters" name="parameters" rows="10"
spellcheck="false">{{ parameters_text }}</textarea>
<p class="field__hint">
A JSON Schema object. Each property becomes a
<code>{{ '{{name}}' }}</code> you can use below, and a
<code>description</code> on each one is worth writing — the model reads
those too.
</p>
</div>
</section>
<section class="card">
<h2 class="card__title">The call</h2>
<div class="field">
<label class="field__label" for="method">Method</label>
<select class="select" id="method" name="method">
{% for method in methods %}
<option value="{{ method }}" {{ 'selected' if method == tool.method }}>{{ method }}</option>
{% endfor %}
</select>
</div>
<div class="field">
<label class="field__label" for="url_template">URL</label>
<input class="input input--mono" id="url_template" name="url_template" required
value="{{ tool.url_template }}"
placeholder="https://api.example.com/weather/{{ '{{city}}' }}">
<p class="field__hint">
Placeholders are filled from the arguments and escaped, so a value
cannot add a path segment or a query of its own. The scheme and host
must be written out — they cannot come from an argument.
</p>
</div>
<div class="field">
<label class="field__label" for="headers">Headers</label>
<textarea class="textarea input--mono" id="headers" name="headers" rows="3"
spellcheck="false"
placeholder="Accept: application/json">{{ headers_text }}</textarea>
<p class="field__hint">One <code>Name: value</code> per line. Placeholders work here too.</p>
</div>
<div class="field">
<label class="field__label" for="body_template">Body</label>
<textarea class="textarea input--mono" id="body_template" name="body_template" rows="4"
spellcheck="false">{{ tool.body_template }}</textarea>
<p class="field__hint">
Ignored for GET. Placeholders are escaped for JSON, so a value cannot
end the string it sits in and add a field.
</p>
</div>
<div class="field">
<label class="field__label" for="timeout">Timeout (seconds)</label>
<input class="input" id="timeout" name="timeout" value="{{ tool.timeout }}" inputmode="numeric">
</div>
</section>
<section class="card">
<h2 class="card__title">Credential</h2>
<div class="field">
<label class="field__label" for="secret_placement">How it is sent</label>
<select class="select" id="secret_placement" name="secret_placement">
{% for value, label in secret_placements %}
<option value="{{ value }}" {{ 'selected' if value == tool.secret_placement }}>
{{ label }}
</option>
{% endfor %}
</select>
</div>
<div class="field">
<label class="field__label" for="secret_name">Header or parameter name</label>
<input class="input input--mono" id="secret_name" name="secret_name"
value="{{ tool.secret_name }}" maxlength="120">
</div>
<div class="field">
<label class="field__label" for="secret">Secret</label>
<input class="input input--mono" id="secret" name="secret" type="password"
autocomplete="off" placeholder="No secret set"
value="{{ unchanged if tool.secret_encrypted else '' }}">
<p class="field__hint">
{% if tool.secret_encrypted %}
Currently <code>{{ masked }}</code>. Leave the dots alone to keep it,
or clear the field to remove it.
{% else %}
Encrypted at rest and never shown again. It is dropped if the endpoint
redirects to another host.
{% endif %}
</p>
</div>
</section>
<section class="card">
<h2 class="card__title">The answer</h2>
<div class="field">
<label class="field__label" for="response_mode">Read the response as</label>
<select class="select" id="response_mode" name="response_mode">
{% for value, label in response_modes %}
<option value="{{ value }}" {{ 'selected' if value == tool.response_mode }}>{{ label }}</option>
{% endfor %}
</select>
</div>
<div class="field">
<label class="field__label" for="response_path">Path into the JSON</label>
<input class="input input--mono" id="response_path" name="response_path"
value="{{ tool.response_path }}" placeholder="data.items.0.title" maxlength="300">
<p class="field__hint">
Dotted; a number indexes a list. Leave empty for the whole document.
A path that leads nowhere gives the whole document rather than nothing.
</p>
</div>
<div class="field">
<label class="field__label" for="max_chars">Most characters to keep</label>
<input class="input" id="max_chars" name="max_chars" value="{{ tool.max_chars }}"
inputmode="numeric">
<p class="field__hint">
Spent out of the context window on every call. The rest is cut off, and
the model is told so.
</p>
</div>
</section>
<section class="card">
<h2 class="card__title">Guidance</h2>
<div class="field">
<textarea class="textarea" name="guidance" rows="4"
placeholder="- Check the weather rather than guessing at it."
>{{ tool.guidance }}</textarea>
<p class="field__hint">
Added to the system message whenever this tool is offered, and nowhere
else. Start a line with <code>- </code> and it joins the list of tool
instructions. Optional: the description above is what makes the tool
usable, this is for habits.
{% if prompt_overridden %}
<br><strong>Someone has overridden this wording under
<a href="/admin/prompts">Prompts</a></strong> — that is what the model
sees, not this.
{% endif %}
</p>
</div>
</section>
<section class="card">
<h2 class="card__title">Availability</h2>
<div class="field">
<div class="checkbox-row">
<label class="checkbox">
<input type="checkbox" name="enabled" value="true" {{ 'checked' if tool.enabled }}>
<span>Enabled — offered in chats</span>
</label>
<label class="checkbox">
<input type="checkbox" name="allow_private" value="true"
{{ 'checked' if tool.allow_private }}>
<span>May reach private and loopback addresses</span>
</label>
</div>
<p class="field__hint">
Leave the second unticked unless this tool points at something on your
own network. It is what stops a tool being aimed at this server, a
router, or a cloud metadata endpoint.
</p>
</div>
<div class="field">
<label class="checkbox">
<input type="checkbox" name="public" value="true" {{ 'checked' if tool.public }}>
<span>Available to everyone</span>
</label>
<p class="field__hint">
Uncheck to restrict this tool to chosen groups. Administrators always
have access.
</p>
</div>
<div class="field">
<span class="field__label">Groups with access</span>
{% if groups %}
<div class="checkbox-row">
{% for group in groups %}
<label class="checkbox">
<input type="checkbox" name="group_ids" value="{{ group.id }}"
{{ 'checked' if group.id in selected_groups }}>
<span>{{ group.name }}</span>
</label>
{% endfor %}
</div>
<p class="field__hint">Ignored while the tool is available to everyone.</p>
{% else %}
<p class="field__hint">
No groups yet — <a href="/admin/groups">create one</a> to restrict access.
</p>
{% endif %}
</div>
<div class="field">
<label class="field__label" for="position">Position</label>
<input class="input" id="position" name="position" value="{{ tool.position }}"
inputmode="numeric">
</div>
</section>
<div class="btn-row">
<button class="btn btn--primary" type="submit">
{{ "Create tool" if is_new else "Save changes" }}
</button>
<a class="btn btn--ghost" href="/admin/tools">Back to all tools</a>
{% if not is_new %}
<button class="btn btn--danger" type="submit" formnovalidate
formaction="/admin/tools/{{ tool.id }}/delete"
data-confirm-button="Delete the tool “{{ tool.name }}”? Chats that used it keep their transcripts.">
Delete
</button>
{% endif %}
</div>
</form>
{% if not is_new %}
<section class="card">
<h2 class="card__title">Try it</h2>
<p class="field__hint">
Calls the <em>saved</em> tool once, so what you see here is what a chat would
get. Nothing is sent to a model.
</p>
<div class="field">
<label class="field__label" for="arguments">Arguments</label>
<textarea class="textarea input--mono" id="arguments" name="arguments" rows="3"
spellcheck="false">{"city": "Minas Tirith"}</textarea>
</div>
<div class="btn-row">
<button class="btn" type="button"
hx-post="/admin/tools/{{ tool.id }}/test"
hx-include="#arguments"
hx-target="#tool-test-result">
{{ icon("refresh", "icon--sm") }} Run once
</button>
</div>
<div id="tool-test-result"></div>
</section>
{% endif %}
{% endblock %}
+110
View File
@@ -0,0 +1,110 @@
{% extends "admin/_layout.html" %}
{% from "_macros.html" import icon %}
{% set section = "tools" %}
{% block title %}Tools - LLeMbas{% endblock %}
{% block heading %}Tools{% endblock %}
{% block admin_content %}
<p class="admin-lede">
HTTP calls a model can make while it answers. Each one is offered to models
marked <strong>Custom tools</strong>, to people who have the permission, and —
if it is restricted — only to the groups you choose. The model decides
<em>when</em> to call it from the description you write, so write that as if
explaining to a colleague what the tool is for.
</p>
{% if saved %}
<div class="alert alert--success">{{ icon("check", "icon--sm") }} <span>{{ saved }}</span></div>
{% endif %}
{% if not total %}
<div class="empty" style="padding: var(--sp-10) 0">
{{ icon("link", "empty__mark") }}
<p class="empty__text">
No tools yet. <a href="/admin/tools/new">Define one</a> — a name, a
description, and a URL with <code>{{ '{{placeholders}}' }}</code> in it.
</p>
</div>
{% else %}
{# Filters are links, so a filtered view is a real URL you can keep or share. #}
<div class="filter-bar">
<div class="filter-tabs">
{% for key, label in filters.items() %}
<a class="filter-tab {{ 'is-active' if key == active_filter }}"
href="/admin/tools?filter={{ key }}{% if q %}&q={{ q|urlencode }}{% endif %}">
{{ label }} <span class="filter-tab__count">{{ counts[key] }}</span>
</a>
{% endfor %}
</div>
<form class="filter-form" method="get" action="/admin/tools">
<input type="hidden" name="filter" value="{{ active_filter }}">
<input class="input" type="search" name="q" value="{{ q }}"
placeholder="Search tools…" aria-label="Search tools">
<button class="btn" type="submit">{{ icon("search", "icon--sm") }} Filter</button>
{% if q or active_filter != "all" %}
<a class="btn btn--ghost" href="/admin/tools">Clear</a>
{% endif %}
<a class="btn btn--primary" href="/admin/tools/new">{{ icon("plus", "icon--sm") }} New tool</a>
</form>
</div>
{% if not tools %}
<div class="empty" style="padding: var(--sp-8) 0">
<p class="empty__text">Nothing matches that filter.</p>
</div>
{% else %}
<div class="model-rows">
{% for tool in tools %}
<div class="model-row {{ 'is-off' if not tool.enabled }}">
<span class="model-row__pos">{{ page_start + loop.index }}</span>
<div class="model-row__main">
<div class="model-row__title">
<a class="model-row__name" href="/admin/tools/{{ tool.id }}/edit">{{ tool.name }}</a>
{% if not tool.enabled %}<span class="badge badge--danger">disabled</span>{% endif %}
{% if not tool.public %}<span class="badge">restricted</span>{% endif %}
{% if tool.allow_private %}<span class="badge">private network</span>{% endif %}
{% if tool.last_error %}<span class="badge badge--danger">last call failed</span>{% endif %}
</div>
<code class="model-row__id">
{{ tool.slug }} · {{ tool.method }} {{ tool.url_template }}
</code>
</div>
<div class="model-row__actions">
<a class="btn btn--sm" href="/admin/tools/{{ tool.id }}/edit">Edit</a>
</div>
</div>
{% endfor %}
</div>
<div class="list-footer">
<span class="text-xs faint">
Showing {{ page_start + 1 }}{{ page_start + tools|length }} of {{ matched }}
{%- if matched != total %} (filtered from {{ total }}){% endif %}
</span>
{% if pages > 1 %}
{% set base = "/admin/tools?filter=" ~ active_filter ~ ("&q=" ~ q|urlencode if q else "") %}
<div class="btn-row">
{% if page > 1 %}
<a class="btn btn--sm" href="{{ base }}&page={{ page - 1 }}">Previous</a>
{% else %}
<span class="btn btn--sm" aria-disabled="true" style="opacity: .45">Previous</span>
{% endif %}
<span class="text-xs faint">Page {{ page }} of {{ pages }}</span>
{% if page < pages %}
<a class="btn btn--sm" href="{{ base }}&page={{ page + 1 }}">Next</a>
{% else %}
<span class="btn btn--sm" aria-disabled="true" style="opacity: .45">Next</span>
{% endif %}
</div>
{% endif %}
</div>
{% endif %}
{% endif %}
{% endblock %}
@@ -6,23 +6,55 @@
answer) and from the stored message afterwards, so the sources behind an
answer stay in the transcript rather than vanishing when the stream ends.
EVERYTHING in here comes from a search provider and is untrusted, exactly as
much as model output is. Jinja autoescaping covers the text; the URL is
checked separately, because `is_linkable` is the only thing standing between
a result carrying a javascript: URL and an anchor pointing at it.
EVERYTHING in here is third-party text and is untrusted, exactly as much as
model output is -- a search provider's results, an administrator's HTTP tool
relaying whatever it was pointed at, an MCP server's reply. Jinja autoescaping
covers it. Two things are handled separately: the URL, because `is_linkable`
is the only thing standing between a result carrying a javascript: URL and an
anchor pointing at it, and `event.text`, which is rendered as preformatted
text and deliberately NOT through services/markdown.py -- markdown is the one
path allowed to emit HTML, and this is the last content that should be given
it.
`kind` says how to label the event. Rows written before it existed have none,
so web_search reads as a search and everything else falls to the generic
branch: an old notes_search event used to claim it had searched the web.
#}
{% for event in tool_events %}
{% set kind = event.kind or ('search' if event.name == 'web_search' else 'tool') %}
<details class="tool-activity {{ 'tool-activity--error' if event.status == 'error' }}">
<summary class="tool-activity__summary">
{{ icon("globe", "icon--sm tool-activity__icon") }}
{% if kind == 'search' %}
{{ icon("globe", "icon--sm tool-activity__icon") }}
{% elif kind == 'custom' %}
{{ icon("link", "icon--sm tool-activity__icon") }}
{% elif kind == 'mcp' %}
{{ icon("server", "icon--sm tool-activity__icon") }}
{% else %}
{{ icon("sparkle", "icon--sm tool-activity__icon") }}
{% endif %}
<span class="tool-activity__label">
{% if event.status == "error" %}
Web search failed
{% elif event.query %}
Searched the web for “{{ event.query }}”
{% if kind == 'search' %}
{% if event.status == "error" %}
Web search failed
{% elif event.query %}
Searched the web for “{{ event.query }}”
{% else %}
Searched the web
{% endif %}
{% else %}
Searched the web
{% set label = event.label or event.name %}
{% if event.status == "error" %}
{{ label }} failed
{% else %}
{{ label }}
{% endif %}
{% if event.query %}
<span class="tool-activity__count">· {{ event.query }}</span>
{% endif %}
{% endif %}
{% if event.results %}
<span class="tool-activity__count">
· {{ event.results | length }} result{{ '' if event.results | length == 1 else 's' }}
@@ -33,21 +65,29 @@
</summary>
<div class="tool-activity__body">
{% if event.detail and kind != 'search' %}
<p class="tool-result__host">{{ event.detail }}</p>
{% endif %}
{% if event.error %}
<p class="tool-activity__error">{{ event.error }}</p>
{% elif not event.results %}
{% elif not event.results and not event.text %}
<p class="tool-activity__error">Nothing was found.</p>
{% endif %}
{% if event.text %}
<pre class="tool-result__text">{{ event.text }}</pre>
{% endif %}
{% for result in event.results %}
<div class="tool-result">
{% set scheme = result.url.split(":")[0] | lower %}
{% set scheme = (result.url or "").split(":")[0] | lower %}
{% if scheme in ("http", "https") %}
<a class="tool-result__title" href="{{ result.url }}"
target="_blank" rel="noopener noreferrer nofollow">{{ result.title }}</a>
{% else %}
{# Not a link. A search result is third-party text and its URL is not
trusted to be safe to click. #}
{# Not a link. A result is third-party text and its URL is not trusted to
be safe to click. #}
<span class="tool-result__title">{{ result.title }}</span>
{% endif %}
<span class="tool-result__host">{{ result.host }}</span>