Houmeres/LLeMbas
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 3 Wiki Activity

Knowledge, notes, memory and skills, and a harness to make them used

Browse Source 
Four places a model can reach for, differing in who writes a record and how it
gets in front of the model.

**Knowledge** is uploaded by a person and searched by the model. It goes through
`services/files.py:prepare` — the same pipeline as a chat attachment — so the
same PDF produces the same text whichever way it arrived, and `Document` carries
the same content columns as `Attachment` for the same reason.

**Notes** are written by the model and edited by you. Too long to inject, so
they are searched.

**Memory** is short facts, and every one of them goes into every request. That
single decision is where the rest of its design comes from: records are capped
short, the block has a budget, there is no search tool because the model is
already looking at them, and they are not shareable — a record about a person is
not content to hand round.

**Skills** are saved procedures. Only the name and description are injected; the
body is fetched when the model decides one applies, which is what makes a
hundred skills affordable. A model may write and revise its own — the safety
story is not a gate but a record: every revision is kept, attributed and
revertible. A model that has just read a hostile page can save a skill that
outlives the conversation, and the honest mitigation is that it is visible and
undoable rather than that it was prevented.

**The harness** is why any of it gets used. A model handed a tools array
ignores it and answers from recall, because nothing in the request suggests
otherwise. `services/harness.py` assembles a preamble from what this chat
actually has: when to reach for each tool, the memories, the skill index.

This is an exception to "system prompts are precedence, not concatenation", and
a deliberate one. That rule governs the three *authored* layers and is
untouched — exactly one still wins. The harness is a different axis: it
describes the machinery rather than the behaviour, nobody authored it, and there
is nothing for it to disagree with. It is prepended to whichever authored prompt
won, in one system message, since several endpoints reject a second.

Supporting changes:

- **Sharing**, in one helper. `visible_to()` is the only definition of who can
  see a library item and every listing and tool goes through it. Sharing grants
  *reading*; two people editing one note with no history and no merge is worse
  than copying it. **Administrators do not bypass this** — they bypass
  permissions elsewhere because an admin can grant themselves those anyway, but
  reading somebody's private notes is a different act.
- **FTS5**, created by `db/migrations.py:ensure_fts` with the triggers an
  external-content index needs. Idempotent, like the column sync beside it.
  Terms are ANDed and then ORed: the caller is usually a model writing a whole
  question, and requiring every word loses the match on one absent term.
- **The attach button is a menu** — file, image, a web page, or a document from
  the library. Attaching a document copies it, because history must not change
  when a document is edited later.
- **A URL fetcher with an SSRF guard.** This server can reach the router, the
  other services on the box and LLeMbas itself, and the address can come from a
  model. Private ranges are refused *after resolution* and redirects are followed
  by hand so every hop is checked. An admin can open it deliberately.
- **Model capabilities split** into protocol support and a toggle per built-in
  tool. Rows predating the split have no `tool_*` keys, and absent counts as on
  when `tools` is on — otherwise an upgrade silently takes web search away from
  every model already configured for it.

Also fixes the test fixture, which built the schema with `create_all` and so ran
against a database without the FTS tables production has; it now runs
`sync_schema`, the same path startup takes.

430 tests, ruff clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Jaroslav Beneš
2026-07-21 19:43:57 +02:00
parent a8b7b5fc14
commit 21001f2eb8
51 changed files with 5135 additions and 157 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/test_tools.py
+55 -18
View File
@@ -104,12 +104,19 @@ def _user(db, user_id):
return db.get(User, user_id)
def test_nothing_is_offered_when_search_is_off(db, user_id):
def _names(offered):
return {tool["function"]["name"] for tool in offered}
def test_web_search_is_absent_when_search_is_off(db, user_id):
"""The library tools do not depend on a search provider, so they stay."""
chat = _chat_with(db, user_id, capabilities={"tools": True})
assert tools_service.enabled_tools(db, chat, _user(db, user_id)) == []
offered = _names(tools_service.enabled_tools(db, chat, _user(db, user_id)))
assert "web_search" not in offered
assert "notes_search" in offered
def test_nothing_is_offered_to_a_model_without_the_tools_capability(db, user_id):
def test_nothing_at_all_without_the_tools_capability(db, user_id):
"""Sending a tools array to an endpoint that does not implement tool calling
fails the entire request, exactly as image parts do without vision."""
settings_store.update(db, {"enabled": True}, key=settings_store.SEARCH)
@@ -120,13 +127,29 @@ def test_nothing_is_offered_to_a_model_without_the_tools_capability(db, user_id)
def test_web_search_is_offered_when_everything_lines_up(db, user_id):
settings_store.update(db, {"enabled": True}, key=settings_store.SEARCH)
chat = _chat_with(db, user_id, capabilities={"tools": True})
offered = tools_service.enabled_tools(db, chat, _user(db, user_id))
assert len(offered) == 1
assert offered[0]["function"]["name"] == "web_search"
assert "web_search" in _names(tools_service.enabled_tools(db, chat, _user(db, user_id)))
def test_nothing_is_offered_when_the_provider_cannot_run(db, user_id, monkeypatch):
def test_a_model_predating_the_split_keeps_its_tools(db, user_id):
"""Rows configured before the per-tool flags existed have no tool_* keys.
Reading absent as off would silently take web search away from every model
already set up for it."""
settings_store.update(db, {"enabled": True}, key=settings_store.SEARCH)
chat = _chat_with(db, user_id, capabilities={"tools": True})
assert "web_search" in _names(tools_service.enabled_tools(db, chat, _user(db, user_id)))
def test_a_family_turned_off_for_the_model_is_withheld(db, user_id):
settings_store.update(db, {"enabled": True}, key=settings_store.SEARCH)
chat = _chat_with(
db, user_id, capabilities={"tools": True, "tool_notes": False}
)
offered = _names(tools_service.enabled_tools(db, chat, _user(db, user_id)))
assert "notes_search" not in offered
assert "web_search" in offered, "turning one family off must not affect another"
def test_web_search_is_withheld_when_the_provider_cannot_run(db, user_id, monkeypatch):
"""Offering a tool that will fail on every call is worse than not offering
it at all."""
monkeypatch.setattr(
@@ -134,7 +157,13 @@ def test_nothing_is_offered_when_the_provider_cannot_run(db, user_id, monkeypatc
)
settings_store.update(db, {"enabled": True}, key=settings_store.SEARCH)
chat = _chat_with(db, user_id, capabilities={"tools": True})
assert tools_service.enabled_tools(db, chat, _user(db, user_id)) == []
assert "web_search" not in _names(
tools_service.enabled_tools(db, chat, _user(db, user_id))
)
def _context(**kwargs):
return tools_service.ToolContext(owner_id="someone", **kwargs)
# --- Running one -------------------------------------------------------------
@@ -144,7 +173,7 @@ async def test_running_web_search_formats_results_for_the_model(monkeypatch):
monkeypatch.setattr("lembas.services.search.run", fake_run)
outcome = await tools_service.run_tool({}, "web_search", '{"query": "mallorn"}')
outcome = await tools_service.run_tool(_context(), "web_search", '{"query": "mallorn"}')
assert "A title" in outcome.content
assert "https://a.test" in outcome.content
assert outcome.event["status"] == "ok"
@@ -161,7 +190,7 @@ async def test_malformed_argument_json_is_treated_as_the_query(monkeypatch):
return []
monkeypatch.setattr("lembas.services.search.run", fake_run)
await tools_service.run_tool({}, "web_search", "mallorn tree")
await tools_service.run_tool(_context(), "web_search", "mallorn tree")
assert seen["query"] == "mallorn tree"
@@ -174,19 +203,19 @@ async def test_a_failed_search_hands_the_model_an_explanation(monkeypatch):
monkeypatch.setattr("lembas.services.search.run", fake_run)
outcome = await tools_service.run_tool({}, "web_search", '{"query": "x"}')
outcome = await tools_service.run_tool(_context(), "web_search", '{"query": "x"}')
assert "rate limiting" in outcome.content
assert outcome.event["status"] == "error"
async def test_an_unknown_tool_is_reported_rather_than_raised():
outcome = await tools_service.run_tool({}, "launch_missiles", "{}")
outcome = await tools_service.run_tool(_context(), "launch_missiles", "{}")
assert "no tool called" in outcome.content
assert outcome.event["status"] == "error"
async def test_a_call_with_no_query_is_reported():
outcome = await tools_service.run_tool({}, "web_search", '{"query": " "}')
outcome = await tools_service.run_tool(_context(), "web_search", '{"query": " "}')
assert outcome.event["status"] == "error"
@@ -220,8 +249,16 @@ def test_the_tool_turn_carries_the_call_id():
}
def test_the_schema_is_valid_json():
"""It is sent verbatim to the endpoint; a schema that will not serialise
def test_every_schema_is_valid_json():
"""They are sent verbatim to the endpoint; a schema that will not serialise
fails every request rather than one."""
json.dumps(tools_service.WEB_SEARCH_SCHEMA)
assert tools_service.WEB_SEARCH_SCHEMA["function"]["parameters"]["required"] == ["query"]
for name, tool in tools_service.REGISTRY.items():
json.dumps(tool.schema)
assert tool.schema["function"]["name"] == name
assert tool.family in tools_service.FAMILIES
def test_every_tool_describes_when_to_use_it():
"""The description is all the model has to decide with."""
for tool in tools_service.REGISTRY.values():
assert len(tool.description) > 40, tool.name