MCP servers, over streamable HTTP

A server is a row with a URL; its tools are discovered by a button and
cached, then offered beside the built-in ones. Written by hand rather than
taken from the reference SDK, because that SDK's transport does its own
connecting -- and the one thing that must not be bypassed is check_url on
every hop. Owning the transport is the point; the framing beside it is the
small part.

Sessions are per call: initialize, initialized, the call, a best-effort
DELETE. Caching one wants an owner, a TTL, eviction, a lock and a shutdown
hook, and the server may expire it under all of that anyway -- ToolContext
is a session-free snapshot precisely so nothing in a tool holds live state.

A server's names and descriptions reach the model as instructions and are
bounded before they do; what it returns is escaped preformatted text, never
markdown. Tools are namespaced per server, so two servers exposing "search"
do not collide and neither shadows a built-in.

Also: a round's calls now run together under a semaphore, results indexed
so each tool turn stays paired with its call, and generation.status names
what is running -- a remote tool is latency-bound, and a silent pause is
what a hang looks like.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

src/lembas/services/tools.py

@@ -715,12 +715,19 @@ def _family_allowed(
 def _row_defs(db: DBSession, user: User | None, *, everything: bool = False) -> list[ToolDef]:
     """Tool definitions built from rows, in the order they claim names.
 
-    Imported here rather than at the top because `custom_tools` needs `ToolDef`
-    from this module.
+    Custom tools first, then MCP servers, because a custom tool's name is
+    written by hand and refused if it collides while an MCP tool's is derived
+    and renamed silently -- the one that can adapt should be the one that has to.
+
+    Imported here rather than at the top because both modules need `ToolDef`
+    from this one.
     """
     from lembas.services import custom_tools
+    from lembas.services.mcp import registry as mcp_registry
 
-    return custom_tools.tool_defs(db, user, everything=everything)
+    custom = custom_tools.tool_defs(db, user, everything=everything)
+    taken = {*REGISTRY, *(tool.name for tool in custom)}
+    return [*custom, *mcp_registry.tool_defs(db, user, everything=everything, taken=taken)]
 
 
 def _book(defs: list[ToolDef]) -> dict[str, ToolDef]:
@@ -944,9 +951,11 @@ def _row_source(db: DBSession):
     it -- the override outlives the row.
 
     Gated on the tool's own family, so the guidance appears exactly when the
-    tool it describes is offered and never otherwise.
+    tool it describes is offered and never otherwise. An MCP server gets one
+    fragment rather than one per advertised tool: forty entries on the prompts
+    page is a page nobody would read.
     """
-    from lembas.db.models import CustomTool
+    from lembas.db.models import CustomTool, McpServer
 
     for row in db.scalars(select(CustomTool).order_by(CustomTool.position, CustomTool.slug)):
         yield prompts_service.Fragment(
@@ -959,6 +968,17 @@ def _row_source(db: DBSession):
             default=row.guidance or "",
         )
 
+    for server in db.scalars(select(McpServer).order_by(McpServer.position, McpServer.slug)):
+        yield prompts_service.Fragment(
+            key=f"tool.mcp_{server.slug}",
+            label=server.name or server.slug,
+            group=prompts_service.GROUP_TOOLS,
+            order=600 + server.position,
+            families=(f"{FAMILY_MCP}:{server.slug}",),
+            hint=f"Appears when any tool from {server.name or server.slug} is offered.",
+            default=server.guidance or "",
+        )
+
 
 __all__ = [
     "FAMILIES",