Finding a thing that does not use your words

Three pieces, and the first one is that they are all optional.

Extraction stops being constants. Upload size, image edge, JPEG quality, PDF
pages, extracted characters, orphan age and the text-extension list are settings
now, read through a process-level snapshot rather than a session -- `prepare` and
everything under it are called from routes, tool runners and the startup sweep,
and several of those have no session in hand. Two things deliberately stayed
constants: the decompression-bomb guard, which is a guard and not a preference,
and ORPHAN_AGE, which would have been evaluated at import if it stayed in the
signature and pinned the shipped 24 hours whatever anybody set.

An embedding model is picked from the models an administrator flagged for it, and
one that has since lost its flag is *named* rather than dropped from the picker:
a setting that vanishes is one nobody can tell from a setting never made. Nothing
here is required. Choosing none means no chunk rows, no requests, and
retrieval.search returning exactly what fts.search_ids returns in exactly that
order -- asserted, because it is what makes this safe to land on an instance that
never asked for it.

The two rankings are fused by reciprocal rank fusion: ranks and not scores,
because bm25 is a corpus-dependent negative and cosine is 0..1, and normalising
them onto one scale means picking a constant nobody can tune without a labelled
set they do not have. RRF's one constant is famously insensitive and degrades to
whichever list is non-empty -- which is what turns "no embedding model" into a
branch that does not exist.

A record scores as its best chunk rather than its average, or a long document
about something else outranks a short one that says the thing. Width and model
are stored beside every vector and a mismatch is skipped, because vectors from
two spaces score against each other perfectly happily and mean nothing -- a
search that works and is wrong is the worst failure this can have, and a model
change now leaves stale rows ignored rather than trusted.

Indexing is fired and forgotten, and how a change is noticed is a session event
rather than a call in each of the ten library writers. That is a departure from
this codebase's taste for explicit seams, for the reason tool_label is a Jinja
global: a step every writer has to remember is one that gets forgotten, and here
forgetting is silent -- the record saves, keyword search still finds it, and only
its recall goes stale. Chunks are embedded before anything is deleted, so a
failure leaves the old index rather than half a new one.

Also: `embeddings` joins the model capabilities, and the three tool flags that
had shipped with no checkbox -- canvas, scheduling and helpers -- have one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

CLAUDE.md

@@ -20,7 +20,7 @@ lembas info                         # paths + counts, useful when confused
 lembas secret-key                   # generate LEMBAS_SECRET_KEY
 lembas create-admin                 # create or promote an admin
 
-pytest                              # 1979 tests, ~2min
+pytest                              # 2024 tests, ~2min
                                     # PLAN.md tracks what is and is not built
 ruff check .                        # lint (line length 100)
 python scripts/build_artwork.py     # regenerate artwork (SVG + PWA icons;
@@ -88,6 +88,7 @@ src/lembas/
     admin_images.py  the ComfyUI, and the workflow templates on it
     admin_prompts.py the prompt fragment editor and its preview
     admin_branding.py  the name, the logo, the wording and the themes
+    admin_extraction.py  what a file may cost, and what finds it afterwards
     branding.py      /branding.css and the assets behind it, both unauthenticated
     admin_suggestions.py  the cards offered on the new-chat screen
     admin_tools.py   custom HTTP tools and MCP servers
@@ -159,6 +160,10 @@ src/lembas/
                      and everything it may not do
     branding.py      whose instance this is: the name, the artwork, the
                      wording and the themes, cached once per process
+    llm/embeddings.py  /v1/embeddings, batched, normalised on the way out
+    library/chunks.py    splitting a record, and packing a vector
+    library/indexing.py  keeping the semantic index current, and rebuilding it
+    library/retrieval.py keywords and meaning, fused
     settings_store.py  runtime instance settings
     canvas.py        what is open in the canvas panel, and where it comes from
     scratch.py       a chat's own working document
@@ -194,6 +199,10 @@ touching the code it names -- these are the same notes, not a summary.
   schedule something wrote a note and said it had).
 - `docs/notes/image-generation.md` -- the ComfyUI workflow with holes in it, what
   substitution walks, the review-and-retry loop, and how a failure reports itself.
+- `docs/notes/search-and-extraction.md` -- extraction limits as a snapshot, why
+  reciprocal rank fusion and not a weight, how a record scores as its best chunk,
+  why vectors from two models never meet, and the session event that notices a
+  library record changing.
 - `docs/notes/branding.md` -- the branding snapshot and why it is a Jinja global,
   where the instance name went and how an upgrade keeps it, how a custom theme
   inherits through `data-base`, and why `/branding.css` is a route.
@@ -1449,6 +1458,14 @@ with nothing streaming is exactly what a hang looks like.
 chunks. `tools.ToolCallAccumulator` rejoins them keyed on `index` — not on
 name, which breaks the moment a model calls one tool twice in a turn.
 
+**Library search is keyword *and* meaning, and neither is a mode.** `fts.search_ids`
+was already the one seam; `library/retrieval.search` sits in front of it and fuses
+its ranking with a vector one by reciprocal rank fusion -- ranks, not scores,
+because bm25 and cosine are not comparable and normalising them means picking a
+constant nobody can tune. With no embedding model configured it returns exactly
+what FTS returned, in that order, and no chunk row is ever written. See
+`docs/notes/search-and-extraction.md`.
+
 **Four stores, four different reasons.** `services/library/` — `documents`
 (uploaded by a person, searched by the model), `notes` (written by the model,
 searched), `memories` (short, and *injected whole* every turn), `skills` (index