Add nullable columns without a default

Deploying knowledge bases showed the migration runner doing the wrong thing:

    ALTER TABLE "documents" ADD COLUMN "base_id" VARCHAR(32) DEFAULT ''

`base_id` is nullable and its absent value is NULL, but the runner derived a
default from the column type and backfilled every existing row with the empty
string. Nothing then matched `base_id IS NULL`, so the startup sweep that files
pre-bases documents into a default base would have skipped all of them and the
documents would have stayed invisible.

Nobody lost anything -- the live instance had no documents yet -- but the fault
is general: any nullable column added from here would arrive as "" rather than
NULL, and every "is this set?" check would be wrong about the rows that predate
it. So a default is now emitted only for NOT NULL columns, where SQLite requires
one.

The sweep also accepts "" as meaning unfiled, since a deployment that upgraded
through the previous release has rows holding it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Jaroslav Beneš
2026-07-21 20:03:01 +02:00
parent 35b9d8c8d2
commit 2a79d962a3
4 changed files with 64 additions and 4 deletions
+20 -2
View File
@@ -272,12 +272,30 @@ def test_an_uploaded_document_always_lands_in_a_base(db, people):
assert document.base_id is not None
def test_documents_predating_bases_are_filed_at_startup(db, people):
@pytest.mark.parametrize("absent", [None, ""])
def test_documents_predating_bases_are_filed_at_startup(db, people, absent):
"""Both spellings of "no base": NULL, and the empty string a release that
added the column with a type-derived default left behind.
The empty string has to be written with foreign keys off, because that is
the only way it could ever have got there -- ALTER TABLE ADD COLUMN does not
check existing rows, but an UPDATE would.
"""
from sqlalchemy import text
document = documents_service.store_upload(
db, owner=people["frodo"], payload=b"x", filename="a.txt"
)
document.base_id = None
db.commit()
db.execute(text("PRAGMA foreign_keys=OFF"))
db.execute(
text("UPDATE documents SET base_id = :value WHERE id = :id"),
{"value": absent, "id": document.id},
)
db.commit()
db.execute(text("PRAGMA foreign_keys=ON"))
db.expire(document)
assert document not in db.scalars(documents_service.visible(db, people["frodo"]))
assert documents_service.sweep_unfiled(db) == 1