Merge v6.7.x into main #36
Loading…
Reference in a new issue
No description provided.
Delete branch "v6.7.x"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Brings main from v6.5.1 up to v6.7.1, covering the whole 6.6 and 6.7 lines:
recall(domain_filter=...); fixes_migrate_indexes(), which had silently failed on every upgradelicence(redistribution rights) on every memory, set at ingestprovenance(retrieved / concluded / asserted) on every memoryrecall()andfind_skills(),weak_matchflag, phantom index guard and drift reporting (#28–#34)Tagged and released as v6.7.1 on the v6.7.x branch. The branch is 15 commits ahead of main with nothing on main it doesn't already have. Full suite passes locally: 1914 passed, 1 skipped.
🤖 Generated with Claude Code
https://claude.ai/code/session_01HWqtyPdN5MHxqoVM3Sj9xD
A project context now declares the kinds of work inside it — python, docker, wcag-accessibility — and recall(query, domain_filter="python") searches every project declaring that domain at once instead of one project at a time. This is the gap compiled skills left open. A skill only exists once its lessons clear the reinforcement gate, so everything below that threshold — the gotcha you have hit twice and never written up — was reachable only by searching the whole store unfiltered. The domain filter reaches those raw memories. One vocabulary, not two: project domains normalise through the same resolve_domain/validate_domain as compiled skills, so `py` on a project and `py` on a skill both become `python` and the two cannot drift into parallel taxonomies. The auto skill scan now ranks a domain higher when projects declare it, boosting candidates only — a domain with no lesson pool would be dropped at the min-pool gate anyway, so it never spends one of the capped candidate slots. Design decisions worth keeping: - Domains route, they never label memories. A project is Python and CSS and Docker at once, so inheriting its domains onto every memory would surface a CSS gotcha in a Python search. The domain narrows candidate projects; the vector search still decides relevance within them. - An unmatched domain is always reported. Filtering on a domain nobody declares runs unscoped with a leading domain_filter_notice, because a global search presented as a scoped one is worse than no filter. An empty domain-and-project intersection returns nothing rather than falling through to a global search — the pipeline reads an empty project list as "search everything", so the tool layer short-circuits. - domains is stored comma-separated, not JSON. A TAG field tokenises on commas, so the JSON-array form used by tags/topics indexes as garbage and @tags:{python} matches nothing. Verified live against valkey-search, including the JSON control case. - project_filter now takes a list and composes with clause-level OR, never in-brace alternation, which returns an empty set on valkey-search. One unsafe value drops push-down for the whole clause rather than filtering on part of the set. A startup migration seeds domains from each project's existing stack string so the filter is not empty on day one after an upgrade. It only writes where the field has never existed, never overwrites, and records an empty marker when a stack yields nothing usable. compile_project_domains() proposes domains from the stack field and the tags that recur across a project's own memories, with the evidence for each, writing only on auto_save=True. The web UI gains domain pills, a filter chip bar, ?domain= filtering, a datalist of known domains, and a Suggest from stack & memories button that fills the field without saving. 1544 tests pass, 100% line coverage on every module touched. Verified end to end against a real valkey-search instance, including a simulated 6.5.1 to 6.6.0 upgrade. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>recall() and find_skills() both filled their result list regardless of whether anything left was relevant, so top_k behaved as a quota rather than a ceiling and an agent got handed noise it then had to reason about. recall gains RECALL_MIN_SCORE (default 0.4), gated on the RAW similarity rather than the adjusted score. Raw similarity is the only number in the pipeline that measures subject match; the multipliers on top of it encode policy (old, abandoned, derived fact), and gating on those would have silently hidden the two classes deliberately scored down — enriched facts at surface 0.5 and abandoned experiences at 0.1 — which are exactly what should still come back when nothing better matches. Abandoned warnings and reinstate candidates are exempt: neither earned its place on vector similarity, so neither should be filtered on one. find_skills gains SKILL_MIN_SCORE (default 0.25) and a per-entry confidence. Skills need their own floor because their discovery vector is long human-written prose and scores much lower than a memory's content. Measured against three compiled skills on a live store: lowest true positive 0.3042 ("modal dialog, keyboard + screen reader" -> wcag), highest false positive 0.2341 ("python" -> preferences). 0.25 sits in that gap with ~0.05 either side. Exact domain hits are identity matches and are never gated. Phantom guard (#28): an index entry whose backing hash is gone comes back from FT.SEARCH as a doc id with no fields. remember() rejects empty content, so empty content in a memory namespace means the record is gone — recall now skips and logs it instead of returning a result with no text. That answers the open question on #28: a phantom could reach a caller, so this was a correctness bug, not only housekeeping. #29: reindex documented three namespaces and handled four. It now handles five (skill included — the SKILL.md body lives in the hash, so the index rebuild is as safe there as anywhere) and says so. The same three-namespace assumption turned up twice more: recall's tool description omitted preference, and memory_audit built its unscoped prefix list by hand and so silently skipped mem:preference: entirely despite accepting it as a filter. Both now derive from the accepted set. FakeEmbedder seeded its per-word RNG on hash(), which Python randomises per process, so the same word embedded differently on every run. Harmless while nothing compared against a threshold; instant flakiness once the floors did. Seeded on crc32 instead. Tests about ranking mechanics rather than relevance take a no_relevance_floor fixture, since the fake's word-overlap similarity is far harsher than all-MiniLM-L6-v2 on the same pairs. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>