ric/omnimem

Watch

Fork 0

record_experience() captures narrative, the skill compiler needs a claim — separate the lesson from what happened #35

New issue

Closed

opened 2026-09-08 15:58:48 +00:00 by ric · 1 comment

ric commented

2026-09-08 15:58:48 +00:00

Owner

Follow-up to #33, which is closed on the diagnostic half. This is the schema half, left open deliberately because it is a design decision with a migration behind it rather than a bug.

The observation, restated

compile_skill(domain='python') found 29 candidates and produced zero rules. The gate was right; the candidates were the problem:

"Community PR merged cleanly despite significant changes to maintenance.py on bot…"
"Leveraged existing find_all_duplicates() and _has_negation_pair() functions from…"

These can never become rules however often something like them recurs, because they aren't the kind of sentence a rule is. The counter-example from #33 is the one that proves it — the valkey-py dropindex() memory whose breakthrough generalises: "when a fake stands in for a component with known divergent behaviour, the fake must reproduce the divergence." That is rule-shaped, and it got that way because whoever wrote it happened to be in a generalising mood.

Why the field produces narrative

record_experience() takes breakthrough: str | None documented as "What finally worked." That prompt asks for the past tense of a specific incident, and gets it. extract_lessons() (memory/skills.py:495) then reads exactly that field and turns it into a do rule.

So the compiler is asking one field to do two jobs: be the record of what happened on this occasion, and be a claim that generalises beyond it. Those are different sentences, and when they conflict the narrative wins because that's what the docstring asked for.

Reinforcement can't rescue this. Clustering two narratives about different incidents finds no recurrence, and if it did, the rule it emitted would be a generalisation nobody made deliberately.

Proposed shape

Add a lesson argument alongside breakthrough, stored as its own field:

breakthrough — what happened, this time. Keeps its current meaning; still what explain_memory and the experience views show.
lesson — the generalisable claim, if there is one. Prompted for separately, and explicitly optional: plenty of real work teaches nothing that transfers, and a required field would just collect narrative wearing a different label.

extract_lessons() prefers lesson and falls back to breakthrough when it's absent, which makes this additive with no migration: every existing memory keeps contributing exactly what it does today, and new ones can do better. gotchas already tends to be written as a claim, so it likely needs nothing.

What to decide

Is lesson optional or prompted-with-a-nudge? Optional is honest and risks the field going unused. The agent instructions are the lever, not validation.
Should the compiler prefer memories that have one? A lesson is a deliberate generalisation, so arguably it should count for more than a breakthrough fallback — but that's a scoring change and #30 is a fresh reminder to land a field and change what reads it in separate releases.
Backfill, or let it accrue? Nothing forces a decision now, but the longer it waits the more experience data sits in the single-field shape — which was your own argument in #33 for deciding early.
Does bless() change? It currently promotes a breakthrough regardless of outcome. If lesson exists, blessing arguably means "this claim generalises", which is closer to what blessing already meant.

Not urgent, but not free either

Nothing is broken. Broad domains will keep compiling nothing and the diagnostics from 6.7.1 now explain why at the point of use. The cost of waiting is only that experience keeps accumulating in a shape the compiler can't use, which is the thing #33 flagged as worth getting ahead of.

Follow-up to #33, which is closed on the diagnostic half. This is the schema half, left open deliberately because it is a design decision with a migration behind it rather than a bug. ## The observation, restated `compile_skill(domain='python')` found 29 candidates and produced zero rules. The gate was right; the candidates were the problem: - "Community PR merged cleanly despite significant changes to maintenance.py on bot…" - "Leveraged existing `find_all_duplicates()` and `_has_negation_pair()` functions from…" These can never become rules however often something like them recurs, because they aren't the kind of sentence a rule is. The counter-example from #33 is the one that proves it — the valkey-py `dropindex()` memory whose breakthrough generalises: *"when a fake stands in for a component with known divergent behaviour, the fake must reproduce the divergence."* That is rule-shaped, and it got that way because whoever wrote it happened to be in a generalising mood. ## Why the field produces narrative `record_experience()` takes `breakthrough: str | None` documented as **"What finally worked."** That prompt asks for the past tense of a specific incident, and gets it. `extract_lessons()` (`memory/skills.py:495`) then reads exactly that field and turns it into a `do` rule. So the compiler is asking one field to do two jobs: be the record of what happened on this occasion, and be a claim that generalises beyond it. Those are different sentences, and when they conflict the narrative wins because that's what the docstring asked for. Reinforcement can't rescue this. Clustering two narratives about different incidents finds no recurrence, and if it did, the rule it emitted would be a generalisation nobody made deliberately. ## Proposed shape Add a `lesson` argument alongside `breakthrough`, stored as its own field: - **`breakthrough`** — what happened, this time. Keeps its current meaning; still what `explain_memory` and the experience views show. - **`lesson`** — the generalisable claim, if there is one. Prompted for separately, and explicitly optional: plenty of real work teaches nothing that transfers, and a required field would just collect narrative wearing a different label. `extract_lessons()` prefers `lesson` and falls back to `breakthrough` when it's absent, which makes this **additive with no migration**: every existing memory keeps contributing exactly what it does today, and new ones can do better. `gotchas` already tends to be written as a claim, so it likely needs nothing. ## What to decide 1. **Is `lesson` optional or prompted-with-a-nudge?** Optional is honest and risks the field going unused. The agent instructions are the lever, not validation. 2. **Should the compiler prefer memories that have one?** A `lesson` is a deliberate generalisation, so arguably it should count for more than a `breakthrough` fallback — but that's a scoring change and #30 is a fresh reminder to land a field and change what reads it in separate releases. 3. **Backfill, or let it accrue?** Nothing forces a decision now, but the longer it waits the more experience data sits in the single-field shape — which was your own argument in #33 for deciding early. 4. **Does `bless()` change?** It currently promotes a `breakthrough` regardless of outcome. If `lesson` exists, blessing arguably means "this claim generalises", which is closer to what blessing already meant. ## Not urgent, but not free either Nothing is broken. Broad domains will keep compiling nothing and the diagnostics from 6.7.1 now explain why at the point of use. The cost of waiting is only that experience keeps accumulating in a shape the compiler can't use, which is the thing #33 flagged as worth getting ahead of.

ric referenced this issue

2026-09-08 15:58:54 +00:00

Broad domains compile from project narrative, not generalisable rules — 'python' can never clear reinforcement #33

ric referenced this issue from a commit

2026-09-15 09:56:42 +00:00

feat: record_experience lesson field, preferred by the skill compiler (#35)

ric closed this issue

2026-09-15 09:56:42 +00:00

ric commented

2026-09-15 09:56:52 +00:00

Author

Owner

Implemented on v7.0.x in 7a4c2ba, for 7.0.0.

The four decisions, as settled:

Optional, nudged. record_experience(..., lesson=None). No validation; the tool docstring and the agent instructions (instructions.py, claude_config/CLAUDE.md) ask for a claim that holds beyond the incident, and to leave it out when nothing transfers.
Preferred, nothing else changed. extract_lessons() uses lesson for the do-rule and falls back to breakthrough. Same outcome gate, same reinforcement weight. A scoring change for lessons would ship on its own.
No backfill. Additive with no migration: a memory without a lesson compiles exactly as before, so existing skills don't move until new lessons arrive.
bless() unchanged. It marks a memory eligible and extract_lessons decides the text, so blessing now promotes the lesson when one exists.

breakthrough keeps its meaning and is still what the experience views show. lesson is in the episodic return fields and the compiler's pool projection (tested, since a field missing from a projection reads as None), and is returned by recall(), recall_detail(), get_experience() and explain_memory(), and shown on the web UI memory detail page.

Tests: tests/test_issue35_lesson.py. Full suite 1930 passed, 1 skipped, memory/ and tools/ at 100%.

Implemented on `v7.0.x` in 7a4c2ba, for 7.0.0. The four decisions, as settled: 1. **Optional, nudged.** `record_experience(..., lesson=None)`. No validation; the tool docstring and the agent instructions (`instructions.py`, `claude_config/CLAUDE.md`) ask for a claim that holds beyond the incident, and to leave it out when nothing transfers. 2. **Preferred, nothing else changed.** `extract_lessons()` uses `lesson` for the do-rule and falls back to `breakthrough`. Same outcome gate, same reinforcement weight. A scoring change for lessons would ship on its own. 3. **No backfill.** Additive with no migration: a memory without a lesson compiles exactly as before, so existing skills don't move until new lessons arrive. 4. **`bless()` unchanged.** It marks a memory eligible and `extract_lessons` decides the text, so blessing now promotes the lesson when one exists. `breakthrough` keeps its meaning and is still what the experience views show. `lesson` is in the episodic return fields and the compiler's pool projection (tested, since a field missing from a projection reads as `None`), and is returned by `recall()`, `recall_detail()`, `get_experience()` and `explain_memory()`, and shown on the web UI memory detail page. Tests: `tests/test_issue35_lesson.py`. Full suite 1930 passed, 1 skipped, `memory/` and `tools/` at 100%.

No labels

No milestone

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

ric/omnimem#35

No description provided.

Rows
Columns