feat: remember_document() tool with chunking strategies #15

New issue

Closed

opened 2026-04-08 19:21:28 +00:00 by ric_harvey · 1 comment

ric_harvey commented

2026-04-08 19:21:28 +00:00

(Migrated from codeberg.org)

Problem

remember() is designed for storing short, focused facts — decisions, experiences, preferences. When callers need to index long-form content (conversation exports, documents, meeting transcripts), they have to implement their own chunking logic before calling remember() multiple times.

This was discovered during the LongMemEval benchmark project, where storing whole conversation sessions (500-2000 words) as single memories resulted in 45% zero recall — Valkey's semantic search couldn't match short questions against large undifferentiated blobs. Switching to turn-pair chunking (one user+assistant exchange per memory) reduced zero recalls to 0% and improved answer quality dramatically.

Proposed solution

Add a remember_document() MCP tool:

remember_document(
    content: str,           # long-form text to index
    chunk_strategy: str,    # "turn_pairs" | "sentences" | "paragraphs" | "fixed_tokens"
    project: str,
    tags: list[str],
    namespace: str = "episodic",
    chunk_size: int = None, # for fixed_tokens strategy
) -> {"keys": list[str], "chunks_stored": int}

Chunking strategies

turn_pairs — splits on User: / Assistant: alternating turns. Each exchange stored as one memory. Best for conversation exports.

sentences — splits on sentence boundaries. Good for articles, notes, documentation.

paragraphs — splits on double newlines. Good for structured documents.

fixed_tokens — splits into chunks of chunk_size tokens with overlap. General purpose fallback.

Why this matters

Callers get correct chunking without reimplementing it
Returns list of inserted keys — solves cleanup problem from issue #14 (no need to search for keys later)
Keeps remember() simple for its primary use case (short facts)
Enables OmniMem to be used as a document indexing layer, not just a fact store

Benchmark evidence

LongMemEval results comparing whole-session vs turn-pair chunking:

Strategy	Avg chunks/item	Zero recalls	Abstention rate
Whole session (v1)	53	45%	66%
Turn pairs (v2)	261	0%	~10%

The improvement is entirely due to chunking granularity — same Valkey search, same recall pipeline, different input preparation.

Issue #14 (bulk delete by project) — remember_document() returning keys makes cleanup trivial

## Problem `remember()` is designed for storing short, focused facts — decisions, experiences, preferences. When callers need to index long-form content (conversation exports, documents, meeting transcripts), they have to implement their own chunking logic before calling `remember()` multiple times. This was discovered during the LongMemEval benchmark project, where storing whole conversation sessions (500-2000 words) as single memories resulted in 45% zero recall — Valkey's semantic search couldn't match short questions against large undifferentiated blobs. Switching to turn-pair chunking (one user+assistant exchange per memory) reduced zero recalls to 0% and improved answer quality dramatically. ## Proposed solution Add a `remember_document()` MCP tool: ```python remember_document( content: str, # long-form text to index chunk_strategy: str, # "turn_pairs" | "sentences" | "paragraphs" | "fixed_tokens" project: str, tags: list[str], namespace: str = "episodic", chunk_size: int = None, # for fixed_tokens strategy ) -> {"keys": list[str], "chunks_stored": int} ``` ### Chunking strategies **`turn_pairs`** — splits on `User:` / `Assistant:` alternating turns. Each exchange stored as one memory. Best for conversation exports. **`sentences`** — splits on sentence boundaries. Good for articles, notes, documentation. **`paragraphs`** — splits on double newlines. Good for structured documents. **`fixed_tokens`** — splits into chunks of `chunk_size` tokens with overlap. General purpose fallback. ### Why this matters - Callers get correct chunking without reimplementing it - Returns list of inserted keys — solves cleanup problem from issue #14 (no need to search for keys later) - Keeps `remember()` simple for its primary use case (short facts) - Enables OmniMem to be used as a document indexing layer, not just a fact store ## Benchmark evidence LongMemEval results comparing whole-session vs turn-pair chunking: | Strategy | Avg chunks/item | Zero recalls | Abstention rate | |----------|----------------|-------------|-----------------| | Whole session (v1) | 53 | 45% | 66% | | Turn pairs (v2) | 261 | 0% | ~10% | The improvement is entirely due to chunking granularity — same Valkey search, same recall pipeline, different input preparation. ## Related - Issue #14 (bulk delete by project) — `remember_document()` returning keys makes cleanup trivial

ric_harvey commented

2026-07-08 11:11:22 +00:00

(Migrated from codeberg.org)

Closing — shipped in v5.0.0. remember_document() is a registered MCP tool with the four chunking strategies (turn_pairs for User:/Assistant: transcripts, sentences, paragraphs, fixed_tokens with configurable size), storing each chunk as an individual memory linked by a shared doc_id with chunk_index/chunk_strategy metadata, plus per-chunk near-duplicate skipping. Driven by exactly the finding in this issue — whole-session storage produced 45% zero-recall on 500–2,000-word inputs while turn-pair chunking dropped that to 0%.

Since then it also feeds the async enrichment queue (v5.2.0, #19) and the supplement-not-replace fact architecture (v5.3.1, #20).

Closing — shipped in v5.0.0. `remember_document()` is a registered MCP tool with the four chunking strategies (`turn_pairs` for User:/Assistant: transcripts, `sentences`, `paragraphs`, `fixed_tokens` with configurable size), storing each chunk as an individual memory linked by a shared `doc_id` with `chunk_index`/`chunk_strategy` metadata, plus per-chunk near-duplicate skipping. Driven by exactly the finding in this issue — whole-session storage produced 45% zero-recall on 500–2,000-word inputs while turn-pair chunking dropped that to 0%. Since then it also feeds the async enrichment queue (v5.2.0, #19) and the supplement-not-replace fact architecture (v5.3.1, #20).

No labels

No milestone

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

ric/omnimem#15

No description provided.

Rows
Columns