feat: built-in query expansion for recall() #16

New issue

Closed

opened 2026-04-08 19:21:47 +00:00 by ric_harvey · 1 comment

ric_harvey commented

2026-04-08 19:21:47 +00:00

(Migrated from codeberg.org)

Problem

Recall quality is significantly limited when the query phrasing doesn't closely match how information was originally stored. A question like "What degree did I graduate with?" scores poorly against a session that says "I studied Business Administration at university" — the vocabulary is different even though the meaning is the same.

The current workaround is to implement query expansion in the calling client, generating multiple search variants and unioning the results. This works but pushes complexity onto every caller.

Evidence from LongMemEval benchmark

Without query expansion, even with good turn-pair chunking, some questions fail to retrieve anything because the question phrasing and stored content don't share vocabulary. With query expansion generating 3 variants per question, recall coverage improves substantially — we went from needing to write ~50 lines of client-side expansion logic to a single flag.

Example:

Question: "What degree did I graduate with?"

Expansions generated:
1. "graduated university degree bachelor master"
2. "studied qualification academic programme"  
3. "college diploma graduation ceremony"

All 4 queries are run, results are unioned by key and re-ranked by top score.

Proposed solution

Add an expand_queries flag to recall() and recall_index():

recall(
    query: str,
    expand_queries: bool = False,   # generate variants and union results
    top_k: int = 10,
    project_filter: str = None,
    ...
)

When expand_queries=True:

Use Claude (via the existing ANTHROPIC_API_KEY integration) to generate N variant queries
Run recall for each variant
Union results, deduplicate by key, re-rank by highest score across variants
Return top top_k

Configuration

RECALL_EXPAND_QUERIES=true    # enable by default for all recall calls
RECALL_EXPAND_COUNT=3         # number of variants to generate (default 3)

Why in OmniMem rather than the client

Query expansion is universally useful — any caller benefits, not just benchmark scripts
OmniMem already has optional Claude integration for RSS summarisation — this reuses the same pattern
Keeps client code simple: one recall() call instead of a loop over variants
OmniMem can cache expansion results for repeated queries

Cost consideration

Each expansion call adds ~1 Claude API call (Haiku is sufficient, ~0.01p per expansion). Only fires when expand_queries=True or the env var is set, so it's opt-in.

Issue #15 (remember_document()) — both issues stem from the LongMemEval benchmark project

## Problem Recall quality is significantly limited when the query phrasing doesn't closely match how information was originally stored. A question like "What degree did I graduate with?" scores poorly against a session that says "I studied Business Administration at university" — the vocabulary is different even though the meaning is the same. The current workaround is to implement query expansion in the calling client, generating multiple search variants and unioning the results. This works but pushes complexity onto every caller. ## Evidence from LongMemEval benchmark Without query expansion, even with good turn-pair chunking, some questions fail to retrieve anything because the question phrasing and stored content don't share vocabulary. With query expansion generating 3 variants per question, recall coverage improves substantially — we went from needing to write ~50 lines of client-side expansion logic to a single flag. Example: ``` Question: "What degree did I graduate with?" Expansions generated: 1. "graduated university degree bachelor master" 2. "studied qualification academic programme" 3. "college diploma graduation ceremony" ``` All 4 queries are run, results are unioned by key and re-ranked by top score. ## Proposed solution Add an `expand_queries` flag to `recall()` and `recall_index()`: ```python recall( query: str, expand_queries: bool = False, # generate variants and union results top_k: int = 10, project_filter: str = None, ... ) ``` When `expand_queries=True`: 1. Use Claude (via the existing `ANTHROPIC_API_KEY` integration) to generate N variant queries 2. Run recall for each variant 3. Union results, deduplicate by key, re-rank by highest score across variants 4. Return top `top_k` ### Configuration ```env RECALL_EXPAND_QUERIES=true # enable by default for all recall calls RECALL_EXPAND_COUNT=3 # number of variants to generate (default 3) ``` ### Why in OmniMem rather than the client - Query expansion is universally useful — any caller benefits, not just benchmark scripts - OmniMem already has optional Claude integration for RSS summarisation — this reuses the same pattern - Keeps client code simple: one `recall()` call instead of a loop over variants - OmniMem can cache expansion results for repeated queries ### Cost consideration Each expansion call adds ~1 Claude API call (Haiku is sufficient, ~0.01p per expansion). Only fires when `expand_queries=True` or the env var is set, so it's opt-in. ## Related - Issue #15 (`remember_document()`) — both issues stem from the LongMemEval benchmark project

ric_harvey commented

2026-07-08 11:11:34 +00:00

(Migrated from codeberg.org)

Closing — shipped in v5.0.0. recall() and recall_index() take an expand_queries flag (default follows RECALL_EXPAND_QUERIES, count via RECALL_EXPAND_COUNT): Claude Haiku generates alternative phrasings, results are unioned across variants and deduplicated, and variants are cached in Valkey under qexp:<sha1> with a 24h TTL so repeated queries don't re-bill the API.

Two refinements since: v5.3.0 gave variant searches the same query-side state/project filter push-down and candidate over-fetch as the main query, and v5.3.1 fixed variants not applying the temporal boost (date-anchored memories ranked lower when found via a variant than via the original query).

Closing — shipped in v5.0.0. `recall()` and `recall_index()` take an `expand_queries` flag (default follows `RECALL_EXPAND_QUERIES`, count via `RECALL_EXPAND_COUNT`): Claude Haiku generates alternative phrasings, results are unioned across variants and deduplicated, and variants are cached in Valkey under `qexp:<sha1>` with a 24h TTL so repeated queries don't re-bill the API. Two refinements since: v5.3.0 gave variant searches the same query-side state/project filter push-down and candidate over-fetch as the main query, and v5.3.1 fixed variants not applying the temporal boost (date-anchored memories ranked lower when found via a variant than via the original query).

No labels

No milestone

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

ric/omnimem#16

No description provided.

Rows
Columns