Repository navigation
Claude Code plugin: record why prompt-time recall injected nothing, and skip prompts with no topic - #57
Merged
Conversation
…pts, keep injections when state writes fail
Prompt-time recall returned {} on every failure path, so a miss looked like "no relevant memories". On the
dispatch fleet every run missed because a run's private HOME hides ~/.config/agent-memory/env: the plugin
found AGENT_MEMORY_URL but no token and silently did nothing.
- Each SessionStart and UserPromptSubmit appends one line to <cacheDir>/diagnostics.jsonl (outcome, reason,
latency, counts; no prompt text, memory content, URLs or credentials), rotated at 256 KiB;
bin/diagnostics.mjs summarizes it.
- A partial configuration (URL without token, or the reverse) gets a SessionStart notice naming what is missing
and AGENT_MEMORY_ENV_FILE.
- The recall search is skipped when the query has no distinctive term (1, 2, 3 / yes / please continue);
queued change notices are still delivered. eval-recall applies the same gate.
- State and offline-cache writes are best-effort in both hooks, and the follower spawn's async error is
handled, so neither can discard an injection the server returned.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Dispatch-Task: agent-memory-recall-silent-misses
… hardened log reader - Bootstrap: a body cut off by the deadline or the network falls back to the offline cache like any other outage, and is labeled timeout or network (with its errno) instead of bad_json; hits that do not fit the budget log over_budget, as recall does. - The partial-config notice falls back to a short sentence when the budget cannot hold the full one, and an explicitly set AGENT_MEMORY_ENV_FILE that yields nothing counts as partial configuration. - Diagnostics: open before mkdir, one fstat, no descriptor leak; the reader refuses symlinks and FIFOs like the writer; summaries use Maps so odd event names cannot break them. - Tests: permission-based write-failure test skips under root; DESIGN.md wording narrowed to what the code does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Dispatch-Task: agent-memory-recall-silent-misses
…maining silent paths - The partial-config notice names the remedy for the reason: an env file for a missing URL or token, the https/loopback rule for a rejected URL, a single-line token for a rejected token. - The reader reads at most the cap's worth from the end of each file. - SessionStart with an unsupported source and the MCP headers helper without a credential each log a line; stateError keeps every distinct code; eval-recall's text report prints the skipped count. - DESIGN.md: `injected` means returned to hook.mjs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Dispatch-Task: agent-memory-recall-silent-misses
…deadline AbortSignal.timeout's timer is unref'd, so with no socket open Node 22's test runner saw an empty event loop and cancelled the rest of the file. Node 24 and later passed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Dispatch-Task: agent-memory-recall-silent-misses
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Dispatch-Task: agent-memory-recall-silent-misses
kriszyp
marked this pull request as ready for review
November 17, 2026 14:27
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
⊙ Problem
Prompt-time recall printed nothing on any failure (missing credentials, a 401, a timeout), so a miss looked like "no relevant memories".
On the dispatch fleet every dev-agent run missed. Each run gets a private
HOME, which hides~/.config/agent-memory/env, so the plugin found a URL but no token. And a prompt like1, 2, 3recalled an unrelated memory.💡 Solution
diagnostics.jsonl(plus latency and hit count when a request ran), never prompt text or memory content;bin/diagnostics.mjssummarizes it.Fleet recall comes back with dispatch task
dispatch-dev-prompt-recall-subject, which passesAGENT_MEMORY_ENV_FILEto runs.🤔 Should one short term (
fix CI,PR 2368) trigger a search?No:
hasDistinctiveTermneeds one non-conversational term of 4+ characters (2 in CJK scripts). MCPrecallstill serves such prompts; retuning is one function.🤔 Should a never-configured install log every prompt?
Yes: when env vars are lost (a GUI launch, a harness), it is the only record of why nothing happened. The alternative logs only half-found configurations.
⚖️ Alternatives
score— reciprocal rank fusion is rank only: any top hit scores at least 1/61.lmdb,ECONNREFUSED).Framing-Verdict: better-alternative-exists — resolved: dispatch has its own task; one-term gate adopted.
🤔 Should the plugin find the token itself when
HOMEis replaced?No: the passwd home would bypass dispatch's deliberate private
HOME, and temp-HOMEtests would read a developer's real token. The harness passesAGENT_MEMORY_ENV_FILEinstead.🔧 Changes
One line per hook run
diagnostics.mjs:53
persistkeeps the injection when a write failshooks.mjs:81
Files
claude-plugin/lib/diagnostics.mjs(new): size-limited writer, reader and summary.claude-plugin/lib/hooks.mjs:recordedHookwraps both hooks and writes the line; the gate sits before the search only; the partial-config notice picks its remedy by reason; bootstrap falls back to the offline cache on a cut-off body;persistguards the state and offline-cache writes.claude-plugin/lib/config.mjs:unconfiguredReasonandpartiallyConfigured.claude-plugin/lib/follower.mjs: guardsmkdirand listens for the spawn'serror.claude-plugin/lib/cache.mjs:sessionKey, the state-file key the log reuses.claude-plugin/bin/diagnostics.mjs(new): the reader CLI.claude-plugin/bin/headers.mjs: logsmcp_headerswhen it has no credential.scripts/eval-recall.mjs: applies the same gate; gated rows count as skipped misses, as empty queries already did.DESIGN.md: the log, credential lookup and best-effort persistence invariants.claude-plugin/README.md: the Diagnostics section,AGENT_MEMORY_ENV_FILEandAGENT_MEMORY_CACHE_DIR.CHANGELOG.md.Product and architecture tour: outcomes, credentials, the search gate, persistence
Product and architecture tour
Why a recall injected nothing
What does a reader of the log learn about a prompt that got no memories?
{}: unconfigured, 401, timeout, network error, bad JSON, corrupt state, or a thrown write.injected,empty,deduped,over_budget,http_<status>,timeout,error(with a reason),skipped,disabledorunconfigured.diagnostics.jsonl.1at 256 KiB. It is openedO_NOFOLLOW | O_NONBLOCK, blocking planted symlinks or FIFOs.node <plugin root>/bin/diagnostics.mjsprints counts by outcome and latency percentiles;--tail Nand--jsonadd detail.Credentials under a replaced HOME
What does a dispatch run see now, before the dispatch task lands?
Where the hooks look for a token
The plugin reads only the environment,
AGENT_MEMORY_ENV_FILE, or the env file underHOME. It now reports partial configurations.flowchart LR Env[Token in environment] -->|no| File[AGENT_MEMORY_ENV_FILE or HOME env file] File -->|found| Ok[Hooks run] File -->|missing| Half{URL or env file set?} Half -->|yes| Notice[Session notice and unconfigured log line] Half -->|no| Quiet[Plugin not set up: log line only]Prompts with no distinctive term
Which prompts still search?
1, 2, 3,yes,abc,PR 2368please continue,sounds goodlmdb,ECONNREFUSED,db migration修复,복제Writes that fail no longer drop an injection
What happens when
state.jsonor the offline cache cannot be written?erroris now handled; unhandled, it ended the hook as an uncaught exception.✅ Verification
npm test522/522 on Node 22 and 26,npm run test:integration81/81, format clean. A 65-termw0 w1 …probe now leads with a real word (memory.test.ts).hook.mjswith a privateHOMEand the token in a host env file. Base prints{}with no trace; here it shows the notice and logsunconfigured/missing_token. Base also injects for1, 2, 3and drops an injection on a failed state write.Hook wall time, base vs this branch (real
hook.mjsprocesses, all sessions launched at once, 300 ms mock server, median of 3 runs)New tests cover each outcome (timeout via a hung server and a cut-off body), the size cap and rotation, symlink refusal, the gate with notices, and write failures.
1, 2, 3Misses appear only when server latency nears the 1500 ms recall timeout (24 of 24 injected at 1200 ms, 0 at 1450 ms). Host: 12 cores, load about 1 before each run, turbo on, not under bench-run.
Docs: the plugin README is the user-facing doc; HarperFast/documentation has no agent-memory pages.
🤖 Generated by Claude Opus 5.5 with Claude Code; posted via @kriszyp.
Related PRs: none found
Complexity: medium
Deferred findings
f13fce661c1ce140minor reliability —claude-plugin/lib/config.mjs— fleet recall and logs still lost until dispatch passes ENV_FILE and CACHE_DIR — filed Claude Code plugin recall: known limitations to revisit #5822787b9104bcb85cminor correctness —claude-plugin/lib/hooks.mjs— gate skips 3-letter technical terms and ticket ids — filed Claude Code plugin recall: known limitations to revisit #580a343f75dd9acc8dminor reliability —claude-plugin/lib/hooks.mjs— recall timeout not user-visible — filed Claude Code plugin recall: known limitations to revisit #58b2ce4a1deceba3e4minor performance —claude-plugin/lib/diagnostics.mjs— per-hook synchronous log I/O incl. never-configured installs — filed Claude Code plugin recall: known limitations to revisit #58a4e19c49e2a32c0anit data-integrity —claude-plugin/lib/hooks.mjs— injectedIds persisted before stdout delivery — filed Claude Code plugin recall: known limitations to revisit #5813200ad74e488cd7nit reliability —claude-plugin/lib/hooks.mjs— partial-config notice dropped under small valid budget — filed Claude Code plugin recall: known limitations to revisit #585eb55a4f19fe0766nit maintainability —scripts/eval-recall.mjs— gated rows averaged as recall 0; text report omits skipped count — filed Claude Code plugin recall: known limitations to revisit #581256a04eb6f14fe7nit reliability —claude-plugin/lib/hooks.mjs— diagnostic 'injected' logged before stdout delivers the injection — filed Claude Code plugin recall: known limitations to revisit #58Origin — the dispatch brief this PR was written from
Find and fix why the Claude plugin's prompt-time recall (UserPromptSubmit, claude-plugin/lib/hooks.mjs handleUserPromptSubmit) intermittently injects nothing, and make such misses observable. Every failure path returns {} silently today (401/403, non-OK status, timeout, fetch error, corrupt session state), so a miss looks identical to 'no relevant memories'. Step 1: add a bounded, append-only diagnostics log under the plugin cacheDir (one JSON line per recall: timestamp, session id hash, outcome = injected|empty|deduped|http_|timeout|error, latency ms, hit count; no prompt text, no memory content, no credentials), rotated or size-capped, plus a way to read it (e.g. a small script or a documented path). Step 2: use it plus a reproduction to find the root cause of the intermittent misses described in context and fix it. Step 3: when the query is too short or generic to be meaningful (e.g. a prompt like '1, 2, 3'), recall should inject nothing rather than the nearest unrelated memory: apply a minimum relevance score or minimum query content, whichever matches how the server scores results.
Acceptance
Unit tests for each recall outcome being logged (including timeout and http error), for the log size cap, and for the short/generic-query case injecting nothing. A reproduction (test or script) of the intermittent miss that fails before the fix and passes after, or, if the root cause is server latency, a measured latency breakdown in the PR with the chosen fix (e.g. timeout/budget change, query shortening, or server-side change) justified by the numbers. Existing unit and integration tests pass.
Dispatch: task
agent-memory-recall-silent-misses· queued by kris-session · ran by claude/opus/xhigh · worker kzyp-xps-1Ready-For-Promotion: 8ce60a2 urgency=normal late=folded
Review-Coverage: authored=claude; ran=cursor-composer,gemini,codex,gemini-carbon,cursor-grok; late=gemini-carbon:adjudicated,cursor-grok:adjudicated; adjudicated=domain; declined=muse; rounds=5; full=2 @ 8ce60a2
Review-Attention: read ~6m @ 8ce60a2