Skip to content

Let another plugin scan memories and add bootstrap sections and search annotations - #45

Merged
kriszyp merged 8 commits into
mainfrom
feat/summary-extension-points
Nov 13, 2026
Merged

kriszyp merged 8 commits into
mainfrom
feat/summary-extension-points

Conversation

@kriszyp

@kriszyp kriszyp commented Nov 11, 2026 •

Copy link
Copy Markdown
Member
A datefruit basket weighed against its limit while the son tags fruit in line for a map-maker robot

⊙ Problem

A separate summary-tree plugin (#36) needs to read every memory and add topic summaries to bootstrap and to search hits. Today nothing outside agent-memory's guarded routes may read the Memory table, and no other plugin can add to those responses.

💡 Solution

Three exports let another app reach the running plugin, like registerMemorySink:

  • scanMemories() walks stored memories oldest first, resumable after any row. Trusted plugin code only; no route reaches it.
  • registerBootstrapSection() adds a bootstrap section after pinned and needsReview, within the budget left. Items arrive as memories entries without an id, so the Claude Code plugin shows them unchanged.
  • registerSearchAnnotator() adds hit[name] to q searches (REST, MCP recall) without changing which hits return or their order.

A throwing contributor is logged and left out.

⚖️ Alternatives

🤔 Hooks inside agent-memory, not routes wrapped by the plugin

The planning review asked whether the plugin could wrap agent-memory's routes at the same URL instead. Ruled out: the package exports no route class, configure() re-mounts routes when urlPath changes, and handlers return serialized responses. Each of the three surfaces would also redo budget packing and failure handling.

🤔 Contributors are trusted, like sinks

agent-memory catches a contributor's throw or rejection and checks that section items are strings. It does not guard against other malformed values, or stop contributors mutating the hits they are handed. That follows the brief's trust level and a steer from the queueing session. Two consequences are tracked in #40.

Other options
  • Sections return only { text }: the Claude client skips any record that does not fit its own framed budget. A single block sized to the remaining budget is therefore dropped whole whenever a pinned entry comes before it, while items in priority order lose only their tail. The companion plugin is also briefed with both forms.
  • The plugin serves its own endpoint, and clients call both: existing clients would need a change.
  • A generic response hook: it could reorder or drop memories, or overrun the budget.
  • Decoration in Harper core: it would hit the same serialized-response problem, and need a Harper newer than the ^5.3.1 peer range.

🔧 Changes

Product and architecture tour: where contributed output enters bootstrap and search (one diagram)

Product and architecture tour

Bootstrap sections

What does a session see at startup once a summary plugin is installed?

Before After
pinned, needsReview, recent, relevant: memories only. The same, with each registered section's items between needsReview and recent, inside the same budget.

One budget, packed in order

Sections render after pinned and needsReview are packed. Each gets the remaining budget while recent and relevant keep loading.

flowchart LR
    P[pinned] --> N[needsReview] --> S[sections] --> R[recent] --> Q[relevant]
Loading

A contributed entry is { title, content, section }, built by agent-memory: no id, so clients never watch or cache it, and no author, scopes or revision.
An item that does not fit is skipped, never truncated. A section that throws is left out and logged.

Search annotations

What does a search hit carry once an annotator is registered?

Example: a hit with its topic

A summary plugin registers an annotator named summaries that returns a Map from hit id to the topic nodes holding that memory.

Outcome: GET /Memory/?q=deploy returns each mapped hit with a summaries field. The hits, their order and their other fields are unchanged.

Annotators run only for q searches (REST and MCP recall); listings and bootstrap are not annotated.
Each gets a frozen copy of the hit list and must not change the hit objects in it.

Reading every memory

How does a derived store build itself and catch up?

scanMemories walks stored rows oldest first in pages of 100, resuming strictly after a given row; it is a walk, not a snapshot or a change feed.
A builder registers its sink before walking, compares revision, and walks again with includeInactive after any gap in which its sink did not run.

Bootstrap

  • Bootstrap keeps origin/main's single four-way Promise.all when no section is registered. With sections, they render after pinned and needsReview pack while recent and relevant still load. Each item that does not fit is skipped, never truncated. chars is measured by agent-memory, never read from the contributor (a change from the brief).
  • bootstrapItems accepts { text } or an array of { title?, content }. renderBootstrapSections gives each section one try/catch and a log line at most once a minute.
  • With watch=true, only memory ids are watched, and the Claude client neither watches nor caches contributed entries (tested end to end through its hooks).

🤔 Sections go before recent

The brief says "after pinned and needsReview, before relevant" and does not mention recent. recent holds up to 50 of the caller's own memories, enough to fill the Claude client's 4,000-character budget, so a section placed after it would rarely show.

Search

  • runAnnotators hands every annotator one frozen copy of the hit list, then merges by position in agent-memory's own list. Called from REST and MCP recall, only when q is set. With nothing registered, no promise is awaited.
  • A name may not be a hit field or score (register checks this, and stores a frozen copy of what was registered).

🤔 Annotators skip listings and bootstrap's relevant search

The order=created listing feeds copy-memories, whose import rejects unknown fields. Bootstrap has its own extension point.

Scan and across apps

  • The scan reads in pages of 100 with enforceExecutionOrder, reusing ?order=created's tie-break. It selects the view's fields plus embedding and embeddingModel, and copies the cursor's Date before the consumer sees the row. DESIGN states that it is a walk, not a snapshot or a change feed.
  • MemoryRuntime.scanMemories and the domain lists are optional members, so domains and runtimes from 0.1.4 still work.

🤔 Warning beside a 0.1.4 plugin

A 0.1.4 plugin never reads sections or annotators. warnIfOldRuntime warns once per runtime. When the registration comes first, noteExtensionRegistered turns the domain's runtime into an accessor, because 0.1.4 publishes with a plain assignment. Reads and writes behave as before.

Docs

✅ Verification

Performance: no extra work when nothing is registered, beyond one registry length check.

Framing-Verdict: option-set-too-narrow — resolved by overruling the two missing alternatives with the facts above.

Refs #36

🤖 Generated by Claude Code (Claude Opus 5.5); posted via @kriszyp.

Related PRs: #41 overlaps (merged; this extends its runtime registry and the same DESIGN.md section)
Complexity: complicated

Deferred findings

  • 8adc8f46716c1c03 minor correctness — src/extensions.ts — running plugin no longer rechecks reserved hit fields and section names at dispatch across versions — NOT FILED
  • 45ab778fcd9fd78b minor reliability — src/extensions.ts — sparse render array escapes validation and fails bootstrap — NOT FILED
Origin — the dispatch brief this PR was written from

Add the agent-memory extension points a separately packaged summary-tree plugin needs (issue #36), so that plugin can build topic summaries over memories and serve them in bootstrap and beside search hits without reading the Memory table directly. Three additions, all exported from the package index and reachable across apps through the existing per-worker runtime registry (src/runtime.ts domain), with the same rules as registerMemorySink: (1) scanMemories(): a read-only async iterable over Memory rows for trusted in-process derived builders (initial build and catch-up after restart); (2) registerBootstrapSection(section): lets another plugin contribute a named section to GET /Memory/bootstrap (and the MCP equivalent if bootstrap is exposed there), placed after pinned and needsReview and before relevant, rendered per caller with the resolved principal and within the caller's character budget; (3) registerSearchAnnotator(annotator): lets another plugin attach per-hit data to search results (GET /Memory/?q= and MCP recall), given the resolved principal and the hits. Update DESIGN.md (Extension points, Search and bootstrap) in the same PR. A companion task (summary-tree-agent-memory-plugin, repo HarperFast/summary-tree) builds the consumer against this API; keep the contract below unless the plan finds a concrete problem, and if you change it say so in the PR body.

Acceptance

Unit tests: scanMemories yields active/unexpired rows by default, every row with includeInactive, resumes from after without skipping or repeating rows that share a createdAt; a registered bootstrap section appears in the right position, respects the budget, and a throwing section is omitted without failing bootstrap; a search annotator's data appears per hit under its name, cannot change hit set or order, and a throwing annotator is skipped. A cross-app test (same pattern as the existing cross-copy runtime tests) shows a second copy of the package registering a section and an annotator that the running plugin uses. DESIGN.md updated. Existing tests green on Node 22 and 24. Do not touch the Claude Code plugin client or the release.

Dispatch: task agent-memory-summary-extension-points · queued by kris-session · ran by claude/opus/xhigh · worker kzyp-xps-1

Ready-For-Promotion: 23dc59d urgency=normal late=none

Review-Coverage: authored=claude; ran=codex,cursor-composer,gemini; adjudicated=domain; declined=cursor-grok,muse; rounds=7; full=1 @ 23dc59d

Review-Attention: read ~7m (decisions: do-less-alternative, requirement-shift, load-order-containment, trusted-contributors, contributor-owns-authorization, registering-copy-name-check, runtime-accessor, perf-cost) @ 23dc59d

kriszyp and others added 8 commits November 11, 2026 10:00
…h annotations

scanMemories() walks stored memory rows in createdAt, id order for trusted
in-process builders, paged and resumable. registerBootstrapSection() adds a
section to GET /Memory/bootstrap after pinned and needsReview, within the
budget they left, as id-less memories entries clients frame as stored data.
registerSearchAnnotator() attaches per-hit data to q searches (REST and MCP
recall) without being able to change the hits. A throwing contributor is
logged and left out. All three reach the running plugin from another app
through the per-worker runtime registry. Refs #36.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Dispatch-Task: agent-memory-summary-extension-points
Register a frozen copy of each section and annotator, so the name checked at
registration is the one dispatch uses. Validate rendered items with
Array.from, so a sparse array is caught inside the section's own boundary.
Copy the scan cursor's Date before yielding the page. Skip the annotation
await entirely when nothing is registered. Warn about an older running
plugin also when the copy first calls it, once per runtime.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Dispatch-Task: agent-memory-summary-extension-points
…r annotators

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Dispatch-Task: agent-memory-summary-extension-points
…older plugin publishing late

Bootstrap with no registered section keeps one four-way Promise.all. A
section or annotator registered before any plugin published makes the
domain's runtime an accessor, so an older plugin's plain publish runs the
same old-runtime check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Dispatch-Task: agent-memory-summary-extension-points
…oads from escaping Promise.all

The running plugin skips an annotator whose name is a hit field in its
version, or a section named like a built-in, since a copy of another version
validated the name against its own lists. The recent and relevant loads are
async, so a synchronous query throw rejects instead of leaving the other
loads' rejections unobserved.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Dispatch-Task: agent-memory-summary-extension-points
Annotators get a frozen copy of the hit list holding the result objects, so
they still cannot add, drop or reorder hits, but agent-memory no longer
deep-clones the hits, JSON-copies annotation values, reads Maps through
Map.prototype, checks for sparse item arrays, or rechecks contributor names
in the running plugin. Failures (a throw or a rejection) are still contained
and logged. The recent and relevant bootstrap loads return to the shape
origin/main has.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Dispatch-Task: agent-memory-summary-extension-points
…her bootstrap load's rejection

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Dispatch-Task: agent-memory-summary-extension-points
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Dispatch-Task: agent-memory-summary-extension-points
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant