Skip to content

feat(feed): add a "Search everything" button to the admin feed - #84

Merged
junaiddshaukat merged 1 commit into
mainfrom
feat/feed-search-everything
Nov 1, 2026
Merged

junaiddshaukat merged 1 commit into
mainfrom
feat/feed-search-everything

Conversation

@junaiddshaukat

Copy link
Copy Markdown
Member

Summary

Adds Search everything to /admin/dashboard/feed. One click runs every search in the collector's list once (about 97 today: 40 topic searches plus one per member), so it covers all the posts search can reach. It replaces the fixed 40-search "Deep search".

  • Same pipeline as Find new posts. It calls the same endpoint and collector: new posts are saved as live, posts already in the database are skipped, and missing excerpts are filled in.
  • Batches of 40 keep each request well inside Vercel's 5-minute limit. Progress shows after each batch ("Searching… 80 of 97 done, 31 new posts so far").
  • Confirms first with the search count and cost ("about 97 searches and 485 Olostep credits").
  • Stops early if a whole batch fails, for example when Olostep credits run out.
  • One full pass of the list as it stood at the start. Members found during the run are searched on the next run.
  • Find new posts is unchanged (5 searches).

Test plan

  • tsc and next lint clean
  • Headless Chrome with mocked API responses (no database, no Olostep, no credits):
    • Search everything showed the right confirmation, sent batches of 40 + 40 + 17 = 97, and reported "97 searches, 9 new posts, 3 updated"
    • Find new posts still sends one default request (5 searches)
  • The server path is unchanged and already verified on production by Find new posts (5 searches added 24 posts)

🤖 Generated with Claude Code

Replace the fixed 40-search "Deep search" with "Search everything", which
runs every search in the list once (about 97 today), so one click covers
all the ground search can reach.

- Runs in batches of 40 to stay well inside Vercel's 5-minute limit, and
  shows progress after each batch
- Uses the same endpoint and collector as "Find new posts": new posts are
  saved as live, existing ones are skipped, missing excerpts are filled
- Confirms first with the number of searches and Olostep credits
- Stops early if a whole batch fails, e.g. when credits run out

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@junaiddshaukat
junaiddshaukat merged commit 2094569 into main Nov 1, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant