Repository navigation
feat: NX-16342 detect and update outdated CLI tools on every setup run - #146
Conversation
Each installer registers an update check. Setup compares installed and latest versions in parallel, lists outdated tools, and prompts to update them (default Yes). Go tools read versions from binary build info and compare against the tag go install @latest resolves to (or HEAD for untagged repos). Homebrew and npm tools use their manager's own latest version. Failed lookups are skipped with a warning.
There was a problem hiding this comment.
Code Review
This pull request introduces a comprehensive version-checking and self-updating mechanism for various CLI tools, centralizing the logic in a new lib/utils/versions.js utility and integrating it across installers and the main CLI flow. Feedback on the implementation focuses on enhancing robustness: ensuring strict Semver 2.0.0 compliance regarding leading zeroes in pre-release identifiers, wrapping JSON.parse in a try-catch block to handle malformed Homebrew output, adding authorization headers to GitHub API requests to prevent rate-limiting, and defensively wrapping tool updates in a try-catch block.
|
Ratatoskr reviewed this pull request. Changes requested on Finished 2026-11-11 12:42 UTC. |
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request introduces a robust update-checking mechanism for various CLI tools managed by the application, implementing version comparison, registry checks (Homebrew, npm, Go, GitHub releases), and interactive update prompts. Feedback on the new lib/utils/versions.js utility suggests adding a mandatory User-Agent header to GitHub API requests to avoid 403 errors, and adding a falsy guard for the binary argument in commandVersion to prevent execution errors.
fank
left a comment
There was a problem hiding this comment.
REQUEST_CHANGES — 1 verified update-path defect.
Review details
Required correction
[P2] Update the binary that was checked, not a different installation — lib/installers/atl.js:397–402 (also the n8n/esq/discord registrations).
findAtlBinary() explicitly supports ~/.local/bin/atl and /usr/local/bin/atl, but the new updater delegates to installAtl(), whose go install writes to GOBIN or GOPATH/bin (normally ~/go/bin). When the detected older binary remains first on PATH, accepting the update reports success and caches current, yet normal configuration still executes the old binary. The next run offers the same update again. This leaves the original docs/binary mismatch unresolved for these supported installation locations.
Reproduced in an isolated temporary HOME with stubbed git/go and the real registry/installer flow: active ~/.local/bin/atl was v1.0.0; latest was v1.1.0; offerUpdate('atl') returned updated; v1.1.0 appeared in ~/go/bin/atl; the active binary remained v1.0.0 and the next check again reported v1.0.0 → v1.1.0.
Make the update target the detected installation (or explicitly migrate selection/PATH), and verify the selected binary after updating before declaring it current. Add a regression test with the active binary outside the Go install destination; the existing tests stub update() and do not exercise this mismatch.
Verification
Reviewed head: 976ef220621f1b63f16a2fd5724649cf59a1eaa7.
The checkout has no installed dependencies, so its initial npm test failed with missing chalk/inquirer. Without modifying it, exported this exact commit to /tmp/pr146-validation-M278HL (a temporary validation directory), installed dependencies there using Node 24.21.0, and ran:
npm ci --ignore-scripts— succeeded.npm test— 35 passed.npm run lint— passed.npm run check-format— passed.node /tmp/pr146-repro.mjs— reproduced the update-path defect above with local stubs only.
No real package upgrades, authentication flows, or live-system configuration changes were run. CI lint/format/CodeQL checks were successful; License Check was skipped.
Earlier discussions
GraphQL enumeration included all four threads and all their comments; neither connection required another page. 0 unresolved; 0 resolved by this review. All four were already resolved and were authored by another reviewer, not authenticated reviewer fank:
- Semver leading-zero discussion: no additional blocker identified for the supported versions.
- Malformed Homebrew JSON: verified the parse is now caught and its regression test passes.
- GitHub API authentication: optional resilience improvement, not a blocker.
- Defensive update catch: no additional current throwing updater established.
No replies or thread-state changes were made.
Also cover the update listing and skipped-lookup warning with tests, and document that installers must be imported before checkForUpdates().
go install now targets the directory of the active binary via GOBIN, so a binary in ~/.local/bin or /usr/local/bin is replaced instead of a second copy landing in GOPATH/bin behind it. offerUpdate re-checks the tool after an update and reports it as failed when it is still outdated.
|
FIXED in ad60444: Ratatoskr review, "Update the binary that was checked, not a different installation".
|
|
FIXED: the change request announced in Ratatoskr's status comment is addressed in ad60444. Details: reply to the review. |
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request introduces a comprehensive update-checking mechanism for installed CLI tools by adding a new versions.js utility and integrating it across various installer modules and the main CLI flow. It also includes robust test suites for these new features. The review feedback highlights two valuable improvements: wrapping fs.realpathSync in a try-catch block within goToolUpdate to ensure process.env.GOBIN is safely restored even if resolving a symlink fails, and utilizing GITHUB_TOKEN or GH_TOKEN in latestGithubRelease requests to prevent hitting GitHub API rate limits.
…s current The up-to-date banner now requires every installed tool to be verified current. Release comparisons in the npm, gcx and ovhcloud checks throw on non-semver versions, so they show up as skipped.
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request introduces a comprehensive update-checking mechanism for the CLI tools. It adds a new utility module lib/utils/versions.js to handle version parsing, semver comparison, and latest-version lookups across multiple sources (Git tags, Homebrew, npm, and GitHub releases). It integrates these checks into the main CLI flow and individual tool installers, allowing users to see update badges and receive interactive prompts to update outdated tools. The review feedback suggests enhancing the GitHub API lookup by supporting GITHUB_TOKEN or GH_TOKEN environment variables to prevent rate-limiting issues.
|
🔏 Readiness attested — the Readiness summaryPR Readiness Check (#146, 2542dd6) Fixes made during readiness:
Not exercised for real: brew upgrade and npm install -g updates. The Go update path was exercised live (atl v1.13.0 → v1.15.0). |
|
Ratatoskr reviewed this pull request. Changes requested on Finished 2026-11-11 13:06 UTC. |
fank
left a comment
There was a problem hiding this comment.
REQUEST_CHANGES — 1 blocking correctness finding.
Review details
Reviewed head 2542dd68314809b13dd9d4f3cfbb848cd417b75b.
Finding
[P2] Refresh Homebrew metadata before declaring tools current — lib/utils/versions.js:175–179
The only Homebrew lookup is brew info, which does not trigger Homebrew's normal auto-update. Consequently, an installed tool matching cached metadata is reported as current even when brew upgrade would refresh metadata and install a newer release. This affects gh, sqlcmd, hcloud and brew-managed gcx/ovhcloud. In particular, the third-party OVH tap can remain stale indefinitely if the developer only runs setup; repeated setup runs never reach brew upgrade because no update prompt is offered.
Homebrew explicitly enables auto-update for outdated/upgrade, but not info (command classification). API-backed metadata also has a different cache lifetime outside auto-update commands. Please refresh metadata once per setup run within the timeout/error policy, or use a lookup that participates in Homebrew's normal freshness checks, before trusting outdated: false. Add coverage distinguishing stale metadata from a verified-current installation; the existing fake-brew test only checks parsing a supplied flag.
Verification
- In a temporary archive of the exact head, using Node 24.21.0:
npm ci --ignore-scripts,npm test,npm run lint, andnpm run check-formatall succeeded. 42 tests passed. The default shell initially lacked npm; the installed Node toolchain was then added to PATH. - GitHub checks were all passing when inspected.
- No real package upgrades or interactive account/configuration changes were executed. Homebrew freshness behavior was checked against upstream source, not a local Homebrew installation.
- The reviewed checkout remains unchanged.
Earlier discussions
Enumerated all review threads and their comments through GraphQL; neither connection required another page. All nine existing threads were already resolved; 0 unresolved, 0 newly resolved, 0 replies posted. No thread mutations were needed (existing discussions).
brew info never refreshes Homebrew metadata, so a stale tap reported tools as current. brew outdated refreshes on Homebrew's auto-update schedule and reflects what brew upgrade would install; pinned packages count as current. brew info still decides whether brew manages a tool. Calls are serialized to avoid contending for Homebrew's update lock.
|
FIXED in 86b4e33: Ratatoskr review on 2542dd6, "Refresh Homebrew metadata before declaring tools current".
|
|
Ratatoskr reviewed this pull request. Changes requested on Finished 2026-11-11 13:13 UTC. |
|
FIXED: the change request in Ratatoskr's second status comment is addressed in 86b4e33. Details: reply. |
|
/gemini review |
fank
left a comment
There was a problem hiding this comment.
REQUEST_CHANGES — 1 blocking correctness finding.
Review details
Finding
[P2] Match tapped Homebrew formula names — lib/utils/versions.js:219–222
brew outdated --json=v2 uses the formula's full name for non-core taps (for example, grafana/grafana/gcx), whereas this predicate compares it to the unqualified gcx. Consequently, an installed, outdated tapped formula is treated as current: no badge or update prompt appears. This affects the supported Homebrew-managed gcx path and other registered tools installed from taps.
Reproduced with a stubbed brew: info reports installed gcx 1.3.1, and outdated lists name: "grafana/grafana/gcx", current_version: "1.7.1", pinned: false; await brewPackageInfo('gcx') incorrectly returns { installed: '1.3.1', latest: '1.3.1', outdated: false }. Homebrew's outdated.rb emits f.full_name for formulae, unlike cask tokens.
Match the resolved full formula name (or normalize the returned formula name appropriately), and add a regression test with a qualified formula name. The existing tapped-cask test does not exercise this case.
Verification
Reviewed head 86b4e33b61d1691394c62455c62d5453514f9f12. In a temporary copy created with git archive HEAD (review checkout unchanged), ran:
npm ci --ignore-scripts: succeeded.npm test: 47 passed, 0 failed.npm run lint: passed.npm run check-format: passed.- Isolated fake-Homebrew reproduction described above: confirmed the incorrect current verdict.
No real package upgrades, authentication flows, or browser installation were executed. GitHub checks inspected: lint/format, license and Actions analysis passed; JavaScript/TypeScript analysis was pending, and the CodeQL summary was skipped. Pending CI is not the reason for this verdict.
Earlier discussions
Enumerated review threads and comments through GraphQL; neither connection required another page. All nine existing threads were already resolved, so there were no unresolved findings to follow up, replies to post, or thread resolutions to perform.
There was a problem hiding this comment.
Code Review
This pull request introduces a comprehensive update-checking utility (lib/utils/versions.js) and integrates it across various CLI tool installers to check for and offer updates during setup. It also adds extensive test coverage for these new version-checking features. The review feedback highlights three key areas for improvement: updating the regular expression in extractVersion to correctly support hyphens in prerelease tags, adding a User-Agent header to GitHub API requests to prevent rate-limiting or 403 errors, and wrapping fs.realpathSync in a try-catch block to handle potential file system exceptions gracefully.
|
FIXED in a2014bf: Ratatoskr review on 86b4e33, "Match tapped Homebrew formula names".
|
|
FIXED: the change request in Ratatoskr's third status comment is addressed in a2014bf. Details: reply. |
|
Ratatoskr reviewed this pull request. Approved on Finished 2026-11-11 13:36 UTC. |
|
🔏 Readiness attested — the Readiness summaryPR Readiness Check (#146, a2014bf) Refresh fixes:
Open: fank's CHANGES_REQUESTED review state remains until Ratatoskr re-reviews a2014bf. |
fank
left a comment
There was a problem hiding this comment.
APPROVE — 0 blocking findings; earlier update-path and Homebrew findings are fixed.
Review details
Reviewed head a2014bf5c3df6cb96750a35c2595dda095c87125 against base 0dd6979ea801164dd06384d5520d2886e1b715ad, including both entry points and installer integration. No remaining actionable blocking defects identified.
Earlier findings rechecked
- Go update destination:
goToolUpdate()now targets the resolved active binary directory using GOBIN and restores the previous environment;offerUpdate()checks the result again.test/go-tool-update.test.jsexercises an active binary outside GOPATH/bin and passes. - Homebrew metadata freshness: checks now use
brew outdated, participating in Homebrew's normal auto-update schedule. The stale-info regression passes; verified the command's freshness behavior against upstream Homebrew source. - Tapped formula names: the package-specific query now reads the returned entry without incorrectly comparing its qualified name to an unqualified name. Tests cover
grafana/grafana/gcxand both cask name forms.
Verification
The default shell lacked npm, and the checkout had no installed dependencies. Using the installed Node 24.21.0 toolchain, exported the exact head with git archive HEAD into a temporary directory and ran:
npm ci --ignore-scripts— succeeded.npm test— 49 passed, 0 failed.npm run lint— passed.npm run check-format— passed.
All GitHub checks inspected were passing. No real package upgrades, account authentication/configuration changes, or browser installations were performed. Those live integration paths remain unverified locally. The reviewed checkout is unchanged.
Discussion follow-up
GraphQL enumeration returned all 12 review threads and their comments; neither connection required pagination. All were already resolved: 0 unresolved, 0 newly resolved, 0 replies posted. No thread-state mutations were appropriate. Existing discussions: PR review threads.
|
ADVISORY: Ratatoskr's approval of a2014bf confirms the three earlier findings are fixed. Nothing further to address. |
ktaganova
left a comment
There was a problem hiding this comment.
Ran the full flow locally on macOS (Node 24, Homebrew 6) with expect driving the prompts — works as described.
npm test49/49; read-only check lists the expected tools, esq reported current- Decline path (
--gh), current tool (--esq, no prompt), replace-hook path viaconfigureShellEnvwith a discord replace hook (one prompt, none on re-entry) — all ✅ - Real updates (noted as not exercised in the PR):
npm install -g(playwright 1.61.1 → 1.65.0, matching chromium-1254 installed in the same run) andbrew upgrade(sqlcmd 1.9.0 → 1.10.0) both work. Also Go (atl v1.14.0 → v1.15.0, auth untouched) and the gcx GitHub-release download (1.0.0 → 1.7.1, context kept). A re-check afterwards reports all four current.
Offline test note: an HTTP proxy alone doesn't make the machine offline. With url.ssh://git@github.com/.insteadof https://github.com/ in the git config, git ls-remote goes over SSH and ignores the proxy, and Node 24's fetch ignores HTTPS_PROXY unless NODE_USE_ENV_PROXY=1 is set. In my first proxy-only run, only npm was skipped. With NODE_USE_ENV_PROXY=1 plus a dead GIT_SSH_COMMAND, the behavior is correct: a skip warning, brew still reports, the Reinstall (y/N) fallback appears, and the run takes about 14s.
Nothing blocking; three small inline notes.
- Single-tool flags (--gh, --atl, --llm, ...) no longer wait on every lookup; offerUpdate checks the selected tool lazily and warns once if it fails. - The full check hints that Homebrew may refresh its metadata first. - GitHub release versions drop the leading v, matching installed versions.
|
/gemini review |
|
Thanks for the review and for running the real updates.
|
There was a problem hiding this comment.
Code Review
This pull request introduces a comprehensive update-checking mechanism for various CLI tools, implementing version comparisons, latest-version lookups (via Git tags, Homebrew, npm, and GitHub releases), caching, and interactive update prompts. It integrates these checks across multiple installer modules and adds thorough unit tests. The review feedback suggests two key improvements in lib/utils/versions.js: adding a required User-Agent header to GitHub API requests to ensure reliability, and wrapping the tool.isInstalled() check inside the try...catch block in evaluate to handle potential exceptions and undefined tools more robustly.
ktaganova
left a comment
There was a problem hiding this comment.
Verified 85edc28 locally: npm test 52/52, eslint/prettier clean. Homebrew hint shows on the full check; --gh/--esq check only their own tool (~2s instead of the full sweep); offline --discord warns and falls back to the Reinstall prompt; GitHub release versions come back without the v (1.7.1). LGTM.
Summary
Setup now detects outdated CLI tools on every run and offers to update them. Until now, tools were installed only when missing, while their docs were rewritten on every run, so agents read docs for commands the installed binary lacked.
lib/utils/versions.js(new): version comparison (semver incl. prerelease/pseudo-version ordering), time-boxed latest-version lookups, a per-run result cache, and the default-YesofferUpdate()prompt.registerUpdatableTool(). "Latest" is always what that tool's update command would install, so an update cannot leave a tool permanently "outdated":atl,n8nctl,esq,discordctl): installed version fromgo version -m <binary>; latest = highest release tag viagit ls-remote(whatgo install …@latestresolves to), or default-branch HEAD for untagged repos (discordctl, compared by pseudo-version commit). Update = existinggo installflow. When no verdict is possible (no Go toolchain,(devel)build, lookup failed), the old "Reinstall?" prompt (default No) is kept.gh,sqlcmd,hcloud, brew-installedgcx, caskovhcloud):brew outdated --json=v2, which refreshes Homebrew's metadata on its normal auto-update schedule; pinned packages count as current; update =brew upgrade.m365,playwright):npm ls -gvsnpm view <pkg>@latest; update =npm install -g <pkg>@latest. Playwright updates before the browser check so browsers match the new version.gcxin~/.local/bin,ovhcloudvia install.sh): latest GitHub release; update reruns the existing installer.--fullandconfigureShellEnv()inlib/index.jscheck every tool once up front; the status display and tool selection appendupdate available (a → b). Single-tool flags (--gh,--atl, …) check only that tool when it is configured, and--llmchecks nothing.configureShellEnv()offers the update before running a replace hook. environment-setup replacesatl,discordandhcloud, and its discord hook bypassesconfigureDiscordCli().offerUpdateasks at most once per tool per run, so the atl hook re-enteringconfigureAtlassianCli()doesn't prompt twice.GIT_TERMINAL_PROMPT=0and SSHBatchMode, so a passphrase or host-key prompt fails the check instead of hanging setup. Every lookup has a 10s timeout.Acceptance criteria
checkForUpdates()for the menu,--fullandconfigureShellEnv(); single-tool flags check their own toolofferUpdate(),formatUpdateBadge()goToolCheck,pickGoLatest,isGoBuildOutdatedbrewUpgrade,npmInstallLatest; apt/dnf/pacman installs are not checked (no lookup matching what their upgrade installs)bin/cli.js,lib/index.jsatllisted and updated after default-Yes promptgo version -mconfirms), auth untouched; next run reports atl currentreworded 2026-11-12: "dev" builds are checked against the version recorded in the binary and flagged when older than the latest release tagn8nctl,esq,discordctl"dev" builds recognized as outdatedv1.1.3= latest tag, so it is current (#12)Testing
npm test: 52 pass. Newtest/versions.test.jscovers semver/prerelease ordering, pseudo-versions,(devel), tag selection (peeled, v2+ and prerelease tags), the HEAD fallback,brewPackageInfowith a stubbedbrew(incl. malformed output), thecheckForUpdateslisting, skipped-lookup warning and up-to-date summary, andofferUpdateoutcomes with a stubbed prompt: default Yes, one prompt per run after a decline or failure, no prompt when current, unknown or absent.npx eslint .andprettier --check .are clean.fetch/AbortSignalwere added to the ESLint globals (Node ≥ 24 built-ins).brew outdatedfirst runs Homebrew's auto-update, which the check line now mentions):esqv1.1.3 = current;hcloud/ovhcloudnot installed.)⚠ Skipped update check for atl, n8nctl, m365, esq, discordctl, playwright (lookup failed — offline?). Brew-managed tools are still checked from local tap metadata. Worst-case duration is about 11s (npm retries until the timeout).configureShellEnv()status table shows· update available (a → b)on outdated tools.npm start -- --atl, Enter at the update prompt →go install …@v1.15.0, then normal auth check.commit <hash> (<date>).brew upgradeandnpm install -gupdates.