Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,7 @@ For the multi-repository case the model clones what is not checked out locally,

**Scan**

Standard or Deep. The whole repository, selected paths, your uncommitted changes, or everything since a branch or commit. Set a time limit if you want one. Two scans can run at once.
Standard or Deep. The whole repository, selected paths, your uncommitted changes, or everything since a branch or commit. For each scan you can pick the reasoning effort, a cost limit and a time limit. Two scans can run at once.

**Review findings**

Expand All @@ -130,7 +130,7 @@ Hand any finding to the conversation and have the model explain it.
</table>

<p align="center">
<img src="docs/images/desktop-running.png" alt="The Desktop panel while a scan is running, with a progress bar and an estimated cost" width="520">
<img src="docs/images/desktop-running.png" alt="The Desktop panel while a scan is running: the scan list, a progress bar, an estimated cost, and the activity log showing what the scan is doing" width="520">
</p>

## Configuration
Expand Down
4 changes: 4 additions & 0 deletions docs/CONFIG.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,10 @@ Tell it the window your provider gives that model, and when to compact:

Use the limit your provider reports for the model (the error message states it), and set the compaction limit to roughly 80% of it. These are native Codex settings, so they also work in `~/.codex/config.toml`, including inside a profile.

## Settings for one scan

The scan review lets you change the reasoning effort, cost limit, time limit and model for that scan only. In chat, ask for it: "scan this on low effort, stop at $5". Anything you leave alone comes from the files above.

## Time limits

`scan.maxTimeMinutes` sets how long a scan investigates. A Standard scan has no limit by default and treats the value as a target: the model paces itself, and writing the report can run past it. A Deep scan stops discovery at the limit; without one the SDK's own default applies (96 hours). You can also set it for one scan in the scan review.
Expand Down
Binary file modified docs/images/desktop-finding.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/desktop-running.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/terminal-findings.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
13 changes: 13 additions & 0 deletions scripts/ui-preview/main.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -65,11 +65,23 @@ const run = (id, status, focus) => ({
estimatedCostUsd: 0.24,
telemetry: {
activity: [
{
id: id + '-note',
kind: 'message',
status: 'completed',
label: 'Scan update',
detail:
'The strongest candidate so far is a missing tenant check on project updates. Checking whether a member of another workspace can reach it.',
paths: [],
startedAt: new Date(t - 6000).toISOString(),
updatedAt: new Date(t - 5000).toISOString(),
},
{
id: id + '-reading',
kind: 'command',
status: status === 'running' ? 'running' : 'completed',
label: 'Reading source',
detail: 'sed -n "1,260p" src/api/projects.ts',
paths: ['src/api/projects.ts'],
worker: 1,
startedAt: new Date(t - 12000).toISOString(),
Expand All @@ -80,6 +92,7 @@ const run = (id, status, focus) => ({
kind: 'command',
status: 'completed',
label: 'Searching source',
detail: 'rg -n "tenantId" src/api',
paths: ['src/api'],
worker: 2,
startedAt: new Date(t - 14000).toISOString(),
Expand Down
16 changes: 11 additions & 5 deletions scripts/ui-preview/test-hierarchy.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,17 +40,21 @@ async def server_event():
await expect(page.get_by_role('heading', name='Reviewing source', exact=True)).to_be_visible()
await expect(page.locator('progress')).to_have_attribute('value', '3')
await expect(page.get_by_text('Partial estimate USD 0.24', exact=True).first).to_be_visible()
await expect(page.locator('.gs-workers')).not_to_be_visible()
# Activity is open while a scan runs, and leads with the scan's latest note.
await expect(page.locator('.gs-workers')).to_be_visible()
await expect(page.locator('.gs-update')).to_contain_text('strongest candidate')
await expect(page.get_by_text('3 of 10 reviews', exact=True)).to_be_visible()
await shot('hierarchy-running-desktop')

# Keyboard disclosure, then a server event: progress updates and the disclosure stays open.
# Closing it from the keyboard sticks across a server event; reopening works too.
await page.locator('.gs-activity > summary').focus()
await page.keyboard.press('Enter')
await expect(page.locator('.gs-workers')).to_be_visible()
await expect(page.locator('.gs-workers')).not_to_be_visible()
await server_event()
await expect(page.locator('progress')).to_have_attribute('value', '4')
await expect(page.get_by_text('4 of 10 reviews', exact=True)).to_be_visible()
await expect(page.locator('.gs-workers')).not_to_be_visible()
await page.keyboard.press('Enter')
await expect(page.locator('.gs-workers')).to_be_visible()
await page.locator('.gs-info > summary').click()
await expect(page.get_by_text('Measured subtotal only', exact=False)).to_be_visible()
Expand All @@ -69,8 +73,10 @@ async def server_event():
await page.locator('.gs-coverage > summary').click()
await expect(page.get_by_text('Production identity-provider configuration was not available', exact=False)).to_be_visible()

# A running scan opens with only its activity showing.
await select('Review upload handling')
assert await page.locator('.gs-detail details[open]').count() == 0
assert await page.locator('.gs-detail details[open]').count() == 1
await expect(page.locator('.gs-activity')).to_have_attribute('open', '')
assert await page.locator('progress').count() == 0
await expect(page.get_by_role('heading', name='Building threat model')).to_be_visible()
await page.get_by_role('button', name='New scan', exact=True).click()
Expand Down Expand Up @@ -105,7 +111,7 @@ async def server_event():
await page.locator('.gs-coverage > summary').click()
await expect(page.get_by_text('The scan did not report what it reviewed.', exact=True)).to_be_visible()
assert not errors, errors
print(json.dumps({'passed': ['running stage and phase progress', 'collapsed diagnostics', 'keyboard disclosure', 'server event updates progress and keeps disclosure', 'selection resets disclosure', 'findings above fold', 'finding evidence', 'coverage details', 'unknown progress', 'new scan dismissal', 'cancel affects selected scan only', 'narrow no overflow', 'finalization', 'deep consolidation', 'failed scan', 'informational-only result', 'missing coverage', 'no browser errors'], 'screenshots': str(EVIDENCE)}))
print(json.dumps({'passed': ['running stage and phase progress', 'activity open while running', 'keyboard disclosure', 'server event updates progress and keeps disclosure', 'selection resets disclosure', 'findings above fold', 'finding evidence', 'coverage details', 'unknown progress', 'new scan dismissal', 'cancel affects selected scan only', 'narrow no overflow', 'finalization', 'deep consolidation', 'failed scan', 'informational-only result', 'missing coverage', 'no browser errors'], 'screenshots': str(EVIDENCE)}))
await browser.close()


Expand Down
16 changes: 16 additions & 0 deletions src/core/run-telemetry.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@ export interface ScanActivityEntry {
kind: 'command' | 'tool' | 'reasoning' | 'message';
status: 'running' | 'completed' | 'failed';
label: string;
/** What the step said, for the user's own screen. Never returned to the session model. */
detail?: string;
paths: string[];
worker?: number;
startedAt: string;
Expand Down Expand Up @@ -133,6 +135,17 @@ function activityLabel(kind: ScanActivityEntry['kind'], description: string): st
if (executable === 'git') return 'Repository command';
return kind === 'tool' ? 'Tool call' : 'Command';
}
/** One redacted line of what a step said. Opaque tool payloads are reduced to the tool's name. */
function activityDetail(kind: ScanActivityEntry['kind'], description: string): string | undefined {
const text = safeReportText(description, 4000).replace(/\s+/g, ' ').trim();
if (!text) return;
if (kind === 'tool') {
const [name = '', payload = ''] = text.split(' · ', 2);
if (name === 'spawn_agent') return 'Starting a worker';
if (/\S{60,}/.test(payload)) return name;
}
return text.length > 400 ? `${text.slice(0, 399)}…` : text;
}
export function recordActivity(
telemetry: ScanTelemetry,
value: CallbackValue<'onActivity'>,
Expand All @@ -159,6 +172,9 @@ export function recordActivity(
kind: value.kind,
status: value.status,
label: activityLabel(value.kind, value.description ?? ''),
...(activityDetail(value.kind, value.description ?? '')
? { detail: activityDetail(value.kind, value.description ?? '') }
: {}),
paths: safeActivityPaths(value.paths, repository),
...(worker === undefined ? {} : { worker }),
startedAt: previous?.startedAt ?? now,
Expand Down
26 changes: 22 additions & 4 deletions src/core/scan-plan.ts
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,10 @@ export interface ScanInputs {
contextFiles?: string[];
/** Minutes of investigation. Advisory for Standard; a discovery deadline for Deep. */
maxTimeMinutes?: number;
/** For this scan only, instead of the configured model, reasoning effort and cost limit. */
model?: string;
reasoningEffort?: string;
maxCostUsd?: number;
/** Continue an interrupted Deep scan instead of starting a new one. Other inputs are ignored. */
resumeRunId?: string;
}
Expand Down Expand Up @@ -79,6 +83,12 @@ function parseInputs(value: unknown): ScanInputs {
!['undefined', 'string'].includes(typeof data.repository) ||
!['undefined', 'string'].includes(typeof data.scanPrompt) ||
!['undefined', 'string'].includes(typeof data.changes) ||
!['undefined', 'string'].includes(typeof data.model) ||
!['undefined', 'string'].includes(typeof data.reasoningEffort) ||
!(
data.maxCostUsd === undefined ||
(typeof data.maxCostUsd === 'number' && data.maxCostUsd > 0)
) ||
!strings(data.scopePaths) ||
!strings(data.contextFiles) ||
![undefined, 'standard', 'deep'].includes(data.mode as string) ||
Expand All @@ -88,7 +98,7 @@ function parseInputs(value: unknown): ScanInputs {
)
)
throw new SecurityInputError(
'Scan inputs are repository, scopePaths, changes, scanPrompt, contextFiles, mode (standard or deep) and maxTimeMinutes.',
'Scan inputs are repository, scopePaths, changes, scanPrompt, contextFiles, mode (standard or deep), maxTimeMinutes, model, reasoningEffort and maxCostUsd.',
);
if (data.changes && (data.scopePaths as string[] | undefined)?.length)
throw new SecurityInputError('Scan either paths or changes, not both.');
Expand Down Expand Up @@ -215,11 +225,19 @@ export async function prepareScan(
const scanSettings = {
...settings.scan,
...(input.maxTimeMinutes ? { maxTimeMinutes: input.maxTimeMinutes } : {}),
...(input.maxCostUsd ? { maxCostUsd: input.maxCostUsd } : {}),
};
const codex = {
...settings.codex,
...(input.model?.trim() ? { model: input.model.trim() } : {}),
...(input.reasoningEffort?.trim()
? { model_reasoning_effort: input.reasoningEffort.trim() }
: {}),
};
if (purpose === 'validation') delete scanSettings.maxCostUsd;
const changes = input.changes?.trim() || undefined;
const target = await scanTarget({ scopePaths: input.scopePaths, changes });
return withSdk(settings.codex, async (client, sdk) => {
return withSdk(codex, async (client, sdk) => {
const preflight = await client.preflight(repository, {
...scanSettings,
auth: settings.auth,
Expand All @@ -236,11 +254,11 @@ export async function prepareScan(
scanPrompt: input.scanPrompt ?? '',
contextFiles: contextFiles.map((path) => ({ path })),
settings: {
codex: (await routeFrom(sdk, settings.codex, preflight)).route,
codex: (await routeFrom(sdk, codex, preflight)).route,
scan: { ...scanSettings, mode: preflight.mode },
},
globalConfigFile: settings.path,
sdk: { codexOverrides: settings.codex, auth: settings.auth },
sdk: { codexOverrides: codex, auth: settings.auth },
expiresAt: new Date(Date.now() + 10 * 60_000).toISOString(),
mock,
// No undefined members: the host only carries JSON values over RPC.
Expand Down
15 changes: 12 additions & 3 deletions src/gui/RunActivity.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@ import type { ScanState } from '../types';
import {
activityActor,
activityRows,
activityText,
latestUpdate,
activityStatusLabel,
deepProgressLabel,
dispatchLabel,
Expand All @@ -21,8 +23,15 @@ import { clockTime } from './format';
export function RunActivity(props: { run: ScanState }) {
const run = () => props.run;
return (
<details class="gs-section gs-activity" aria-label="Scan activity">
<details
class="gs-section gs-activity"
aria-label="Scan activity"
open={run().status === 'running'}
>
<summary>Activity</summary>
<Show when={latestUpdate(run())}>
<p class="gs-update">{latestUpdate(run())}</p>
</Show>
<Show when={workerSummary(run()) || dispatchLabel(run())}>
<p class="gs-muted">
{[workerSummary(run()), dispatchLabel(run())].filter(Boolean).join(' · ')}
Expand All @@ -41,7 +50,7 @@ export function RunActivity(props: { run: ScanState }) {
{(activity) => (
<li data-activity-id={activity.id}>
<span class="gs-worker-name">{activityActor(activity.worker)}</span>
<span class="gs-worker-label">{activity.label}</span>
<span class="gs-worker-label">{activityText(activity)}</span>
<span class="gs-status" data-status={activity.status}>
{activityStatusLabel(activity.status, run())}
</span>
Expand All @@ -59,7 +68,7 @@ export function RunActivity(props: { run: ScanState }) {
<li>
<time dateTime={activity.updatedAt}>{clockTime(activity.updatedAt)}</time>
<span>
{activityActor(activity.worker)} · {activity.label} ·{' '}
{activityActor(activity.worker)} · {activityText(activity)} ·{' '}
{activityStatusLabel(activity.status, run())}
<Show when={activity.paths.length}>
<span class="gs-mono gs-muted gs-block">{activity.paths.join(', ')}</span>
Expand Down
51 changes: 49 additions & 2 deletions src/gui/ScanComposer.tsx
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import { createSignal, Show } from 'solid-js';
import { createSignal, For, Show } from 'solid-js';
import { asState } from '../types';
import type { ScanInputs, ScanPlan } from '../core/scan-plan';
import { splitPaths, userError } from '../scan-display';
import { reasoningEfforts, splitPaths, userError } from '../scan-display';
import { ScanReview } from './ScanReview';
import type { SecurityPort, SecurityStore } from './store';

Expand Down Expand Up @@ -41,6 +41,9 @@ export function ScanComposer(props: {
initial?.contextFiles.map((file) => file.path).join('\n') ?? '',
);
const [minutes, setMinutes] = createSignal(initial?.inputs.maxTimeMinutes?.toString() ?? '');
const [effort, setEffort] = createSignal(initial?.inputs.reasoningEffort ?? '');
const [cost, setCost] = createSignal(initial?.inputs.maxCostUsd?.toString() ?? '');
const [model, setModel] = createSignal(initial?.inputs.model ?? '');
const [preview, setPreview] = createSignal<ScanPlan | undefined>(
props.edit ? undefined : initial,
);
Expand Down Expand Up @@ -73,6 +76,9 @@ export function ScanComposer(props: {
contextFiles: splitPaths(documents()),
...(mode() !== 'default' ? { mode: mode() as 'standard' | 'deep' } : {}),
...(Number(minutes()) > 0 ? { maxTimeMinutes: Number(minutes()) } : {}),
...(effort() ? { reasoningEffort: effort() } : {}),
...(Number(cost()) > 0 ? { maxCostUsd: Number(cost()) } : {}),
...(model().trim() ? { model: model().trim() } : {}),
};
const next = asState(await props.port.prepare(input));
if (!next.plan) throw new Error('The server did not return a scan preview.');
Expand Down Expand Up @@ -141,6 +147,33 @@ export function ScanComposer(props: {
</select>
<span class="gs-help">Deep takes longer and costs more.</span>
</label>
<label>
Reasoning effort
<select
value={effort()}
onChange={(e) => setEffort(e.currentTarget.value)}
aria-label="Reasoning effort"
>
<option value="">Default from settings</option>
<For each={reasoningEfforts}>
{(level) => <option value={level}>{level}</option>}
</For>
</select>
<span class="gs-help">Lower is faster and cheaper; higher looks harder.</span>
</label>
<label>
Cost limit <span class="gs-muted">US dollars, optional</span>
<input
type="number"
min="0.01"
step="any"
value={cost()}
onInput={(e) => setCost(e.currentTarget.value)}
placeholder="Default from settings"
aria-label="Cost limit in US dollars"
/>
<span class="gs-help">Stops the scan when its estimated cost passes this.</span>
</label>
<label>
Time limit <span class="gs-muted">minutes, optional</span>
<input
Expand Down Expand Up @@ -209,6 +242,20 @@ export function ScanComposer(props: {
<Show when={scope() === 'uncommitted' || scope() === 'since'}>
<p class="gs-help">Change scans run at Standard depth.</p>
</Show>
<details class="gs-disclosure" open={!!model()}>
<summary>Model</summary>
<label>
Model
<input
value={model()}
onInput={(e) => setModel(e.currentTarget.value)}
placeholder="Default from settings"
aria-label="Model"
spellcheck={false}
/>
<span class="gs-help">For this scan only. Your provider must offer it.</span>
</label>
</details>
<details class="gs-disclosure">
<summary>Context files</summary>
<label>
Expand Down
15 changes: 15 additions & 0 deletions src/rpc.ts
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,21 @@ export const scanInputSchema = {
description:
'Minutes to spend investigating. Standard scans have no limit unless this is set, and treat it as a target; Deep scans stop discovery at it.',
},
model: {
type: 'string',
description: 'Model for this scan only. Omit to use the configured model.',
},
reasoningEffort: {
type: 'string',
description:
'Reasoning effort for this scan only, such as low, medium, high or xhigh. Lower is faster and cheaper. Omit to use the configured effort.',
},
maxCostUsd: {
type: 'number',
exclusiveMinimum: 0,
description:
'Stop this scan when its estimated cost passes this many US dollars. Omit to use the configured limit, if any.',
},
resumeRunId: {
type: 'string',
description:
Expand Down
Loading
Loading