Skip to content

feat(security): audit log detections, alert rules that count audit events - #2532

Open
Makisuo wants to merge 4 commits into
audit-trail/vercel-gcpfrom
audit-trail/detections
Open

Makisuo wants to merge 4 commits into
audit-trail/vercel-gcpfrom
audit-trail/detections

Conversation

@Makisuo

@Makisuo Makisuo commented Nov 12, 2026 •

Copy link
Copy Markdown
Collaborator

Fourth layer of the unified audit log, on top of #2530: detections. An org admin can alert on the audit log.

What a detection is

"N or more events matching these filters arrived in the last W minutes." It is an alert rule of a new managed signal type, audit, so it gets the alert pipeline as it is: incidents, severities, destinations, silences, the Alerts hub inbox.

It is managed the way an uptime monitor's or an SLO's rule is. It cannot be created, edited or listed through the generic alert rule APIs, the alert editor, the MCP tools or the infrastructure-as-code provider. It is written only through new endpoints on the audit trail API, and those need audit_log:read, which only org admins hold.

What it may and may not carry

Alert rules and incidents are synced to every member's browser and sent to shared destinations, so a detection is built to hold nothing a member may not see:

  • Filters are provider, category, outcome and an exact action. There is no actor or IP filter, so no rule names a person.
  • There is no grouping, so no incident key or notification title carries an email or an address.
  • An incident and its notifications say how many events matched and over which window. They never include an event. The incident page links to the audit log, which only admins can open.

What a member does see: the detection's name, its threshold and that it fired.

How it counts

  • Events are counted by when they arrived (IngestedAt), not when they happened, so an event a provider delivers late still alerts. OccurredAt is bounded to a week before the window so partitions prune.
  • Distinct event ids, so a redelivery inside the window counts once.
  • The count is a fact even at zero: an empty window is healthy, not "no data", so nothing is held open by the liveness probe.
  • Checked every minute. One check at the count opens the incident and one below it resolves it; a reminder goes out once a day while it stays open.

Changes

  • Domain: audit in AlertSignalType and MANAGED_SIGNAL_TYPES; AuditDetectionFilters; the detection wire schemas and four endpoints under /internal/audit-trail/detections.
  • Postgres: one nullable column, alert_rules.audit_filters_json (migration 20261112150508_alert_rules_audit_filters).
  • Warehouse: auditTrailCountQuery, routed to the managed warehouse like the list query.
  • Backend: AuditDetectionsService (list, get, create, update, remove; 50 per org; destinations and the name checked under the org's alert rule lock), AuditTrailService.count, and a branch in AlertsService.evaluateRule beside the other managed types.
  • API: handlers gated on audit_log:read; each change is recorded in Maple's own audit log as alert_rule.created/updated/deleted.
  • Everything that enumerates signal types now knows audit: labels, the v2 read-scope switch, the incident page, the iOS formatter and the generated iOS OpenAPI file, the webhook reference.

There is no page for this yet. The Detections page in the Security hub is the next layer.

Worth knowing before merging

  1. Older iOS builds. The generated Swift enum is closed, so an installed build fails to decode a /v2 response that contains an audit incident. anomaly and slo_burn_rate had the same exposure when they were added. Nothing creates an audit rule until the page ships.
  2. Deleting a detection deletes its incidents with it, open ones included, and sends no "resolved" notification. That is what deleting an SLO or a rule does today.
  3. A redelivery that arrives after the window has passed counts again, because it has a later arrival time until the table merges it away. The effect is a second incident for the same event. Providers redeliver within seconds in practice.
  4. Migration. Additive and nullable; the prd deploy applies it.

Verified

  • Backend: the audit trail folder and AlertsService.audit.test.ts, 56 tests; 25 alert, audit and anomaly files, 342 tests; the service against PGlite covers create, list, update, remove, the cap, an unknown destination and a duplicate name.
  • API: the audit trail routes, 16 tests (a non-admin gets the forbidden error on every endpoint, an admin round-trips a detection); the v2 read-scope switch, 15; the alert routes that build AlertsService, 62.
  • Domain 1240 tests, db parity and migrations 17, query-engine catalog and SQL baseline, web-data alerts and collections 203, the incident page scene tests 18, the MCP descriptor and contract tests.
  • gen:anticipated-errors:check, ios:openapi:check and ui:patterns:check are up to date. oxlint -c .oxlintrc.effect.json is clean on the changed files. bun typecheck passes, 57 of 57.

Not verified: the Swift change is not compiled. The count query has not run against a real ClickHouse, only compiled and decoded through the fake warehouse.


Autofix with [code]smith
Open in [code]smith · Need help on this PR? Tag @codesmith-bot with what you need.

…n and the count query

An audit log detection is a managed alert rule with signal type audit: N or more
matching events arrived in the last W minutes. This adds the wire schemas and
endpoints on the audit trail API, the nullable audit_filters_json column on
alert_rules, and auditTrailCountQuery, which counts distinct event ids by
IngestedAt and bounds OccurredAt so partitions prune.
…ents

A detection is a hidden alert rule with signal type audit, written only through
the audit trail API by callers who hold audit_log:read. AuditDetectionsService
owns the rule row the way an SLO owns its burn-rate rule: destinations and the
name are checked under the org's alert rule lock, an edit ends the tick's claim,
and a delete takes the rule's state, deliveries and incidents with it.

The evaluator counts the events that arrived in the rule's window through
AuditTrailService.count. A count is a fact even at zero, so an empty window is
healthy and resolves, and a count that could not be read is an evaluation
failure that leaves an open incident open. Notifications carry the count as
their detail and never an event.
The rule fields and the row read back; the detections service over PGlite
(create, list, update, remove, the cap, an unknown destination, a taken name,
another org's ids); the count query through the fake warehouse; a scheduler
tick that stays healthy below the count, opens at it, resolves below it and
holds an open incident while the count cannot be read; and the routes, where a
caller without audit_log:read is refused on every endpoint.
@coderabbitai

coderabbitai Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 1a376441-67db-4395-8801-72d94655c0d1

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@maple-review-bot

This comment has been minimized.

@maple-review-bot maple-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 inline note from Maple's review. The score and summary are in the review comment above.

… who can open it

Every member can read an incident, and the audit log needs audit_log:read.
A member who followed the link was sent home with no explanation. The note
now says only org admins can open the log, and the link shows for them.
@maple-review-bot

maple-review-bot Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

Maple review

🟢 Confidence 10/10 · safe to merge. Detections are org-scoped, admin-gated and covered by service, HTTP, evaluator and scene tests; the earlier wrong-link defect is fixed and pinned.
no findings · tests covered · risk medium · 2/2 new units observable

Adds audit log detections: an audit managed signal type whose hidden rule counts matching events, four admin-only endpoints on the audit trail API, a warehouse count query and the incident-page copy. The earlier defect — a link that sent members home — is fixed and tested; safe to merge.

  • audit joins AlertSignalType and MANAGED_SIGNAL_TYPES, so every generic rule API refuses it
  • AuditDetectionsService writes the hidden rule, capped at 50 per org
  • AlertsService.evaluateRule counts distinct arrived events and returns before the query path
  • Incident page offers the audit log link only to audit_log:read holders

Before merge

Fixed since the last review

  • ✅ F1 · Open audit log sends non-admin members home from an audit incident

c592466 · What was checked, coverage and telemetry details are on the Maple / review check. Reply "won't fix" to dismiss a finding, or mention @maple-review-bot to ask about one.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant