Skip to content

fix(tunnelconfig): prune rules removed from Ingresses - #50

Open
etomm wants to merge 1 commit into
StringKe:mainfrom
etomm:fix/tunnel-prune-removed-ingress-paths
Open

etomm wants to merge 1 commit into
StringKe:mainfrom
etomm:fix/tunnel-prune-removed-ingress-paths

Conversation

@etomm

@etomm etomm commented Nov 18, 2026

Copy link
Copy Markdown

Fixes the stale-rule behaviour described in #49, where a rule for a path removed from an Ingress stays in the tunnel configuration and the edge keeps returning 502 for it.

The plain rebuild path does prune a removed path, so this PR addresses the failure modes that make the stale rule stick permanently in the aggregated Ingress/<ns>/<class-config> source.

Root causes

  1. Lost update in the tunnelconfig controller. updateConfigMap wrote the whole configuration snapshot it had parsed at the start of the reconcile back into the ConfigMap. If the Ingress controller pruned a source while the push was in flight, the tunnelconfig controller reverted the prune and recorded its own hash as Synced. Every later reconcile then short-circuited on LastHash == ComputeHash() and SyncStatus == "Synced", so both the ConfigMap and Cloudflare kept the removed rule indefinitely. This matches the "same version reported across many reconciles" observation in the issue.
  2. The hash included bookkeeping fields. ComputeHash hashed Sources including per-source UpdatedAt, so it changed on every write and could not be used to detect actual content changes reliably.
  3. A deletion observed as "object is gone" was a no-op. handleIngressDeletion returned immediately, so when the finalizer had already been removed (or the delete event was missed) nothing rebuilt the aggregate and there was no later event to fix it.
  4. An empty aggregate could not be pushed. The catch-all rule was only appended when at least one rule existed, so an aggregate that lost all its sources produced an invalid ingress list and the previously pushed configuration stayed active.

Changes

  • internal/controller/tunnelconfig/controller.go
    • persist only the sync bookkeeping fields, on a freshly read ConfigMap, instead of writing back a stale payload;
    • always terminate the pushed ingress list with the http_status:404 catch-all so the configuration is valid even with zero sources.
  • internal/controller/tunnelconfig/types.go
    • ComputeHash now hashes only the effective configuration (in stable source order) and ignores UpdatedAt, Generation and the sync status fields.
  • internal/controller/ingress/controller.go
    • handleIngressDeletion rebuilds every TunnelIngressClassConfig from the live Ingresses;
    • a failed rebuild now sets State=error and Ready=False on the TunnelIngressClassConfig instead of leaving Ready=True with stale rules.

Tests

New tests cover:

  • pruning the aggregate when a path is removed from an Ingress;
  • rebuilding after a deletion that was only observed as "object is gone";
  • a failed rebuild surfacing as an error condition on the class config;
  • the status update not reverting a concurrently written (pruned) source;
  • hash stability against bookkeeping writes and hash change on rule removal;
  • the catch-all being present when all sources are gone.

go build ./..., go vet and the affected package tests pass. The envtest suite under internal/controller was not run here because the etcd binaries are not available on my machine; this branch does not touch it.

Notes

The issue also asks whether an Ingress update that only removes a path triggers a rebuild: it does - the watch has no predicates and the aggregate is rebuilt on every reconcile. The permanent staleness came from the defects above, not from a missing rebuild trigger.

Refs #49

Removed Ingress paths could stay in the pushed tunnel configuration forever
(edge 502 on a dead route). Three independent defects contributed:

- The tunnelconfig controller wrote the whole configuration snapshot it had
  parsed at the start of the reconcile back into the ConfigMap, reverting any
  source updated concurrently by the Ingress controller. The reverted snapshot
  then matched the recorded LastHash, so every later reconcile short-circuited
  and the stale rule was never pushed away again. Only the sync status fields
  are now written, on a freshly read ConfigMap.
- ComputeHash included per-source UpdatedAt/Generation, so it changed on every
  bookkeeping write and could not reliably detect content changes. It now hashes
  only the effective configuration (in stable source order), so a removed rule
  always changes the hash and an unchanged aggregate does not force a push.
- Deletions observed only as "object is gone" (handleIngressDeletion) were a
  no-op, leaving the aggregate of every class config stale with no event left to
  rebuild it. Every TunnelIngressClassConfig is now rebuilt from the live
  Ingresses.

Also surface rebuild failures on the TunnelIngressClassConfig (State=error and
Ready=False) instead of leaving Ready=True with stale rules, and always end the
pushed ingress list with the catch-all rule so an aggregate that lost all its
sources still produces a valid configuration.

Related to StringKe#49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant