Repository navigation
Guard source captures against visitor bot-check tokens - #10830
Merged
Merged
Conversation
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Hex digits, "-", "$" and "." encode only to base64 letters and digits, so a UUID-dollar-IPv4 token never contains "+" or "/". The capture guard counted both as token characters and missed a token beside them: in a URL path, after a JSON-escaped slash, or after a plus sign. The token class now holds letters and digits only. The encoded-token test wraps each boundary token in ten page contexts, and two near misses, each one character short of a token, pass in every context. The privacy property samples the same contexts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A percent-encoded URL puts "%2F" or "%3D" beside a token. The escape's hex digits join the token's letters and digits, so the guard missed the token. The guard now also scans each capture after one round of percent-decoding and reports a token found in either form once. The encoded-token and near-miss tests gain a percent-encoded URL path and query. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
State in the capture README which runs the guard decodes, where it finds tokens, and what it misses: IPv6, payloads with extra decoded bytes, and tokens joined to other letters or digits. Add a raw query string to the encoded-token contexts, since the README names it. Join a hard-wrapped CONTRIBUTING paragraph into one line to match its neighbours. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis
marked this pull request as ready for review
November 16, 2026 03:52
The README's example of a token joined to letters or digits named a bare slash, which the guard treats as a boundary. It now names the JSON Unicode escape for a slash, whose last hex digit joins the token. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The guard matches bot-manager markers in a capture's raw bytes only, and base64 visitor tokens in the raw bytes and after one round of percent-decoding. Move the percent-decoding clause in CONTRIBUTING.md and the guard's docstring so it covers tokens alone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Author
|
US + core hub merge audit, us#10830 at
🤖 Generated with Claude Code |
This was referenced Nov 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Guard source captures before contributors publish visitor bot-check data. The check scans every file under
source_conflicts/sources/for bot-manager markers and base64 UUID-dollar-IPv4 tokens. Diagnostics give the path and match kind without captured values.When a site serves bot-check scripts, contributors should extract pandoc text and include the source URL and extraction method.
Invariants and validation
+and/as at any byte outside the base64 alphabet, in the raw bytes and after one round of percent-decoding. Hex digits,-,$and.encode only to base64 letters and digits, so a token never contains+or/.Narrowing the base64 window to 64–67 characters before padding fails every complete-token regression for the shortest and longest payloads. Restoring
/and+to the token alphabet fails the URL-path, plus-sign and escaped-slash regressions. Dropping the percent-decoded scan fails the percent-encoded URL regressions. Removing the pattern's lookbehind or its lookahead leaves all 146 capture tests passing, so no regression pins the rule that a decoded run has no letter or digit on either side and no further=after it.test_source_conflicts.pycollects 251 tests, 146 of them for this check, and the property file caps each of its two properties at 32 examples. CI's Rest Python shards run both files.Pull request run 43910613214 at
8043062a4fpassed all 37 jobs on GitHub's merge with main8e1837790b.Methodology choices
__cf_chland_cf_chl_opt.=after their padding and tokens joined to other letters or digits, such as after a\u002Fescape or in a URL percent-encoded twice, remain outside this check.Microsimulation impact
This PR changes a code-health test file, a property test, capture guidance and a changelog fragment, and no formula, parameter or policy test, so it changes no model output.
impact: none: this PR changes no formula, parameter or policy test
axiom: n/a: this change adds source capture privacy checks and guidance without changing encoded provisions.
🤖 Generated with Claude Code