Skip to content

Preserve leading U+FEFF in decoded strings - #70

Merged
blikblum merged 2 commits into
masterfrom
copilot/fix-decode-stream-leading-bom
Nov 17, 2026
Merged

blikblum merged 2 commits into
masterfrom
copilot/fix-decode-stream-leading-bom

Conversation

@blikblum

@blikblum blikblum commented Nov 17, 2026 •

Copy link
Copy Markdown
Member

DecodeStream.readString() strips a leading U+FEFF from UTF-8 and UTF-16 strings, preventing encoded strings and binary fields from round-tripping correctly.

  • Decoding: Configure TextDecoder to preserve leading U+FEFF.
  • Coverage: Add explicit-byte and round-trip cases for UTF-8, UTF-16LE, UCS-2, and UTF-16BE.
const string = new r.String(r.uint8, 'utf16be');
string.fromBuffer(string.toBuffer('\uFEFFA')); // '\uFEFFA'

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
@blikblum blikblum changed the title [WIP] Fix DecodeStream.readString to preserve leading U+FEFF Preserve leading U+FEFF in decoded strings Nov 17, 2026
@blikblum
blikblum marked this pull request as ready for review November 17, 2026 05:23
@blikblum
blikblum merged commit a2b2a99 into master Nov 17, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

DecodeStream.readString strips a leading U+FEFF from UTF-8 and UTF-16 strings

2 participants