Repository navigation
Fix data-loss bugs from code review; speed and size optimizations - #11
Merged
Merged
Conversation
9bdd392 set IS_DIRTY inside run_module_by_index, which load and save also use for MODDET/MODTOK/MODDIS, so every BASIC or ML load and every BASIC save left the buffer marked unsaved. Mark it in run_selected_module () and do_search_replace (only when MODSFR reports an edit) instead. Also from the same path: - MODDSK's open-file return now asks about unsaved changes first; the chosen name goes to a pending buffer so a save from that prompt still uses the document's own name. - A module $01 error is no longer shown twice. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With an empty replacement, a second RETURN deleted the match, so a plain find-next destroyed text. RETURN now replaces only from the REPLACE field (an empty replacement still deletes there); from the FIND field it steps to the next match. The duplicate empty-replace path is gone, since sfr_replace_at_match already handles length 0. Report edits through MOD_SFR_EDITED so the editor marks the buffer dirty only on a real edit. Also fix sfr_move_gap case A, which moved the block but left GAP_END unchanged (latent: the search never moves the gap backwards today). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Cancelling left MOD_STATUS at $FF, which the editor shows as MODULE ERROR. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MODTOK stores the text after REM (to end of line) and DATA (to an unquoted ':') untokenized, but MODDET expanded every $80-$D8 byte as a keyword. Each BASIC save runs MODTOK, writes, then MODDET, so graphics or shifted letters in a REM or DATA line turned into keywords on every save (10 REM <shift-*> became 10 REM TAN), permanently. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
After MODDIS, load_run_moddis wrote 25 sentinel CRs and set GAP_START = GAP_END = work_buf. That made the whole buffer the document: the disassembly, the CRs, then stale bytes. With no gap every keystroke was dropped, and F5 saved the junk too. run_module_by_index already places the gap at MOD_NEW_END..work_buf_end, as for a BASIC load; keep that. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ists Pass 2 rewrites in place, with the output's head start over the staged input equal to the free gap. When numbers grow (lines 1,2,3 -> 10,20,30) in a nearly full buffer, the output passed the read position and overwrote source not yet read, with no error. Pass 2 now runs once without writing and checks every output byte against the read position. If it would catch up, the staging copy is undone and the module returns $01 with the buffer unchanged. renumber_ref also continues past ',' so ON X GOTO/GOSUB a,b,c renumbers every target, not just the first. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
BSS isn't cleared at load, so COMPL_ACTIVE started as whatever was in RAM. If CTRL+I was the first key and that byte was non-zero, completion took the cycling path with a garbage token and rewound the gap by a garbage length, below work_buf. Also run load_settings after init_settings in both start paths (it's a stub today, but the old order would have discarded its values). Note: editor.asm also gains the FNAME_STASH/DSK_PEND buffers used by the loadsave/modules fixes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… a failed load Save: a serial drive reports write protect, disk full and similar errors only on channel 15, so the save looked clean and cleared IS_DIRTY. Read the status after CLOSE and treat 20+ as an I/O error, leaving the buffer dirty. Also don't append the final CR past work_buf_end when the buffer is exactly full. Load: IS_BASIC and IS_NEW_FILE were cleared before OPEN, and the filename the prompt overwrote was never restored. So a mistyped F3 on a BASIC document turned it into plain text under a bad name, and the next save wrote untokenized text. Each success path already sets the flags; a failure before any byte arrives now restores the name and leaves the document alone. A failure part-way keeps what arrived as an unnamed, dirty text document, so a save asks for a name. The name stash gets its own buffer, since IO_NAME_BUF is rebuilt for the F6 open. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The output file was opened with a bare name, so re-assembling to an existing name failed with 63 FILE EXISTS, reported only on the error channel, and the old file stayed while the assembler said OK. Disk full and write protect were likewise invisible: emit_byte masked READST with the read-side bits ($42). Open as "@0:name", mask the write-side bits ($83), and read channel 15 after CLOSE; codes 20+ give I/O ERROR. The test harness's fake drive now models the error channel, the @0: prefix and a full disk. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
After an overflow, emit_byte freezes DST_PTR at the relocated source, so every later line's LINK_PTR pointed into the user's text, and the patch overwrote its first two bytes. modtok then restored that damaged copy, so a too-large BASIC save also corrupted the start of the buffer. modsct also dropped a final line that had no trailing CR: it was never terminated, and its $0000 link ended the program. It now closes the line the way modtok's @eof_line does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
tests/test_modules.py runs MODTOK, MODDET and MODREN against a gap buffer: the tokenize round trip (REM, DATA, strings), the overflow restore, ON...GOTO lists, a split gap, and the renumber overtake guard. CI never ran tests/test_include.py either; build.yml now runs both. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MAIN ran to $D000, so the editor plus its 24 K BSS buffer could grow into MOD_HI ($A000) or MOD_LO ($C000) and still link; the first module load would then overwrite it. It ends at $8CD3 today. Cap it at $A000 so overgrowth fails the link instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
expand_token, expand_extended_token, tokenize_line and tl_match_kw had no callers anywhere in the tree (tokenizing moved to the MODTOK/MODSCT modules; the viewport renders plain text). EXT_X_SAVE, POP_CHAR_TMP, MOD_LOADED and SETTING_TOKENIZE_SAVE were written or declared but never read, and NMI_VEC/NMI_VEC_HI were never used. The kw_strtab/kw_len_tab tables stay: colorize.asm's col_match_kw still uses them. Also drop the jmp rv_next_row that jumped to the very next instruction. Saves 464 bytes of image on both C64 and C128 (plus 4 BSS bytes). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…once before the read loop The GAP_START = <ptr>, GAP_END = work_buf_end sequence was spelled out eight times. It is now one subroutine with two entries (gap_at_buf_ptr loads BUF_PTR into X/A, gap_at_xa takes the pointer in X/A); tail positions jmp to it and load_run_moddis's fallback falls into it. The three load success paths (F6 source, BASIC, text/binary) all cleared IS_BASIC/IS_NEW_FILE/IS_DIRTY and set the gap from BUF_PTR first, so that is done once after CLOSE and only the BASIC path sets IS_BASIC. None of the call sites relied on X afterwards. The READST at the top of @read_loop could only be non-zero on the first pass: the loop branches back only after the post-CHRIN READST returned 0 and nothing in between touches ST. It now runs once before the loop, so an empty file / immediate EOF or error still reads nothing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- do_new_file, setup_screen_blank: reset the gap via gap_at_xa. - do_new_file: drop the render_status call; modal_exit repaints the status row after the viewport anyway. - warm_start: drop lda BOOT_FNAME_LEN / beq editor_ready, which fell through to editor_ready either way. - setup_screen: it duplicated setup_screen_blank (screen clear, gap, TOP_LINE, LEFT_COL, IS_BASIC); call it, then copy the test text and set GAP_START from BUF_PTR. - modal_exit is now a second label on render_status instead of a jmp. - init_settings: one lda #0 for FNAME_LEN/IS_BASIC/IS_DIRTY. - settings_write_label: X was never changed between stx/ldx SAVED_X, the second ldy POP_LBL_IDX was a repeat, and clc/adc #0 was a no-op. SAVED_X is only ever read right after its own store elsewhere. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The SRC_STACK + depth*6 frame-base multiply was copied five times (.include push, src_at_end, skip_to_next_line, fill_line_buf, pop_src_frame). Factor it into src_frame_ptr / src_top_frame_ptr, which return the frame base in TMP2 and preserve X and Y (the .include push keeps the filename length in X across it). The frames never cross a page (now asserted), so the hi byte is a constant and the old carry propagation goes away. src_at_end and skip_to_next_line used TMP3 for the pointer; they now use TMP2 like the others. Both are called only from run_pass's line loop, where TMP2 is not live, and skip_to_next_line already clobbered TMP2 through fill_line_buf on the same (depth > 0) path. Saves 63 bytes; no behavior change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
col_match_kw ran at every word start, filled the 9-byte lookahead and then
walked all 89 kw_strtab entries, so plain identifiers like A, I or X$ paid
for the whole table. That scan was the bulk of colorize_row's cost per row.
Only keywords that start with the current letter can ever match, so the
scan now walks just that letter's group from a first-letter index:
kw_let_start (26 bytes, $FF for letters no keyword starts with) points into
kw_cand_tok/kw_cand_off, which list the keywords grouped by first letter and
in token order within each group. Those are exactly the entries the old scan
could have matched, in the same order, so the first full match (INPUT#
before INPUT, GOTO before GO, the trailing-letter rejection) is unchanged.
A first-char mismatch ends the group; a sentinel entry ("+") closes the
last one. Letters with no keyword (B H J K Q X Y Z) return before the
lookahead fill.
The index is generated at assemble time: kw_strtab entries now go through a
KW macro that emits the same bytes as before and records each entry's
offset and first char, so the index cannot drift from the table. kw_strtab
and kw_len_tab (shared with tab completion and tl_match_kw) are byte-for-byte
unchanged.
Cost: 190 bytes of tables plus 18 bytes of code. A typical 24-row BASIC
listing drops from about 17.3K to 7.1K cycles per row (py65). A py65 harness
comparing old and new builds over about 26K rows (every keyword, its
prefixes and suffixes, gap positions, mid-line starts, horizontal scroll,
random bytes) on both C64 and C128 builds gave identical color RAM.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
find_mapping ran a linear scan of the old->new table for every line header and every GOTO/GOSUB/THEN target, and pass 2 now runs twice (dry rehearsal + rewrite), so on a 400-line program the lookups were most of the run time. The table is built in source order, so whenever the program's line numbers are strictly ascending it is sorted and duplicate-free. Pass 1 now tracks that (store_entry compares each old number with the previous one) and find_mapping binary-searches in that case. Out-of- order or duplicate numbers fall back to the unchanged linear scan, which returns the FIRST matching entry; with unique sorted keys the binary search finds that same entry, so the output is identical either way. The search works on byte offsets that are multiples of 4, so mid = ((lo + hi) / 2) & ~3 needs no index->offset shifts. 399-line test (py65): 6.27M -> 1.62M steps, 18.07M -> 5.44M cycles. Code grows 167 bytes; the C128 build still links with 12 bytes to spare. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
sfr_find_next recomputed limit = total - find_len on every outer step (sfr_vbyte clobbers sfr_tmp, where it lived) and folded the pattern character on every comparison. Now the limit is computed once into its own variable, the pattern is folded once per search into sfr_pat_buf, and the first character is compared before entering the inner loop, so the common miss skips the index bookkeeping. Matching is unchanged: fold(buf) is compared with fold(pat) through the same sfr_fold. sfr_flash_found / _not_found / _full were the same code with a different message and color; they are now one sfr_flash taking the message offset in X and the color in A. Search over a ~6 KB program (py65, whole module run): 1.74M -> 1.20M cycles case-insensitive, 1.46M -> 1.06M case-sensitive. Code shrinks 32 bytes on both targets. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
run_module grows a keys= option: GETIN ($FFE4) is trapped and returns the scripted PETSCII codes one per call (then STOP), and the jiffy clock at $A2 ticks so the status flash's timed wait ends. The new search/replace test covers find, find-next, the cursor as the search start, case folding and F1, not-found, a pattern longer than the text, replace, delete-by-empty-replacement and the gap-full guard. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each of the 12 set_err_* entry points was a 10-byte copy of "load the message address into TMP, JMP set_err_common". The messages are now one contiguous block (err_msgs, byte-for-byte unchanged), so an entry only has to name its message's offset: LDA #offset, then a $2C byte that turns the next entry's LDA into the operand of a BIT abs, falling through to set_err_common, which forms TMP = err_msgs + offset itself. The BIT only reads RAM at $xxA9 with xx < $A0 (asserted), so it has no side effects on either target. The offset travels in A, which the stubs always clobbered; X stays untouched and Y is still clobbered only when the error is actually recorded. ASM_ERR is only ever $00 or $FF, so set_err_common tests it with BIT + BMI to keep A, preserving the first-error-wins rule. Saves 83 bytes. A differential run of 25 sources (every error message, first-error-wins with two errors, include errors, a full symbol table) against the previous build gives identical status, message, line, output file and screen output. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
sfr_fold tested $7B before $C1, so every shifted letter ($C1-$DA, how the editor stores SHIFT+letter) took the "above lowercase" exit unfolded, and case-insensitive search never matched shifted text, despite the comment saying it did. Test the shifted range first. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SRC_DEPTH, SRC_STACK, LINE_BUF_END and LINE_BUF live at hard-coded $BF00-$BF91, inside the $A000-$BFFF MAIN area modasm.cfg gives the code, so nothing stopped the image from growing over them: the first .include would then overwrite code. Mark the end of the image (MODASM_CODE_END) and assert at link time that it stays below SRC_DEPTH. Checked by padding the code so it ends at $BF3C: ld65 now stops with the assert instead of linking. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The 13-way cmp/bne/jmp chain on the addressing mode becomes an RTS-trick jump table (mode_tab only holds 0-12, so the unreachable fall-off case goes). The runs of `jsr emit_dst` used for padding become emit_spaces (count in X), the handlers share their "pad N and finish" tails, the three absolute-operand handlers share op_dollar_word, and ZP,X / ZP,Y share one body. Module shrinks 2312 -> 2011 bytes (C64), 2340 -> 2032 (C128). Output is unchanged: old and new builds were run on py65 over every opcode (alone, with six operand pairs and four load addresses, plus one stream of all 256), 300 random binaries (some split across the gap) and 20 output-overflow runs, for both targets (C128 with FAR_API stubs); status, MOD_NEW_END, the output text and all memory outside the module image matched in all 6465 cases. Also correct the spinner comments: it toggles every 16 instructions between white ($01) and black ($00), not every 256 / $01 XOR $07, and emit_dst clobbers Y rather than preserving everything. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Block counts: dsk_print_decimal (blocks free) only handled up to 999 - from 1000 the hundreds digit ran past '9' into ':' and beyond - and dsk_print_blk_count (per file) ignored the high byte entirely, so a 300-block file showed as 44. Both are replaced by dsk_print_u16, one 16-bit routine that prints 0-65535 right-aligned in a 5-column field ending where the old 3-digit field ended (col 31 for blocks free, col 22 for files), so counts below 1000 sit where they did. Leading zeros are now blanks: 0-9 print as " 5" instead of the old " 05". Format ID: DSK_INPUT_BUF was 16 bytes but the format prompt stores the 2-char disk ID at +16/+17, overwriting DSK_TRUNCATED and the next byte. It is now 18 bytes. That next byte, DSK_ERROR_CODE, was only ever used as an X spill in dsk_draw_one_entry and is no longer needed (below), so it goes and BSS grows by one byte net. Dedupe: dsk_cache_ptr is now `lda DSK_ENTRY_COUNT` falling into dsk_entry_ptr, which no longer uses DSK_TMP (dsk_read_entries keeps the block count there across the call). dsk_draw_one_entry computes the screen row pointer once and derives the colour RAM pointer by adding >(COLOR-SCREEN) to the high byte (asserted page-aligned) instead of running the x40 multiply three times, and clears and colours the row in one loop; dsk_color_row goes. CODE shrinks 3253 -> 2990 bytes, the PRG 3255 -> 2992 (C64) / 3250 -> 2987 (C128). Checked on py65 for both targets: dsk_print_u16 called directly for 0, 9, 10, 99, 100, 300, 664, 999, 1000, 1328, 9999, 10000, 65535 and every 7th value up to 65535; old vs new dsk_draw_one_entry over 1500 random cache/selection/viewport states (screen and colour RAM identical except the block-count field, which differs only for counts the old code got wrong or zero-padded, and is correct); old vs new dsk_draw_info for blocks free 0-1100 and large values (identical except cols 27-31, same rule). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
sym_lookup walked every 10-byte SYM_TABLE entry (about 50 cycles each, up to 400 of them) for every label reference in both passes and for every definition's duplicate check. On a 300-label, 900-reference source that was half of all assembly time, and two thirds when the names share a prefix (L0001, L0002, ...) so the compare runs deeper. sym_define now also stores an 8-bit hash of the name (h = h*5 + c over the 8 bytes) in a parallel SYM_HASH array, and sym_lookup sweeps those bytes (14 cycles per symbol, one 256-symbol page per (ptr),y pass), doing the full 8-byte compare only on a hash match. Equal names always hash equal, so the index is a pure filter. SYM_TABLE's layout, the duplicate check, the table-full check and error reporting are unchanged; names in the table are unique (duplicates are rejected and pass 2 defines nothing), so sweeping from the top down finds the same entry. X is still preserved; sym_define no longer touches TMP. SYM_HASH lives at $BD00-$BE8F with its scratch bytes at $BF32-$BF35, the same kind of free RAM above the image as SRC_STACK; the link guard now stops the code at $BD00 (2.9 K of headroom left). Benchmark (py65, 300 labels x 3 references, both passes): random names 29,140,046 -> 20,144,549 cycles (-31%) L0000..L0299 41,022,708 -> 20,025,811 cycles (-51%) Cycles inside the symbol routines: 14.0M -> 5.0M and 26.1M -> 5.2M. Costs 86 bytes. A differential run of 25 sources against the previous build gives identical status, message, line, output and screen; the new test case 15 covers lookups across the 256-symbol page boundary, a duplicate caught on page 2, and the 401st-label overflow, and also passes on the pre-change build. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Tokenizing or detokenizing a 10 KB program took 8-12 seconds on a C64, most of it in three places: - try_keyword rejected ~85 table entries on their first char for every letter in the source. The table now lives in kwtab.inc (shared by modtok and modsct, which had byte-identical copies) with a first-letter index generated at assemble time; only that letter's entries are tried, in table order, so the longest-first result is unchanged. One quirk is kept on purpose: when a shifted source letter equals an entry's final char the original walk ran on past the entry and resumed its scan off entry boundaries. The index hands that case to the original linear walk, so even those inputs give the same token. The table bytes and their order are unchanged, and in modtok the table is still the last thing in the image. - moddet's emit_keyword skipped (token - $80) entries byte by byte with a JSR and PHA/PLA per byte. kw_off, a one-byte-per-token offset table generated from the entry labels, finds the entry directly. - The relocate / linearize / restore copies decremented or incremented a 16-bit pointer and counter per byte. They now copy page-wise with Y as the index, in the same direction as before (descending where the destination is above the source, ascending where below); buf_lda and buf_sta still take the same pointers, so the C128 FAR_* helpers apply. Cycles for a 10,015-byte program (py65, C64 build): modtok 11.2M -> 2.2M, moddet 7.8M -> 1.5M, modsct 11.9M -> 3.0M. Equivalence was checked by running the old and new builds (C64 and C128 modtok/moddet, C64 modsct with an emulated REU) side by side over ~6,500 inputs: one representative of each of the 25,002 behaviour classes of the old try_keyword that stay inside the table, every keyword and shifted abbreviation, random BASIC with strings, REM, DATA, ?, pi, long lines and split gaps, page-boundary lengths, unaligned buffers and the overflow/restore paths. Buffers, status and REU contents were byte-identical. tests/test_modules.py gains every-keyword, shifted-abbreviation and page-boundary cases. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d table When a shifted source letter equalled a keyword's final character (T<shift-O> against TO, F<shift-N> against FN), the bit-7 compare saw an exact match and kept comparing into the next table bytes, off entry boundaries. The linear walk it fell into could read past the $FF sentinel into whatever memory followed the module, and run for millions of instructions or produce garbage, from an everyday abbreviation habit. A shifted letter only ends an abbreviation before a keyword's final char, so that entry simply doesn't match and the search moves to the next candidate. With the misaligned walk unreachable, it's removed, and so is the KW_XSAVE store that only it read. F<shift-N> stays literal as before; T<shift-O> now does too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes the top-tier bugs from a full-codebase review (several of them lose or corrupt user data), plus a round of speed and size optimizations. Every fix that a test can reach has a py65 regression test that fails on the old code.
Bug fixes
Data loss / corruption
10 REM <shift-*>became10 REM TANon every save, permanently.work_buf, so typing was dropped and F5 saved junk.sfr_move_gap.@0:so re-assembling replaces the file.Smaller fixes
ON X GOTO a,b,crenumbers every target.DSK_INPUT_BUF.Optimizations
Each was checked for byte-identical output against the old build on large py65 corpora, except the intended disk-count fix.
Link-time guards: the C64 editor's MAIN now ends at
MOD_HI_BASE($A000), and modasm's code can't grow into its working RAM.Tests and CI
tests/test_modules.pyruns the real built modules on py65: tokenize round trips, overflow restore, renumber, scripted search/replace sessions, and abbreviations.tests/test_include.pygains error-channel,@0:and symbol-table cases.build.ymlnow runs both suites. CI never ran the tests before.Reviewer notes
@locallabels, mid-source.orggaps,make_petproject.shrun from another directory, andmake_disk.pyskipping missing PRGs.🤖 Generated with Claude Code