gh-159207: Store Tachyon samples in bounded columnar blocks by ivonastojanovic · Pull Request #160313 · python/cpython

ivonastojanovic · 2026-10-22T19:00:04Z

Tachyon's binary format currently stores all samples in a single compressed stream. Each record contains its thread and interpreter IDs, timestamp, status, and stack change, so values such as the thread and interpreter IDs are written repeatedly across records. The existing RLE encoding avoids some of this when a stack repeats, and compression reduces it further, but the format still mixes fields with very different patterns in the same stream.

More importantly, this means replay has to decompress the complete sample stream before it can process the first sample. The temporary buffer therefore grows with the length of the capture, even though replay only needs a small part of the sample data at a time.

This PR changes the binary format to split samples into bounded, independently decodable blocks. Replay can decompress and process one block at a time, then reuse the same buffers for the next block. Within each block, sample fields are also separated into columns, allowing repeated values to be removed before compression and each type of data to use an encoding that better matches its structure.

This change focuses on profiles exported to regular files. Live streaming remains separate work in #145464. Regular files can keep shared string and frame tables at the end of the file, while a live stream cannot refer to tables that have not been sent yet. For streaming, we may instead need to include the required string and frame data in each block so that it can be decoded as soon as it arrives.

Implementation

The new file layout is:

header | sample block | sample block | ... | string table | frame table | stats | footer

Each sample block has its own header followed by six columns:

block header
    | stream table
    | stream IDs
    | timestamp residuals
    | status bitmaps
    | stack instructions
    | frame IDs

Samples keep their original order across these columns, but no longer need a complete record containing every field. Values shared across a block move into its header or tables, while the per-sample columns only store what changes. The block header contains the sample count, base timestamp, frame-ID width, status masks, and the size and encoding of each column.

Column	What it stores	What this avoids
Stream table	Each `(thread_id, interpreter_id)` pair used by the block, once	Repeating both full IDs in every record
Stream IDs	A small stream-table index for each sample	Carrying the full thread and interpreter IDs with each sample
Timestamp residuals	Difference from the expected timestamp for that stream	Storing larger timestamp differences alongside other fields
Status bitmaps	One bitmap for each status bit that varies	Writing one complete status byte per sample; common bits are stored once in the header
Stack instructions	How to repeat, change, or reuse a stack	Writing frames that are unchanged or already available in the block
Frame IDs	Referenced frame-table indices at the smallest width needed by the block	Using a larger representation for every frame ID

The writer first reserves space for the file header. It then collects samples into a block and starts a new one when the decoded column data reaches 1 MiB or the block reaches 4,096 unique (thread_id, interpreter_id) pairs. Once sampling finishes, it writes the shared string and frame tables, statistics, and footer, then seeks back to fill in the file header.

Strings and frames remain shared across the whole file, preserving deduplication across the complete capture. These tables still grow with the number of unique strings and frames, but temporary decoded sample data is now bounded by the size of one block rather than the total number of samples.

Timestamp and stack state are tracked independently for each stream. Timestamps are stored as residuals from the expected timestamp, so samples arriving at the normal interval usually store zero. Status bits shared by all samples in a block are stored once in the block header, while only varying bits need per-sample bitmaps.

Stacks use three instructions:

Instruction	Operands	Meaning
`REPEAT`	`count`	Reuse this stream's previous stack for `count` samples
`REPLACE_SUFFIX`	`keep_count`, `new_count`	Keep the common outer frames and append `new_count` frame IDs
`REUSE`	`offset`, `count`	Copy a complete stack already stored in the block's frame-ID column

REPEAT is tracked per stream, so samples from other streams can appear between repetitions. The other instructions either write only the frames that changed or reuse a complete stack already stored in the block. Frame IDs use between one and four bytes and are stored in byte planes to improve compression.

Each column is stored as raw bytes, a single repeated byte, or zstd level 3, whichever is smallest. The file-wide string and frame tables similarly choose between raw and zstd.

After discussing this with @pablogsal, we decided not to keep support for version 1 profiles since this change will be shipped in 3.16.

The public BinaryWriter, BinaryReader, and collector APIs remain unchanged, except that the get_stats() dictionaries now report reuse_records instead of pop_push_records, since the pop-push encoding no longer exists.

Issue: Tachyon: use bounded columnar blocks for binary profiles #159207

ivonastojanovic · 2026-10-22T19:00:28Z

cc @maurycy

Tachyon currently stores all samples in a single compressed stream. Replay has to decompress the complete sample section before it can process any samples, so temporary memory usage grows with the length of the capture. The stream also mixes data with different patterns and repeatedly stores values such as thread and interpreter IDs. Change the file format to split samples into independently decodable blocks, each limited to 1 MiB of decoded data. Strings and frames remain shared across the whole file, preserving deduplication across the complete capture. Within each block, separate the sample data into columns for stream identifiers, timestamps, statuses, stack changes, and frame indices. Store each thread and interpreter pair once and have samples refer to those pairs by index. Encode timestamps relative to the expected sampling time for each stream, and store status bits common to all samples in the block once while packing the bits that vary into bitmaps. Encode stacks by reusing unchanged stacks, retaining common outer frames when only part of a stack changes, and referring back to complete stacks already seen in the block. Store frame indices using the smallest byte width needed by the block. These encodings remove repeated values and group similar data together before compression. Compress each column independently as raw bytes, a repeated byte, or zstd, whichever is smallest. The block header contains the information needed to locate, validate, and decode these columns independently. Replay can then decode one block at a time and reuse its buffers for subsequent blocks. This bounds temporary sample decompression memory by the block size rather than the total capture length, although the shared string and frame tables still grow with the number of unique entries. This change only applies to profiles exported to files. Live streaming will need a different approach because it cannot rely on string and frame tables written at the end of the file, and is left for pythongh-145464.

ivonastojanovic · 2026-10-22T22:16:11Z

I did some benchmarking while working on this, so leaving the results here for reference.

For the main comparison, I profiled test_list, test_dict, test_tokenize, test_exceptions, test_syntax, and test_threading with non-debug builds on an Apple Silicon Mac. The number of samples was slightly different between runs, so I compared bytes per sample instead of the final file size.

Compression	`main`	This PR	With compressed tables
None	2.4121	1.6643 (-31.0%)	N/A
zstd	0.3186	0.2787 (-12.5%)	0.2572 (-19.3%)

I also experimented with compressing the file-wide string and frame tables, so I included those numbers in the last column. It helps a bit more with zstd, but I left it out of this PR since the change is already pretty large. We can always add that separately.

I also tried two smaller programs with zstd to see how the format behaves with different workloads. One does JSON and zlib work across eight threads, so repeated stacks are interleaved across several streams. The other runs SQLite queries and processes the results in Python, so it mostly stays on one stream and moves between a small set of stacks.

Workload	`main`	This PR	With compressed tables
JSON/zlib threads	0.4659	0.2508 (-46.2%)	0.2652 (-43.1%)
SQLite queries	1.3747	1.0991 (-20.0%)	1.0896 (-20.7%)

The small differences between the two block-format results are just normal run-to-run variation. These are also bytes per sample.

I also checked a few synthetic stack patterns with zstd: one stable stack, stacks where only the inner frames change, complete stacks shared across threads, and a bank of randomly selected stacks.

Workload	`main`	This PR	With compressed tables
Stable stacks	1.6 KiB	1.4 KiB (-13.9%)	1.2 KiB (-28.6%)
Common outer frames	323.1 KiB	129.1 KiB (-60.0%)	108.7 KiB (-66.4%)
Shared complete stacks	518.2 KiB	125.2 KiB (-75.8%)	50.8 KiB (-90.2%)
Random stack bank	734.6 KiB	3,506.1 KiB (+377.3%)	3,385.9 KiB (+360.9%)

The random stack case is the obvious regression. The frame IDs do not compress particularly well, and repeated stacks can end up in different blocks, so we lose the ability to reuse them across block boundaries.

The main thing I wanted to improve with this change was replay memory, though. With the previous format, peak memory grew from 31.0 MiB at 100,000 samples to 59.4 MiB at one million samples. With the block reader, it stayed at around 26.3 MiB.

ivonastojanovic · 2026-10-23T16:40:11Z

+typedef struct {
+    uint32_t offset;
+    uint32_t count;
+} Repeat;


I also tried with just REPEAT, but REPEAT, count is smaller and generally faster, especially without compression

ivonastojanovic · 2026-10-23T16:51:18Z

+    ByteBuffer columns[COLUMN_COUNT];
+    const uint32_t *stream_ids;
+    const uint64_t *timestamps;
+    const uint8_t *statuses;


Before we used one byte per sample, so for 1000 samples the old format uses 1000 bytes. With bitmaps, one varying flag only needs 127 bytes. Even in the worst case, where all eight flags vary, we need 1002 bytes, so it's basically the same size as before.

ivonastojanovic · 2026-10-23T16:57:20Z

+/* Sample blocks */
+#define BLOCK_MAGIC             "TBLK"
+#define BLOCK_MAGIC_SIZE        (sizeof(BLOCK_MAGIC) - 1)
+#define MAX_BLOCK_SIZE          (1024 * 1024)


I also experimented with larger 8 MiB blocks, but overall 1 MiB worked better. The 8 MiB blocks only helped meaningfully in the synthetic case with repetition across many samples, since more stack history stayed within the same block.

maurycy

@ivonastojanovic Thank you! This is the top of my queue. Two extremely quick notes.

pablogsal · 2026-10-23T17:38:19Z

I took a first look at the PR. A few ideas for this change or later PRs:

Add a small, bounded cache in the reader that maps frame IDs to immutable FrameInfo objects. Reuse these objects across samples, while each callback still receives a fresh frame list. I suspect this will reduce the CPU spent on repeated object allocation.
For long captures, avoid a whole-file preload. On Linux, remove eager page population and release pages after the reader finishes with them. On Windows, replace the whole-file buffer with reads of individual blocks. This looks useful for keeping resident memory low as captures grow.
Give each hash bucket in the writer cache room for two or four stacks. Currently, stacks that map to the same bucket replace each other, even if both recur. Keep the total number of slots unchanged. I expect fewer collisions to discard useful entries.
The reuse cache also excludes stacks encoded as changes to a previous stack. For a repeated stack, consider a full-stack record so later occurrences can use REUSE. That gives us another opportunity for compression, provided the later savings cover the larger initial record.
Base the block cutoff on decoded column size, rather than the larger temporary buffers. Keep a separate memory limit for those buffers. I think this will give us fuller blocks and less overhead per sample. The trade-off to check is whether larger flushes cause longer pauses.

Update the replay progress bar every 1,000 decoded samples instead of only once per block. This matches the previous behavior and prevents the bar from appearing stuck while large blocks are replayed. Clarify in NEWS that Python 3.16 cannot read profiles written in the previous binary format.

Replay currently builds a new Python FrameInfo object every time a frame appears in a stack. These objects are immutable, so keep a small, bounded cache on the reader and reuse them while still creating a new frame list for each stack.

The stack reuse cache currently keeps one stack at each hash position, so a collision immediately evicts the previous stack. Alternating between two colliding stacks therefore prevents either from being reused. Let each cache bucket remember two stacks while keeping the cache the same overall size. Lookups check both stacks, and inserting a third stack replaces the older one.

ivonastojanovic · 2026-10-23T22:17:47Z

I took a first look at the PR. A few ideas for this change or later PRs:

Add a small, bounded cache in the reader that maps frame IDs to immutable FrameInfo objects. Reuse these objects across samples, while each callback still receives a fresh frame list. I suspect this will reduce the CPU spent on repeated object allocation.

Done. On a two-million-sample repeated-frame profile, median replay time dropped from 0.98s to 0.83s (about 15%), while CPU time dropped by about 23%.

For long captures, avoid a whole-file preload. On Linux, remove eager page population and release pages after the reader finishes with them. On Windows, replace the whole-file buffer with reads of individual blocks. This looks useful for keeping resident memory low as captures grow.

I'll handle this in a follow-up PR. I don't have a good way to test on Windows right now, so I'll open one next week.

Give each hash bucket in the writer cache room for two or four stacks. Currently, stacks that map to the same bucket replace each other, even if both recur. Keep the total number of slots unchanged. I expect fewer collisions to discard useful entries.

Done.

The reuse cache also excludes stacks encoded as changes to a previous stack. For a repeated stack, consider a full-stack record so later occurrences can use REUSE. That gives us another opportunity for compression, provided the later savings cover the larger initial record.

I'll experiment with this in a follow-up as well, since this PR is already getting pretty large.

Base the block cutoff on decoded column size, rather than the larger temporary buffers. Keep a separate memory limit for those buffers. I think this will give us fuller blocks and less overhead per sample. The trade-off to check is whether larger flushes cause longer pauses.

rather than the larger temporary buffers

Follow up.

So @pablogsal, I would leave this as it is for now and experiment with a few things in follow-ups:

Points 2, 4, and 5 that you mentioned here
Compressing the string and frame tables, or potentially compressing the whole file at the end

Let me know if you think any of these should be fixed in the first round.

The latest writer-cache change mainly improved shared complete stacks, reducing this PR's result from 125.2 KiB in the earlier run to 82.8 KiB; the other synthetic results stayed roughly the same.

Compression	`main` (bytes/sample)	This PR (bytes/sample)
None	2.4121	1.6687 (-30.8%)
zstd	0.3186	0.2986 (-6.3%)

Workload	`main` (bytes/sample)	This PR (bytes/sample)
JSON/zlib threads	0.4659	0.2700 (-42.0%)
SQLite queries	1.3747	1.0715 (-22.1%)

Synthetic stack pattern	`main`	This PR
Stable stacks	1.6 KiB	1.4 KiB (-13.9%)
Common outer frames	323.1 KiB	128.6 KiB (-60.2%)
Shared complete stacks	518.2 KiB	82.8 KiB (-84.0%)
Random stack bank	734.6 KiB	3,506.1 KiB (+377.3%)

maurycy · 2026-10-24T05:43:05Z

@ivonastojanovic I started writing lots of inline comments but I realized it's getting very long, so following Pablo's format:

I'd check if buffering the first sample has implications for some formats. I'm thinking of GeckoCollector here in particular. The tests should check this, too!
Perhaps choose_stack_encoding() can be gotten rid of. That's just a matter of style, though.
There's a wider discussion about counts vs. timestamps. See the trick from gh-149718: Aggregate same stack frames in Tachyon in some collectors #149719 but in the binary format. I think of pstats, collected and flamegraph here. Two notes: this can combine non-consensutive occurences, and maybe REUSE could identify these before even create frames. That's a general question about decode_block(): how lazy decoding (materialization) can be?
Similarly: https://github.com/python/cpython/pull/160313/changes#diff-dfdcb94821b5902e4d751f5d78e385c19c1efa6ef2ed808e358741e41fda369bR1063-R1067 maybe there's no need to decode / memset?
Maybe in build_frame_list() we can read in the reverse order, without temporary stacks in flush_stream_batch().
Each stream retains a prev_stack buffer until writer destruction. Each buffer reserves frame IDs. When a stream enters a new block, block_add_stream() resets its stack depth and timestamp. The previous block's stack contents are no longer needed. Maybe we can we keep the ID registry global, but keep stack, timestamp, and repeat state only for streams in the current block? This also avoids reset growing linearly.
Thought experiment: the stream count gives us the bit width needed for the indices. With four streams, we only need two bits instead of a byte.
Perhaps we can compare independent zstd blocks with the columnar format. Then try REUSE, repeat counts and the bit/byte planes separately. I'm curious how much comes from splitting the data into blocks, how much from each encoding. File size, write/replay CPU time, peak memory, distribution of flush times etc. I can do this if you want!

Thank you! I promise more!

The block reader buffered the first sample for every reconstructed stack. A later stream could therefore be delivered before the earliest sample, causing Gecko to use the wrong replay time origin and produce negative timestamps. Emit newly reconstructed stacks immediately and reserve batching for repeated stacks. This restores the previous replay behavior while keeping repeat batches efficient.

Choose and write stack instructions in write_stack() instead of passing the decision and four related values through a separate helper. This keeps each encoding path together and removes an unreachable fallback.

Constant stream ID and timestamp residual columns normally contain the same one-byte varint for every sample. The reader expanded that byte into a per-sample buffer and then decoded the same value repeatedly. Use the stored byte directly for these columns and exclude it from the shared block buffer. Other constant columns continue through the general expansion path.

The reader copied every reconstructed stack into a temporary reversed array before creating the collector's frame list. Walk the block's outermost-first stack backwards while building the list instead. This removes the copy while preserving innermost-first frame lookup and frame-cache behavior.

Stack delta state resets at each block boundary, but it was stored on file-wide thread entries. As a result, each thread kept a previous-stack buffer until the writer was destroyed, and block reset scanned every thread seen in the capture. Keep previous stacks, timestamps, and repeat state in the open block's stream entries. File-wide thread entries now only store identity and their current block index, while reset visits only streams from the block being flushed.

The stack cache hashed stacks with Py_HashBuffer(), which depends on the per-process hash secret. Since the hash picks which stacks stay cached for REUSE, the same samples produced a different file in each run.

ivonastojanovic · 2026-10-26T20:55:22Z

@ivonastojanovic I started writing lots of inline comments but I realized it's getting very long, so following Pablo's format:

I'd check if buffering the first sample has implications for some formats. I'm thinking of GeckoCollector here in particular. The tests should check this, too!

Oh this was a really good point! Fixed it now, I also found a bug in GeckoCollector (already on main): we're not updating last_sample_time properly, so open markers can close too early when batches arrive out of order. I'll open an issue for that one and make a fix.

Perhaps choose_stack_encoding() can be gotten rid of. That's just a matter of style, though.

There's a wider discussion about counts vs. timestamps. See the trick from gh-149718: Aggregate same stack frames in Tachyon in some collectors #149719 but in the binary format. I think of pstats, collected and flamegraph here. Two notes: this can combine non-consensutive occurences, and maybe REUSE could identify these before even create frames. That's a general question about decode_block(): how lazy decoding (materialization) can be?

Good idea, but it changes how collectors get called, so I'd rather do it in a follow-up. Lazier decode_block() fits there too.

Similarly: https://github.com/python/cpython/pull/160313/changes#diff-dfdcb94821b5902e4d751f5d78e385c19c1efa6ef2ed808e358741e41fda369bR1063-R1067 maybe there's no need to decode / memset?

Maybe in build_frame_list() we can read in the reverse order, without temporary stacks in flush_stream_batch().

Each stream retains a prev_stack buffer until writer destruction. Each buffer reserves frame IDs. When a stream enters a new block, block_add_stream() resets its stack depth and timestamp. The previous block's stack contents are no longer needed. Maybe we can we keep the ID registry global, but keep stack, timestamp, and repeat state only for streams in the current block? This also avoids reset growing linearly.

Thought experiment: the stream count gives us the bit width needed for the indices. With four streams, we only need two bits instead of a byte.

Tried it: with zstd it saves 0–12% depending on the thread count, uncompressed 22–89%. Since zstd is the default, I'll send it as a follow-up.

Perhaps we can compare independent zstd blocks with the columnar format. Then try REUSE, repeat counts and the bit/byte planes separately. I'm curious how much comes from splitting the data into blocks, how much from each encoding. File size, write/replay CPU time, peak memory, distribution of flush times etc. I can do this if you want!

Yup, we should definitely do this! Would you be OK with doing it in follow-up PRs? I'd rather not make this one even bigger :D I could open drafts stacked on this one, but I think it's better to get this one through a round or two of review first, so the follow-ups don't need many changes afterwards.

Thank you! I promise more!

Thanks @maurycy for the great review! I've fixed most of the points and replied to the ones I haven't addressed yet. I agree with those too, but since this PR is already quite large, I'd rather handle them in follow-ups. What do you think?

bedevere-app Bot added the type-feature A feature request or enhancement label Oct 22, 2026

bedevere-app Bot mentioned this pull request Oct 22, 2026

Tachyon: use bounded columnar blocks for binary profiles #159207

Open

ivonastojanovic added 2 commits October 22, 2026 15:06

Clean up binary format tests

8865260

ivonastojanovic force-pushed the tachyon-block-format branch from 71c18bd to 8865260 Compare October 22, 2026 22:09

ivonastojanovic added the topic-profiling label Oct 22, 2026

ivonastojanovic marked this pull request as ready for review October 22, 2026 22:17

ivonastojanovic requested review from AA-Turner, emmatyping, erlend-aasland, itamaro and pablogsal as code owners October 22, 2026 22:17

bedevere-app Bot added the awaiting review label Oct 22, 2026

ivonastojanovic added the performance Performance or resource usage label Oct 22, 2026

ivonastojanovic commented Oct 23, 2026

View reviewed changes

maurycy reviewed Oct 23, 2026

View reviewed changes

Comment thread Misc/NEWS.d/next/Library/2026-10-20-12-49-32.gh-issue-159207.Lccezg.rst Outdated

Comment thread Modules/_remote_debugging/binary_io_reader.c Outdated

ivonastojanovic added 3 commits October 23, 2026 10:54

fixup! fixup! Store Tachyon samples in bounded columnar blocks

874f7d5

ivonastojanovic added 3 commits October 26, 2026 18:20

fixup! Store Tachyon samples in bounded columnar blocks

0c3614a

Choose and write stack instructions in write_stack() instead of passing the decision and four related values through a separate helper. This keeps each encoding path together and removes an unreachable fallback.

ivonastojanovic added 3 commits October 26, 2026 18:20

fixup! Store Tachyon samples in bounded columnar blocks

b2b10db

The stack cache hashed stacks with Py_HashBuffer(), which depends on the per-process hash secret. Since the hash picks which stacks stay cached for REUSE, the same samples produced a different file in each run.

Navigation Menu

Repository navigation

gh-159207: Store Tachyon samples in bounded columnar blocks - #160313

gh-159207: Store Tachyon samples in bounded columnar blocks#160313
ivonastojanovic wants to merge 12 commits into
python:mainfrom
ivonastojanovic:tachyon-block-format

ivonastojanovic commented Oct 22, 2026 •

edited

Loading

Uh oh!

ivonastojanovic commented Oct 22, 2026

Uh oh!

ivonastojanovic commented Oct 22, 2026

Uh oh!

ivonastojanovic Oct 23, 2026

Uh oh!

ivonastojanovic Oct 23, 2026

Uh oh!

ivonastojanovic Oct 23, 2026

Uh oh!

maurycy left a comment •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

pablogsal commented Oct 23, 2026

Uh oh!

ivonastojanovic commented Oct 23, 2026

Uh oh!

maurycy commented Oct 24, 2026 •

edited

Loading

Uh oh!

ivonastojanovic commented Oct 26, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

ivonastojanovic commented Oct 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Implementation

Uh oh!

ivonastojanovic commented Oct 22, 2026

Uh oh!

ivonastojanovic commented Oct 22, 2026

Uh oh!

ivonastojanovic Oct 23, 2026

Choose a reason for hiding this comment

Uh oh!

ivonastojanovic Oct 23, 2026

Choose a reason for hiding this comment

Uh oh!

ivonastojanovic Oct 23, 2026

Choose a reason for hiding this comment

Uh oh!

maurycy left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

pablogsal commented Oct 23, 2026

Uh oh!

ivonastojanovic commented Oct 23, 2026

Uh oh!

maurycy commented Oct 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ivonastojanovic commented Oct 26, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ivonastojanovic commented Oct 22, 2026 •

edited

Loading

maurycy left a comment •

edited

Loading

maurycy commented Oct 24, 2026 •

edited

Loading