Skip to content

rest: make StdioSession work in a forked child - #237

Merged
ThomasWaldmann merged 1 commit into
borgbackup:mainfrom
ThomasWaldmann:stdio-close-after-fork
Oct 29, 2026
Merged

ThomasWaldmann merged 1 commit into
borgbackup:mainfrom
ThomasWaldmann:stdio-close-after-fork

Conversation

@ThomasWaldmann

@ThomasWaldmann ThomasWaldmann commented Oct 29, 2026 •

Copy link
Copy Markdown
Member

Fixes borgbackup/borg#10566: a daemonized borg mount (2.0.0b24+, rest/ssh stdio repo) never exits after borg umount.

Cause

borg mount opens the store and then forks to daemonize. The StdioSession stderr drain thread does not survive fork(), so in the daemon:

  • close() hangs: at fork time the drain thread was blocked in read() on the server's stderr, holding the BufferedReader's internal lock. process.stderr.close() waits for that lock forever (the reported hang).
  • nothing drains the server's stderr: server messages get lost (no Remote: ... debug lines, no stderr tail in error messages), and once the pipe buffer (64 KiB on Linux) is full, the server blocks writing stderr and stops answering requests.

Fix

The session records the pid that started the drain thread. When request() or close() runs in a different process (a forked child), _check_fork():

  1. wraps an os.dup() of the stderr fd in a new reader (fresh, unheld lock) and starts a new drain thread on it,
  2. closes the old reader's unbuffered raw file (no lock involved), which also makes the old reader count as closed, so neither we nor its finalizer close it via the stuck lock.

close() then uses the normal path. Same-process behaviour is unchanged (one os.getpid() per request). This is done lazily rather than via os.register_at_fork(), because at-fork hooks also run in Popen(..., preexec_fn=...) children before exec, where starting a thread is unsafe.

Remaining limitation (as before): the server is a child of the original process, not of the daemon, so wait() in the daemon cannot see its exit code.

Tests

  • New test_stdio_session_works_in_forked_child mimics daemonizing: a child opens the session and forks again, the intermediate parent exits, and the grandchild does a request where the server writes ~200 KB to stderr, then closes. It passes with this fix and hangs (killed after 10 s, test fails) both without any fix and with only a non-blocking close.
  • Linux (Debian 13), borg master + borgstore 0.7.0, daemonized borg mount over ssh://: without the fix the daemon stays stuck in stderr.close() after borg umount; with it, it exits ~2 s after umount, no leftover processes, repo lock released. With borg serve --debug, the daemon's server stderr pipe holds 0 unread bytes with the fix vs. stale unread output without it.

🤖 Generated with Claude Code

A daemonized "borg mount" opens the store, then forks. The stderr drain
thread does not survive fork(), so in the daemon:

- nothing drains the server's stderr any more: its messages get lost and
  the server blocks once the pipe buffer is full.
- the old thread was blocked reading stderr at fork time, holding the
  buffered reader's lock, so closing that reader waited forever
  (borgbackup/borg#10566).

When used in a different process than the one that started the stderr
thread, read a dup of the stderr fd with a new reader and start a new
drain thread. Close the old reader's unbuffered raw file, which takes no
lock and also makes the old reader count as closed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ThomasWaldmann ThomasWaldmann changed the title rest: StdioSession.close() must not hang in a forked child rest: make StdioSession work in a forked child Oct 29, 2026
@ThomasWaldmann
ThomasWaldmann merged commit f6e41c1 into borgbackup:main Oct 29, 2026
9 checks passed
@ThomasWaldmann
ThomasWaldmann deleted the stdio-close-after-fork branch October 29, 2026 22:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

borg2: daemonized "borg mount" hangs forever after "borg umount"

1 participant