Skip to content

Avoid a crash in forked children on macOS with the binary gem - #767

Open
dduugg wants to merge 3 commits into
ged:masterfrom
dduugg:macos-krb5-fork-crash
Open

dduugg wants to merge 3 commits into
ged:masterfrom
dduugg:macos-krb5-fork-crash

Conversation

@dduugg

@dduugg dduugg commented Oct 28, 2026 •

Copy link
Copy Markdown

Fixes #663 for the binary gems. #538 and #311 are related, but this patch can't reach them (see the end).

Problem

On macOS, a child forked from a Ruby process can crash when it makes a TCP connection attempt with the default gssencmode=prefer. A closed port is enough, so no server or credentials are needed:

require "pg"
attempt = -> { PG.connect(host: "127.0.0.1", port: 1, connect_timeout: 2) rescue PG::ConnectionBad }
attempt.call # leave this out for the second case below
pid = fork { attempt.call; exit!(0) }
Process.wait(pid)
p $?.termsig

There are two failure modes, depending on whether the parent made an attempt before forking:

  • The parent attempted a connection first, for example an app server whose master connected before forking workers, or a parallel test runner. The child gets SIGSEGV. Every crash report I collected for this case has the same stack:

    PQconnectPoll > init_allowed_encryption_methods > select_next_encryption_method > pg_GSS_have_cred_cache >
    gss_acquire_cred > ... > acquire_init_cred > krb5_cccol_have_content > krb5_cccol_cursor_next >
    api_macos_ptcursor_next > get_primary_name > xpc_connection_resume > _xpc_connection_activate_if_needed >
    os_log_type_enabled > _os_log_preferences_refresh > SIGSEGV
    
  • The child is the first to attempt one, as under Spring (segfault with macos precompiled binary when using with rails spring #663). The objc runtime aborts the child with +[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called. ... Crashing instead., from cc_initialize() under api_macos_ptcursor_next(). A Ruby process is always multithreaded when it forks, so this happens unless OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES is set.

libpq probes for GSSAPI credentials on each TCP attempt under gssencmode=prefer. krb5 built for darwin defaults to its API: ccache (krb5 3bb429b), which reaches the credential cache service through Apple's CCAPI framework and XPC. Neither is safe in a child forked from a multithreaded process.

Change

  • ports/patches/krb5/1.22.2/0003-Avoid-the-macOS-API-ccache-in-forked-children.patch records the pid when the library loads. In every child forked after require "pg":

    • the API ccache reports its collection as empty;
    • resolving or opening a cache fails with KRB5_FCC_NOFILE, including handles the parent opened;
    • destroying an inherited handle leaves the parent's cache in place, and closing one leaves its CCAPI objects unreleased.

    So the child never calls the CCAPI framework or XPC. Processes that never fork behave as before.

  • krb5/krb5#1558 proposes a narrower version upstream, which records the pid at the first API ccache use. That keeps the API ccache working in children of single-threaded C programs, but it leaves the child-first abort. Every Ruby process is multithreaded at fork, so the gem's copy records it at load. The patch header links #1558.

  • Two specs in spec/pg/connection_spec.rb, limited to binary gems, connect in a forked child: one after the parent connected, and one forked from a fresh Ruby process that never connected, with OBJC_DISABLE_INITIALIZE_FORK_SAFETY unset. Both pin gssencmode=prefer, clear KRB5CCNAME and KRB5_CONFIG, and time out after 60 seconds.

Behavior change for Kerberos users: a forked child no longer sees credentials in the API ccache, including a cache named with KRB5CCNAME=API:<name>. With gss authentication the child fails as if it had no credentials, and under gssencmode=prefer it connects without GSSAPI encryption. A file ccache still works in children: kinit -c FILE:/tmp/krb5cc_$(id -u) with KRB5CCNAME=FILE:/tmp/krb5cc_$(id -u).

Still not covered: a process that loads pg only after it was itself forked from a multithreaded parent, such as a Puma worker whose master didn't load the app, can still hit the abort. Loading pg before forking avoids it, as does OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES.

Testing

On macOS 27.0.1 arm64, with the arm64-darwin gem cross-built with rake-compiler-dock 1.11.1 and the repro above:

Gem Parent connects, then child Only the child connects Only the child, OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES
pg 1.7.0, released SIGSEGV SIGABRT ok
this PR's first two commits (pid at first use) ok SIGABRT ok
this PR ok ok ok
  • Ruby 3.4.12 and 4.0.5 give the same results. The parent-first SIGSEGV happens with either setting of the env var.
  • Both new specs fail against the released 1.7.0 gem (SIGSEGV and SIGABRT) and pass with this PR.
  • The full suite passes against this PR's gem, except one #inspect example, run twice through connection_sync_spec.rb, whose user=\w* regex doesn't match a local user name containing a dot. It fails without this PR too.
  • CI on my fork (binary-gems.yml): without 0003 (run), the child-first spec fails in all four macOS test jobs (macOS 26 arm64 and macOS 15 Intel, Ruby 3.1 and 4.0). The parent-first spec fails on macOS 26 every time, but on macOS 15 Intel in only one of four runs. With this PR (run), all four macOS test jobs pass. In both runs, the GLIBC, yugabyte and Ubuntu Ruby 3.1 jobs failed before testing on the ubuntu-26.04 runner image (docker-compose: command not found, and no Ruby 3.1.7 there); the same jobs passed in master's latest run.

Release builds: the ports/*-<platform>.installed checkpoints don't track patch contents, so a tree that has built pg before keeps its old krb5 and libpq. Remove ports/krb5-*.installed and ports/postgresql-*.installed before building a release with this patch.

Related

Workarounds until a release: gssencmode=disable (or PGGSSENCMODE=disable), a Unix-socket host, or KRB5CCNAME=FILE:<path>. For the abort only, OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES.

The first two commits record the pid at first use, and the third moves it to load time. I can squash them if you prefer.

The krb5 bundled into the macOS binary gems defaults to its API
credential cache, which looks up the primary cache over XPC. libpq
probes for GSSAPI credentials on each TCP connection attempt while
gssencmode=prefer, so a process that connects, forks and connects again
crashes the child in xpc_connection_resume(). See ged#663.

Patch krb5 so the API ccache records the pid of its first use and, in a
forked child, reports no caches instead of calling the CCAPI framework
or XPC. Add a spec that connects in a forked child after the parent
connected. It fails with SIGSEGV against an unpatched arm64-darwin gem.
@dduugg
dduugg marked this pull request as ready for review October 28, 2026 02:30
Handles and cursors the parent opened reached the CCAPI framework in a
forked child too. Opening, initializing or destroying them there now
fails as not found, and closing them leaves their CCAPI objects
unreleased. This matches the version proposed to krb5 upstream.
The krb5 fork guard recorded the pid at the first API ccache use, so
it missed a child that was the first to use it. That child is what
ged#663 reports: the objc runtime aborts it with "+[__NSCFConstantString
initialize] may have been in progress in another thread when fork()
was called", unless OBJC_DISABLE_INITIALIZE_FORK_SAFETY is set. Every
Ruby process is multithreaded when it forks, so this hits Spring and
any other fork where the parent hadn't connected yet.

Record the pid when the library loads, so that every child forked
after `require "pg"` skips the API ccache. Kerberos through the API
ccache is then unavailable in forked children; a file ccache
(KRB5CCNAME=FILE:<path>) still works there. A process that loads pg
only after it was itself forked from a multithreaded parent can still
hit the abort.

Add a spec for the child-first case. It forks from a fresh Ruby
process with OBJC_DISABLE_INITIALIZE_FORK_SAFETY unset. Both fork
specs now pin gssencmode=prefer, clear KRB5CCNAME and KRB5_CONFIG,
print the child's error, and time out after 60 seconds.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

segfault with macos precompiled binary when using with rails spring

1 participant