Repository navigation
Fix GUI startup ENOMEM on small-RAM machines with an OpenMP OpenBLAS - #4733
Merged
Merged
Conversation
An OpenMP build of OpenBLAS reserves 128 MiB buffers at load, about 1 GiB of private writable memory per CPU. The preview imports numpy, which loads it, so every GUI with a preview holds that reservation. With the default overcommit mode a fork() must account it again, and on a machine with less RAM than that the kernel refuses: __vm_enough_memory: pid: 2586, comm: axis, bytes: 10737418240 not enough memory for the allocation OSError: [Errno 12] Cannot allocate memory The GUIs that fork after the import fail to start: axis, gmoccapy and qtvcp for the POSTGUI_HALFILE, and qtdragon also in QtWebEngine. Reported by NTULINUX. OMP_NUM_THREADS=1 drops the reservation (8.5 GB to 1.2 GB VmSize at 8 CPUs). OPENBLAS_NUM_THREADS=1 does not. LinuxCNC does no BLAS work that needs threads. A value the user set is kept.
axis, gmoccapy, gscreen and qtvcp ran POSTGUI_HALFILE and POSTGUI_HALCMD through os.spawnvp(), which forks the whole GUI process. A fork must account the GUI's private writable memory again, and with the preview's numpy that can be gigabytes of reservation (see the OMP_NUM_THREADS commit). subprocess.call() starts the child with vfork/posix_spawn, which shares the parent's memory until exec and accounts nothing. The return value is the same: the exit status, or minus the signal. gmoccapy's comment on SIGTERM lost in a fork hook no longer points at the postgui spawns; the check stays for any other fork.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
NTULINUX reported on Discord that on a Gentoo VM with 4 GB, sim axis lathe, gmoccapy lathe and qtdragon lathe fail to start, while plain axis starts:
I reproduced it in his VM image. It is not a leak. Gentoo's OpenBLAS (OpenMP build, loaded through flexiblas) reserves 128 MiB buffers when it loads, about 1 GiB of private writable memory per CPU: 4.3 GB of VmSize with 4 CPUs, 8.5 GB with 8. Resident memory stays around 36 MB. The preview imports numpy, which loads it, so every GUI with a preview holds that reservation. With the default overcommit mode a
fork()has to account it again, and when it is larger than RAM plus swap the kernel refuses. In his VMpython3 -c 'import numpy, os; os.fork()'fails on its own.The GUIs that fail are the ones that fork after numpy is loaded:
POSTGUI_HALFILEthroughos.spawnvp(), which forks the GUI. Plain axis has no postgui file, so it never forks.It surfaced now because the preview started importing numpy with the OpenGL 3.3 rewrite.
Two commits:
scripts/linuxcncsetsOMP_NUM_THREADS=1unless the user already set it. That drops the reservation to 1.2 GB at 8 CPUs and fixes all three GUIs, qtdragon included. LinuxCNC does no BLAS work that needs threads.OPENBLAS_NUM_THREADS=1does not help with this build.subprocess.call()instead ofos.spawnvp().subprocessstarts the child with vfork/posix_spawn, which shares the parent's memory until exec, so the GUI's size no longer matters. The return value is the same (exit status, or minus the signal). On its own it fixes axis and gmoccapy but not qtdragon, so it does not replace the first commit; it just stops copying a large GUI for every postgui file.Tested in his Gentoo VM (Python 3.14, OpenBLAS 0.3.34, 4 GB, 8 CPUs), lathe sims of axis, gmoccapy and qtdragon:
OMP_NUM_THREADS=1subprocessonlyOn Debian 13 the patched axis runs the sim lathe postgui file and links its pyvcp pins as before.
The OpenBLAS reservation itself is a Gentoo packaging matter; any numpy program that forks on a small machine hits it. Until this lands,
export OMP_NUM_THREADS=1before starting LinuxCNC works around it.