Skip to content

COMP: Compile each ImageProcessing filter executor once - #1963

Merged
imikejackson merged 3 commits into
BlueQuartzSoftware:developfrom
joeykleingers:topic/ip-compile-time
Nov 13, 2026
Merged

imikejackson merged 3 commits into
BlueQuartzSoftware:developfrom
joeykleingers:topic/ip-compile-time

Conversation

@joeykleingers

Copy link
Copy Markdown
Contributor

Summary

The NXImageProcessing filters included one 5,891-line header, ImageProcessingFilterUtilities.hpp, that held all of the executor templates and 23 engine headers. Each filter translation unit compiled the executor code again for the same type sets. For example, the Black Top Hat, White Top Hat, Morphological Gradient and grayscale opening and closing objects each held about 2.3 MB of the same machine code. This change compiles each executor family once, in its own source file in the NXImageProcessing plugin. The executor bodies, type sets, error codes and messages do not change. 110 files changed, +5,889 / −5,178 lines.

1. Executors move into the plugin

  • src/Plugins/NXImageProcessing/src/NXImageProcessing/utils/ImageFilterExecutors.hpp declares the 30 executors that always use their default type set as exported plain functions (NXIMAGEPROCESSING_EXPORT).
  • src/Plugins/NXImageProcessing/src/NXImageProcessing/utils/*ImageFilterExecutors.cpp (11 files, one for each engine family: mask, adaptive histogram equalization, morphology, object morphology, binary morphology, reconstruction, regional extrema, distance map, Gaussian, component and watershed) define them and instantiate each type set once.
  • src/Plugins/NXImageProcessing/src/NXImageProcessing/utils/ImageFilterTemplateExecutors.hpp keeps the 7 executors whose template arguments change with the caller (point-wise, two-pass, box-neighborhood, contour, histogram, axis projection and finite-difference) as header templates.
  • src/Plugins/NXImageProcessing/src/NXImageProcessing/utils/ImageFilterExecutorDetail.hpp holds the helpers that more than one family or a test uses.

Each filter includes only the executor header that it calls. Helpers that only one family uses stay in that family's source file with internal linkage.

2. Shared header reduced to preflight code

  • src/simplnx/Utilities/ImageProcessing/ImageProcessingFilterUtilities.hpp (5,891 → 770 lines) keeps only the type-set policies, the input checks, the preflight helpers and SaturateCastFromDouble. It no longer includes engine headers.
  • src/simplnx/Utilities/ImageProcessing/ImageProcessingOperationTypes.hpp holds MorphOp, ObjectMorphOp, ReconstructOp and RegionalExtremaOp. The engine headers include it, so their users see the same enums.

Only NXImageProcessing sources include ImageProcessingFilterUtilities.hpp, so no other plugin needs a change. The unused StreamBinaryThreshold helper is removed.

3. Export check

  • src/Plugins/NXImageProcessing/test/ImageFilterExecutorsLinkTest.cpp takes the address of each of the 30 exported executors through a volatile pointer. A missing export macro then fails the link of NXImageProcessingUnitTest on every platform, also on Windows, where a missing __declspec(dllexport) would otherwise only show in a caller outside the plugin.

Measured effect

Serial compile CPU time for each object, macOS arm64, AppleClang 21, Release:

  • NXImageProcessing plugin objects: 793 s → 502 s CPU (−37%; −33% after a correction for machine load measured on unchanged control objects).
  • Duplicate weak machine code in the plugin objects: 19.1 MB → 6.6 MB (−65%).
  • Mean filter object: 7.6 s → 3.2 s. The longest object is now the Gaussian executor family (41 s), which compiles in parallel with the filters.
  • NXImageProcessingUnitTest objects: no significant change.

This is about 4–5% of the compile work of a full simplnx and DREAM3D-NX build.

Test plan

  • macOS arm64 (AppleClang), in-core Release with SIMPLNX_TEST_ALGORITHM_PATH=0: full build passes; all 1117 ctest -R ImageProcessing tests pass.
  • Linux (GCC, Ubuntu 24.04) Release: build passes; all 1117 ctest -R ImageProcessing tests pass.
  • Windows 11 (MSVC) Release: build passes; 1116 of 1117 ctest -R ImageProcessing tests pass. The link test confirms that all 30 executors are exported. NXImageProcessing_test_profile_isolation fails on that machine only because its build folder path is longer than the 260-character Windows path limit; the test does not use the changed code, and it passes on macOS and Linux.
  • macOS arm64 out-of-core Release (with the out-of-core storage plugin): all 1268 ctest -R ImageProcessing tests pass.
  • CI passes on all platforms.

Related Issue

  • This PR has no related issue

@joeykleingers joeykleingers self-assigned this Nov 12, 2026
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

@imikejackson imikejackson left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified this is a pure code move: executor bodies, type sets and error codes are unchanged, and the only new code is what the move needs (declarations, exports, includes, CMake lists, link test). Rebased on develop after #1942 (one include added to Algorithms/NormalizeImage.cpp) and updated four test comments to the new SafeBorderPadRadius location. In-core: ImageProcessing tests 1128/1128 pass. Out-of-core: ImageProcessing ooc-labeled tests 140/140 pass.

joeykleingers and others added 3 commits November 12, 2026 22:19
The NXImageProcessing filters included one 5,891-line header that held
all of the executor templates and 23 engine headers. Each filter
compiled again the executor code that it shared with other filters.
For example, five morphology filters each compiled the same 2.3 MB of
machine code. This change gives each executor family one translation
unit in the NXImageProcessing plugin.

* Move the 30 executors that use only their default type set to
  exported functions in NXImageProcessing/utils/ImageFilterExecutors.hpp
  with one .cpp file for each engine family.
* Move the 7 executors whose template arguments change with the caller
  to NXImageProcessing/utils/ImageFilterTemplateExecutors.hpp.
* Move the helpers that more than one family or a test uses to
  NXImageProcessing/utils/ImageFilterExecutorDetail.hpp.
* Move MorphOp, ObjectMorphOp, ReconstructOp and RegionalExtremaOp to
  the new ImageProcessingOperationTypes.hpp. The engine headers include
  it, so their users see the same enums.
* Keep only the shared preflight helpers, input checks and type-set
  policies in ImageProcessingFilterUtilities.hpp (770 lines).
* Add ImageFilterExecutorsLinkTest, which references each exported
  executor, so a missing export fails the link on every platform.
* Remove StreamBinaryThreshold, which had no caller.

The executor bodies, type sets, error codes and messages do not change.

Signed-off-by: Joey Kleingers <joey.kleingers@bluequartz.net>
SafeBorderPadRadius moved to NXImageProcessing/utils/ImageFilterExecutorDetail.hpp.

Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
PR BlueQuartzSoftware#1947 added ValidateMorphologyLayout preflight checks to the morphology filters. They reached it through ImageProcessingFilterUtilities.hpp, which no longer includes MorphologyEngine.hpp. Move the two overloads and their error codes, unchanged, into MorphologyLayout.hpp so the filters can include them without the morphology engine.

Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
@imikejackson

Copy link
Copy Markdown
Contributor

/azp run

@imikejackson
imikejackson enabled auto-merge (squash) November 13, 2026 03:30
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

@imikejackson

Copy link
Copy Markdown
Contributor

@joeykleingers I rebased this PR onto develop tonight so it could merge with the other approved PRs. #1947 merged first and conflicted with it, so here is what changed on your branch.

Rebase onto #1947 (folded into 224d476aa)

  • MorphologyEngine.hpp and ObjectMorphologyEngine.hpp: kept all of BUG: Validate morphology kernel radii and bound equalization loops #1947's additions and applied only your enum move to ImageProcessingOperationTypes.hpp.
  • ImageProcessingFilterUtilities.hpp: kept your smaller header. BUG: Validate morphology kernel radii and bound equalization loops #1947 added one ValidateMorphologyLayout check at the start of 7 Execute*Morphology* functions. Each check moved, unchanged, into the function's new home in utils/: MorphologyImageFilterExecutors.cpp, BinaryMorphologyImageFilterExecutors.cpp, ObjectMorphologyImageFilterExecutors.cpp and ReconstructionImageFilterExecutors.cpp. AddMorphologyLayoutGeometryContext stays once in ImageProcessingFilterUtilities.hpp.
  • Apart from those 7 moved checks, the rebased diff matches your original diff line for line.

New commit 5ab1f1dab "COMP: Move morphology layout checks into a small header"
#1947 also added ValidateMorphologyLayout preflight checks to 16 morphology filters. They reached it through ImageProcessingFilterUtilities.hpp -> MorphologyEngine.hpp, which this PR removes, so the rebased branch did not compile. I didn't want to add MorphologyEngine.hpp back to the filters, because that would undo the compile-time gain. So the two ValidateMorphologyLayout overloads and the error codes -88005 to -88009 moved, unchanged, into a new simplnx/Utilities/ImageProcessing/MorphologyLayout.hpp. MorphologyEngine.hpp includes it. The filters include it instead of the engine headers, and five filters that had included an engine header directly now include only the small one.

Other

  • 01a2820b9 (the DOC commit): removed a Co-Authored-By trailer and added a Signed-off-by. The code didn't change.
  • Linux gcc-13 ci-linux-x64 build: all 3971 tests pass.

If you would rather ValidateMorphologyLayout live somewhere else, a follow-up PR is fine.

@imikejackson
imikejackson merged commit 263031b into BlueQuartzSoftware:develop Nov 13, 2026
5 checks passed
@imikejackson
imikejackson deleted the topic/ip-compile-time branch November 13, 2026 04:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants