Skip to content

feat: Build on UBI 10 and fix the Hadoop native build against OpenSSL 3 - #1534

Open
razvan wants to merge 16 commits into
mainfrom
feat/ubi10-hadoop
Open

razvan wants to merge 16 commits into
mainfrom
feat/ubi10-hadoop

Conversation

@razvan

@razvan razvan commented Jun 16, 2026 •

Copy link
Copy Markdown
Member

Description

Fixes: stackabletech/issues#845

Moves stackable-base and stackable-devel to ubi10/ubi-minimal, and makes the Hadoop native code build and work against RHEL 10's OpenSSL 3.5.

What changes

  • stackable-base, stackable-devel: ubi10/ubi-minimal@sha256:ab67cd02caf8706e34a695a1432ebe965934f6128ec352a6c3f4048b43a4fb5e, expected certs package ca-certificates-2026.2.92_v9.0.319-100.3.el10_2.

  • java-base installs gzip.
    tar -z needs it, and the hadoop final stage unpacks async-profiler with curl | tar -xz.
    On UBI 9 every image built on java-base had gzip, because krb5-workstation pulls in pam, which pulls in cracklib, which requires gzip; on UBI 10 krb5-workstation no longer depends on pam.
    It goes into java-base rather than the hadoop image so that every Java image keeps what it had on UBI 9, and images that never had it do not gain it.
    The installed version is gzip-1.13-4.el10_2, which contains the fixes for CVE-2026-41991 and CVE-2026-41992 (RHSA-2026:61625).

  • hadoop 3.4.3 and 3.5.0: three patches for HADOOP-19872:

    The touched files are byte-identical in 3.4.3, 3.5.0 and trunk.
    The two follow-up patches are not part of HADOOP-19872: Exclude Engine API when building with OpenSSL 3.x apache/hadoop#8550.

  • hadoop 3.3.6, which hive 3.1.3 and 4.0.0 build against:

Why the cipher needs the guard as well

apache/hadoop#8550 alone makes the build pass, but OpensslCipher.initIDs still resolves ENGINE_by_id and ENGINE_free at runtime.
RHEL 10's headers define OPENSSL_NO_ENGINE, yet its libcrypto still exports both symbols, so the lookup succeeds there by accident.
Against a libcrypto without them, initIDs throws UnsatisfiedLinkError and the AES/CTR codec falls back to JceAesCtrCryptoCodec with only a warning, so encrypted data transfer keeps working and nothing reports that native crypto is gone.
With the guard, libhadoop.so references no ENGINE symbol at all.

Testing

Built with boil build hadoop=3.4.3,3.5.0.

In both images:

  • hadoop checknative -a reports openssl: true /lib64/libcrypto.so (openssl-libs 3.5.8).
  • strings libhadoop.so shows no ENGINE_ symbol.
  • A native AES/CTR round trip through OpensslAesCtrCryptoCodec with OpensslSecureRandom: 1 MiB, ciphertext identical to JCE's AES/CTR/NoPadding, decrypts back to the input.

hadoop 3.3.6 builds and passes the same three checks: openssl: true /lib64/libcrypto.so, no ENGINE_ string, and the AES/CTR round trip matching JCE.
zstd and bzip2 report false in checknative, as they do in the 26.7.0 hadoop image.

hive 3.1.3 builds against the patched 3.3.6, and schematool -dbType derby -initSchema completes in the image.
Its checknative reports openssl: false because the hive image has no libcrypto.so symlink; the UBI 9 hive images behave the same, so this is not new.

On a k3d cluster with the 26.7.0 operators, the HDFS kerberos test (both realms, ZooKeeper 3.9.5, OPA 1.16.2, dfs.encrypt.data.transfer with AES/CTR) passes for 3.4.3 and 3.5.0.
Its datanodes log OpensslAesCtrCryptoCodec, which they do not when the codec falls back to JCE.

Known UBI 10 differences

  • gzip, see above.
  • tput is not installed, so the hadoop scripts print tput: command not found in places. Cosmetic.
  • Vector keeps the el9 inotify-tools RPM; it installs and inotifywait runs on UBI 10.

Open question

This changes the base of every image, not only Hadoop.
#1349 asks whether UBI 10 images should be built alongside the UBI 9 ones or replace them, and that is not settled.
Only hadoop 3.4.3 and 3.5.0 ran on a cluster; 3.3.6 and hive 3.1.3 were built and checked in the image only.
No image other than hadoop and hive was built or tested on UBI 10.

Definition of Done Checklist

Note

Not all of these items are applicable to all PRs, the author should update this template to only leave the boxes in that are relevant.

Please make sure all these things are done and tick the boxes

  • Changes are OpenShift compatible
  • All added packages (via microdnf or otherwise) have a comment on why they are added
  • Add an entry to the CHANGELOG.md file
  • Integration tests ran successfully
TIP: Running integration tests with a new product image

The image can be built and uploaded to the kind cluster with the following commands:

boil build <IMAGE> --image-version <RELEASE_VERSION> --strip-architecture --load
kind load docker-image <MANIFEST_URI> --name=<name-of-your-test-cluster>

See the output of boil to retrieve the image manifest URI for <MANIFEST_URI>.

@razvan razvan self-assigned this Jun 16, 2026
The hadoop final stage unpacks async-profiler with `curl | tar -xz`, which needs gzip.
On UBI 9 every image built on java-base had it, because krb5-workstation pulls in pam, which pulls in cracklib, which requires gzip.
On UBI 10 krb5-workstation no longer depends on pam, so nothing installs gzip.
It goes into java-base so that the Java images keep what they had on UBI 9, and images that never had it do not gain it.
apache/hadoop#8550 alone leaves OpensslCipher resolving ENGINE_by_id and ENGINE_free in initIDs.
RHEL 10's headers define OPENSSL_NO_ENGINE, but its libcrypto still exports both symbols, so the lookup succeeds there by accident.
Against a libcrypto without them, initIDs throws UnsatisfiedLinkError and the AES-CTR codec falls back to JCE with only a warning.
The two follow-up patches guard the remaining ENGINE paths in OpensslSecureRandom and OpensslCipher, and fix the misspelt ENGINE_by_free on Windows.
GCC 14, which UBI 10 ships, turns implicit-function-declaration into an error by default.
OpensslSecureRandom.c defines locks_setup and locks_cleanup outside the version guard that declares the dlsym_CRYPTO_* lock functions they call, so hadoop-common's native build fails on UBI 10.
Only OpenSSL before 1.1 calls them.
3.4.3 and 3.5.0 already contain HADOOP-19551, which restores the old default with -Wno-error=implicit-function-declaration.
The libstdc++ headers of GCC 14, which UBI 10 ships, no longer pull in <cstdint> transitively, so the libhdfspp public headers that use uint16_t, uint32_t and uint64_t without including it fail to compile on UBI 10.
3.4.3 and 3.5.0 already contain HDFS-17226, which adds the include.
Hive 3.1.3 and 4.0.0 build against Hadoop 3.3.6, so 3.3.6 has to build on UBI 10 too.
Its OpensslSecureRandom.c is identical to 3.4.3's and fails to compile against headers that define OPENSSL_NO_ENGINE in the same way.
OpensslCipher.c in 3.3.6 never looks up an ENGINE, because engine selection and SM4 only arrived in 3.4, so the third patch applied to 3.4.3 and 3.5.0 has nothing to guard here.
@lfrancke lfrancke changed the title fix: hadoop build with openssl 3 on UBI 10 feat: Build on UBI 10 and fix the Hadoop native build against OpenSSL 3 Oct 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Hadoop on UBI10

2 participants