Repository navigation
Conversation
The hadoop final stage unpacks async-profiler with `curl | tar -xz`, which needs gzip. On UBI 9 every image built on java-base had it, because krb5-workstation pulls in pam, which pulls in cracklib, which requires gzip. On UBI 10 krb5-workstation no longer depends on pam, so nothing installs gzip. It goes into java-base so that the Java images keep what they had on UBI 9, and images that never had it do not gain it.
apache/hadoop#8550 alone leaves OpensslCipher resolving ENGINE_by_id and ENGINE_free in initIDs. RHEL 10's headers define OPENSSL_NO_ENGINE, but its libcrypto still exports both symbols, so the lookup succeeds there by accident. Against a libcrypto without them, initIDs throws UnsatisfiedLinkError and the AES-CTR codec falls back to JCE with only a warning. The two follow-up patches guard the remaining ENGINE paths in OpensslSecureRandom and OpensslCipher, and fix the misspelt ENGINE_by_free on Windows.
GCC 14, which UBI 10 ships, turns implicit-function-declaration into an error by default. OpensslSecureRandom.c defines locks_setup and locks_cleanup outside the version guard that declares the dlsym_CRYPTO_* lock functions they call, so hadoop-common's native build fails on UBI 10. Only OpenSSL before 1.1 calls them. 3.4.3 and 3.5.0 already contain HADOOP-19551, which restores the old default with -Wno-error=implicit-function-declaration.
The libstdc++ headers of GCC 14, which UBI 10 ships, no longer pull in <cstdint> transitively, so the libhdfspp public headers that use uint16_t, uint32_t and uint64_t without including it fail to compile on UBI 10. 3.4.3 and 3.5.0 already contain HDFS-17226, which adds the include.
Hive 3.1.3 and 4.0.0 build against Hadoop 3.3.6, so 3.3.6 has to build on UBI 10 too. Its OpensslSecureRandom.c is identical to 3.4.3's and fails to compile against headers that define OPENSSL_NO_ENGINE in the same way. OpensslCipher.c in 3.3.6 never looks up an ENGINE, because engine selection and SM4 only arrived in 3.4, so the third patch applied to 3.4.3 and 3.5.0 has nothing to guard here.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fixes: stackabletech/issues#845
Moves
stackable-baseandstackable-develtoubi10/ubi-minimal, and makes the Hadoop native code build and work against RHEL 10's OpenSSL 3.5.What changes
stackable-base,stackable-devel:ubi10/ubi-minimal@sha256:ab67cd02caf8706e34a695a1432ebe965934f6128ec352a6c3f4048b43a4fb5e, expected certs packageca-certificates-2026.2.92_v9.0.319-100.3.el10_2.java-baseinstalls gzip.tar -zneeds it, and the hadoop final stage unpacks async-profiler withcurl | tar -xz.On UBI 9 every image built on
java-basehad gzip, becausekrb5-workstationpulls in pam, which pulls in cracklib, which requires gzip; on UBI 10krb5-workstationno longer depends on pam.It goes into
java-baserather than the hadoop image so that every Java image keeps what it had on UBI 9, and images that never had it do not gain it.The installed version is
gzip-1.13-4.el10_2, which contains the fixes for CVE-2026-41991 and CVE-2026-41992 (RHSA-2026:61625).hadoop
3.4.3and3.5.0: three patches for HADOOP-19872:OpensslSecureRandom.cwhen OpenSSL is built without it;OpensslSecureRandom;ENGINE_by_idandENGINE_freelookups inOpensslCipher.c, plus the WindowsENGINE_by_freetypo.The touched files are byte-identical in 3.4.3, 3.5.0 and trunk.
The two follow-up patches are not part of HADOOP-19872: Exclude Engine API when building with OpenSSL 3.x apache/hadoop#8550.
hadoop
3.3.6, which hive3.1.3and4.0.0build against:Its
OpensslSecureRandom.cis byte-identical to 3.4.3's, and itsOpensslCipher.cnever looks up an ENGINE, because engine selection arrived in 3.4, so the third patch has nothing to guard;GCC 14 makes implicit function declarations an error, which
OpensslSecureRandom.ctrips over in the pre-1.1 lock helpers, and its headers no longer pull in<cstdint>for libhdfspp.Why the cipher needs the guard as well
apache/hadoop#8550 alone makes the build pass, but
OpensslCipher.initIDsstill resolvesENGINE_by_idandENGINE_freeat runtime.RHEL 10's headers define
OPENSSL_NO_ENGINE, yet itslibcryptostill exports both symbols, so the lookup succeeds there by accident.Against a
libcryptowithout them,initIDsthrowsUnsatisfiedLinkErrorand the AES/CTR codec falls back toJceAesCtrCryptoCodecwith only a warning, so encrypted data transfer keeps working and nothing reports that native crypto is gone.With the guard,
libhadoop.soreferences no ENGINE symbol at all.Testing
Built with
boil build hadoop=3.4.3,3.5.0.In both images:
hadoop checknative -areportsopenssl: true /lib64/libcrypto.so(openssl-libs 3.5.8).strings libhadoop.soshows noENGINE_symbol.OpensslAesCtrCryptoCodecwithOpensslSecureRandom: 1 MiB, ciphertext identical to JCE'sAES/CTR/NoPadding, decrypts back to the input.hadoop
3.3.6builds and passes the same three checks:openssl: true /lib64/libcrypto.so, noENGINE_string, and the AES/CTR round trip matching JCE.zstdandbzip2reportfalseinchecknative, as they do in the 26.7.0 hadoop image.hive
3.1.3builds against the patched 3.3.6, andschematool -dbType derby -initSchemacompletes in the image.Its
checknativereportsopenssl: falsebecause the hive image has nolibcrypto.sosymlink; the UBI 9 hive images behave the same, so this is not new.On a k3d cluster with the 26.7.0 operators, the HDFS kerberos test (both realms, ZooKeeper 3.9.5, OPA 1.16.2,
dfs.encrypt.data.transferwith AES/CTR) passes for 3.4.3 and 3.5.0.Its datanodes log
OpensslAesCtrCryptoCodec, which they do not when the codec falls back to JCE.Known UBI 10 differences
tputis not installed, so the hadoop scripts printtput: command not foundin places. Cosmetic.inotify-toolsRPM; it installs andinotifywaitruns on UBI 10.Open question
This changes the base of every image, not only Hadoop.
#1349 asks whether UBI 10 images should be built alongside the UBI 9 ones or replace them, and that is not settled.
Only hadoop
3.4.3and3.5.0ran on a cluster;3.3.6and hive3.1.3were built and checked in the image only.No image other than hadoop and hive was built or tested on UBI 10.
Definition of Done Checklist
Note
Not all of these items are applicable to all PRs, the author should update this template to only leave the boxes in that are relevant.
Please make sure all these things are done and tick the boxes
TIP: Running integration tests with a new product image
The image can be built and uploaded to the kind cluster with the following commands:
See the output of
boilto retrieve the image manifest URI for<MANIFEST_URI>.