[janhoy]

When solrSecurity.probesRequireAuth is set, the generated probe command piped the output of 'solr api' through 'grep -v JAVA_TOOL_OPTIONS' to keep the JVM's "Picked up JAVA_TOOL_OPTIONS: ..." line, which contains the basic auth credentials, out of the probe output that Kubernetes surfaces in pod events.

A pipeline exits with the status of its last command, so the exit code of 'solr api' was discarded and grep's 0 was reported instead. The probes therefore always succeeded, and a pod whose credentials Solr rejects with a 401 was still reported as healthy and ready.

Capture the output in a variable instead of piping, save the exit status, then filter and exit with the saved status. POSIX sh only, so this does not rely on 'pipefail', which dash does not support.

Fixes #799

[danielmeint]

Found this as well and it seems more straightforward to solve this via bin/solr env vars:

SOLR_AUTH_TYPE=basic SOLR_AUTHENTICATION_OPTS="-Dbasicauth=$(cat <dir>/username):$(cat <dir>/password)" solr api -get "<url>"

Passing -Dbasicauth via JAVA_TOOL_OPTIONS also seems to break on Solr 10; and it avoids the "Picked up JAVA_TOOL_OPTIONS" banner

[janhoy]

🤖 Thanks @danielmeint — you're right, and for a sharper reason than "JAVA_TOOL_OPTIONS is awkward": bin/solr derives the SolrJ auth mechanism from SOLR_AUTH_TYPE, and that mapping changed in Solr 10, while we hardcode the 9-era one.

Solr	what SOLR_AUTH_TYPE=basic sets
8.11, 9.11	-Dsolr.httpclient.builder.factory=…impl.PreemptiveBasicAuthClientBuilderFactory (what we hardcode today)
10.1, 11.0	-Dsolr.solrj.http.jetty.customizer=…jetty.PreemptiveBasicAuthClientCustomizer

Verified against authenticated Solr instances (blockUnknown: true, probing /solr/admin/info/system):

mechanism	8.11	9.11	10.1	(main)
JAVA_TOOL_OPTIONS + -Dbasicauth + builder factory (today)	✅ banner, creds in output	✅ banner, creds in output	❌ 401	❌
SOLR_AUTH_TYPE=basic + SOLR_AUTHENTICATION_OPTS="-Dbasicauth=u:p"	✅	✅	✅	✅ (deprecation WARN)
SOLR_AUTH_TYPE=basic + SOLR_SECURITY_AUTH_BASICAUTH_CREDENTIALS=u:p (env only)	❌	❌	✅	✅
solr api --credentials u:p	flag absent	flag absent	✅	✅

Every working row exits 0 on success and 1 on a wrong password, with no banner and no credentials in the output — so #799 is fixed by construction, with no output filter and no saved-exit-status dance.

Plan, assuming this merges after #827:

1. Pass the credentials through bin/solr's own env vars and drop the pipeline entirely, so the probe exit status is the CLI's: SOLR_AUTH_TYPE=basic SOLR_AUTHENTICATION_OPTS="-Dbasicauth=$(cat …/username):$(cat …/password)" solr api <url-flag> "<url>".
2. Remove what that makes dead: the hardcoded -Dsolr.httpclient.builder.factory sysprop, the grep -v JAVA_TOOL_OPTIONS filter, the probe_output/probe_status capture, and the unit tests asserting the filter.
3. Drop Add Solr 10 compatibility support #827's --credentials branch for Solr 10 — one mechanism now covers 8 through 11, and --credentials puts the credentials in the process arguments anyway.
4. Keep Add Solr 10 compatibility support #827's major-version gate for the URL flag only, which is the one thing that genuinely needs branching: -get on 8, --solr-url on 9+ (-get is deprecated for removal since 9.7 and rejected outright in 10).

Optional, in the same PR or later:

5. On Solr 10+, set the credentials via the SOLR_SECURITY_AUTH_BASICAUTH_CREDENTIALS env var instead of a sysprop, which keeps them out of the java argv (both SOLR_AUTHENTICATION_OPTS and --credentials show up in ps inside the pod). It does not work on 8/9, so it needs the version gate.

Worth noting either way: the credentials stop reaching the probe output that Kubernetes surfaces in pod events, which was the more serious exposure in #799 — argv is readable only with a shell in the pod, pod events with namespace read access.