[wu-sheng]

Share the Lettuce batch span among all commands of the batch

Follow-up to #843, which left three limitations of the Lettuce/BATCH_WRITE span. #843 hung that span on the last
command of the batch that was not done yet, so:

1. A failure of any other command of the batch was missed. For example, an earlier INCR on a string key got
   ERR value is not an integer, while the batch span was a success.
2. The span ended too early when the last command was cancelled. The other commands were still waiting for their
   replies.
3. A cancellation while the batch was being written could leave the batch with no span, or with two. No span: the
   chosen command is cancelled before its span is set. Two spans: a delegate writer chooses another command after the
   outer writer's choice is cancelled.

Fix

• One span shared by every command of a write (RedisWriteSpan):
  • Each command of the batch holds a share of the span. Its share is taken once, atomically, by whichever runs first:
    complete(), completeExceptionally(), cancel(), the writer's re-check of a command that is already done, or
    the writer's exception handler.
  • The span finishes when the last share is released. The writer holds a share of its own until every command has
    one, so the span cannot finish while the batch is still being set up.
  • Any failure makes the span an error. Only the first failure is logged, so a batch that fails as a whole (closed
    connection, timeouts) does not log one error per command. A later exception still makes the span an error, unless
    it is one of the agent's ignored exceptions (StatusCheckService).
  • A single command uses the same class, with one share.
• Nested writers reuse the span. Expiry, listener, master/replica and pub/sub writers delegate the write. If any
  command of the collection already has a span, the delegate only sets the peer. This also covers the cluster ASK
  write [ASKING, cmd].
• Cluster batches. ClusterDistributionChannelWriter splits a batch by slot and writes each slot's commands to its
  node as a batch of their own. So the cluster writer no longer traces a collection: each slot's batch gets its own
  span with its node's peer, and keyless commands keep their single-command spans. This is the same span count as
  before.
  • The class is package-private, so the plugin matches it by name.
• Error replies of reactive commands. A Redis error reply completes a reactive command through complete(), with
  the error left in its output. Before this change, such a span stayed a success. The complete() interceptor now
  checks CommandOutput#hasError().
  • Async and sync commands already recorded error replies, because AsyncCommand completes its inner command
    exceptionally first.

The span still records only cache.type and cache.cmd=BATCH_WRITE. It carries no list of the batch's commands.

Tests

• lettuce-scenario and lettuce-6.5.x-scenario add:

  • A batch [INCR on a string key, SET]. Its span must be an error, carrying one log of the
    RedisCommandExecutionException.
  • A reactive INCR on the same key. Its span must be an error with the reply message.

• RedisChannelWriterInterceptorTest adds unit tests for the cases a scenario cannot reproduce. Each of the four batch
  tests below fails against main's code:

  • A batch whose last command is cancelled first still ends with the other command.
  • An earlier command's error reply fails the batch span.
  • Failures are logged once.
  • An ignored first exception does not hide a later failure.
  • A command cancelled while the batch is traced leaves one span.
  • A delegate writer keeps the outer writer's span.

• Ran locally for every version in the four Lettuce support-version.list files (16 versions, Lettuce 5.0.5 to
  6.7.1), with the mock collector, the agent, redis:3.2.9-alpine and the validator. All pass.

• Add a unit test to verify that the fix works.

• Explain briefly why the bug exists and how to fix it.

• Update the CHANGES log.