[grepjava]

perf client stamps and reads message times with System.nanoTime(), while perf producer and perf consumer use the MicrosClock wall clock. When perf client consumes messages sent by perf producer, the latency comes out negative, the histogram throws out of onMessage, and the message is redelivered. The run stalls and never ends.

All three commands now take their clock from MicrosClock.timeProvider, including the fallback and its warning. It has to be the wall clock, as it's the only one that means anything across processes. perf client latencies are therefore measured on the wall clock now, like those of the other two commands.

Wall clocks can still differ between hosts or step back, so a negative latency is now recorded as 0 instead of throwing, for the receive latency and for the send latency.

MessageListenerBenchmarkTest checks a message stamped ahead of the receiver's clock and one stamped behind it. The first fails without the fix, with Histogram recorded value cannot be negative.

Tested against a broker built from main, changing only the CLI jar, with 100000 messages sent by perf producer --persistent --message-count 100000, then perf client --persistent --duration 20 --warmup 5 on the same queue:

• Without the fix: still running at a 60 second timeout, with 16567 of these exceptions.
• With the fix: ended normally with no exceptions, 225478 sent and 325478 received, which is the 100000 plus those sent.

https://issues.apache.org/jira/browse/ARTEMIS-6355

[clebertsuconic]

I understand the change.. but I believe perf client is not supposed to run in parallel with producer and consumer, right?

Also: the new storage I'm working on will highly benefit from parallel queues, parallel producers... we should find a way to measure that.

[grepjava]

Not in parallel, no. It was sequential: perf producer filled a backlog and finished, then perf client started on the same queue, for the paging tests on #6838. It reads the producer's messages, stamped by the other clock, and the run never ends. That's why I used perf producer and perf consumer side by side for those tests, as they share the clock.

If you don't want to support that mix, the negative latency handling still seems worth having: wall clocks can differ between hosts, so perf producer and perf consumer on two hosts can hit the same exception, and a negative latency shouldn't cause a redelivery.

For parallel queues and producers, the connection tests I posted on #6838 are one way to measure it: perf client --num-destinations 16 --producers 1 --consumers 1 --threads 4 at 6, 12 and 24 connections. More connections made no difference on my box, at the default flush period or at 0.1 ms, but that's 4 shared CPUs with PostgreSQL on the same disk, so it can't show what a real database server would do. With #6870 in, --tx also lets the consumers commit in batches, so I can add that to the runs if it's useful.