JBTM-4038 JGroupsSlots and JGroupsRaftSlots benchmarks - #239
Conversation
📝 WalkthroughWalkthroughAdds JGroups and JGroups Raft configurations, Maven dependency wiring, three JMH slot-store benchmarks, transaction rollback cleanup, report filtering, slot-count headroom, and benchmark CSV row-count validation. ChangesJGroups benchmark support
Sequence Diagram(s)sequenceDiagram
participant JMHRunner
participant RaftChannels
participant JGroupsRaftSlots
participant Leader
participant Follower
JMHRunner->>RaftChannels: create and connect three channels
RaftChannels->>RaftChannels: elect leader
JMHRunner->>JGroupsRaftSlots: initialize node slot stores
Leader->>Follower: replicate pre-populated slots
JMHRunner->>Follower: readFromFollower()
JMHRunner->>Leader: writeToLeader()
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/scripts/benchmark.sh:
- Line 113: Update BM5’s JMH selector pattern so it matches the newly added
JGroupsSlotsBenchmark and JGroupsRaftSlotsBenchmark benchmarks, including their
testJGroupsSlotsStore and testJGroupsRaftSlotsStore methods, while retaining the
existing StoreBenchmark matches and expected count of 7.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 6999bbdf-b1be-4c03-ae87-b1357153804c
📒 Files selected for processing (6)
.github/scripts/benchmark.shnarayana/ArjunaJTA/jta/etc/jgroups-raft.xmlnarayana/ArjunaJTA/jta/etc/jgroups.xmlnarayana/ArjunaJTA/jta/pom.xmlnarayana/ArjunaJTA/jta/tests/classes/com/arjuna/ats/jta/xa/performance/JGroupsRaftSlotsBenchmark.javanarayana/ArjunaJTA/jta/tests/classes/com/arjuna/ats/jta/xa/performance/JGroupsSlotsBenchmark.java
3a97f23 to
c984b72
Compare
c0a91a7 to
5af708c
Compare
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/scripts/benchmark.sh:
- Line 113: Update the BM5 benchmark selector configuration to use a lower bound
of 9 instead of 8, ensuring all four added StoreBenchmark methods are required
while preserving the existing selector.
In
`@narayana/ArjunaJTA/jta/tests/classes/com/arjuna/ats/jta/xa/performance/JGroupsRaftSlotsStoreBenchmark.java`:
- Around line 90-115: Remove the stale “configure 3 node cluster” TODO from the
setup method in JGroupsRaftSlotsStoreBenchmark, since
JGroupsRaftClusterStoreBenchmark now covers that configuration. Leave the
existing single-node benchmark setup unchanged.
In
`@narayana/ArjunaJTA/jta/tests/classes/com/arjuna/ats/jta/xa/performance/JTAStoreBase.java`:
- Around line 113-127: Update the transaction setup and cleanup around
tm.begin() in JTAStoreBase so rollback occurs only when this method successfully
started the transaction. Track that state with a local transactionStarted flag,
set it after begin succeeds, and guard both the catch-path rollback and
finally-block rollback with it, preserving suppressed-exception handling.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 5676c249-55bc-4a74-9f89-97b5e1e0fa1b
📒 Files selected for processing (8)
.github/scripts/benchmark.shnarayana/ArjunaJTA/jta/etc/jgroups-raft.xmlnarayana/ArjunaJTA/jta/etc/jgroups.xmlnarayana/ArjunaJTA/jta/pom.xmlnarayana/ArjunaJTA/jta/tests/classes/com/arjuna/ats/jta/xa/performance/JGroupsRaftClusterStoreBenchmark.javanarayana/ArjunaJTA/jta/tests/classes/com/arjuna/ats/jta/xa/performance/JGroupsRaftSlotsStoreBenchmark.javanarayana/ArjunaJTA/jta/tests/classes/com/arjuna/ats/jta/xa/performance/JGroupsSlotsStoreBenchmark.javanarayana/ArjunaJTA/jta/tests/classes/com/arjuna/ats/jta/xa/performance/JTAStoreBase.java
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@narayana/ArjunaJTA/jta/tests/classes/com/arjuna/ats/jta/xa/performance/JGroupsRaftSlotsStoreBenchmark.java`:
- Around line 135-144: Update testJGroupsRaftSlotsStore so RollbackException is
not counted as a successful JMH operation: add and increment JMH auxiliary
counters for committed and rolled-back executions, and publish committed
throughput separately while recording rollback failures. Do not let the
exception handler return as an unclassified successful invocation; preserve the
existing transaction invocation and Blackhole consumption for committed results.
In
`@narayana/ArjunaJTA/jta/tests/classes/com/arjuna/ats/jta/xa/performance/JTAStoreBase.java`:
- Around line 120-127: Update the finally-block cleanup in JTAStoreBase so it
runs only when the transaction was started, and wrap both tm.getTransaction()
and rollback failures as best-effort cleanup. Preserve the original exception
while attaching any cleanup failure via addSuppressed, including failures
retrieving the transaction.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 8ffea7d3-2358-4ec7-87ae-a20ba7385b4f
📒 Files selected for processing (3)
narayana/ArjunaJTA/jta/tests/classes/com/arjuna/ats/jta/xa/performance/JGroupsRaftSlotsStoreBenchmark.javanarayana/ArjunaJTA/jta/tests/classes/com/arjuna/ats/jta/xa/performance/JGroupsSlotsStoreBenchmark.javanarayana/ArjunaJTA/jta/tests/classes/com/arjuna/ats/jta/xa/performance/JTAStoreBase.java
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/scripts/benchmark.sh:
- Line 113: Update the BM5 benchmark pattern to expect 11 CSV rows, accounting
for the committed and rolledBack auxiliary rows emitted by
JGroupsRaftSlotsStoreBenchmark while preserving the existing benchmark
selection.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: cd24571f-5b77-47d5-b175-7e85520ddcfd
📒 Files selected for processing (3)
.github/scripts/benchmark.shnarayana/ArjunaJTA/jta/tests/classes/com/arjuna/ats/jta/xa/performance/JGroupsRaftSlotsStoreBenchmark.javanarayana/ArjunaJTA/jta/tests/classes/com/arjuna/ats/jta/xa/performance/JTAStoreBase.java
…uster of JGroupsRaftSlots (using SHARED_LOOPBACK)
…ounts and has headroom and ensure that the slot config for numberOfSlots is applied SlotStoreEnvironmentBean (required because the SlotStoreAdaptor no-arg constructor creates the SlotStore with SlotStoreEnvironmentBean).
…d, so at high thread counts (e.g. -t 1600) the slot pool fills faster than transactions complete. Cap slot allocation to the THREADS constant and swallow RollbackException in the benchmark method so -foe true JMH flag does not kill the entire JMH run.
JGroupsRaftSlotsStoreBenchmark: replace bh.consume(false) on RollbackException with @AuxCounters so JMH reports committed and rolledBack throughput separately instead of inflating the primary score with failed operations. benchmark.sh: bump store benchmark count to 9 for JGroupsRaftClusterStoreBenchmark."
| throw new Error(e); | ||
| } catch (Exception e) { | ||
| try { | ||
| tm.rollback(); |
There was a problem hiding this comment.
This is very interesting. If I am correctly understanding we are also able to count the rolled back transactions with the benchmark now.
There was a problem hiding this comment.
We can, it's nice isn't it. We can count pretty much any numeric type and have it reported as a separate benchmark run.
|
Thanks @mmusgrov for your PR! I believe letter E is showing results of 'JGroupsRaftSlotsStoreBenchmark.testJGroupsRaftSlotsStore:rolledBack' which is 0, but JGroupsRaftSlotsStoreBenchmark.testJGroupsRaftSlotsStore is not listed instead (which means it is not visible in the png file). I can read '"Param: allowDirtyReads"' in the run output, I noticed allowDirtyReads is not a Param, which is great, but I don't understand how that works. Do the throughputs listed set is as false or true? |
I don't understand the output, it looks like a bug in the pre-existing io.narayana.perf.product.ProductComparison.java and io.narayana.perf.product.ReportGenerator.java logic - the store comparison bencharks are getting conflated with the JTA product comparisons when they are completely independent from each other. I will investigate and get back to you.
This was a difficult one to answer since multiple factors could be at play and I'm not a performance expert: hardware differences, fsync performance is critical (eg on tmpfs it's a no-op which obviously is unsuitable for transaction logs), error margins on the JMH measurements etc. but I'll provide my analysis with the caveat that it's by no means definitive: Even though slot store can process reads and writes concurrently Raft serializes through a single thread. Therefore
The CI benchmarks seem to corroborate this rationale: throughput increases at a faster rate as more threads are added with JGroupsSlotsStore - the crossover happening somewhere between 24 and 240 threads On my laptop both stores scale similarly with thread count. At 1 and 240 threads they're roughly equal and at 24 threads ReplCache has the edge. The error margins are large (especially ReplCache's ±36,010 at 240 threads), so the differences aren't statistically significant. A google search reports that JGroups Raft supports [quote] "batching multiple pending proposals and entries into fewer network/log append operations, relying on underlying transport optimization and the architectural message bundling/batching capabilities provided by the core JGroups framework". So this will account for some of the performance benefits seen by Raft which might account for some of the higher than expected throughput. Some other things to consider (though I'm not sure how to relate them to the varying benchmark results on CI and my laptop) are:
The JMH The reason I did this is I wanted to characterise the overhead of routing reads and writes through the Raft leader: readFromFollower + allowDirtyReads=true is ~800x faster on my laptop with "-t 4 -f 1 -wi 3 -i 5 -r 1" than the other three combinations which makes sense because "reads are served from each node’s local in-memory state machine" so readers avoid the Raft protocol overhead of getting the value from the leader. readFromLeader + allowDirtyReads=true|false makes little difference because the leader can consult the local in-memory state machine. The kind of layout of the JMH results should include a column called (allowDirtyReads), on my laptop, for example: I will say that benchmarking on CI and laptops has always been fraught with issues. |
Thanks @mmusgrov for clarifying! This was surely useful. The PR looks good and it would be great to improve the performance output in the future (especially the png file which is not so clear IMO). Having the allowDirtyReads would be optimal or alternatively having methods like writeToLeaderWithallowDirtyReadsFalse can be good too. [UPDATE] For 'allowDirtyReads'=true I would expect to see similar performance for 'write' and improved performance for 'readFromFollower' executions as the reading is done in the local cache. |
marcosgopen
left a comment
marcosgopen
left a comment
There was a problem hiding this comment.
Thanks @mmusgrov . I am approving but I mention I am not an expert on the jgroup configuration file so I suppose those file have the default/recommended configuration for this performance testing. Additionally, as I mentioned above, we have space for improvements in the performance output files (readability) in the future.
…the counters for committed and rolledBack and not summing to the total number of transactions because the metric includes the warmup time whereas the primary benchmark only reports during the measurement window.
…coded Comparison.length() and skip JMH sub-metrics (:committed, :rolledBack) that were overwriting aggregate results
@marcosgopen I raised an issue for this problem and pushed a fix to this PR since it needs the fix |
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@narayana/ArjunaJTA/jta/tests/classes/com/arjuna/ats/jta/xa/performance/JGroupsRaftClusterStoreBenchmark.java`:
- Around line 118-160: Replace the fixed 500 ms sleep after the leader
pre-population loop with an explicit wait that verifies followerSlots has
applied all PREPOPULATE_SLOTS entries before benchmarking begins. Use the
existing JGroupsRaftSlots state or application/index inspection APIs available
in the class, and fail or time out clearly if follower replication does not
complete; keep readFromFollower() from starting until the follower is caught up.
In
`@narayana/ArjunaJTA/jta/tests/classes/com/arjuna/ats/jta/xa/performance/JTAStoreBase.java`:
- Around line 109-115: In the catch block where tm.rollback() is called
unconditionally, add a guard condition to check that tm.getTransaction() is
non-null before attempting the rollback. When tm.commit() throws
RollbackException, the transaction manager removes the thread association in a
finally block before propagating the exception, so calling tm.rollback()
unconditionally triggers an IllegalStateException. Only call tm.rollback() if
the transaction is still associated with the current thread, which preserves
cleanup for cases where the transaction remains active due to enlistment
failures.
In
`@narayana/ArjunaJTA/jta/tests/classes/io/narayana/perf/product/ReportGenerator.java`:
- Around line 137-138: The key used in row.addColumn is constructed from only
the class name extracted from prod, which causes different benchmark methods and
parameter values to collide under the same key, allowing later rows to overwrite
earlier ones. Augment the key passed to row.addColumn to include the benchmark
method name and any relevant parameter values (such as allowDirtyReads) along
with the class name, ensuring each unique combination of class, method, and
parameters produces a distinct key that preserves all benchmark series in the
report.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 9ff8c15c-c43a-4d44-b788-423e22945212
📒 Files selected for processing (9)
.github/scripts/benchmark.shnarayana/ArjunaJTA/jta/etc/jgroups-raft.xmlnarayana/ArjunaJTA/jta/etc/jgroups.xmlnarayana/ArjunaJTA/jta/pom.xmlnarayana/ArjunaJTA/jta/tests/classes/com/arjuna/ats/jta/xa/performance/JGroupsRaftClusterStoreBenchmark.javanarayana/ArjunaJTA/jta/tests/classes/com/arjuna/ats/jta/xa/performance/JGroupsRaftSlotsStoreBenchmark.javanarayana/ArjunaJTA/jta/tests/classes/com/arjuna/ats/jta/xa/performance/JGroupsSlotsStoreBenchmark.javanarayana/ArjunaJTA/jta/tests/classes/com/arjuna/ats/jta/xa/performance/JTAStoreBase.javanarayana/ArjunaJTA/jta/tests/classes/io/narayana/perf/product/ReportGenerator.java
| waitForLeader(channels, 10_000); | ||
|
|
||
| // Phase 2: Create JGroupsRaftSlots with pre-configured channels | ||
| for (int i = 0; i < 3; i++) { | ||
| configs[i] = new JGroupsRaftStoreEnvironmentBean(); | ||
| configs[i].setJGroupsConfigFileName("jgroups-raft.xml"); | ||
| configs[i].setNodeAddress(NODE_NAMES[i]); | ||
| configs[i].setClusterName(clusterName); | ||
| configs[i].setCacheName(clusterName); | ||
| configs[i].setStoreDir(STORE_BASE_DIR + "/" + NODE_NAMES[i]); | ||
| configs[i].setNumberOfSlots(NUM_SLOTS); | ||
| configs[i].setRaftMembers(RAFT_MEMBERS); | ||
| configs[i].setRaftLogFsync(false); | ||
| configs[i].setRaftTimeout(5000); | ||
| configs[i].setAllowDirtyReads(allowDirtyReads); | ||
| configs[i].setPreConfiguredChannel(channels[i]); | ||
| configs[i].setPreConfiguredStateMachine(stateMachines[i]); | ||
|
|
||
| allSlots[i] = new JGroupsRaftSlots(); | ||
| allSlots[i].init(configs[i]); | ||
| } | ||
|
|
||
| // Identify leader and a follower | ||
| for (JGroupsRaftSlots slots : allSlots) { | ||
| String role = slots.getRole(); | ||
| if (Role.Leader.name().equals(role) && leaderSlots == null) { | ||
| leaderSlots = slots; | ||
| } else if (Role.Follower.name().equals(role) && followerSlots == null) { | ||
| followerSlots = slots; | ||
| } | ||
| } | ||
|
|
||
| if (leaderSlots == null || followerSlots == null) { | ||
| throw new IllegalStateException("Cluster did not form correctly: leader=" + leaderSlots + " follower=" + followerSlots); | ||
| } | ||
|
|
||
| // Pre-populate slots via the leader | ||
| for (int i = 0; i < PREPOPULATE_SLOTS; i++) { | ||
| leaderSlots.write(i, SLOT_DATA, true); | ||
| } | ||
|
|
||
| // Brief pause for replication to followers | ||
| TimeUnit.MILLISECONDS.sleep(500); |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
slots_file="$(fd -a -t f '^JGroupsRaftSlots\.java$' . | head -n 1)"
if [[ -z "${slots_file}" ]]; then
echo "JGroupsRaftSlots.java was not found." >&2
exit 1
fi
# Inspect whether write completion requires every follower to apply the entry.
ast-grep outline "${slots_file}" --items all --match JGroupsRaftSlots
rg -n -C 12 '\b(write|read)\s*\(|ReplicatedStateMachine|allowDirtyReads|preConfigured' "${slots_file}"
# Inspect the pinned Raft dependency and related benchmark setup.
rg -n -C 4 'jgroups-raft|JGroupsRaftClusterStoreBenchmark|PREPOPULATE_SLOTS|waitForLeader' \
narayana/ArjunaJTA/jta/pom.xml \
narayana/ArjunaJTA/jta/tests/classes/com/arjuna/ats/jta/xa/performanceRepository: jbosstm/performance
Length of output: 194
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "Repo files matching JGroupsRaftSlots:"
git ls-files | rg '(^|/)JGroupsRaftSlots\.java$|JGroupsRaftClusterStoreBenchmark\.java$|jgroups-raft|JGroupsRaft' | head -200
echo
echo "Benchmark file outline:"
benchmark_file="$(git ls-files | rg 'JGroupsRaftClusterStoreBenchmark\.java$' | head -n 1 || true)"
if [[ -n "${benchmark_file}" ]]; then
wc -l "${benchmark_file}"
ast-grep outline "${benchmark_file}" --match JGroupsRaftClusterStoreBenchmark || true
sed -n '1,240p' "${benchmark_file}"
fiRepository: jbosstm/performance
Length of output: 9459
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "Find JGroupsRaftSlots source:"
git ls-files | rg '(^|/)JGroupsRaftSlots\.java$' | while read -r path; do
echo "--- $path"
wc -l "$path"
ast-gps="$(ast-grep outline "$path" --items all --match JGroupsRaftSlots 2>/dev/null || true)"
if [[ -n "${ast-gps}" ]]; then
echo "$ast-gps"
fi
echo
rg -n -C 14 '\b(write|read)\s*\(|applySnapshot|applyCommit|allowDirtyReads|leader|follower|preConfigured' "$path"
done
echo
echo "Find related sources:"
git ls-files | rg 'objectstore/slot/jgroups|ReplicatedStateMachine|JGroupsRaft' | head -100Repository: jbosstm/performance
Length of output: 187
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "Find JGroupsRaftSlots references:"
rg -n 'class JGroupsRaftSlots|new JGroupsRaftSlots\(|interface JGroupsRaftSlots|implements JGroupsRaftSlots|JGroupsRaftSlots' .
echo
echo "Find ReplicatedStateMachine references:"
rg -n 'class ReplicatedStateMachine|ReplicatedStateMachine<|allowDirtyReads|applyCommit\(|leaderCommit|commitIndex' .
echo
echo "List files under narayana ArjunaJTA path:"
git ls-files narayana/ArjunaJTA | rg 'JGroups|Raft|Slots|ReplicatedStateMachine|objectstore' | head -200Repository: jbosstm/performance
Length of output: 6313
🌐 Web query:
JGroups Raft ReplicatedStateMachine write returns after follower application commitIndex leader commit
💡 Result:
In JGroups Raft, a write request (such as put or remove in ReplicatedStateMachine) returns to the caller only after the leader has successfully replicated the log entry to a majority of the cluster members and committed it [1][2]. Specifically, the process follows these requirements of the Raft consensus algorithm: 1. Leader Reception: The leader receives the write request from the client [1][2]. 2. Replication: The leader appends the entry to its persistent log and asynchronously replicates it to the other nodes (followers) in the cluster [1][2]. 3. Majority Agreement: The leader waits until the entry has been successfully persisted by a majority of the cluster members [1][2]. 4. Commit and Response: Once the majority acknowledgment is received, the leader updates its commitIndex, applies the entry to its state machine, and finally returns the result of the operation to the client [1][2]. The client does not need to wait for individual followers to explicitly report their local application of the commitIndex; the consensus guarantee is satisfied once the leader confirms that a majority of the cluster has persisted the entry in their logs [1][2]. If the leader cannot replicate the entry to a majority (e.g., due to network partitions or insufficient nodes), the request will time out rather than returning a successful result [2].
Citations:
- 1: https://research.redhat.com/blog/2024/03/20/correctness-in-distributed-systems-the-case-of-jgroups-raft/
- 2: https://www.infoq.com/articles/JGroups-raft-Primer/
🌐 Web query:
JGroups Raft ReplicatedStateMachine allowDirtyReads read stale follower implementation
💡 Result:
In JGroups Raft, the ReplicatedStateMachine allows users to trade off linearizability for performance by enabling "dirty reads," where read operations are served locally from a node's state machine rather than undergoing the full Raft consensus process [1][2]. Key aspects of this implementation include: 1. Method Configuration: You can toggle this behavior using the ReplicatedStateMachine.allowDirtyReads(boolean) method [1][2]. By default, early versions of components like CounterService allowed dirty reads to improve response times, but this behavior can be explicitly disabled (e.g., allowDirtyReads(false)) to force quorum reads [1][2]. 2. Consensus Mechanism: When allowDirtyReads is set to false (or when not enabled), read requests are routed through the RAFT protocol and require a majority consensus to complete, ensuring linearizable reads at the cost of higher latency [1]. 3. Learner Nodes: Newer versions of JGroups Raft have introduced the role of "learner" nodes [3][4]. Learner nodes are designed to scale read operations significantly by serving dirty reads from their local state machine without participating in the quorum for entry commitment or leader election [3][4]. 4. Consistency Trade-offs: Enabling dirty reads means the returned value may be stale because the local node may not have processed the most recent committed updates from the Raft leader [1][2]. Note: Some search results returned information regarding ArangoDB's allowDirtyReads feature [5][6][7]; this is distinct from the JGroups Raft implementation. Ensure your configuration specifically targets the JGroups-based API [1].
Citations:
- 1: https://jgroups-extras.github.io/jgroups-raft/manual/index.html
- 2: https://belaban.github.io/jgroups-raft/manual/index.html
- 3: https://github.com/jgroups-extras/jgroups-raft/blob/bc1702d66d6d741105d97e34fe8902b1eb8c05c8/doc/design/LearnerNodes.adoc
- 4: Introduce learner role jgroups-extras/jgroups-raft#354
- 5: https://github.com/arangodb/arangojs/blob/v10.3.1/src/queries.ts
- 6: https://arangodb.github.io/arangojs/8.1.0/types/database.QueryOptions.html
- 7: https://github.com/arangodb/go-driver/blob/v1.6.9/context.go
🌐 Web query:
"source" "JGroupsRaftSlots" "JGroupsRaftStoreEnvironmentBean" "allowDirtyReads"
💡 Result:
In JGroups-raft, the allowDirtyReads(boolean) method is used to configure whether read operations on a state machine (such as a CounterService) are allowed to return potentially stale data [1][2]. When allowDirtyReads is set to true (the default in some versions), read requests are executed locally against the node's state machine without going through the Raft consensus process [1][2][3]. This provides lower latency but risks returning stale values because the local state machine may not yet have applied all committed writes [1][2][3]. When set to false, read requests are routed through the Raft protocol, requiring a majority (quorum) to complete [1]. This ensures linearizable reads (always returning the most recent committed value) but incurs higher latency due to the consensus overhead [1]. The allowDirtyReads configuration is typically accessed via state machine implementations like CounterService [1][2]. While you mentioned JGroupsRaftStoreEnvironmentBean, note that the core configuration for read consistency is primarily managed at the state machine or RaftHandle level rather than through an environment bean of that specific name [1][4]. {value: "In JGroups-raft, allowDirtyReads(boolean) controls read consistency. true enables faster, local, potentially stale reads, while false forces linearizable, quorum-based reads.", citations: [1][2][3], confidence: 0.9}
Citations:
- 1: https://jgroups-extras.github.io/jgroups-raft/manual/index.html
- 2: https://belaban.github.io/jgroups-raft/manual/index.html
- 3: https://research.redhat.com/blog/2024/03/20/correctness-in-distributed-systems-the-case-of-jgroups-raft/
- 4: https://github.com/jgroups-extras/jgroups-raft/blob/main/doc/manual/overview.adoc
Wait for follower replication before the read benchmark starts.
leadserSlots.write(...) returns after Raft replication/commit, not after followerSlots has applied every index. A fixed 500 ms pause does not establish that the selected follower has applied the PREPOPULATE_SLOTS writes. Add an explicit follower-application wait before JMH measurement starts, so readFromFollower() does not measure stale reads or a 0-hit path.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@narayana/ArjunaJTA/jta/tests/classes/com/arjuna/ats/jta/xa/performance/JGroupsRaftClusterStoreBenchmark.java`
around lines 118 - 160, Replace the fixed 500 ms sleep after the leader
pre-population loop with an explicit wait that verifies followerSlots has
applied all PREPOPULATE_SLOTS entries before benchmarking begins. Use the
existing JGroupsRaftSlots state or application/index inspection APIs available
in the class, and fail or time out clearly if follower replication does not
complete; keep readFromFollower() from starting until the follower is caught up.
| } catch (Exception e) { | ||
| try { | ||
| tm.rollback(); | ||
| } catch (Exception suppressed) { | ||
| e.addSuppressed(suppressed); | ||
| } | ||
| throw e; |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Avoid a second rollback after tm.commit() has finished the transaction.
When tm.commit() throws RollbackException, the configured JBTM-4038 transaction manager removes the thread association before it propagates that exception. At Line 111, tm.rollback() then throws IllegalStateException, and Line 113 adds it as suppressed. This adds failure-path work to each commit-time rollback and changes the benchmark total and rolledBack throughput. (raw.githubusercontent.com)
Call tm.rollback() only if tm.getTransaction() is still non-null. This preserves cleanup for enlistment failures that leave the transaction active.
Proposed fix
} catch (Exception e) {
try {
- tm.rollback();
+ if (tm.getTransaction() != null) {
+ tm.rollback();
+ }
} catch (Exception suppressed) {
e.addSuppressed(suppressed);
}#!/bin/bash
set -euo pipefail
impl="$(fd -a '^TransactionImple\.java$' . | sed -n '1p')"
base="$(fd -a '^BaseTransaction\.java$' . | sed -n '1p')"
# Expect: commitAndDisassociate removes the association in a finally block.
if [[ -n "$impl" ]]; then
ast-grep outline "$impl" --items all
rg -n -C 14 'commitAndDisassociate|removeTransaction' "$impl"
fi
# Expect: TransactionManager.rollback rejects an absent thread association.
if [[ -n "$base" ]]; then
ast-grep outline "$base" --items all
rg -n -C 8 '\brollback\s*\(' "$base"
fi🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@narayana/ArjunaJTA/jta/tests/classes/com/arjuna/ats/jta/xa/performance/JTAStoreBase.java`
around lines 109 - 115, In the catch block where tm.rollback() is called
unconditionally, add a guard condition to check that tm.getTransaction() is
non-null before attempting the rollback. When tm.commit() throws
RollbackException, the transaction manager removes the thread association in a
finally block before propagating the exception, so calling tm.rollback()
unconditionally triggers an IllegalStateException. Only call tm.rollback() if
the transaction is still associated with the current thread, which preserves
cleanup for cases where the transaction remains active due to enlistment
failures.
| // the class names follow the format: <product><PATTERN2> e.g. NarayanaComparison or DiskSlotsStoreBenchmark | ||
| row.addColumn(prod.substring(0, prod.length() - PATTERN2.length()), tput.doubleValue()); |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift
Preserve benchmark method and parameter identity.
prod contains only the class name. Row.addColumn uses it as the map key. Therefore, JGroupsRaftClusterStoreBenchmark.readFromFollower and writeToLeader, including both allowDirtyReads values, use the same JGroupsRaftCluster key for one thread count. Later rows overwrite earlier rows. The table and PNG then lose benchmark series.
Include the benchmark method and parameter values in the series key, or create separate report rows before calling row.addColumn.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@narayana/ArjunaJTA/jta/tests/classes/io/narayana/perf/product/ReportGenerator.java`
around lines 137 - 138, The key used in row.addColumn is constructed from only
the class name extracted from prod, which causes different benchmark methods and
parameter values to collide under the same key, allowing later rows to overwrite
earlier ones. Augment the key passed to row.addColumn to include the benchmark
method name and any relevant parameter values (such as allowDirtyReads) along
with the class name, ensuring each unique combination of class, method, and
parameters produces a distinct key that preserves all benchmark series in the
report.
2 participants
https://redhat.atlassian.net/browse/JBTM-4038
NARAYANA_REPO=mmusgrov
NARAYANA_BRANCH=JBTM-4038
NY_BRANCH=JBTM-4038
THREAD_COUNTS=1 24 240
COMPARE_STORES=y
COMPARE_IMPLEMENTATIONS=n
COMPARE_TRANSPORTS=n
COMPARE_JOURNAL_PARAMETERS=n
JDK25