What happened
On conforma/cli PR #3418, two review runs were triggered: Run 1 (29493548960, July 16) reviewed the initial commit (f9b84cd) and approved. Run 2 (29731575429, July 20) was triggered when the author merged main into their branch (commit 7d2b698 'Merge branch conforma:main into EC-1912-validate-intoto-type'). The merge introduced no changes to PR-specific files — the PR diff (attestation.go and attestation_test.go) was identical between both runs. Run 2 consumed tokens (cache_read=211,227, cache_create=63,212, out=6,321) and produced the same approval with the same findings, then failed at the post-script due to the 422 bug (see fullsend-ai/agents#430).
What could go better
This is a known pattern tracked in #4991. The pre-script should detect that the only new commit since the prior review is a merge from the base branch (main), compare the PR diff against the previously reviewed diff, and skip the review when unchanged. This would have saved the entire run — approximately 4 minutes of compute time plus wasted tokens. The pattern is common: developers frequently merge main into their branch to resolve conflicts or update CI, without changing any PR-specific code.
Proposed change
File this as additional evidence on #4991. The pre-script should implement a diff-based skip mechanism: (1) detect if all new commits since the prior review SHA are merge commits from the base branch, (2) compute the PR diff (git diff main...HEAD) and compare against the prior review's diff, (3) skip the review if the diffs are identical. The prior review SHA was empty (PRIOR_REVIEW_SHA was blank in the run logs), which suggests the pre-fetch step may also need improvement to recover the SHA from the previous review's status comment.
Validation criteria
When #4991 is implemented, verify that merge-from-base commits on PRs with a prior review do not trigger a new review run. Specifically: (1) a PR with an existing review should not get re-reviewed when the author merges main, (2) the skip should be logged clearly in the status comment, (3) actual code changes in new commits should still trigger re-review.
Generated by retro agent from conforma/cli#3418
What happened
On conforma/cli PR #3418, two review runs were triggered: Run 1 (29493548960, July 16) reviewed the initial commit (f9b84cd) and approved. Run 2 (29731575429, July 20) was triggered when the author merged main into their branch (commit 7d2b698 'Merge branch conforma:main into EC-1912-validate-intoto-type'). The merge introduced no changes to PR-specific files — the PR diff (attestation.go and attestation_test.go) was identical between both runs. Run 2 consumed tokens (cache_read=211,227, cache_create=63,212, out=6,321) and produced the same approval with the same findings, then failed at the post-script due to the 422 bug (see fullsend-ai/agents#430).
What could go better
This is a known pattern tracked in #4991. The pre-script should detect that the only new commit since the prior review is a merge from the base branch (main), compare the PR diff against the previously reviewed diff, and skip the review when unchanged. This would have saved the entire run — approximately 4 minutes of compute time plus wasted tokens. The pattern is common: developers frequently merge main into their branch to resolve conflicts or update CI, without changing any PR-specific code.
Proposed change
File this as additional evidence on #4991. The pre-script should implement a diff-based skip mechanism: (1) detect if all new commits since the prior review SHA are merge commits from the base branch, (2) compute the PR diff (git diff main...HEAD) and compare against the prior review's diff, (3) skip the review if the diffs are identical. The prior review SHA was empty (PRIOR_REVIEW_SHA was blank in the run logs), which suggests the pre-fetch step may also need improvement to recover the SHA from the previous review's status comment.
Validation criteria
When #4991 is implemented, verify that merge-from-base commits on PRs with a prior review do not trigger a new review run. Specifically: (1) a PR with an existing review should not get re-reviewed when the author merges main, (2) the skip should be logged clearly in the status comment, (3) actual code changes in new commits should still trigger re-review.
Generated by retro agent from conforma/cli#3418