Skip to content

Run e2e tests on push to main instead of on every PR#3417

Open
jsmid1 wants to merge 1 commit into
conforma:mainfrom
jsmid1:EC-1930
Open

Run e2e tests on push to main instead of on every PR#3417
jsmid1 wants to merge 1 commit into
conforma:mainfrom
jsmid1:EC-1930

Conversation

@jsmid1

@jsmid1 jsmid1 commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

The cli-e2e PipelineRun runs the conforma/e2e-tests pipeline which takes ~25 minutes per run. Moving it from pull_request to push trigger means e2e tests run post-merge on main.

Ref: https://issues.redhat.com/browse/EC-1930

@jsmid1
jsmid1 requested a review from cuipinghuo July 16, 2026 08:10
@coderabbitai

coderabbitai Bot commented Jul 16, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@jsmid1, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 20 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Enterprise

Run ID: d86091c5-4b35-4ce1-8003-fd2980cb4c4e

📥 Commits

Reviewing files that changed from the base of the PR and between 33b9c36 and ae7c2c2.

📒 Files selected for processing (1)
  • .tekton/cli-e2e-push.yaml
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@fullsend-ai-review

fullsend-ai-review Bot commented Jul 16, 2026

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 8:11 AM UTC · Completed 8:14 AM UTC
Commit: 87c4a29 · View workflow run →

@fullsend-ai-review

fullsend-ai-review Bot commented Jul 16, 2026

Copy link
Copy Markdown

Review

Verdict: approve · no blocking findings

Summary

This PR moves the cli-e2e PipelineRun trigger from pull_request to push on main, reducing ~25-minute e2e test overhead on every PR. The change is a single file rename with four targeted edits:

Change Rationale
File rename cli-e2e-pull-request.yamlcli-e2e-push.yaml Matches naming convention (cli-main-push.yaml)
Removed pull_request_number annotation Push events don't carry a PR number
cancel-in-progress truefalse Consistent with existing push-trigger pattern
CEL expression "pull_request""push" Core trigger change
Resource name cli-e2e-on-pull-requestcli-e2e-on-push Name reflects new trigger

Dimension analysis

  • Correctness — All edits are internally consistent. The resulting YAML matches the conventions established by cli-main-push.yaml (no PR-number annotation, cancel-in-progress: false, push CEL expression). No template variables are broken; {{revision}}, {{target_branch}}, and {{source_url}} remain valid for push events. The rest of the spec (params, pipelineRef, serviceAccountName) is untouched.

  • Security — No security impact. This is a CI trigger configuration change; no credentials, RBAC rules, or secret references are modified.

  • Intent & coherence — The change precisely matches the stated intent (Ref: EC-1930). Moving expensive e2e tests from pre-merge to post-merge is an intentional CI optimization tradeoff. The scope is minimal — one file, one concern.

  • Style/conventions — Follows the repository's existing .tekton/ naming and annotation patterns. The push-variant conventions are an exact match with cli-main-push.yaml.

  • Documentation — No documentation updates needed for a CI configuration change internal to .tekton/.

  • Cross-repo contracts — No APIs, schemas, or interfaces are affected. The referenced conforma/e2e-tests pipeline and its parameters are unchanged.

Previous run

Review

Verdict: Approve

Summary

This PR moves the e2e test PipelineRun trigger from pull_request to push on main, reducing CI time for PRs by ~25 minutes per run. The e2e tests will now run post-merge instead of on every PR.

Changes reviewed

Single file renamed: .tekton/cli-e2e-pull-request.yaml.tekton/cli-e2e-push.yaml

Change Rationale
Removed pull_request_number annotation Not applicable for push events
cancel-in-progress: "true""false" Each push to main is a distinct merged commit; canceling prior runs would skip validating those commits
CEL expression: event == "pull_request"event == "push" Core trigger change
name: cli-e2e-on-pull-requestcli-e2e-on-push Consistent rename

Analysis

  • Correctness: All four metadata changes are internally consistent. The spec section (params, pipelineRef, taskRunTemplate) is unchanged. The naming convention matches the existing cli-main-push.yaml. No dangling references to the old filename or PipelineRun name exist elsewhere in the repo.
  • Security: No security implications. No changes to secrets, RBAC, or permissions.
  • Scope: Tightly scoped — single file, single purpose, matching the stated intent and the referenced issue (EC-1930).
  • Conventions: File and resource naming follows the established pattern (cli-main-push.yaml / cli-main-on-push).
  • Documentation: No documentation updates needed for this CI configuration change.

fullsend-ai-review[bot]

This comment was marked as outdated.

@fullsend-ai-review fullsend-ai-review Bot added the ready-for-merge All reviewers approved — ready to merge label Jul 16, 2026
@codecov

codecov Bot commented Jul 16, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

Flag Coverage Δ
acceptance 54.25% <ø> (-0.01%) ⬇️
generative 16.80% <ø> (ø)
integration 27.97% <ø> (ø)
unit 71.74% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

cuipinghuo
cuipinghuo previously approved these changes Jul 16, 2026
The cli-e2e PipelineRun runs the conforma/e2e-tests pipeline which
takes ~25 minutes per run. Moving it from pull_request to push trigger
means e2e tests run post-merge on main, while fast checks (unit tests,
lint, acceptance) continue to gate PRs.

Ref: https://issues.redhat.com/browse/EC-1930
@fullsend-ai-review

fullsend-ai-review Bot commented Jul 17, 2026

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 12:32 PM UTC · Completed 12:35 PM UTC
Commit: 87c4a29 · View workflow run →

@fullsend-ai-review fullsend-ai-review Bot added ready-for-merge All reviewers approved — ready to merge and removed ready-for-merge All reviewers approved — ready to merge labels Jul 17, 2026
@jsmid1

jsmid1 commented Jul 21, 2026

Copy link
Copy Markdown
Contributor Author

Hi @cuipinghuo, I rebased the PR. Could you approve again?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-merge All reviewers approved — ready to merge size: XS

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants