Skip to content

Fail mark-for-deployment when re-deploying same version without --wait-for-deployment - #4307

Open
cuza wants to merge 8 commits into
masterfrom
u/cuza/PAASTA-18862
Open

Fail mark-for-deployment when re-deploying same version without --wait-for-deployment#4307
cuza wants to merge 8 commits into
masterfrom
u/cuza/PAASTA-18862

Conversation

@cuza

@cuza cuza commented May 7, 2026

Copy link
Copy Markdown
Member

Prevent deployment when attempting to redeploy the same version without the --wait-for-deployment flag.

Comment thread paasta_tools/cli/cmds/mark_for_deployment.py
Comment thread paasta_tools/cli/cmds/mark_for_deployment.py Outdated
Comment thread paasta_tools/cli/cmds/mark_for_deployment.py Outdated
@cuza
cuza force-pushed the u/cuza/PAASTA-18862 branch from 1a04794 to ccc3674 Compare June 16, 2026 13:05
@cuza
cuza force-pushed the u/cuza/PAASTA-18862 branch from ccc3674 to 0110d7e Compare June 26, 2026 20:07
@cuza
cuza requested a review from nemacysts June 26, 2026 20:58

@nemacysts nemacysts left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mostly just questions - i think this is basically exactly what we want otherwise!

Comment thread paasta_tools/cli/cmds/mark_for_deployment.py
Comment thread paasta_tools/cli/cmds/mark_for_deployment.py Outdated
Comment on lines +9 to +24
@patch(
"paasta_tools.cli.cmds.check_deploy_health.check_if_instance_is_done",
autospec=True,
)
@patch(
"paasta_tools.cli.cmds.check_deploy_health.get_instance_configs_for_service_in_deploy_group_all_clusters",
autospec=True,
)
@patch(
"paasta_tools.cli.cmds.check_deploy_health.get_currently_deployed_version",
autospec=True,
)
@patch(
"paasta_tools.cli.cmds.check_deploy_health.validate_service_name",
autospec=True,
)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i need to finish my auto-fix pre-commit branch, but can we swap these for context-manager patches? (that's what we normally do for new code since then you don't have to remember how the decorator order matches the param order :p)

...although, maybe this matter less now that llms write a lot of our tests :p

Comment thread tests/cli/test_cmds_check_deploy_health.py Outdated
Comment thread paasta_tools/cli/cmds/check_deploy_health.py Outdated
Comment thread paasta_tools/cli/cmds/check_deploy_health.py
Co-authored-by: Luis Pérez <luisp@yelp.com>
print(deployment_version)
print("Continuing anyway.")
print(f"Warning: {deploy_group} already set to {deployment_version}")
if not args.block:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Security Issue: Early-return path when re-deploying the same version without --wait-for-deployment bypasses the can_user_deploy_service LDAP authorization check at line 550. An unauthorized user can write a deploy tag by re-issuing a mark-for-deployment for the currently deployed version without --wait-for-deployment, skipping the LDAP group membership verification entirely.

Severity: HIGH
Category: authorization_bypass
Tool: ClaudeCode AI Security Analysis

Exploit Scenario: An attacker or unauthorized user who knows the currently deployed version (sha + image_version) of a service can run paasta mark-for-deployment -s <service> -l <deploy-group> -c <current-sha> --image-version <current-image-version> without --wait-for-deployment. Because the version matches old_deployment_version, the code takes the early return at line 514, calling mark_for_deployment() to force-push the deploy git tag and returning success — without ever reaching can_user_deploy_service() at line 550 which checks LDAP group membership. This could be chained: an attacker first re-deploys the same version to confirm access, then races a version change (or exploits a TOCTOU window) to deploy an unauthorized version.

Recommendation: Move the authorization check (can_user_deploy_service) before the same-version early-return block, or call it within the early-return path before invoking mark_for_deployment(). The authorization check should gate ALL paths that write deploy tags.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm, i can't remember if writing a deploy tag in this way will trigger a bounce - if it does, we probably do want to make sure we've done the can_user_deploy_service check

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nemacysts
mark_for_deployment does call trigger_deploys(), but since it's force-pushing the same SHA to the same tag, deployd sees no version change

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm, on_enter_start_deploy() calls mark_for_deployment() which calls get_paasta_tag_from_deploy_group() which adds the current timestamp to the tag - i'm admittedly a little rusty on the nitty gritty internals of how the tag handling works w.r.t. what is a noop and causes a bounce, but i'm a little worried that this will allow folks to bounce services that they shouldn't be able to bounce since the tag won't be exactly the same

@cuza
cuza requested a review from nemacysts July 31, 2026 13:36
@cuza
cuza marked this pull request as ready for review July 31, 2026 13:36
@cuza
cuza requested a review from a team as a code owner July 31, 2026 13:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants