Skip to content

feat(copilot): verify managed MCP connections#5596

Open
j15z wants to merge 4 commits into
stagingfrom
feat/mothership-mcp-verification
Open

feat(copilot): verify managed MCP connections#5596
j15z wants to merge 4 commits into
stagingfrom
feat/mothership-mcp-verification

Conversation

@j15z

@j15z j15z commented Jul 11, 2026

Copy link
Copy Markdown
Collaborator

Summary

Mothership can now distinguish “the MCP server record was saved” from “the server is actually usable.” MCP add operations and connection-affecting edits perform a live tools/list verification and return a structured result to the agent, including tool count, authorization requirements, or a concise failure reason.

A failed verification does not delete the newly created server, so the agent can report the credential or connection problem without losing the user’s configuration. Disabled servers and cosmetic-only edits skip the network handshake explicitly instead of producing a false failure or waiting for a timeout.

Validation

  • Focused MCP service and manage_mcp_tool suites passed (18 tests).
  • Biome and git diff --check passed for the changed files.
  • Full TypeScript checking could not be used in the isolated dev worktree because workspace package links resolved through another branch’s node_modules, producing unrelated package-version errors. The changed TypeScript paths compiled through the focused Vitest runs.

Post-Deploy Monitoring & Validation

  • Inspect Mothership MCP create/edit runs for the returned verification object and compare it with subsequent workflow MCP usage.
  • Healthy signal: successful servers report verified: true with a tool count; authorization and transport failures are reported explicitly without deleting the server.
  • Failure signal: agent responses claim an MCP server is usable when verification failed, or add/edit calls acquire unexpected 30-second latency for disabled/cosmetic changes.
  • Roll back if verification blocks successful server creation or materially increases add/edit failure rates. Owner: Mothership agent team during the first 24 hours.

Compound Engineering
Codex

@vercel

vercel Bot commented Jul 11, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs Skipped Skipped Jul 13, 2026 9:07pm

Request Review

@cursor

cursor Bot commented Jul 11, 2026

Copy link
Copy Markdown

PR Summary

Medium Risk
Adds live MCP handshakes on copilot add/edit (up to timeout latency) and changes OAuth/cache/status handling; mis-tuned skip logic could hide real failures or add unexpected latency.

Overview
Copilot manage_mcp add/edit responses now include a structured verification object so the agent can tell “saved” from “reachable,” without rolling back creates on failed checks.

After add (unless enabled: false), the handler runs a live tools/list via mcpService.verifyServerConnection and returns tool count, requiresAuthorization, or a truncated error while still succeeding when the row was persisted. After edit, verification runs only when the server is enabled and orchestration reports connectionChanged; disabled servers and cosmetic/echoed config edits return skipped with server_disabled or connection_unchanged.

hasMcpServerConnectionChanged (deep header compare) drives connectionChanged on updates and narrows workspace cache invalidation to real connection changes. McpOauthRedirectRequired is treated like other auth-pending paths so discovery/verification do not negative-cache OAuth redirects.

Reviewed by Cursor Bugbot for commit 256e22c. Bugbot is set up for automated code reviews on this repo. Configure here.

@greptile-apps

greptile-apps Bot commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds live MCP server verification to managed MCP add and edit flows. The main changes are:

  • New structured verification results for verified, failed, and skipped checks.
  • Live tools/list verification after adds and connection-affecting edits.
  • Explicit skips for disabled servers and cosmetic-only edits.
  • Focused tests for the management tool and MCP service verification paths.

Confidence Score: 5/5

This looks safe to merge.

  • No blocking issues found in the changed code.
  • The verification gates match the connection-affecting fields exposed by the management tool.
  • Failed verification returns structured output without deleting the saved server.

Important Files Changed

Filename Overview
apps/sim/lib/copilot/tools/handlers/management/manage-mcp-tool.ts Adds verification to MCP add/edit responses and skips checks for disabled or connection-unchanged states.
apps/sim/lib/mcp/service.ts Adds verifyServerConnection using force-refreshed tool discovery with structured success and failure output.
apps/sim/lib/mcp/types.ts Adds the discriminated verification result type used by the service and management tool.
apps/sim/lib/copilot/tools/handlers/management/manage-mcp-tool.test.ts Adds focused coverage for add/edit verification behavior and skipped verification cases.
apps/sim/lib/mcp/service.test.ts Adds coverage for successful verification and structured non-throwing verification failures.

Reviews (1): Last reviewed commit: "feat(copilot): verify managed MCP connec..." | Re-trigger Greptile

Comment thread apps/sim/lib/mcp/service.ts
Comment thread apps/sim/lib/mcp/service.ts
j15z added a commit that referenced this pull request Jul 13, 2026
- persist failed verification status before returning
- classify OAuth redirect failures as authorization-required
- cover status persistence and immediate OAuth retry
Comment thread apps/sim/lib/mcp/service.ts
j15z added a commit that referenced this pull request Jul 13, 2026
- use the shared OAuth-required classifier in aggregate discovery
- keep server summaries consistent for redirect-required auth
- cover both sibling paths with focused regressions
Comment thread apps/sim/lib/copilot/tools/handlers/management/manage-mcp-tool.ts
Comment thread apps/sim/lib/mcp/service.ts
Comment thread apps/sim/app/api/billing/switch-plan/route.ts Outdated
const quotaCheck = await checkStorageQuotaForBillingContext(
storageBillingContext,
fileSize ?? 0
)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mothership multipart still billed

Medium Severity

Multipart initiate now resolves resolveStorageBillingContext and enforces checkStorageQuotaForBillingContext for every non-exempt context, including mothership. Presigned mothership uploads in this same change set are explicitly unbilled and skip payer storage accounting, so large mothership attachments can unexpectedly consume workspace storage quota while small ones do not.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 4b9857f. Configure here.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Large mothership attachments can unexpectedly consume workspace storage quota while small ones do not.

Needs human: this multipart file appears in the PR only because the head diverges from the force-updated dev history (68 base-only and 62 head-only commits). A normal base merge produced 15 semantic conflicts across unrelated billing, uploads, UI, and Copilot code, so I aborted it rather than change storage behavior outside this MCP PR. Options are to rebase/cherry-pick the three MCP fix commits onto current dev and force-push (recommended), which should remove this upload diff, or address the upload issue separately on dev. Leaving this thread open until the branch history is repaired.

j15z added 4 commits July 13, 2026 13:12
- persist failed verification status before returning
- classify OAuth redirect failures as authorization-required
- cover status persistence and immediate OAuth retry
- use the shared OAuth-required classifier in aggregate discovery
- keep server summaries consistent for redirect-required auth
- cover both sibling paths with focused regressions
@j15z j15z changed the base branch from dev to staging July 13, 2026 20:15
@j15z j15z force-pushed the feat/mothership-mcp-verification branch from 4b9857f to 256e22c Compare July 13, 2026 21:07
@j15z j15z requested a review from Sg312 July 13, 2026 21:08

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

There are 2 total unresolved issues (including 1 from previous review).

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 256e22c. Configure here.

error: message,
}
}
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Failed verify poisons discovery cache

Medium Severity

verifyServerConnection probes via discoverServerTools, whose failure path always calls markServerUnhealthy for non-auth errors. A failed add/edit verification therefore writes the 2-minute negative cache even though the failure is already returned inline. Fixing credentials outside an MCP edit (for example adding a missing env var) leaves discovery in cooldown until TTL expiry.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 256e22c. Configure here.

>,
oauthCredentialsChanged = false
): boolean {
return (

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Sg312 Does this seem like the correct way to be checking if oauth creds changed?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant