Lambda-Runtime-Invocation-Id by GarrettBeatty · Pull Request #2508 · aws/aws-lambda-dotnet

GarrettBeatty · 2026-07-31T17:09:38Z

Issue #, if available:

Description of changes:

What this adds

Support for Lambda's cross-wiring-on-invoke-id protection in the .NET runtime client (Amazon.Lambda.RuntimeSupport).

The Runtime API now sends a new per-invoke header, Lambda-Runtime-Invocation-Id, on GET /runtime/invocation/next. This change makes the .NET runtime client:

Read that header from /next.
Echo it back on POST /response and POST /error — only when it was present (backward compatible).
On an ErrorRuntimeInvokeTimeout rejection (HTTP 410 Gone), log via the internal logger and continue to the next invocation instead of crashing.

Why it's needed (plain terms + example)

Lambda reuses an execution environment for many invocations, and two behaviors combine badly:

When an invocation times out, Lambda reports the timeout but does not kill the still-running handler — your code keeps executing in the background.
Responses were matched to invocations only by the request id in the URL path.

Because customers can supply their own request id (and upstream retries can reuse it), this race was possible:

1. Invoke A (requestId = abc) times out. Its handler keeps running.
2. Invoke B arrives reusing requestId = abc, and starts running.
3. A's handler finally finishes and POSTs its result to /invocation/abc/response.
4. Lambda matches "abc" to the currently-active invoke — B —
   and delivers A's response to B's caller.   ← WRONG: cross-wired response

The fix gives every invocation a unique nonce (Lambda-Runtime-Invocation-Id) that is separate from the reusable request id. The runtime echoes it back; if it doesn't match the invocation the platform currently has active (or that invocation already timed out), the platform rejects the response with a 410 instead of misdelivering it. This PR is the .NET runtime's half of that contract.

Backward compatible: if the header is absent from /next (older platform), the runtime doesn't echo anything and behaves exactly as before.

Testing

Unit tests — 8/8 passing

Amazon.Lambda.RuntimeSupport.UnitTests, covering: invocation-id echoed on /response and /error; header omitted when absent (back-compat); 410 → RuntimeApiInvokeTimeoutException; and the bootstrap loop swallowing that exception (log + continue) on both the /response and /error paths.

Passed!  - Failed: 0, Passed: 8, Skipped: 0, Total: 8

Manual / integration testing

How this was tested. I used a custom RIE (one that implements the invoke timeout and 410 rejection) to run the real .NET runtime client, triggered invocations with curl, and captured the client's socket traffic with strace to confirm the header is actually on the wire. The scenarios below were run against the .NET 10 build of the client; the happy-path echo was also re-verified against the .NET 8 build, since the library targets both.

In the captures below, recvfrom = bytes the client received from the API (the /next response) and sendto = bytes the client sent back (its /response or /error). Note the echoed Lambda-Runtime-Invocation-Id is a different value from the request id in the URL path — that's the whole point: the id is a separate per-invoke nonce, not the reusable request id.

1. Normal invocation still works, and the id round-trips

A handler that just returns a value (input "ping" → "pong"), HTTP 200. Confirms the new header handling didn't break the normal path, and that the client reads the id from /next and echoes it on the response:

recvfrom (from /next):     Lambda-Runtime-Invocation-Id: 3ebffaf2-...bb5f
sendto   (to /response):   POST .../invocation/f0c37d75-.../response
                           Lambda-Runtime-Invocation-Id: 3ebffaf2-...bb5f   ← same value echoed

2. The id is echoed on the error path too

A handler that throws, so the client reports the failure to /error instead of /response. Confirms the echo happens on the error path as well:

recvfrom (from /next):   Lambda-Runtime-Invocation-Id: 8c40a5da-...5f59
sendto   (to /error):    POST .../invocation/a026d6ce-.../error
                         Lambda-Runtime-Invocation-Id: 8c40a5da-...5f59   ← same value echoed

3. When a timed-out response is rejected (410), the client logs it and keeps going

This is the core behavior. I needed the invocation to time out while the handler was still running, so that the handler finishes and posts its response after the platform already gave up on it. To arrange that, I used a handler that deliberately runs for a fixed ~10 seconds regardless of the Lambda deadline:

// busy-runs ~10s on its own timer, ignoring the function timeout
var source = new CancellationTokenSource(TimeSpan.FromSeconds(10));
var token = source.Token;
while (!token.IsCancellationRequested) { /* do work */ }
return /* "success" */;

Setting the function timeout to 2 seconds guarantees the timeout fires first. Observed sequence:

t=2s   platform times out the invoke → caller gets Sandbox.Timedout; the invoke id is remembered
t=10s  handler finishes; client posts its (now stale) response
       platform: "response is too late for timed out invoke"  →  HTTP 410
       client:  "[Info] Invocation <id> timed out before its response was submitted;
                 the Runtime API rejected it. Continuing to the next invocation."

The client logged the rejection and went back to asking for the next invocation — the process stayed alive and did not crash. (Before this change, that 410 would have surfaced as an unhandled error.)

4. The id is echoed on the response-streaming path too

Response streaming sends the reply a different way — it builds the HTTP request by hand over a raw socket (chunked transfer), separately from the normal buffered path — so the echo had to be added there independently and is worth verifying on its own. Using a streaming handler (writes "Hello, World!"), HTTP 200. Confirms the id is echoed on the streaming response:

recvfrom (from /next):     Lambda-Runtime-Invocation-Id: 0f7ea9af-...c4f4
sendto   (to /response):   POST .../invocation/2370c6e9-.../response
                           Lambda-Runtime-Function-Response-Mode: streaming
                           Transfer-Encoding: chunked
                           Lambda-Runtime-Invocation-Id: 0f7ea9af-...c4f4   ← same value echoed

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

RuntimeApiHeaders exposes InvocationId (null when the header is absent, preserving backward compatibility). Surfaced on LambdaContext for the invoke loop to echo back on the response and error calls.

InternalRuntimeApiClient adds the Lambda-Runtime-Invocation-Id request header on /response and /error when present, added to the per-request message so concurrent invocations cannot echo each other's id. HTTP 410 Gone (Runtime.InvokeTimeout) maps to a new standalone RuntimeApiInvokeTimeoutException instead of the generic unexpected-status throw. Adds invocationId overloads to RuntimeApiClient and the public IRuntimeApiClient interface (following the SnapStart precedent of extending the runtime API client surface); existing overloads delegate with a null invocation id.

LambdaBootstrap threads the invocation id into the response and error POSTs and catches RuntimeApiInvokeTimeoutException before the general handler, logging via the internal logger and looping back to /next in both on-demand and multi-concurrency modes. The catch encloses both the response and error POST paths so a 410 on either does not crash the on-demand process. Updates the streaming test doubles to shadow the new invocationId overloads so LambdaBootstrap dispatches to their tracking logic.

Threads the invocation id through ResponseStreamFactory and ResponseStreamContext into RawStreamingHttpClient, which writes the Lambda-Runtime-Invocation-Id header into the raw HTTP header block before the terminating blank line and only when present.

Amazon.Lambda.RuntimeSupport, Minor increment.

The Release build treats the missing param tag as CS1573; add it so the build stays warning-clean.

Copilot

🟡 Not ready to approve

The PR introduces a breaking public interface change (new members on IRuntimeApiClient) and has a couple of fixable implementation issues (redundant computation + missing CR/LF sanitization when echoing invocation id into raw HTTP headers).

Once you've addressed the issues Copilot identified, you can request another Copilot review.

This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.

Pull request overview

Adds support in Amazon.Lambda.RuntimeSupport for Lambda’s cross-wiring-on-invoke-id protection by propagating the per-invocation Lambda-Runtime-Invocation-Id header to /response, /error, and streaming responses, and by treating HTTP 410 (invoke timeout rejection) as non-fatal in the invoke loop.

Changes:

Plumbs Lambda-Runtime-Invocation-Id from /next through context/bootstrap into buffered, error, and streaming response paths (echoed only when present).
Introduces 410→RuntimeApiInvokeTimeoutException handling and updates the bootstrap loop to log-and-continue instead of crashing.
Updates/extends unit tests and test doubles to cover invocation-id propagation and 410 behavior.

File summaries

File	Description
Libraries/test/Amazon.Lambda.RuntimeSupport.Tests/Amazon.Lambda.RuntimeSupport.UnitTests/TestHelpers/TestStreamingRuntimeApiClient.cs	Updates streaming test client to accept/record invocation id.
Libraries/test/Amazon.Lambda.RuntimeSupport.Tests/Amazon.Lambda.RuntimeSupport.UnitTests/TestHelpers/TestRuntimeApiClient.cs	Adds invocation-id-aware overloads and 410 simulation toggles for tests.
Libraries/test/Amazon.Lambda.RuntimeSupport.Tests/Amazon.Lambda.RuntimeSupport.UnitTests/TestHelpers/TestMultiConcurrencyRuntimeApiClient.cs	Adds invocation-id overloads to multi-concurrency test client.
Libraries/test/Amazon.Lambda.RuntimeSupport.Tests/Amazon.Lambda.RuntimeSupport.UnitTests/TestHelpers/NoOpInternalRuntimeApiClient.cs	Updates internal client stub signatures for invocation id.
Libraries/test/Amazon.Lambda.RuntimeSupport.Tests/Amazon.Lambda.RuntimeSupport.UnitTests/StreamingE2EWithMoq.cs	Captures invocation id in streaming E2E test clients.
Libraries/test/Amazon.Lambda.RuntimeSupport.Tests/Amazon.Lambda.RuntimeSupport.UnitTests/RuntimeApiClientTests.cs	Adds unit tests for invocation-id echo and 410 behavior.
Libraries/test/Amazon.Lambda.RuntimeSupport.Tests/Amazon.Lambda.RuntimeSupport.UnitTests/ResponseStreamFactoryTests.cs	Ensures invocation id is passed into streaming start.
Libraries/test/Amazon.Lambda.RuntimeSupport.Tests/Amazon.Lambda.RuntimeSupport.UnitTests/LambdaResponseStreamingCoreTests.cs	Updates streaming client override signature for invocation id.
Libraries/test/Amazon.Lambda.RuntimeSupport.Tests/Amazon.Lambda.RuntimeSupport.UnitTests/LambdaContextTests.cs	Adds tests for reading invocation id from headers into context.
Libraries/test/Amazon.Lambda.RuntimeSupport.Tests/Amazon.Lambda.RuntimeSupport.UnitTests/LambdaBootstrapTests.cs	Adds tests for invocation-id echo and swallowing 410 in bootstrap.
Libraries/src/Amazon.Lambda.RuntimeSupport/Context/LambdaContext.cs	Exposes invocation id (internally) via context for bootstrap/streaming.
Libraries/src/Amazon.Lambda.RuntimeSupport/Client/RuntimeApiHeaders.cs	Parses/stores `Lambda-Runtime-Invocation-Id` from Runtime API headers.
Libraries/src/Amazon.Lambda.RuntimeSupport/Client/RuntimeApiClient.cs	Adds invocation-id-aware overloads for response/error/streaming.
Libraries/src/Amazon.Lambda.RuntimeSupport/Client/IRuntimeApiClient.cs	Adds new invocation-id-aware members to the public interface.
Libraries/src/Amazon.Lambda.RuntimeSupport/Client/InternalClientAdapted.cs	Echoes invocation id on `/response` and `/error`; throws on 410 Gone.
Libraries/src/Amazon.Lambda.RuntimeSupport/Bootstrap/ResponseStreaming/ResponseStreamFactory.cs	Passes invocation id into streaming start via context.
Libraries/src/Amazon.Lambda.RuntimeSupport/Bootstrap/ResponseStreaming/ResponseStreamContext.cs	Stores invocation id for streaming responses.
Libraries/src/Amazon.Lambda.RuntimeSupport/Bootstrap/ResponseStreaming/RawStreamingHttpClient.cs	Adds invocation id header to raw streaming HTTP request.
Libraries/src/Amazon.Lambda.RuntimeSupport/Bootstrap/LambdaBootstrap.cs	Plumbs invocation id through invoke loop and swallows 410 via log+continue.
.autover/changes/4133a18f-b6f9-4f24-886e-217756fbb670.json	Changelog entry for invocation-id support and 410 behavior.

Review details

Files reviewed: 20/20 changed files
Comments generated: 5
Review effort level: Lite

We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.

GarrettBeatty · 2026-07-31T18:02:24Z

+            // Echo the per-invocation id back for cross-wiring protection. Only sent when the
+            // Runtime API provided it on /next.
+            if (!string.IsNullOrEmpty(invocationId))
+                headers.Append($"{RuntimeApiHeaders.HeaderInvocationId}: {invocationId}\r\n");
            headers.Append("\r\n");


should never happen

GarrettBeatty · 2026-07-31T18:02:01Z

+                        // Echo the per-invocation id back for cross-wiring protection. Only sent when the
+                        // Runtime API provided it on /next; added to this request message (never the shared
+                        // HttpClient) so concurrent invocations cannot echo each other's id.
+                        if (!string.IsNullOrEmpty(invocationId))
+                            request_.Headers.TryAddWithoutValidation(RuntimeApiHeaders.HeaderInvocationId, invocationId);
+


this should never happen

GarrettBeatty · 2026-07-31T18:02:14Z

+                    // Echo the per-invocation id back for cross-wiring protection. Only sent when the
+                    // Runtime API provided it on /next; added to this request message (never the shared
+                    // HttpClient) so concurrent invocations cannot echo each other's id.
+                    if (!string.IsNullOrEmpty(invocationId))
+                        request_.Headers.TryAddWithoutValidation(RuntimeApiHeaders.HeaderInvocationId, invocationId);
+


might be overkill should never happen

GarrettBeatty · 2026-07-31T17:50:22Z

+        /// <summary>
+        /// Report an invocation error as an asynchronous operation, echoing the invocation id for
+        /// cross-wiring protection.
+        /// </summary>
+        /// <param name="awsRequestId">The ID of the function request that caused the error.</param>
+        /// <param name="invocationId">The unique-per-invocation id to echo back to the Runtime API. When null, the header is not sent.</param>
+        /// <param name="exception">The exception to report.</param>
+        /// <param name="cancellationToken">The optional cancellation token to use.</param>
+        /// <returns>A Task representing the asynchronous operation.</returns>
+        /// <exception cref="RuntimeApiInvokeTimeoutException">The invocation timed out before the error was submitted (cross-wiring protection).</exception>
+        Task ReportInvocationErrorAsync(string awsRequestId, string invocationId, Exception exception, CancellationToken cancellationToken = default);
+


this interface should really have been internal. not worried about this as per norm

The non-invocationId overload computed ExceptionInfo.GetExceptionInfo but never used the result, then delegated to the invocationId overload which recomputes it. Drop the dead local; addresses PR review feedback.

…changes The Annotations source-generator tests compile the real InternalClientAdapted.cs and assert exact source coordinates in it. The cross-wiring changes shifted RuntimeApiSerializationContext and the JsonSerializerContext.Default usages down, so update the expected line numbers to match (columns unchanged). Verified the full Annotations suite passes 583/583 on net8.0 and net10.0.

GarrettBeatty added 7 commits July 30, 2026 17:42

Read Lambda-Runtime-Invocation-Id header from /next

0334c94

RuntimeApiHeaders exposes InvocationId (null when the header is absent, preserving backward compatibility). Surfaced on LambdaContext for the invoke loop to echo back on the response and error calls.

Add change file for cross-wiring invocation-id support

f8d7ffb

Amazon.Lambda.RuntimeSupport, Minor increment.

Add missing invocationId XML param doc on ErrorWithXRayCauseAsync

2e31307

The Release build treats the missing param tag as CS1573; add it so the build stays warning-clean.

Remove redundant comments in RawStreamingHttpClient

8ed9c6a

GarrettBeatty requested a review from Copilot July 31, 2026 17:15

Copilot started reviewing on behalf of GarrettBeatty July 31, 2026 17:16 View session

Copilot AI reviewed Jul 31, 2026

View reviewed changes

Remove redundant ExceptionInfo computation in ReportInvocationErrorAsync

c26abf0

The non-invocationId overload computed ExceptionInfo.GetExceptionInfo but never used the result, then delegated to the invocationId overload which recomputes it. Drop the dead local; addresses PR review feedback.

GarrettBeatty changed the title ~~Feature/crosswire invocation~~ Lambda-Runtime-Invocation-Id Jul 31, 2026

GarrettBeatty marked this pull request as ready for review July 31, 2026 19:25

GarrettBeatty requested review from a team as code owners July 31, 2026 19:25

GarrettBeatty requested a review from normj July 31, 2026 19:25

normj approved these changes Jul 31, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Lambda-Runtime-Invocation-Id - #2508

Lambda-Runtime-Invocation-Id#2508
GarrettBeatty wants to merge 9 commits into
devfrom
feature/crosswire-invocation-id

GarrettBeatty commented Jul 31, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

GarrettBeatty Jul 31, 2026

Uh oh!

GarrettBeatty Jul 31, 2026

Uh oh!

GarrettBeatty Jul 31, 2026

Uh oh!

GarrettBeatty Jul 31, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

GarrettBeatty commented Jul 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What this adds

Why it's needed (plain terms + example)

Testing

Unit tests — 8/8 passing

Manual / integration testing

1. Normal invocation still works, and the id round-trips

2. The id is echoed on the error path too

3. When a timed-out response is rejected (410), the client logs it and keeps going

4. The id is echoed on the response-streaming path too

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Not ready to approve

Uh oh!

Uh oh!

GarrettBeatty Jul 31, 2026

Choose a reason for hiding this comment

Uh oh!

GarrettBeatty Jul 31, 2026

Choose a reason for hiding this comment

Uh oh!

GarrettBeatty Jul 31, 2026

Choose a reason for hiding this comment

Uh oh!

GarrettBeatty Jul 31, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

GarrettBeatty commented Jul 31, 2026 •

edited

Loading