docs(executorch): note the single-CUDA-stream requirement for coalesced .pte

[Conarnar]

A coalesced TensorRT + CUDA .pte runs on multiple backend delegates, each of which
enqueues async GPU work on its own CUDA stream by default. Because separate streams
are unordered, a consuming delegate can read a producing delegate's output before it
is ready -- an intermittent race that shows up as wrong results or an illegal memory
access.

Document that unifying all delegates onto one caller-provided CUDA stream is the
runner's responsibility, and that a runner reading delegate outputs between calls
(e.g. an autoregressive decode loop) must synchronize that stream before reading.

[shoumikhin]

Thanks for documenting the single-stream requirement for coalesced programs.

Heads up that the shared caller-stream integration for the TensorRT delegate is landing
separately in #4421. Once it merges, this doc could point at it and name the concrete
CallerStreamGuard API that a runner scopes around execution, which would make the
guidance easier to apply. Nothing to change now, mainly flagging so the two stay in sync.

[shoumikhin]

Correction to my earlier comment: the shared caller-stream work is #4454, not #4421. I pointed at the wrong number, #4421 is closed and was superseded, so the link above is a dead end. Sorry for the misdirection.

Nothing here conflicts with it: #4454 does not touch saving_models.rst.

One suggestion once #4454 lands. It adds a "Caller-stream contract for the TensorRT
backend" section to cpp/src/torch_tensorrt/executorch/README.md covering the same
ground, including the requirement that every delegate in the process observe the same
caller-stream state. So rather than describing the mechanism in two places, this
section could name the concrete API and link there:

#include <executorch/extension/cuda/caller_stream.h>

executorch::extension::cuda::CallerStreamGuard guard(stream);
// ... run the program ...

That keeps the "why" here, where a reader saving a coalesced .pte will meet it, and
leaves the exact contract in one place so the two cannot drift. Your call whether to
do that now or in a follow-up after #4454 merges.