[AMD] [AGENTX] GLM-5.2 MXFP4 MI355X AgentX with EAGLE MTP (MTP-only) / 新增 GLM-5.2 MXFP4 MI355X SGLang EAGLE MTP AgentX 配置（仅 MTP）

[ajith-sirra-amd]

Adds the single-node AgentX (agentic-coding) recipe for GLM-5.2 (MXFP4) on MI355X with SGLang, MTP only.

Updated per @functionstackx's review: instead of adding MTP as a second arm, the recipe is now spec-decode only. Per MODELS.md, agentic-coding recipes are run and published with speculative decoding enabled rather than as an STP/MTP A/B, and GLM-5.2 agentic non-MTP is deprecated after 2026-08-03 — so the non-MTP arm is neither wired into the master config nor kept as a separate script. Rebased on main.

What's here

File	Change
benchmarks/single_node/agentic/glm5.2_fp4_mi355x_sglang_mtp.sh	New (renamed from ..._sglang.sh). _mtp is the suffix runners/launch_mi355x-amds.sh resolves for SPEC_DECODING=mtp.
configs/amd-master.yaml	Config key glm5.2-fp4-mi355x-sglang-agentic-mtp, spec-decoding: mtp.
perf-changelog.yaml	Entry filled in, PR link added.

MTP

GLM-5.2 ships its own nextn head, so SGLang EAGLE runs off the checkpoint with no external draft model:

--speculative-algorithm EAGLE --speculative-num-steps 3 \
--speculative-eagle-topk 1 --speculative-num-draft-tokens 4

3 speculative tokens per verification step — the same shape as the B300 sibling #2447 and the GLM-5.2 GB300 dynamo-sglang agentic recipes.

The draft MoE backend is deliberately left unset. GLM-5.2's nextn layer is unquantized bf16, so it cannot inherit the target model's MXFP4 MoE runner once EP8 puts an all-to-all in the path — but upstream's _deepseek_spec_moe_resolution gates its fix-up hook on is_hip(), so ROCm resolves it automatically. This is the one place this recipe diverges from the CUDA sibling, which has to pass --speculative-moe-a2a-backend none / --speculative-moe-runner-backend triton explicitly.

Synthetic acceptance length

Throughput runs pin acceptance to the golden AL, per the AgentX policy in golden_al_distribution/README.md:

SGLANG_SIMULATE_ACC_LEN=2.99
SGLANG_SIMULATE_ACC_METHOD=match-expected
SGLANG_SIMULATE_ACC_TOKEN_MODE=real-draft-token

EVAL_ONLY runs leave it off — simulated acceptance commits drafted tokens regardless of the target logits, so the eval would score ~0.

2.99 is the GLM-5.2 curve at num_speculative_tokens=3, thinking_on, from SPEED-Bench coding (speedbench-al.yml run 28058352479). The curve file itself is committed by #2447 (golden_al_distribution/glm5.2_mtp.yaml) rather than duplicated here; it is not on main yet, so this PR should land after #2447. The curve was collected on the FP8 checkpoint — one curve per model, and the MXFP4 checkpoint ships the same nextn head.

Image

lmsysorg/sglang-rocm:v0.5.16-rocm720-mi35x-20260728, unchanged from the first revision and verified on Docker Hub (pushed 2026-07-28). v0.5.16 is the first release that reads SGLANG_SIMULATE_ACC_TOKEN_MODE — ACC_LEN / ACC_METHOD exist further back, so an older ROCm image would silently honor only two thirds of the contract.

Serve shape

ROCm low-latency arm, unchanged from the first revision outside the speculative and simulated-acceptance blocks: TP8 + EP8, --kv-cache-dtype fp8_e4m3, tilelang DSA prefill/decode, glm47 tool-call parser, glm45 reasoning parser, --chunked-prefill-size 131072 at conc ≤ 16, HiCache host-DRAM offload (ratio 1.5 / write_through / direct / page_first_direct).

One deliberate change: --mem-fraction-static is held at 0.80 across the whole grid instead of taking the +0.05 the non-speculative shape could afford at conc ≤ 16. The nextn layer's weights and its own KV pool come out of the same static budget, and graph capture is over 4-token verification batches rather than 1-token decodes; an OOM mid-warmup costs the whole sweep point, and HiCache's host tier (ratio is relative to the device pool) absorbs the slightly smaller HBM KV pool.

Mooncake stays out of the master config as an offload backend on this SKU — the HIP/TCP transfer engine is not stable on gfx950 under sustained agentic load — but the branch stays in the script for on-node experiments.

Search space

- { tp: 8, ep: 8, spec-decoding: mtp, kv-offloading: dram, kv-offload-backend: { name: hicache }, conc-list: [1, 4, 8, 12, 16] }

The AgentX MTP concurrency grid: steps of at least 2 (single-step sampling can't separate configurations by more than run-to-run noise on the agentic corpus) and a hard stop at conc 16.

Validation

• bash -n clean; configs/amd-master.yaml / perf-changelog.yaml parse.
• generate_sweep_configs.py test-config and process_changelog.py against origin/main both select exactly the 5 expected jobs (glm5.2_tp8_conc{1,4,8,12,16}_kvdram-hicache_spec-mtp).
• validate_perf_changelog.py passes.
• Launcher resolution simulated: benchmarks/single_node/agentic/glm5.2_fp4_mi355x_sglang_mtp.sh.
• No references to the old script / config-key names remain.

Sweep results to follow.

[github-actions]

Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase As a PR reviewer and CODEOWNER, I have reviewed this and have.

For PR verification, add the full-sweep-fail-fast label (strongly recommended) to this PR — the benchmark sweep only runs on labeled PRs. Use full-sweep-enabled only if you need matrix jobs to keep running past a failure.

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs

---

感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 As a PR reviewer and CODEOWNER, I have reviewed this and have。

如需进行 PR 验证，请为此 PR 添加 full-sweep-fail-fast 标签（强烈推荐）— 基准测试 sweep 仅在带有标签的 PR 上运行。仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled。

PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档

[functionstackx]

hi @ajith-sirra-amd thanks for your PR! can u add MTP to it?

Per https://github.com/SemiAnalysisAI/InferenceX/blob/main/MODELS.md , as agreed upon by AMD @andyluo7 , in order to be efficient with the compute we have, we will be doing mtp on only since everyone in prod deploys with MTP.

This will cut our queue times by 50% by only doing spec decode benchmarks