bug: `adk eval` misparses Windows absolute eval-set paths as case selectors

[sgoel2be24-cyber]

Describe the bug

On Windows, adk eval incorrectly parses absolute eval-set file paths.

The CLI supports optionally selecting individual cases using:

<eval-set path or ID>:<comma-separated case IDs>

The parser treats every colon as the case-selector separator, so the colon in a Windows drive prefix such as C:\\ is mistaken for that separator.

For example, C:\tmp\agent\eval.evalset.json is parsed approximately as:

{"C": [r"\tmp\agent\eval.evalset.json"]}

instead of:

{r"C:\tmp\agent\eval.evalset.json": []}

The problem also prevents case selectors from working with Windows absolute paths.

Steps to reproduce

uv run pytest -q -p no:cacheprovider `
  tests/unittests/cli/utils/test_cli_tools_click.py::test_cli_eval_with_eval_set_file_path

The existing regression fails because the CLI writes no eval result:

assert len(eval_set_results) == 1
E assert 0 == 1

Direct parser demonstration:

uv run python -c "from google.adk.cli.cli_eval import parse_and_get_evals_to_run as parse; inputs=[r'C:\tmp\agent\eval.evalset.json',r'C:\tmp\agent\eval.evalset.json:case1',r'C:\tmp\agent\eval.evalset.json:case1,case2']; [print(repr(value), '->', repr(parse([value]))) for value in inputs]"

Actual result

'C:\\tmp\\agent\\eval.evalset.json'
-> {'C': ['\\tmp\\agent\\eval.evalset.json']}

'C:\\tmp\\agent\\eval.evalset.json:case1'
-> {'C': ['\\tmp\\agent\\eval.evalset.json']}

'C:\\tmp\\agent\\eval.evalset.json:case1,case2'
-> {'C': ['\\tmp\\agent\\eval.evalset.json']}

The drive letter becomes the eval-set identifier and requested case IDs are lost.

Expected result

{r"C:\tmp\agent\eval.evalset.json": []}
{r"C:\tmp\agent\eval.evalset.json": ["case1"]}
{r"C:\tmp\agent\eval.evalset.json": ["case1", "case2"]}

Existing behavior should remain unchanged:

{"/tmp/agent/eval.evalset.json": ["case1", "case2"]}
{"my_eval_set": ["case1", "case2"]}

Environment

• OS: Windows 10, build 19045
• Python: 3.11.15
• uv: 0.11.29
• Local google-adk: 2.4.0
• Latest upstream main
• Commit: 4a84d8a459853b039d3774da27e00b57b4d51aa5

No model, credentials, or external service is involved.

Root cause

parse_and_get_evals_to_run() uses unrestricted splitting:

eval_set = input_eval_set.split(":")[0]
evals = input_eval_set.split(":")[1].split(",")

The first colon in a Windows path is the drive-letter separator. When a selector colon is also present, that selector is beyond index 1 and is ignored.

Proposed narrow solution

Make parsing aware of a leading Windows drive prefix:

1. Preserve the drive-designator colon.
2. Split only at a subsequent selector delimiter.
3. Preserve the full drive-qualified path when no selector is present.
4. Keep POSIX-path and eval-set-ID behavior unchanged.

The solution should not depend solely on the file already existing, so nonexistent paths retain useful file-not-found diagnostics.

Regression-test plan

Add platform-independent parser tests for Windows absolute paths without selectors, with one selector, and with multiple selectors, plus POSIX paths and eval-set IDs with selectors. Use Windows-style strings even on Linux so existing CI covers the regression. Retain the existing CLI-level test shown above.

Conflict check

I searched open and closed issues and pull requests for the parser name, Windows absolute eval-set paths, adk eval Windows paths, drive-letter colon parsing, and the focused test name. I found no issue or PR addressing this root cause.

PR #6166 is a separate Windows wildcard-expansion change and explicitly notes that this eval test fails independently on unmodified main. Issue #2602 concerns pytest-driven result persistence and has a different root cause.

Contribution

I am willing to contribute a small, focused fix with these regression tests after maintainer approval.

[surajksharma07]

@sgoel2be24-cyber

We tried a narrow fix in parse_and_get_evals_to_run: check for a leading drive prefix (letter + : + \ or /) and only look for the selector colon after that.
That handles all five cases you listed while leaving POSIX paths and eval-set IDs untouched.

Could you try that same approach locally and confirm it holds for you ?

[sgoel2be24-cyber]

Thanks, @surajksharma07 — I tested the suggested approach locally on Windows against upstream commit fd006db9.

I implemented the proposed narrow approach: detect a leading alphabetic drive prefix followed by :\ or :/, then search for the selector delimiter only after that prefix.

The results hold for me:

• 9 direct parser regression cases passed
• the previously failing test_cli_eval_with_eval_set_file_path now passes
• the complete parser test module passes: 11 tests
• the adjacent CLI test module passes: 42 tests

The verified cases include:

• Windows backslash paths with and without selectors
• Windows forward-slash paths
• lowercase drive letters
• one and multiple selectors
• POSIX paths
• eval-set IDs

Existing blank-selector filtering, multiple-input handling, and duplicate merging remain unchanged. The implementation does not depend on the file existing.

One local tooling caveat: two repository pre-commit hooks use POSIX executable/shebang paths that native-Windows pre-commit could not launch. Their underlying checks passed when run directly through Git Bash and uv; Ruff, isort, Pyink, whitespace checks, end-of-file checks, and git diff --check all passed.

I currently have the focused implementation and tests on a local branch, uncommitted and unpushed. I can open a PR if this implementation direction looks good.