Skip to content

RustPython fuzzing: generic hostile modes + panic-site dedup (core) - #226

Merged
devdanzin merged 1 commit into
mainfrom
rustpython-fuzzing-core
Jul 19, 2026
Merged

RustPython fuzzing: generic hostile modes + panic-site dedup (core)#226
devdanzin merged 1 commit into
mainfrom
rustpython-fuzzing-core

Conversation

@devdanzin

Copy link
Copy Markdown
Owner

Why

The first two RustPython fleets converged fast on the same ~9 crashes, and neither --tsan (Rust's Arc/Send/Sync prevent the C-style races TSan hunts) nor --oom (Rust OOM aborts rather than exposing continuable error paths) applies. Every RustPython finding is instead a .unwrap()/.expect()/panic!/unchecked-index reachable from Python, or unguarded native recursion — all in the ~96 Rust-implemented modules.

This PR adds four generic hostile capabilities (they help any alternative interpreter — RustPython, PyPy) in core fusil, plus an in-loop dedup catalog so fleets keep only NEW panic sites. RustPython-specific packaging (curated module list, known-panic suppression) will land in a separate fusil_rustpython_plugin.

What

New modes / options

  • --concurrency-stress — emit the --tsan shared-object/many-thread stress region without a TSan build or the free-threaded/--with-thread-sanitizer preflight (reuses --tsan-threads/-iterations/-shared-objects). The GIL guard is hard only under --tsan; under --concurrency-stress it warns and runs serialised — threading teardown/re-entrancy bugs surface via context switches (the class that produced the RustPython _thread RefCell double-borrow).
  • --new-uninit — for each type T discovered in the session's module(s), including hidden result types reached via a bounded call sweep (e.g. type(re.match('a','a')), never a module attribute), build obj = T.__new__(T) and hammer its protocol slots (subscript, iter/next, call, number, comparison, buffer, context-manager) plus every dir() method with hostile args. Protocol slots that touch payload without re-validating segfault a type-confused instance where ordinary methods downcast-fail cleanly.
  • --modules-file FILE — read a curated module list (one per line, # comments), bypassing discovery/blacklists; unions with --modules.
  • RUST_BACKTRACE=1 in the target env (harmless no-op for CPython/PyPy) for richer panic frames.

Dedup enginefusil/python/rustpython_dedup.py, mirroring tsan_dedup.py/oom_dedup.py:

  • parse_report → panic-site signature crates/<path>.rs:<line> (drops the column, normalises absolute checkout paths to the crates/ tail, handles the newer thread 'NAME' (tid) panicked form and worker panics printed mid-stdout).
  • RustPyDeduper.decide(keep, label): catalog hit → RUSTPY-00NN, new panic → rustpyNEW, bare segfault → rustpySEGV (or a gdb-resolved top frame via an injectable resolver). Wired via --rustpython-dedup-catalog/-keep/-prune on the existing application.session_keep_policy hook.

Testing

  • Pure-Python engine unit-tested in tests/python/test_rustpython_dedup.py (23 tests).
  • The two new mode flags are set False in test_oom_fuzz's MagicMock fixture (a bare Mock attr is truthy and would divert generation into the stress region — the same hazard the existing o.tsan = False guards).
  • Verified end-to-end against RustPython 0.5.0: --new-uninit reproduces the re.Match.__new__ subscript segfault and a new structseq.__new__ panic face; --concurrency-stress runs the stress region on RustPython with no TSan preflight; the dedup keep-policy labels crash dirs rustpyNEW / RUSTPY-id.
  • ruff check + ruff format --check clean; full unittest suite (1168 tests) green.

Non---tsan/non---new-uninit output is unchanged (both are gated early-return branches).

🤖 Generated with Claude Code

RustPython fleets converge fast on the same ~9 panics/segfaults, and neither
--tsan (Rust's Arc/Send/Sync prevent C-style races) nor --oom (Rust OOM aborts)
applies. Add four generic, alt-interpreter-friendly capabilities in core fusil
(the RustPython packaging will live in a separate plugin), and a dedup catalog so
fleets surface only NEW sites.

New modes / options (fusil/python/__init__.py, write_python_code.py, python_source.py):
- --concurrency-stress: emit the --tsan shared-object/many-thread stress region
  WITHOUT a TSan build/preflight (reuses --tsan-threads/-iterations/-shared-objects).
  The GIL guard is hard only under --tsan; under --concurrency-stress it warns and
  runs serialised (threading teardown/re-entrancy bugs surface via context switches
  -- this is the class that produced the RustPython _thread RefCell double-borrow).
- --new-uninit: for each type T discovered in the session's module(s) -- including
  hidden RESULT types reached via a bounded call sweep (e.g. type(re.match('a','a')),
  never a module attribute) -- build obj = T.__new__(T) and hammer its PROTOCOL slots
  (subscript, iter/next, call, number, comparison, buffer, context-manager) plus every
  dir() method with hostile args. Reproduces the re.Match __new__ subscript segfault
  from the mode alone.
- --modules-file FILE: read a curated module list (one per line, '#' comments),
  bypassing discovery/blacklists; unions with --modules.
- RUST_BACKTRACE=1 in the target env (harmless no-op for CPython/PyPy): richer panic
  frames in the crash dir for dedup + reports.

Dedup engine (fusil/python/rustpython_dedup.py, mirrors tsan_dedup/oom_dedup):
- parse_report -> panic-site signature crates/<path>.rs:<line> (drop the column;
  normalise absolute checkout paths to the crates/ tail; handle the newer
  'thread NAME (tid) panicked' form and worker panics printed mid-stdout).
- RustPyDeduper.decide -> (keep, label): catalog hit -> RUSTPY-00NN, new panic ->
  rustpyNEW, bare segfault -> rustpySEGV (or a gdb-resolved top frame via an injectable
  resolver). Wired via --rustpython-dedup-catalog/-keep/-prune on the existing
  application.session_keep_policy hook.

Pure-Python engine unit-tested in tests/python/test_rustpython_dedup.py (23 tests).
The two new mode flags are set False in test_oom_fuzz's MagicMock fixture (a bare
Mock attr is truthy and would divert generation into the stress region, same hazard
the existing o.tsan = False guards). Verified end-to-end against RustPython 0.5.0:
--new-uninit reproduces the re.Match segfault and (bonus) a new structseq __new__
panic face; the dedup keep-policy labels crash dirs rustpyNEW / RUSTPY-id. ruff +
1168-test suite green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant