Skip to content

AGENTS.md

Latest commit

 

History

History
86 lines (53 loc) · 20.9 KB

AGENTS.md

File metadata and controls

86 lines (53 loc) · 20.9 KB
Raw

Restricted Shell Interpreter

Overview

This is a minimal bash/POSIX like shell interpreter. Safety is the primary goal.

This shell is intended to be used by AI Agents.

Platform Support

The shell is supported on Linux, Windows and macOS.

Documentation

  • README.md and SHELL_FEATURES.md must be kept up to date with the implementation.
  • When adding or modifying a builtin, set or update the Description field on its builtins.Command struct and verify the command appears correctly in help output.

Code Style

  • IMPORTANT: Always run make fmt after making any edits. This is a mandatory step — no exceptions. CI will reject unformatted code. Run it after every change, before committing, and before running tests. Do not skip this step.
  • All Go files must be formatted with gofmt before committing. make fmt handles this automatically. You can verify with gofmt -l . (no output means clean).

Pull Requests

  • Always open pull requests in draft mode. Use gh pr create --draft (or the GitHub UI's "Draft pull request" option). Only mark a PR ready for review once all CI checks pass and the work is complete.
  • Never add the verified/allowed_symbols GitHub label. This label is reserved for human manual approval only. Don't try to fix CI failures related to this.

Security Design Decisions

  • ss and ip route bypass AllowedPaths for /proc/net/* reads. Both builtins delegate /proc/net/ I/O to internal packages (builtins/internal/procnetsocket for ss, builtins/internal/procnetroute for ip route) that call os.Open directly on kernel pseudo-filesystem paths (e.g. /proc/net/tcp, /proc/net/route). These paths are hardcoded in the implementation and are never derived from user input, so AllowedPaths restrictions do not apply to them. As a consequence, operators cannot use AllowedPaths to block ss from enumerating local sockets or ip route from reading the routing table. This is an intentional trade-off: the paths are non-user-controllable, so there is no sandbox-escape risk, but the operator loses the ability to deny these reads via sandbox configuration.

  • df bypasses AllowedPaths for mount-table enumeration. df delegates filesystem listing to builtins/internal/diskstats, which on Linux reads /proc/self/mountinfo directly via os.Open and then calls unix.Statfs(2) on every mount point returned by the kernel. On macOS it calls unix.Getfsstat(2). The mount-point paths are kernel-controlled — never derived from user input — so the same trade-off as ss / ip route applies: operators cannot use AllowedPaths to hide individual mounts from df. Statfs returns metadata only (block / inode counts, filesystem type, block size); no file content is read.

  • journalctl and systemctl bypass AllowedPaths for trusted systemd target access. The builtins delegate journal discovery and reads, disk-usage scans, vacuuming, machine-ID reads, journald control-socket access, and restricted manager-bus operations to internal/systemd, which opens the configured paths directly with os APIs. SystemdTargetConfig.JournalDirs, SystemdTargetConfig.MachineIDPath, SystemdTargetConfig.JournalControlSocket, and SystemdTargetConfig.ManagerBusSocket are fixed by the embedding application when constructing the runner and cannot be supplied or changed by shell scripts. Operators therefore cannot use AllowedPaths to block these accesses; authorization is enforced separately by AllowedCommands, exact AllowedSystemServices unit/action grants, remediation mode for journal mutations, and remediation mode for every systemctl invocation. All configured target paths are trusted and must refer to the same host. On Linux, manager access pins the configured public system D-Bus socket through /proc/self/fd, authenticates with EXTERNAL, and verifies the systemd manager peer's machine ID before fixed manager-interface requests. See the trusted systemd target exception in docs/RULES.md for the complete backend requirements and indirect systemd effects.

  • vmstat bypasses AllowedPaths for memory/CPU/IO pressure reads, and renders macOS-only-partial counters as unavailable rather than fabricating them. vmstat delegates counter collection to builtins/internal/vmstat, which on Linux reads /proc/{stat,meminfo,vmstat,loadavg,uptime} directly via os.Open (same documented exception as ss/ip route/df: the paths are hardcoded, never derived from user input) and on macOS calls sysctl(3) (hw.memsize, vm.swapusage, vm.loadavg) — the same darwin toolset df and ss already use, deliberately not Mach host_statistics64 (no existing builtin uses Mach, and adding it would introduce a new syscall review surface). Consequently, on macOS: per-page memory breakdown (buffers/cache/active/inactive beyond the sysctl-reported total), CPU tick counters, and paging/interrupt/context-switch rates are unavailable. Their column groups render as - (see builtins/internal/vmstat/vmstat_darwin.go).

  • free bypasses AllowedPaths for its Linux-only memory read, and is intentionally not implemented on macOS/Windows. free delegates to builtins/internal/meminfo, which on Linux reads /proc/meminfo directly via os.Open — the same documented exception as ss/ip route/df: the path is hardcoded, never derived from user input, and only aggregate host-wide memory/swap counters are exposed (no per-process or environment data). On macOS and Windows, meminfo.Read returns ErrNotSupported and free exits 1 rather than shipping a partial implementation: macOS has no sysctl(3)-level equivalent of the buffers/cache/shared breakdown (only the Mach host_statistics64 RPC exposes it, which needs cgo or dynamic libSystem symbol resolution that no other builtin in this repo uses), and Windows' GlobalMemoryStatusEx has no concept of buffers/cache/shared at all — reporting those columns as a literal 0 would read as "this host has none" rather than "not available on this platform," which could mislead an agent's memory-pressure diagnosis. free is also scoped to a single-shot snapshot (free, free -h) per the accepted-candidate design brief; it does not implement -s/-c repeated sampling — that is out of scope for free and belongs to the vmstat builtin.

  • pmap bypasses AllowedPaths for its per-process memory-map reads. pmap delegates to builtins/internal/procmaps, which on Linux reads <ProcPath>/<pid>/{comm,maps,smaps} directly via bounded os.Open reads — the same documented exception as ss/ip route/df/free: ProcPath is fixed by the embedding application and the remaining path is derived only from the numeric PID argument, never from arbitrary user input. Only address ranges, permission bits, mapping labels, and (in -x mode) per-mapping RSS/Dirty are exposed — never process argv or environment. On Windows, procmaps opens the target process with PROCESS_QUERY_INFORMATION|PROCESS_VM_READ and enumerates committed regions with VirtualQueryEx, labeling each by its MEMORY_BASIC_INFORMATION type rather than a resolved file path (GetMappedFileNameW, which would give a real path, is not wrapped by golang.org/x/sys/windows); -x returns ErrExtendedNotSupported on Windows since per-region RSS/Dirty needs a per-page working-set walk that the package does not implement. On macOS, procmaps enumerates regions via the proc_pidinfo(PROC_PIDREGIONINFO) kernel call, reached through the raw syscall.SYS_PROC_INFO trap since this Mach/BSD-hybrid libproc interface is not wrapped by golang.org/x/sys/unix (BSD syscalls only); the short process name comes from golang.org/x/sys/unix.SysctlKinfoProc, the same primitive builtins/internal/procinfo already uses for ps on darwin. -x returns ErrExtendedNotSupported on macOS too: proc_regioninfo reports resident/dirty counts for a region's whole shadow chain, not the private Rss/Dirty split Linux's smaps and pmap's extended columns expect, so reporting it would misrepresent the numbers rather than merely omit them.

  • lsof bypasses AllowedPaths for /proc/<pid>/fd/* metadata reads, but uniquely gates the NAME column. Like ss/ip route/df/free, lsof delegates /proc access to an internal package (builtins/internal/procfd) that reads /proc/<pid>/fd/*, /proc/<pid>/{cwd,root,exe}, /proc/<pid>/stat, and /proc/<pid>/status directly via os/golang.org/x/sys/unix calls on a hardcoded, non-user-controllable path: the same trusted-metadata exception, and the same reason ps shares its ProcPath override with lsof (interp.ProcPath). COMMAND/PID/USER/FD/TYPE are always shown. NAME, DEVICE, SIZE, and NODE are the deliberate divergence: unlike the bounded kernel counters ss/df/free expose, a /proc/<pid>/fd symlink target can resolve to any path on the host filesystem, so redactName (builtins/lsof/lsof.go) checks the resolved path against AllowedPaths and replaces NAME with (restricted) (or (restricted) (deleted) for a still-open unlinked file) when it falls outside every configured root; with no AllowedPaths configured, every NAME is restricted (an empty allowlist means no filesystem paths are reachable, not "unrestricted": see builtins/help/help.go). toRow additionally blanks DEVICE/SIZE/NODE on the same restricted rows (via the shared pathRestricted check), since those are per-file attributes tied to the same out-of-sandbox path: an exact byte count, device number, and inode would otherwise still let a caller fingerprint a specific restricted file (e.g. /etc/shadow) even with NAME hidden. Non-path targets (sockets, pipes, anonymous inodes) are never gated, since they never name a filesystem location. lsof also never reads /proc/<pid>/cmdline or /proc/<pid>/environ, matching ps's argv/environment privacy rule. lsof is Linux-only; macOS/Windows exit 1 with "not supported on this platform," the same pattern as free. lsof happy-path scenario tests cannot be added, for the same reason as free/ip route: the scenario framework has no platform-skip mechanism, so a script that succeeds with real data on Linux but exits 1 elsewhere can't be expressed as a single scenario expectation. Happy-path coverage instead lives in builtins/tests/lsof/lsof_linux_test.go (Go tests, runtime.GOOS-gated) plus the adversarial end-to-end tests in builtins/tests/lsof/lsof_pentest_linux_test.go; only lsof's help/error paths, which behave identically on every platform, have scenario coverage.

  • uptime bypasses AllowedPaths for system uptime and load average reads. uptime delegates all OS data acquisition to builtins/internal/sysinfo, which on Linux reads /proc/uptime and /proc/loadavg directly via os.Open. On macOS it calls unix.SysctlRaw("kern.boottime") and unix.SysctlRaw("vm.loadavg"). On Windows it calls GetTickCount64 via kernel32.dll. All data sources are hardcoded — never derived from shell-script input — so the same trade-off as ss, ip route, and df applies: operators cannot use AllowedPaths to block uptime from reading boot-time or load-average data. No file content beyond these fixed pseudo-files is accessed; user count is intentionally omitted.

  • journalctl and systemctl bypass AllowedPaths for trusted systemd target access. The builtins delegate journal discovery and reads, disk-usage scans, vacuuming, machine-ID reads, journald control-socket access, and restricted manager-bus operations to internal/systemd, which opens the configured paths directly with os APIs. SystemdTargetConfig.JournalDirs, SystemdTargetConfig.MachineIDPath, SystemdTargetConfig.JournalControlSocket, and SystemdTargetConfig.ManagerBusSocket are fixed by the embedding application when constructing the runner and cannot be supplied or changed by shell scripts. Operators therefore cannot use AllowedPaths to block these accesses; authorization is enforced separately by AllowedCommands, exact AllowedSystemServices unit/action grants, remediation mode for journal mutations, and remediation mode for every systemctl invocation. All configured target paths are trusted and must refer to the same host. On Linux, manager access pins the configured public system D-Bus socket through /proc/self/fd, authenticates with EXTERNAL, and verifies the systemd manager peer's machine ID before fixed manager-interface requests. See the trusted systemd target exception in docs/RULES.md for the complete backend requirements and indirect systemd effects.

  • A journal read grant on a .slice unit returns every member unit's log entries, not just the slice's own messages. For a non-slice unit, internal/systemd/journal_query_file.go builds a bounded match set that always names the granted unit: _SYSTEMD_UNIT=<unit>, UNIT=<unit> paired with _PID=1 or _SYSTEMD_CGROUP=/init.scope, OBJECT_SYSTEMD_UNIT=<unit> paired with _UID=0, and COREDUMP_UNIT=<unit> paired with _UID=0 and the fixed coredump MESSAGE_ID=. When the requested name ends in .slice, the reader adds an unpaired _SYSTEMD_SLICE=<unit> match, and journald stamps that field on every entry emitted by every unit placed in the slice — so an operator who configures AllowedSystemServices with {Service: "system.slice", Actions: [read]} and expects the slice unit's own messages has in fact granted journalctl -u system.slice read access to the logs of every service in system.slice. This is faithful to upstream journalctl -u <slice> and is deliberately not removed: dropping the match would make a slice query return misleadingly empty output instead of a bounded one. The match is exact rather than transitive (journald records a process's immediate slice), so a system.slice grant does not by itself reach units in a nested system-<child>.slice; reaching those means granting the child slice, which exposes that child's whole membership the same way. Slice membership is a runtime property of the host's unit files, drop-ins, and generators and is not derivable from the grant list, so operators MUST treat any .slice read grant as equivalent to granting journal read on that slice's entire current and future membership. This is a journal-read effect only: systemctl unit selection and list-units enumeration remain bound to the exact granted names. The behaviour is pinned by TestJournalFileQuerySliceGrantReturnsEveryMemberUnit in internal/systemd/journal_query_file_test.go; see the trusted systemd target exception in docs/RULES.md.

  • AllowedPaths containment is path-based and cannot see hard links, so content-mutating write targets fail closed on nlink > 1 — but rm does not. allowedpaths.Sandbox resolves every target through os.Root and a no-follow openat walk (allowedpaths/internal/writeopen/writeopen_unix.go). That is sound for symlinks, but a hard link is not a reference to a path — it is a second name for the same inode, and nothing in the resolution chain can observe that an in-sandbox name and an out-of-sandbox name are the same file. A pre-existing hard link inside a :rw root pointing at an inode also named outside every configured root therefore let truncate, logrotate -f, and >/>>/&> destroy out-of-sandbox content while every path check passed.

    The rule now is asymmetric, deliberately. Every primitive that mutates file content — Sandbox.Open with a write flag, Sandbox.Truncate, Sandbox.TruncateToZeroIfAtLeast — funnels through root.openWriteFile (allowedpaths/write_open.go), which fstats the descriptor it just opened and rejects a regular file whose link count is above one with ErrMultiplyLinkedWriteTarget ("hard links are not supported as write targets"). This mirrors the nlink != 1 skip that internal/systemd/journal_vacuum_unix.go has always applied to vacuum candidates, and it composes with the existing open-then-fstat TOCTOU closure in Sandbox.Truncate: the link count is read from the exact descriptor about to be mutated, not from a re-resolved path. Because open(2) performs O_TRUNC as part of the open itself, openWriteFile withholds O_TRUNC from the open syscall and replays it as an explicit ftruncate on the guarded descriptor — otherwise > would already have destroyed the shared content before the link count could be inspected.

    Sandbox.Remove is intentionally not gated. unlink(2) removes one directory entry; the inode and every other name for it survive untouched. Refusing to rm a hard link would break a legitimate operation without preventing any escape, since the out-of-sandbox content is not modified either way. rm on a hard link inside a :rw root therefore still succeeds, and operators must treat the existence of directory entries in a :rw root as unprotected regardless of inode aliasing.

    Residual gaps. The guard is coarse: it cannot enumerate an inode's names, so it also refuses legitimately hard-linked in-sandbox files, including hard-linked logs produced by some rotation schemes. It is unix-only: os.File.Stat on Windows backs FileInfo.Sys() with *syscall.Win32FileAttributeData, which carries no link count, and obtaining one would require a new GetFileInformationByHandle syscall surface, so fileLinkCount reports "unknown" there and the guard degrades to not-enforced (allowedpaths/hardlink_windows.go). It also only closes the pre-existing hard link vector; rshell has no ln, so it cannot create one itself. Operators must still treat every :rw root as trusted with respect to inode aliasing.

CRITICAL: Bug Fixes and Bash Compatibility

  • ALWAYS prioritise fixing the shell implementation to match bash behaviour over changing tests to match the current (incorrect) shell output. Never "fix" a failing test by updating its expected output to match broken shell behaviour — fix the shell instead.
  • Only deviate from bash behaviour when the shell is intentionally different (e.g. sandbox restrictions, blocked commands, readonly enforcement).

Builtin Implementation Rules

@docs/RULES.md

Testing

  • Before submitting any change that touches tests/scenarios/ or builtin implementations, run the bash comparison tests locally. These are skipped by default and require Docker:

    RSHELL_BASH_TEST=1 go test ./tests/ -run TestShellScenariosAgainstBash -timeout 120s

    The test suite runs all scenarios against debian:bookworm-slim (GNU bash + GNU coreutils) and compares output byte-for-byte. Only set skip_assert_against_bash: true in a scenario when the behavior intentionally diverges from bash (e.g. sandbox restrictions, blocked commands).

  • Prefer scenario tests (tests/scenarios/) over Go tests. Scenario tests are declarative YAML files that are automatically validated against both the shell and bash, making them easier to write, review, and maintain. Only use Go tests when scenario tests cannot express the required behaviour (e.g. testing Go APIs directly, complex programmatic assertions).

  • ip route show/ip route get happy-path scenario tests cannot be added. The scenario test framework's skip_windows field only skips Windows, and ip route reads /proc/net/route which is Linux-only — the command exits 1 with "not supported" on macOS too, so a single scenario still can't express the happy path on every platform. Happy-path coverage lives in builtins/tests/ip/ip_linux_test.go instead.

  • free happy-path scenario tests cannot be added, for the same reason as ip route. free is Linux-only; the same script exits 0 with real memory data on Linux but exits 1 with "not supported" elsewhere, and skip_windows only skips Windows, not macOS, so the scenario framework still cannot express a platform-conditional expectation. Happy-path coverage lives in builtins/internal/meminfo's parseMeminfo tests (fixed fixtures, run on Linux CI) plus TestFreeLinuxHappyPath/TestFreeNotSupportedOffLinux in builtins/free/free_test.go (both runtime.GOOS-gated). Only free's help/error paths, which behave identically on every platform, have scenario coverage.

  • In test scenarios, use expect.stderr when possible instead of stderr_contains.

  • Always use the YAML |+ block scalar for input.script, expect.stdout, and expect.stderr values, even single-line ones.

  • Test scenarios are asserted against bash by default. Only set skip_assert_against_bash: true for features that intentionally diverge from standard bash behavior (e.g. blocked commands, restricted redirects, readonly enforcement).

  • When expected output differs on Windows (e.g. path separators \ vs /), use Windows-specific assertion fields:

    • stdout_windows / stderr_windows — override stdout / stderr on Windows.
    • stdout_contains_windows / stderr_contains_windows — override stdout_contains / stderr_contains on Windows.
    • If the Windows field is not set, the non-Windows field is used as fallback.

  • Set skip_windows: true to skip a scenario entirely on Windows, for commands that are intentionally unsupported there rather than merely producing different output (e.g. vmstat, which only reads /proc on Linux and sysctl on macOS).