Skip to content

maskgen: --mintime/--mincomplexity act as maximums, and ?b/?h/?H/?1-?4 masks are costed as 1 - #10

Open
bandrel wants to merge 1 commit into
Hydraze:masterfrom
bandrel:pr/maskgen-statsgen-fixes
Open

maskgen: --mintime/--mincomplexity act as maximums, and ?b/?h/?H/?1-?4 masks are costed as 1#10
bandrel wants to merge 1 commit into
Hydraze:masterfrom
bandrel:pr/maskgen-statsgen-fixes

Conversation

@bandrel

@bandrel bandrel commented Jul 31, 2026

Copy link
Copy Markdown

Two of these are regressions against iphelix/pack@622dd92, so they affect this
fork specifically.

--mintime and --mincomplexity are maximums

(self.mincomplexity is None or mask_complexity <= self.mincomplexity) and \
(self.mintime is None or mask_time <= self.mintime) and \

Upstream has >= for both. As written, asking for a minimum silently returns
exactly the masks you were trying to exclude. minoccurrence and minlength
kept their >=, so only these two are affected.

?b, ?h, ?H and ?1-?4 are costed as complexity 1

getcomplexity handles only l u d s a. Upstream 622dd92 handles
l u d s a b h H 1 2 3 4 (added in "Added the rest of the built-in charsets"
and "Added support for custom character sets"), and the --custom-charset*-len
options are gone from the CLI here entirely.

The consequence is quiet and expensive: a hex mask like ?h?h?h?h?h?h?h?h?h?h
falls to the else branch, which prints a warning but leaves count
unmultiplied. The mask is costed as 1, so it looks instant, sorts to the front
of an --optindex run, and never counts against --targettime. With this PR
that mask correctly costs 16^10 and reports ~18 minutes at 1e9 keys/sec.

Uncostable masks (a literal in a .hcmask, or ?1 with no declared length) are
now rejected with a message rather than admitted with an understated cost.

Masks repeated across input files are not aggregated

self.masks[mask] = dict() is last-write-wins, but every occurrence is added to
total_occurrence. Loading two files that share masks understates coverage —
easy to see with the same file twice, which reported Masks coverage: 50% (6/12) before and 100% (12/12) after.

Runtimes truncate to zero

mask_complexity // self.pps means anything under a second reports 0s. At the
default 1e9 keys/sec that is every mask below a billion candidates:
?d?d?d?d?d?d?d?d (1e8) and ?l?l?l?l?l?l (3.1e8) both showed 0:00:00. That
makes --mintime/--maxtime unable to distinguish them and lets
--targettime accumulate zeros without ever tripping. Complexity stays integral
and the division is now floating point, which also resolves the
# TODO: Something wrong here, complexity and time doesn't match with estimated from policygengetmaskscoverage summed complexity and divided once while
loadmasks divided per mask, so the two paths genuinely disagreed.

Smaller items

  • --targettime wrote the overshooting mask to the output file before breaking,
    so the emitted set always exceeded the budget by one mask.
  • if options.mintime: and friends discard a legitimate 0; --pps 0 was
    accepted and then divided by. Now is not None throughout, and --pps 0 is
    rejected.
  • statsgen's Advanced Masks section hardcoded if count*100//filter_counter > 0 while the other three sections gate on self.hiderare, so sub-1% advanced
    masks were always hidden from the display regardless of the flag. The output
    file was and remains complete.
  • Empty or fully filtered input divided by zero in both tools.
  • statsgen never closed its output file, so a killed run could leave a
    truncated .masks.

Testing

Regression tests for each item are in
bandrel/pack as tests/test_pack.py. Happy
to split this into per-bug PRs if you'd rather review them separately.

Related: #9 (rulegen), iphelix#31 (the subset affecting the
Python 2 original).

…eporting

maskgen --mintime and --mincomplexity compared with <=, so both behaved as
maximums and selected exactly the masks the operator was excluding.

getcomplexity had lost ?b, ?h, ?H and ?1-?4 support along with the
--custom-charset*-len options, so masks using them were costed as
complexity 1: they looked instant, sorted to the front and never counted
against --targettime. Uncostable masks are now rejected rather than
silently understated.

A mask appearing in more than one input file was overwritten rather than
accumulated while the coverage denominator counted every occurrence, so a
complete mask set could report a fraction of its true coverage.

Mask runtimes used integer division, so anything under one second became
0s -- at the default 1e9 keys/sec that is every mask below a billion
candidates, which defeated --targettime and the time filters.

Also:
- --targettime withholds the mask that would overshoot instead of writing
  it and then reporting the overshoot.
- Zero-valued filters are honoured rather than discarded as falsey, and
  --pps 0 is rejected instead of dividing by zero.
- statsgen's Advanced Masks section ignored --hiderare and always hid
  entries under 1%; it is now gated like every other section, and the
  output file still receives the full set.
- Empty or fully filtered input no longer divides by zero.
- statsgen closes its output file on every exit path.

Regression tests are in https://github.com/bandrel/pack.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant