Skip to content

Update dependency chardet to v7 - #133

Open
renovate[bot] wants to merge 1 commit into
mainfrom
renovate/chardet-7.x
Open

Update dependency chardet to v7#133
renovate[bot] wants to merge 1 commit into
mainfrom
renovate/chardet-7.x

Conversation

@renovate

@renovate renovate Bot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

This PR contains the following updates:

Package Change Age Confidence
chardet (changelog) >=5,<7>=7,<8 age confidence

Release Notes

chardet/chardet (chardet)

v7.4.3

Compare Source

Patch release: fixes a crash when input contains null bytes inside a <meta charset> declaration.

Bug Fixes

  • Fixed ValueError: embedded null character crash when input contained a <meta charset> declaration with a null byte in the encoding name (e.g. b'<meta charset="\x00utf-8">'). codecs.lookup() raises ValueError on embedded nulls, and lookup_encoding() was only catching LookupError. Also added defensive ValueError catches in _validate_bytes() and _to_utf8() for completeness. (#​369, thanks @​DRMacIver for the report)

Full Changelog: chardet/chardet@7.4.2...7.4.3

v7.4.2

Compare Source

Patch release: fixes a crash on short inputs and closes a bunch of WHATWG/IANA alias gaps.

Bug Fixes

  • Fixed RuntimeError: pipeline must always return at least one result on ~2% of all possible two-byte inputs (e.g. b"\xf9\x92"). Multi-byte encodings like CP932 and Johab could score above the structural confidence threshold on very short inputs, but then statistical scoring would return nothing, leaving an empty result list instead of falling through to the fallback. (#​367, #​368, thanks @​jasonwbarnett)

Improvements

  • Added ~90 encoding aliases from the WHATWG Encoding Standard and IANA Character Sets registry so that <meta charset> labels like x-cp1252, x-sjis, dos-874, csUTF8, and the cswindows* family all resolve correctly through the markup detection stage. Every alias was driven by a failing spec-compliance test, not speculative. (#​366)
  • Added a spec-compliance test suite covering Python decode round-trips for all 86 registry encodings, WHATWG label resolution, IANA preferred MIME names, and Unicode/RFC conformance (BOM sniffing, UTF-8 boundary cases, UTF-16 surrogate pairs). This is the test suite that would have caught the 7.4.1 BOM bug before release. (#​366)

Full Changelog: chardet/chardet@7.4.1...7.4.2

v7.4.1

Compare Source

Bug Fixes

  • BOM-prefixed UTF-16/32 input now returns utf-16/utf-32 instead of utf-16-le/utf-16-be/utf-32-le/utf-32-be. The endian-specific codecs don't strip the BOM on decode, so callers were getting a stray U+FEFF at the start of their text. BOM-less detection is unchanged. (#​364, #​365)

Full Changelog: chardet/chardet@7.4.0...7.4.1

v7.4.0.post2

Compare Source

v7.4.0.post1

Compare Source

v7.3.0

Compare Source

License

  • 0BSD license — the project license has been changed from MIT to 0BSD, a maximally permissive license with no attribution requirement. All prior 7.x releases should also be considered 0BSD licensed as of this release.

Features

  • Added mime_type field to detection results — identifies file types for both binary (via magic number matching) and text content. Returned in all detect(), detect_all(), and UniversalDetector results. (#​350)
  • New pipeline/magic.py module detects 40+ binary file formats including images, audio/video, archives, documents, executables, and fonts. ZIP-based formats (XLSX, DOCX, JAR, APK, EPUB, wheel, OpenDocument) are distinguished by entry filenames. (#​350)

Bug Fixes

  • Fixed incorrect equivalence between UTF-16-LE and UTF-16-BE in accuracy testing — these are distinct encodings with different byte order, not interchangeable

Performance

  • Added 4 new modules to mypyc compilation (orchestrator, confusion, magic, ascii), bringing the total to 11 compiled modules
  • Capped statistical scoring at 16 KB — bigram models converge quickly, so large files no longer score the full 200 KB. Worst-case detection time dropped from 62ms to 26ms with no accuracy loss.
  • Replaced dataclasses.replace() with direct DetectionResult construction on hot paths, eliminating ~354k function calls per full test suite run

Build

  • Added riscv64 to the mypyc wheel build matrix — prebuilt wheels are now published for RISC-V Linux alongside existing architectures (#​348, thanks @​gounthar)

v7.2.0: chardet 7.2.0

Compare Source

Features

  • Added include_encodings and exclude_encodings parameters to detect(), detect_all(), and UniversalDetector — restrict or exclude specific encodings from the candidate set, with corresponding -i/--include-encodings and -x/--exclude-encodings CLI flags (#​343)
  • Added no_match_encoding (default "cp1252") and empty_input_encoding (default "utf-8") parameters — control which encoding is returned when no candidate survives the pipeline or the input is empty, with corresponding CLI flags (#​343)
  • Added -l/--language flag to chardetect CLI — shows the detected language (ISO 639-1 code and English name) alongside the encoding (#​342)

Fixes

  • Fixed null-separated ASCII data being misdetected as UTF-16-BE (#​346, #​347)

Full changelog: https://chardet.readthedocs.io/en/latest/changelog.html

v7.1.0: chardet 7.1.0

Compare Source

Features

  • Added PEP 263 encoding declaration detection — # -*- coding: ... -*- and # coding=... declarations on lines 1–2 of Python source files are now recognized with confidence 0.95 (#​249)
  • Added chardet.universaldetector backward-compatibility stub so that from chardet.universaldetector import UniversalDetector works with a deprecation warning (#​341)

Fixes

  • Fixed false UTF-7 detection of ASCII text containing ++ or +word patterns (#​332)
  • Fixed 0.5s startup cost on first detect() call — model norms are now computed during loading instead of lazily iterating 21M entries (#​333)
  • Fixed undocumented encoding name changes between chardet 5.x and 7.0 — detect() now returns chardet 5.x-compatible names by default (#​338)
  • Improved ISO-2022-JP family detection — recognizes ESC sequences for ISO-2022-JP-2004 (JIS X 0213) and ISO-2022-JP-EXT (JIS X 0201 Kana)
  • Fixed silent truncation of corrupt model data (iter_unpack yielded fewer tuples instead of raising)
  • Fixed incorrect date in LICENSE

Performance

  • 5.5x faster first-detect time (~0.42s → ~0.075s) by computing model norms as a side-product of load_models()
  • ~40% faster model parsing via struct.iter_unpack for bulk entry extraction (eliminates ~305K individual unpack calls)

New API parameters

  • Added compat_names parameter (default True) to detect(), detect_all(), and UniversalDetector — set to False to get raw Python codec names instead of chardet 5.x/6.x compatible display names
  • Added prefer_superset parameter (default False) — remaps legacy ISO/subset encodings to their modern Windows/CP superset equivalents (e.g., ASCII → Windows-1252, ISO-8859-1 → Windows-1252). This will default to True in the next major version (8.0).
  • Deprecated should_rename_legacy in favor of prefer_superset — a deprecation warning is emitted when used

Improvements

  • Switched internal canonical encoding names to Python codec names (e.g., "utf-8" instead of "UTF-8"), with compat_names controlling the public output format
  • Added lookup_encoding() to registry for case-insensitive resolution of arbitrary encoding name input to canonical names
  • Achieved 100% line coverage across all source modules (+31 tests)
  • Updated benchmark numbers: 98.2% encoding accuracy, 95.2% language accuracy on 2,510 test files
  • Pinned test-data cloning to chardet release version tags for reproducible builds

Full changelog: https://chardet.readthedocs.io/en/latest/changelog.html

v7.0.1

Compare Source

Fixes

  • Fixed false UTF-7 detection of SHA-1 git hashes (#​324, fixing #​323) — requirements files with VCS pins (e.g., +4bafdea3...) were misdetected as UTF-7, breaking tools like tox
  • Fixed _SINGLE_LANG_MAP missing aliases for single-language encoding lookup (e.g., big5big5hkscs)
  • Fixed PyPy TypeError in UTF-7 codec handling

Improvements

  • Retrained bigram models — 24 previously failing test cases now pass
  • Updated language equivalences for mutual intelligibility (Slovak/Czech, East Slavic + Bulgarian, Malay/Indonesian, Scandinavian languages)

New Contributors

  • @​rembish made their first contribution — both reporting the UTF-7 false detection issue and submitting the fix! (#​323, #​324)

v7.0.0

Compare Source

Ground-up, MIT-licensed rewrite of chardet. Same package name, same public API — drop-in replacement for chardet 5.x/6.x. Just way faster and more accurate!

Highlights:

  • MIT license (previous versions were LGPL)
  • 96.8% accuracy on 2,179 test files (+2.3pp vs chardet 6.0.0, +7.7pp vs charset-normalizer)
  • 41x faster than chardet 6.0.0 with mypyc (28x pure Python), 7.5x faster than charset-normalizer
  • Language detection for every result (90.5% accuracy across 49 languages)
  • 99 encodings across six eras (MODERN_WEB, LEGACY_ISO, LEGACY_MAC, LEGACY_REGIONAL, DOS, MAINFRAME)
  • 12-stage detection pipeline — BOM, UTF-16/32 patterns, escape sequences, binary detection, markup charset, ASCII, UTF-8 validation, byte validity, CJK gating, structural probing, statistical scoring, post-processing
  • Bigram frequency models trained on CulturaX multilingual corpus data for all supported language/encoding pairs
  • Optional mypyc compilation — 1.49x additional speedup on CPython
  • Thread-safe detect() and detect_all() with no measurable overhead; scales on free-threaded Python 3.13t+
  • Negligible import memory (96 B)
  • Zero runtime dependencies

Breaking changes vs 6.0.0:

  • detect() and detect_all() now default to encoding_era=EncodingEra.ALL (6.0.0 defaulted to MODERN_WEB)
  • Internal architecture is completely different (probers replaced by pipeline stages). Only the public API is preserved.
  • LanguageFilter is accepted but ignored (deprecation warning emitted)
  • chunk_size is accepted but ignored (deprecation warning emitted)

Configuration

📅 Schedule: (UTC)

  • Branch creation
    • At any time (no schedule defined)
  • Automerge
    • At any time (no schedule defined)

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.


  • If you want to rebase/retry this PR, check this box

This PR was generated by Mend Renovate. View the repository job log.

@renovate

renovate Bot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor Author

⚠️ Artifact update problem

Renovate failed to update an artifact related to this branch. You probably do not want to merge this PR as-is.

♻ Renovate will retry this branch, including artifacts, only when one of the following happens:

  • any of the package files in this branch needs updating, or
  • the branch becomes conflicted, or
  • you click the rebase/retry checkbox if found above, or
  • you rename this PR's title to start with "rebase!" to trigger it manually

The artifact failure details are included below:

File name: uv.lock
Command failed: uv lock --upgrade-package chardet
Using CPython 3.14.6 interpreter at: /opt/containerbase/tools/python/3.14.6/bin/python3
  × No solution found when resolving dependencies for split (markers:
  │ python_full_version >= '3.8' and python_full_version < '3.10'):
  ╰─▶ Because the requested Python version (>=3.8) does not satisfy
      Python>=3.10 and chardet>=7.0.0 depends on Python>=3.10, we can conclude
      that chardet>=7.0.0 cannot be used.
      And because only the following versions of chardet are available:
          chardet<=7.0.0
          chardet==7.0.1
          chardet==7.1.0
          chardet==7.2.0
          chardet==7.3.0
          chardet==7.4.0.post1
          chardet==7.4.0.post2
          chardet==7.4.1
          chardet==7.4.2
          chardet==7.4.3
      and your project depends on chardet>=7, we can conclude that your
      project's requirements are unsatisfiable.

hint: While the active Python version is 3.14, the resolution failed for other Python versions supported by your project. Consider limiting your project's supported Python versions using `requires-python`.
hint: The `requires-python` value (>=3.8) includes Python versions that are not supported by your dependencies (e.g., chardet>=7.0.0 only supports >=3.10). Consider using a more restrictive `requires-python` value (like >=3.10).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant