Skip to content

Hebrew text: model outputs a hallucinated token while Latin text on the same image is read correctly — is Hebrew supported? #90

Description

@aseel1

Summary

On Hebrew-language documents the model produces no usable output: dense full pages return
empty results at every size preset, and on a large-glyph crop it reads the Latin text
correctly while answering the Hebrew text on the same image with an invented,
niqqud-decorated token repeated several times. This looks like Hebrew (Hebr script) is
outside the training distribution. Related: #3 (unanswered language-support question).

Environment

  • baidu/Unlimited-OCR via transformers==4.55.0, trust_remote_code=True, torch_dtype=torch.bfloat16
  • NVIDIA L4 (24 GB), CUDA build of torch 2.13, Linux (GCP Deep Learning VM)
  • model.infer(tokenizer, prompt="<image>\nFree OCR.", image_file=..., output_path=..., save_results=True)

What we tested (controlled)

Input: a scanned Hebrew business invoice (A4, mixed Hebrew/Latin header), rendered at 200 DPI
(1654×2339). Each run in a fresh process.

run config result
full page base_size=1024, image_size=640, crop_mode=True (default/gundam) empty output
full page base_size=1024, image_size=1024, crop_mode=False empty output (3–4 s)
full page base_size=1280, image_size=1280, crop_mode=False empty output (3–4 s)
top-30% header crop (large glyphs) base_size=1280, image_size=1280, crop_mode=False Latin tokens read correctly (Master-Coach, the invoice number 10707), but ALL Hebrew on the crop answered as תְבָרִי לַלְעֹן repeated ×3 — a token that appears nowhere in the image

The Latin success in the same inference rules out an image-quality or harness problem.
An earlier full-page run with the <|grounding|> prompt degenerated into a repeated
<|det|>text [102, 844, 190, 844]<|/det|>[Non-Text] loop until max_length.

Reproduction should work with any Hebrew document image (e.g. render a paragraph of Hebrew
text plus one English word at ~1200 px height and compare which of the two comes back).

Questions

  1. Is Hebrew (or RTL scripts generally) included in the training data / supported language set?
  2. If not, could the README document the supported-language list? (Language Support #3 asks this too.)
  3. If Hebrew is expected to work, is there a recommended config for RTL documents?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions