data ingest: JSON file-bearing manifests bypass the "filename" column preflight (#372-class, JSON path)

[divyasinghds]

Part of epic tracebloc/backend#1142 (Theme D — preflight parity). Follow-up from #372.

LOW/MED. The filename-column contract is now enforced at preflight for CSV manifests (ingestor IngestableRecordsValidator, data-ingestors PR #384; CLI CheckImageFilenameColumn, #373). A file-bearing category can also be ingested from a JSON manifest (source_type: json is independent of data_format), and that path has no equivalent preflight.

Gap

• The ingestor's IngestableRecordsValidator is CSV-only by design — every check (_check_has_rows, _check_referenced_files, and the new _check_filename_column) no-ops on a non-CSV input. So a JSON manifest is never checked for the filename column.
• The transfer read is still the case-sensitive record.get("filename"), so a JSON manifest whose records use Filename (case variant) or a wrong key (image_id) resolves to None and fails cluster-side with exit 9 ("No filename found in record") after the full upload — the exact accept-locally / fail-after-upload gap data ingest: "filename" column resolved case-insensitively at preflight but case-sensitively at transfer (#340-class) #372 closed for CSV.
• The CLI (#373) is labels.csv-only too, so there's no JSON preflight on either side.

Note: data-ingestors #382 briefly covered this via a universal ingest-time remap, but it resolved the asymmetry the wrong way (case-insensitive accept) and was reverted by #384 in favor of the "filename only" (case-sensitive) contract. So JSON is back to long-standing pre-#382 behavior (fail-late), not a new regression.

Preferred fix (ingestor-side)

Extend the ingestor's preflight to the JSON manifest path: peek the first record's keys and apply the same exact-lowercase filename reject _check_filename_column already applies to CSV (case-sensitive, whitespace-lenient), so preflight and transfer agree for JSON too. Handle sparse records the way _resolve_label_column does (retry past a leading object that omits the key). Optionally mirror in the CLI if/when JSON manifests get a dry-run preview.

Repro

tb data ingest <dir> --task image_classification with a JSON manifest whose records use {"Filename": "...", "label": "..."} → dry-run/preflight passes, real ingest fails cluster-side with exit 9.

[LukasWodka]

Closing as completed — the implementing PR merged; only the issue never closed.

Why it stayed open: this repo's default branch was main while the PR merged to develop, and GitHub fires closing keywords only on merges into the default branch. So Closes #N never triggered here, no matter how it was written.

Fixed at the source today (2026-07-25): this repo's default branch is now develop, so closing keywords will work from here on. The PR template was also corrected — it previously suggested Ref tracebloc/other-repo#456 for cross-repo links, and Ref is not a closing keyword.

Closed as part of a board hygiene pass. If any part of this issue is not actually covered by the merged PR, please reopen — it was closed on the strength of a merged reference, not a line-by-line audit.