Skip to content

feat(scripts_bazel): needs.json -> LOBSTER lobster-req-trace converter#654

Draft
FScholPer wants to merge 3 commits into
mainfrom
feat/needs-to-lobster-converter
Draft

feat(scripts_bazel): needs.json -> LOBSTER lobster-req-trace converter#654
FScholPer wants to merge 3 commits into
mainfrom
feat/needs-to-lobster-converter

Conversation

@FScholPer

Copy link
Copy Markdown
Contributor

📌 Description

Adds scripts_bazel/needs_to_lobster.py: a converter that turns the sphinx-needs needs.json into a LOBSTER lobster-req-trace (.lobster) file, so sphinx-needs requirements can be aggregated with the code/test .lobster artifacts by the LOBSTER CLI (lobster-report / lobster-ci-report).

This is the "read-only converter" identified as the way to reuse the existing LOBSTER traceability engine (already used by rules_score / dependable_element in communication and baselibs) without re-implementing traceability checks in docs-as-code and without changing authoring or dashboards:

docs build       ->  sphinx-needs writes needs.json   (single source of truth)
needs_to_lobster ->  needs.json -> <module>.lobster    (lobster-req-trace, v3)
LOBSTER CLI      ->  lobster-report / lobster-ci-report over all *.lobster

RST authoring and sphinx-needs dashboards (needflow, needtable, …) are untouched — LOBSTER only sits downstream of needs.json.

Relationship to eclipse-score/score#3097

score#3097 (DR-009-Infra) is a proposed, still-skeletal decision record (its "Options Considered / Evaluation / Decision" sections are empty). It proposes a different route: extend dependable_element to read .rst, convert needs to TRLC on the fly, then TRLC -> LOBSTER. It does not contain a needs.json -> .lobster converter. This PR implements the direct needs.json -> .lobster path as an alternative that keeps everything Apache-side until the LOBSTER CLI boundary and does not require routing requirements through TRLC.

How it works

  • Reads the already metamodel-validated needs.json.
  • Emits one LOBSTER requirement item per need: tag (<namespace> <id>), location (file ref from docname/lineno), name, refs, just_*, framework=sphinx-needs, kind=<need type>, text, status.
  • satisfies links (and any configurable --up-links) become LOBSTER refs, preserving the process-requirement <-> tool-requirement relationship authored in RST as a LOBSTER up-trace.
bazel run //scripts_bazel:needs_to_lobster -- \
    --needs-json _build/needs.json --output tool_reqs.lobster \
    --types tool_req --up-links satisfies

Model the two sides as separate levels in lobster.conf:

requirements "Process Requirements" { source: "process_reqs.lobster"; }
requirements "Tool Requirements"    { source: "tool_reqs.lobster"; trace to: "Process Requirements"; }

Verified end-to-end with the real LOBSTER CLI: a tool_req__x :satisfies: two gd_req__* produces ref_up on the tool req and ref_down on the process reqs, and lobster-ci-report passes.

Licensing

LOBSTER is AGPLv3. This tool is stdlib-only and never imports LOBSTER — it writes the documented JSON schema and is consumed by the LOBSTER CLI as an external process, keeping the Apache-2.0 code at arm's length. (Making LOBSTER an enforcing CI dependency still needs an Eclipse/legal review; this PR does not add a CI gate.)

🚨 Impact Analysis

  • This change does not violate any tool requirements and is covered by existing tool requirements
  • This change does not violate any design decisions
  • Otherwise I have created a ticket for new tool qualification

New standalone CLI + Bazel target; no changes to the docs build, metamodel, or existing extensions. Draft — opened for discussion of the approach vs. score#3097.

✅ Checklist

  • Added/updated documentation for new or changed features
  • Added/updated tests to cover the changes
  • Followed project coding standards and guidelines

…erter

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@github-actions

github-actions Bot commented Jul 21, 2026

Copy link
Copy Markdown

License Check Results

🚀 The license check job ran with the Bazel command:

bazel run --lockfile_mode=error //src:license-check

Status: ⚠️ Needs Review

Click to expand output
[License Check Output]
Extracting Bazel installation...
Starting local Bazel server (8.6.0) and connecting to it...
INFO: Invocation ID: 0c4a57e2-6f5e-42a5-964b-7afd9d2a14f3
Computing main repo mapping: 
Loading: 
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
    currently loading: src
DEBUG: /home/runner/.bazel/external/rules_python+/python/private/repo_utils.bzl:101:16: 
rules_python:coverage_dep WARNING: rules_python's bundled coverage tool has no wheel for python_version=3.12.13, platform=ppc64le-unknown-linux-gnu. `bazel coverage` will produce empty lcov for py_test targets in this configuration. Either pin python_version to a version in the bundled set (see python/private/coverage_deps.bzl), or configure coverage manually via py_runtime.coverage_tool. See docs/coverage.md.
DEBUG: /home/runner/.bazel/external/rules_python+/python/private/repo_utils.bzl:101:16: 
rules_python:coverage_dep WARNING: rules_python's bundled coverage tool has no wheel for python_version=3.12.13, platform=riscv64-unknown-linux-gnu. `bazel coverage` will produce empty lcov for py_test targets in this configuration. Either pin python_version to a version in the bundled set (see python/private/coverage_deps.bzl), or configure coverage manually via py_runtime.coverage_tool. See docs/coverage.md.
DEBUG: /home/runner/.bazel/external/rules_python+/python/private/repo_utils.bzl:101:16: 
rules_python:coverage_dep WARNING: rules_python's bundled coverage tool has no wheel for python_version=3.12.13, platform=s390x-unknown-linux-gnu. `bazel coverage` will produce empty lcov for py_test targets in this configuration. Either pin python_version to a version in the bundled set (see python/private/coverage_deps.bzl), or configure coverage manually via py_runtime.coverage_tool. See docs/coverage.md.
DEBUG: /home/runner/.bazel/external/rules_python+/python/private/repo_utils.bzl:101:16: 
rules_python:coverage_dep WARNING: rules_python's bundled coverage tool has no wheel for python_version=3.12.13, platform=x86_64-unknown-linux-musl. `bazel coverage` will produce empty lcov for py_test targets in this configuration. Either pin python_version to a version in the bundled set (see python/private/coverage_deps.bzl), or configure coverage manually via py_runtime.coverage_tool. See docs/coverage.md.
Loading: 0 packages loaded
    currently loading: src
WARNING: Target pattern parsing failed.
ERROR: Skipping '//src:license-check': no such target '//src:license-check': target 'license-check' not declared in package 'src' defined by /home/runner/work/docs-as-code/docs-as-code/src/BUILD
ERROR: no such target '//src:license-check': target 'license-check' not declared in package 'src' defined by /home/runner/work/docs-as-code/docs-as-code/src/BUILD
INFO: Elapsed time: 7.298s
INFO: 0 processes.
ERROR: Build did NOT complete successfully
ERROR: Build failed. Not running target

@github-actions

Copy link
Copy Markdown

The created documentation from the pull request is available at: docu-html

…IRECTORY

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
needs_json = _resolve(args.needs_json)
output = _resolve(args.output) if args.output is not None else None

needs_data = json.loads(needs_json.read_text(encoding="utf-8"))
if output is None:
print(payload)
else:
output.write_text(payload + "\n", encoding="utf-8")
…>tool)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

2 participants