diff --git a/src/extensions/docs/index.rst b/src/extensions/docs/index.rst index 0ae5047f4..81134129e 100644 --- a/src/extensions/docs/index.rst +++ b/src/extensions/docs/index.rst @@ -70,6 +70,14 @@ Extensions Architecture and design of the ``score_mounts`` bridge extension. :ref:`Mounts Extension Internals` + .. grid-item-card:: + + Module Verification Report + ^^^ + The ``.. module-verification-report::`` directive that expands + into the standard per-module verification report body. + :ref:`Module Verification Report` + .. toctree:: :maxdepth: 1 @@ -81,3 +89,4 @@ Extensions Extension Guide Sync TOML mounts_internals + module_verification_report diff --git a/src/extensions/docs/module_verification_report.rst b/src/extensions/docs/module_verification_report.rst new file mode 100644 index 000000000..4b9c058e7 --- /dev/null +++ b/src/extensions/docs/module_verification_report.rst @@ -0,0 +1,179 @@ +.. + # ******************************************************************************* + # Copyright (c) 2026 Contributors to the Eclipse Foundation + # + # See the NOTICE file(s) distributed with this work for additional + # information regarding copyright ownership. + # + # This program and the accompanying materials are made available under the + # terms of the Apache License Version 2.0 which is available at + # https://www.apache.org/licenses/LICENSE-2.0 + # + # SPDX-License-Identifier: Apache-2.0 + # ******************************************************************************* + +.. _module_verification_report: + +Module Verification Report extension +==================================== + +``score_module_verification_report`` provides the +``.. module-verification-report::`` directive, which emits the module's +``mod_ver_report`` need. The report body — a feature summary, a component +overview table, and one detailed section per component — is a Sphinx-Needs +content template (``src/needs_templates/mod_ver_report.need``) that the need +selects via ``:template:``. Traceability is resolved by sphinx-needs at render +time: the template only emits ``.. needtable::`` / ``.. needpie::`` widgets +with the right filters. + +The extension is part of the :ref:`score_sphinx_bundle`. +No external config file is required for the common case. + +Typical usage (``verification_report/module_verification_report.rst``): + +.. code-block:: rst + + .. module-verification-report:: + :id: mod_vrep__mymodule__report + :module-id: mod__mymodule + :components: comp__mymodule_a, comp__mymodule_b + :features: feat__mymodule + :safety: QM + :security: YES + :status: valid + :verification-method: test_and_inspection + +.. _mvr_directive: + +Options +------- + +.. list-table:: + :header-rows: 1 + :widths: 22 12 66 + + * - Option + - Required + - Description + + * - ``:id:`` + - yes + - Id of the generated ``mod_ver_report`` need, used verbatim. Must + follow the 3-part scheme the need type requires + (``mod_vrep____``); ``score_metamodel`` validates it + like any other need id. + + * - ``:module-id:`` + - yes + - sphinx-needs id of the ``.. mod::`` need (e.g. ``mod__mymodule``). + Also names the module whose component-id prefix + (``comp___``) the template strips to derive component + slugs and titles. + + * - ``:components:`` + - yes + - Comma-separated list of ``.. comp::`` need ids. Named after the + ``components`` link of the ``mod_ver_report`` need type, which it + populates verbatim. Multi-line values are supported. Optional + ``[version==N]`` qualifiers are stripped. + + * - ``:features:`` + - yes + - Comma-separated list of ``.. feat::`` need ids. Named after the + ``features`` link of the ``mod_ver_report`` need type, which it + populates verbatim. Usually a single id; one ``Feature`` section is + rendered per entry. Not derived from ``:module-id:`` — guessing a + mandatory traceability link would silently produce a dangling link + whenever the guess is wrong. + + * - ``:safety:`` + - yes + - ASIL classification of the module. One of ``QM`` or ``ASIL_B``. + + * - ``:security:`` + - yes + - Whether the module is security-relevant. One of ``YES`` or ``NO``. + + * - ``:status:`` + - yes + - Review status of the report. One of ``valid`` or ``invalid``. + + * - ``:verification-method:`` + - yes + - Free-text description of how the module was verified, e.g. + ``test_and_inspection``. + + * - ``:version:`` + - no + - Version of the emitted ``mod_ver_report`` need. Default: ``1``. + +Metamodel validation +-------------------- + +``:safety:``, ``:security:``, ``:status:``, ``:verification-method:`` and +``:version:`` are not just directive options — the directive uses them to +emit a single sphinx-needs ``mod_ver_report`` need (id taken from +``:id:``, linked ``belongs_to`` the module's +``.. mod::`` need). ``:components:`` and ``:features:`` are passed straight +through to the need's ``components`` and ``features`` links, which +``metamodel.yaml`` declares mandatory and types to ``comp`` / ``feat``. This +need type, its id format and the allowed values for each option are declared +in ``score_metamodel``'s ``metamodel.yaml`` (``mod_ver_report`` entry). + +Every generated need is checked against that definition by the +``score_metamodel`` Sphinx extension as part of the regular build. If any +value does not match the expected pattern (e.g. ``:safety: ASIL_D``, which +is not one of ``QM``/``ASIL_B``), a mandatory option is missing, or the id +does not follow the required ``____`` +scheme, ``score_metamodel`` reports a warning. Since the documentation +build runs Sphinx with ``-W`` (warnings treated as errors), any such +mismatch aborts the build instead of silently producing an inconsistent +report. + +The report template +------------------- + +The body lives in ``src/needs_templates/mod_ver_report.need``, a Jinja +template rendered by Sphinx-Needs. Two properties of that mechanism shape it: + +*Templates render during the read phase*, when the need is created and the +needs graph does not exist yet. The template therefore never looks other needs +up. It reads ``belongs_to`` / ``components`` / ``features`` off the need +itself, derives component slugs and titles from the ids by string +manipulation, and leaves everything else to ``needtable`` / ``needpie``, which +resolve at write time. + +*A need's content cannot open new sections* — docutils rejects them with +"Unexpected section title". The template is therefore applied as +``:post_template:``, not ``:template:``: post-content is placed after the need +at document level, where real section headings work. That is what gives the +report its TOC entries and per-component navigation. Each component section is +additionally a stable link target (``comp-``). + +The heading levels are ``-`` for ``Feature`` and ``Components``, ``~`` for +``Component Overview`` and each component, and ``^`` for the subsections +within a feature or component. + +Graph consistency +----------------- + +Because the need records which architecture needs the report describes, it can +be cross-checked against them. ``score_metamodel``'s +``check_mod_ver_report_links`` graph check enforces, per report: + +#. The need's ``components`` and the module's ``:includes:`` must be the same + set. The report and the module are two independent statements about which + components make up the module; if they disagree, one of them is stale. Both + directions are reported, independently. +#. Every feature a listed component ``belongs_to`` must itself appear in + ``:features:``. A report spanning several features is fine; a component + whose feature the report never mentions is not, because the feature-level + statistics would silently omit it. + +Ids in ``:components:`` or ``:features:`` that do not resolve to a need are +reported as well. Every problem is reported as a warning rather than raised, +so one build surfaces all of them. + +Like every other graph check, it can be disabled or run in isolation via the +``score_metamodel_checks`` config value, e.g. +``score_metamodel_checks = "check_mod_ver_report_links"``. diff --git a/src/extensions/score_metamodel/checks/mod_ver_report_checks.py b/src/extensions/score_metamodel/checks/mod_ver_report_checks.py new file mode 100644 index 000000000..8fe1b45af --- /dev/null +++ b/src/extensions/score_metamodel/checks/mod_ver_report_checks.py @@ -0,0 +1,175 @@ +# ******************************************************************************* +# Copyright (c) 2026 Contributors to the Eclipse Foundation +# +# See the NOTICE file(s) distributed with this work for additional +# information regarding copyright ownership. +# +# This program and the accompanying materials are made available under the +# terms of the Apache License Version 2.0 which is available at +# https://www.apache.org/licenses/LICENSE-2.0 +# +# SPDX-License-Identifier: Apache-2.0 +# ******************************************************************************* +"""Graph checks for ``mod_ver_report`` needs. + +A ``mod_ver_report`` need declares the module it belongs to (``belongs_to``) +and the architecture needs it describes (``components`` and ``features``, both +mandatory links). Its body is rendered by the ``mod_ver_report`` content +template from those same fields. + +Because all of that lives in the needs graph, the report can be validated +against the needs it claims to describe: + +1. ``components`` and the module's ``includes`` must name the *same* set. The + report and the module are two independent statements about which components + make up the module — if they disagree, one of them is stale. Both + directions are reported, and independently: a report that skips a component + of its module is exactly as wrong as one that describes a component the + module does not have. +2. Every feature a listed component ``belongs_to`` must itself be listed in + ``features``. A report spanning several features is fine — what is not fine + is a component whose feature the report never mentions, because the + feature-level statistics then silently omit it. + +Everything here reports through :class:`CheckLogger` rather than raising: a +malformed report must not abort the whole docs build, and the author needs to +see every problem in one run, not just the first. +""" + +from __future__ import annotations + +from score_metamodel import ( + CheckLogger, + graph_check, +) +from sphinx.application import Sphinx +from sphinx_needs.data import NeedsView +from sphinx_needs.need_item import NeedItem + + +def _linked_ids(need: NeedItem, link: str) -> list[str]: + """Return the ids linked via *link*, or an empty list. + + A declared but unset link yields ``[]``. The ``or []`` also covers a need + type that does not declare *link* at all, where the lookup yields ``None``. + """ + return need.get(link) or [] + + +def _join(ids: list[str]) -> str: + """Render a list of need ids for a warning message.""" + return ", ".join(f"`{i}`" for i in sorted(ids)) + + +def _resolve( + report: NeedItem, + link: str, + all_needs: NeedsView, + log: CheckLogger, +) -> list[NeedItem]: + """Resolve the ids linked via *link* to needs, warning about unknown ones.""" + resolved: list[NeedItem] = [] + for need_id in _linked_ids(report, link): + target = all_needs.get(need_id) + if target is None: + log.warning_for_need( + report, f"`{link}` references `{need_id}`, which is not a known need." + ) + continue + resolved.append(target) + return resolved + + +def _check_component_parity( + report: NeedItem, module: NeedItem, log: CheckLogger +) -> None: + """The report's ``components`` and the module's ``includes`` must match.""" + module_components = set(_linked_ids(module, "includes")) + report_components = set(_linked_ids(report, "components")) + module_id = module["id"] + + # The two directions are independent problems, so they are reported + # independently — a report that lists a stale component must still be told + # about the component it is missing. + missing_from_report = module_components - report_components + if missing_from_report: + log.warning_for_need( + report, + f"does not list {_join(list(missing_from_report))} under " + f"`components`, but `{module_id}` `includes` " + f"{'them' if len(missing_from_report) > 1 else 'it'}. The " + "verification report must describe every component of the module.", + ) + + missing_from_module = report_components - module_components + if missing_from_module: + log.warning_for_need( + report, + f"lists {_join(list(missing_from_module))} under `components`, but " + f"`{module_id}` does not `includes` " + f"{'them' if len(missing_from_module) > 1 else 'it'}.", + ) + + +def _check_features_included( + report: NeedItem, + features: list[NeedItem], + components: list[NeedItem], + log: CheckLogger, +) -> None: + """Every feature a listed component belongs to must be listed too. + + A component may belong to more than one feature, and a report may span + more than one feature, so this compares the *full* set of features reached + through the components against the set the report declares. + """ + listed_feature_ids = {feature["id"] for feature in features} + + # feature id -> the listed components that belong to it. Keyed by feature + # so the warning can name both the feature that is missing and the + # components that pointed at it. + unlisted: dict[str, list[str]] = {} + for component in components: + for feature_id in _linked_ids(component, "belongs_to"): + if feature_id not in listed_feature_ids: + unlisted.setdefault(feature_id, []).append(component["id"]) + + for feature_id in sorted(unlisted): + components_str = _join(unlisted[feature_id]) + log.warning_for_need( + report, + f"does not list `{feature_id}` under `features`, but " + f"{components_str} " + f"{'belong' if len(unlisted[feature_id]) > 1 else 'belongs'} to it.", + ) + + +@graph_check +def check_mod_ver_report_links( + app: Sphinx, + all_needs: NeedsView, + log: CheckLogger, +) -> None: + """Validate that every ``mod_ver_report`` agrees with the needs it describes.""" + reports = all_needs.filter_is_external(False).filter_types(["mod_ver_report"]) + + for report in reports.values(): + components = _resolve(report, "components", all_needs, log) + features = _resolve(report, "features", all_needs, log) + modules = _resolve(report, "belongs_to", all_needs, log) + + _check_features_included(report, features, components, log) + + if not modules: + # `belongs_to` is a mandatory link: the option checks report it + # missing, and _resolve already warned about an unresolvable id. + # Nothing left to compare the components against. + continue + if len(modules) > 1: + log.warning_for_need( + report, + f"`belongs_to` names {len(modules)} modules " + f"({_join([m['id'] for m in modules])}); a verification report " + "describes exactly one module.", + ) + _check_component_parity(report, modules[0], log) diff --git a/src/extensions/score_metamodel/metamodel.yaml b/src/extensions/score_metamodel/metamodel.yaml index 33b088d52..f794d8dbd 100644 --- a/src/extensions/score_metamodel/metamodel.yaml +++ b/src/extensions/score_metamodel/metamodel.yaml @@ -988,11 +988,12 @@ needs_types: mandatory_links: # req-Id: tool_req__docs_verification_report_need belongs_to: mod + components: comp + features: feat optional_links: # req-Id: tool_req__docs_verification_report_need contains: ANY evidence: ANY - covers: ANY realizes: workproduct tags: - verification_report @@ -1142,6 +1143,20 @@ needs_extra_links: incoming: evidence_for outgoing: evidence + # req-Id: tool_req__docs_verification_report_need + # Mandatory links of `mod_ver_report`: the architecture needs a module + # verification report describes. They must be declared here as well, not just + # in the need type's `mandatory_links` — sphinx-needs only creates a + # directive option for links listed in `needs_extra_links`, and silently + # drops the value of any option it does not know. + components: + incoming: component_reported_by + outgoing: components + + features: + incoming: feature_reported_by + outgoing: features + ############################################################## # Graph Checks diff --git a/src/extensions/score_module_verification_report/BUILD b/src/extensions/score_module_verification_report/BUILD new file mode 100644 index 000000000..7d90f76c6 --- /dev/null +++ b/src/extensions/score_module_verification_report/BUILD @@ -0,0 +1,55 @@ +# ******************************************************************************* +# Copyright (c) 2026 Contributors to the Eclipse Foundation +# +# See the NOTICE file(s) distributed with this work for additional +# information regarding copyright ownership. +# +# This program and the accompanying materials are made available under the +# terms of the Apache License Version 2.0 which is available at +# https://www.apache.org/licenses/LICENSE-2.0 +# +# SPDX-License-Identifier: Apache-2.0 +# ******************************************************************************* +load("@aspect_rules_py//py:defs.bzl", "py_library") +load("@docs_as_code_hub_env//:requirements.bzl", "all_requirements") +load("//:score_pytest.bzl", "score_pytest") + +filegroup( + name = "sources", + srcs = glob(["*.py"]), +) + +filegroup( + name = "tests", + srcs = glob(["tests/*.py"]), +) + +filegroup( + name = "all_sources", + srcs = [ + ":sources", + ":tests", + ], + visibility = ["//visibility:public"], +) + +py_library( + name = "score_module_verification_report", + srcs = [":sources"], + imports = ["."], + visibility = ["//visibility:public"], + deps = all_requirements + [ + "@score_docs_as_code//src/helper_lib", + ], +) + +score_pytest( + name = "score_module_verification_report_tests", + size = "small", + srcs = glob(["tests/*.py"]), + # test_needs_template.py renders the shipped ``mod_ver_report.need`` + # template, so it must be present in the test's runfiles. + data = ["@score_docs_as_code//src/needs_templates:files"], + deps = [":score_module_verification_report"], + pytest_config = "//:pyproject.toml", +) diff --git a/src/extensions/score_module_verification_report/__init__.py b/src/extensions/score_module_verification_report/__init__.py new file mode 100644 index 000000000..2e3426fcd --- /dev/null +++ b/src/extensions/score_module_verification_report/__init__.py @@ -0,0 +1,52 @@ +# ******************************************************************************* +# Copyright (c) 2026 Contributors to the Eclipse Foundation +# +# See the NOTICE file(s) distributed with this work for additional +# information regarding copyright ownership. +# +# This program and the accompanying materials are made available under the +# terms of the Apache License Version 2.0 which is available at +# https://www.apache.org/licenses/LICENSE-2.0 +# +# SPDX-License-Identifier: Apache-2.0 +# ******************************************************************************* +"""Sphinx extension providing the module verification report directive. + +Usage in RST:: + + .. module-verification-report:: + :id: mod_vrep__baselibs__report + :module-id: mod__baselibs + :features: feat__baselibs + :components: comp__baselibs_json, comp__baselibs_containers + :safety: ASIL_B + :security: YES + :status: valid + :verification-method: test_and_inspection + +The directive emits a single sphinx-needs ``mod_ver_report`` need. Its body — +feature summary and statistics, component overview, and one section per +component — comes from the ``mod_ver_report`` content template +(``src/needs_templates/mod_ver_report.need``), which Sphinx-Needs renders from +the need's own fields. + +Consistency of the need with the rest of the graph — does the module +``includes`` exactly the components the report lists? does every listed +component ``belongs_to`` a listed feature? — is validated by +``score_metamodel``'s ``check_mod_ver_report_links`` graph check. +""" + +from __future__ import annotations + +from typing import Any + +from .directive import ModuleVerificationReportDirective + + +def setup(app: Any) -> dict: + app.add_directive("module-verification-report", ModuleVerificationReportDirective) + return { + "version": "1.0", + "parallel_read_safe": True, + "parallel_write_safe": True, + } diff --git a/src/extensions/score_module_verification_report/directive.py b/src/extensions/score_module_verification_report/directive.py new file mode 100644 index 000000000..ba11c9724 --- /dev/null +++ b/src/extensions/score_module_verification_report/directive.py @@ -0,0 +1,153 @@ +# ******************************************************************************* +# Copyright (c) 2026 Contributors to the Eclipse Foundation +# +# See the NOTICE file(s) distributed with this work for additional +# information regarding copyright ownership. +# +# This program and the accompanying materials are made available under the +# terms of the Apache License Version 2.0 which is available at +# https://www.apache.org/licenses/LICENSE-2.0 +# +# SPDX-License-Identifier: Apache-2.0 +# ******************************************************************************* +"""The ``.. module-verification-report::`` Sphinx directive.""" + +from __future__ import annotations + +from docutils import nodes +from docutils.statemachine import ViewList +from sphinx.util.docutils import SphinxDirective + +# ``:post_template:`` is a Sphinx-Needs core option, so score_metamodel's +# option check accepts it on a metamodel-defined need type. It selects +# ``src/needs_templates/mod_ver_report.need``, which renders the whole report +# body from the need's own fields — this directive emits the need, nothing more. +# +# Post-content, not content: a need's *content* cannot open new sections +# ("Unexpected section title"), so a ``:template:`` body could only use +# ``.. rubric::`` and would produce no TOC entries. Post-content is placed +# after the need at document level, where real section headings work. +NEEDS_TEMPLATE_NAME = "mod_ver_report" + +MOD_VER_REPORT_TEMPLATE = """\ +.. mod_ver_report:: {title} + :id: {report_id} + :post_template: {template_name} + :version: {version} + :safety: {safety} + :security: {security} + :status: {status} + :verification_method: {verification_method} + :belongs_to: {module_id} + :components: {components} + :features: {features} + +""" + +# Every option the directive requires. Each maps onto a mandatory option or +# link of the ``mod_ver_report`` need type (see metamodel.yaml), so none of +# them can be defaulted: guessing a traceability link would silently produce a +# dangling one whenever the guess is wrong. +_REQUIRED_OPTIONS = ( + "id", + "module-id", + "components", + "features", + "safety", + "security", + "status", + "verification-method", +) + + +def _join_ids(ids_str: str) -> str: + """Normalise a comma-separated option value onto a single line. + + Multi-line values are supported — docutils folds them into one string with + newlines, which would break the emitted option. Version qualifiers such as + ``[version==1]`` are passed through: Sphinx-Needs parses them itself, and + stripping them here would silently drop the constraint. + """ + return ", ".join(part.strip() for part in ids_str.split(",") if part.strip()) + + +def _report_title(module_short: str) -> str: + """Derive the report's human-readable title from the module slug. + + Only the display title is derived. The need's id comes from ``:id:`` and is + passed through untouched, so the author stays in control of it. + """ + return f"{module_short.replace('_', ' ').title()} Verification Report" + + +class ModuleVerificationReportDirective(SphinxDirective): + """Emit the ``mod_ver_report`` need for one module. + + Usage:: + + .. module-verification-report:: + :id: mod_vrep__mymodule__report + :module-id: mod__mymodule + :components: comp__mymodule_a, comp__mymodule_b + :features: feat__mymodule + :safety: QM + :security: YES + :status: valid + :verification-method: test_and_inspection + + Every option is required. The directive is a shorthand: ``:id:`` becomes + the need's id verbatim, the title is derived from ``:module-id:``, and the + rest is passed straight through. + + The report *body* is not generated here — the emitted need selects the + ``mod_ver_report`` content template, which Sphinx-Needs renders from the + need's own fields. + """ + + required_arguments = 0 + optional_arguments = 0 + option_spec = {opt: str for opt in _REQUIRED_OPTIONS} | {"version": str} + has_content = False + + def run(self) -> list[nodes.Node]: + missing = [opt for opt in _REQUIRED_OPTIONS if not self.options.get(opt)] + if missing: + # Report here, where the author can see which directive is at + # fault, rather than as a metamodel warning about a generated need. + return [ + self.state_machine.reporter.error( + "module-verification-report: missing mandatory option(s) " + f"{', '.join(':' + m + ':' for m in missing)} required to " + "generate the mod_ver_report need", + line=self.lineno, + ) + ] + + module_id = self.options["module-id"] + module_short = ( + module_id[len("mod__") :] if module_id.startswith("mod__") else module_id + ) + rst_text = MOD_VER_REPORT_TEMPLATE.format( + title=_report_title(module_short), + report_id=self.options["id"], + template_name=NEEDS_TEMPLATE_NAME, + version=self.options.get("version", "1"), + safety=self.options["safety"], + security=self.options["security"], + status=self.options["status"], + verification_method=self.options["verification-method"], + module_id=module_id, + components=_join_ids(self.options["components"]), + features=_join_ids(self.options["features"]), + ) + + view_list = ViewList() + for lineno, line in enumerate(rst_text.splitlines()): + view_list.append(line, "", lineno) + + # A plain nested_parse is enough: the emitted block is a single + # directive with no section titles. + container = nodes.container() + container.document = self.state.document + self.state.nested_parse(view_list, self.content_offset, container) + return container.children diff --git a/src/extensions/score_module_verification_report/tests/test_directive.py b/src/extensions/score_module_verification_report/tests/test_directive.py new file mode 100644 index 000000000..1ba9ea1b2 --- /dev/null +++ b/src/extensions/score_module_verification_report/tests/test_directive.py @@ -0,0 +1,168 @@ +# ******************************************************************************* +# Copyright (c) 2026 Contributors to the Eclipse Foundation +# +# See the NOTICE file(s) distributed with this work for additional +# information regarding copyright ownership. +# +# This program and the accompanying materials are made available under the +# terms of the Apache License Version 2.0 which is available at +# https://www.apache.org/licenses/LICENSE-2.0 +# +# SPDX-License-Identifier: Apache-2.0 +# ******************************************************************************* +"""Unit tests for :mod:`score_module_verification_report.directive`. + +The directive needs a full Sphinx environment to instantiate, so the pure +helpers and the emitted RST are tested in isolation. +""" + +from __future__ import annotations + +from src.extensions.score_module_verification_report.directive import ( + _REQUIRED_OPTIONS, + MOD_VER_REPORT_TEMPLATE, + NEEDS_TEMPLATE_NAME, + _join_ids, + _report_title, +) + +# --------------------------------------------------------------------------- +# _join_ids +# --------------------------------------------------------------------------- + + +def test_join_single_id(): + assert _join_ids("comp__mymod_json") == "comp__mymod_json" + + +def test_join_normalises_spacing_and_order(): + assert ( + _join_ids("comp__mymod_json,comp__mymod_bits") + == "comp__mymod_json, comp__mymod_bits" + ) + + +def test_join_folds_multiline_values_onto_one_line(): + """docutils folds a multi-line option value into one string with newlines. + + They must not survive into the emitted option or the RST breaks. + """ + result = _join_ids("comp__m_json,\n comp__m_result\n") + assert result == "comp__m_json, comp__m_result" + assert "\n" not in result + + +def test_join_preserves_version_qualifiers(): + """Sphinx-Needs parses ``id[version==N]`` itself; stripping loses it.""" + assert ( + _join_ids("comp__m_json[version==1], comp__m_result") + == "comp__m_json[version==1], comp__m_result" + ) + + +def test_join_skips_empty_entries(): + assert _join_ids("comp__m_json, , ") == "comp__m_json" + assert _join_ids("") == "" + assert _join_ids(" , ") == "" + + +def test_join_is_type_agnostic(): + """The same helper serves :components: and :features:.""" + assert _join_ids("feat__one, feat__two") == "feat__one, feat__two" + + +# --------------------------------------------------------------------------- +# _report_title +# --------------------------------------------------------------------------- + + +def test_title_derived_from_module_slug(): + assert _report_title("baselibs") == "Baselibs Verification Report" + + +def test_title_title_cases_multiword_modules(): + assert _report_title("my_module") == "My Module Verification Report" + + +def test_id_is_never_derived(): + """The need id comes from the author's :id:, never from the module slug.""" + import inspect + + from src.extensions.score_module_verification_report import directive + + source = inspect.getsource(directive.ModuleVerificationReportDirective.run) + assert 'report_id=self.options["id"]' in source + assert "mod_vrep__" not in source + + +# --------------------------------------------------------------------------- +# Required options +# --------------------------------------------------------------------------- + + +def test_required_options_cover_every_mandatory_field_and_link(): + """One list drives both the option spec and the missing-option error.""" + assert _REQUIRED_OPTIONS == ( + "id", + "module-id", + "components", + "features", + "safety", + "security", + "status", + "verification-method", + ) + + +# --------------------------------------------------------------------------- +# Emitted RST +# --------------------------------------------------------------------------- + + +def _render(**overrides: str) -> str: + fields = dict( + title="Demo Verification Report", + report_id="mod_vrep__demo__report", + template_name=NEEDS_TEMPLATE_NAME, + version="1", + safety="QM", + security="YES", + status="valid", + verification_method="test_and_inspection", + module_id="mod__demo", + components="comp__demo_a, comp__demo_b", + features="feat__demo", + ) + fields.update(overrides) + return MOD_VER_REPORT_TEMPLATE.format(**fields) + + +def test_emitted_need_carries_every_field(): + out = _render() + for expected in ( + ".. mod_ver_report:: Demo Verification Report", + ":id: mod_vrep__demo__report", + ":post_template: mod_ver_report", + ":version: 1", + ":safety: QM", + ":security: YES", + ":status: valid", + ":verification_method: test_and_inspection", + ":belongs_to: mod__demo", + ":components: comp__demo_a, comp__demo_b", + ":features: feat__demo", + ): + assert expected in out + + +def test_emitted_need_is_only_the_need(): + """The body comes from the content template, not from here.""" + out = _render() + assert "needtable" not in out + assert "needpie" not in out + + +def test_every_option_stays_inside_the_directive_block(): + body = [line for line in _render().splitlines() if line.strip()] + assert body[0].startswith(".. mod_ver_report::") + assert all(line.startswith(" :") for line in body[1:]) diff --git a/src/extensions/score_module_verification_report/tests/test_needs_template.py b/src/extensions/score_module_verification_report/tests/test_needs_template.py new file mode 100644 index 000000000..2c140944a --- /dev/null +++ b/src/extensions/score_module_verification_report/tests/test_needs_template.py @@ -0,0 +1,204 @@ +# ******************************************************************************* +# Copyright (c) 2026 Contributors to the Eclipse Foundation +# +# See the NOTICE file(s) distributed with this work for additional +# information regarding copyright ownership. +# +# This program and the accompanying materials are made available under the +# terms of the Apache License Version 2.0 which is available at +# https://www.apache.org/licenses/LICENSE-2.0 +# +# SPDX-License-Identifier: Apache-2.0 +# ******************************************************************************* +"""Tests for the ``mod_ver_report`` Sphinx-Needs content template. + +The template is rendered by Sphinx-Needs from the need's own fields, using +MiniJinja. These tests render it directly with the same engine and the same +context shape, so a broken template fails here instead of in a docs build. +""" + +from __future__ import annotations + +from pathlib import Path + +import pytest +from sphinx_needs._jinja import render_template_string + +TEMPLATE = ( + Path(__file__).resolve().parents[3] / "needs_templates" / "mod_ver_report.need" +) + + +def _render(**overrides: object) -> str: + context: dict[str, object] = { + "id": "mod_vrep__baselibs__report", + "title": "Baselibs Verification Report", + "belongs_to": ["mod__baselibs"], + "components": ["comp__baselibs_json", "comp__baselibs_bit_manipulation"], + "features": ["feat__baselibs"], + } + context.update(overrides) + return render_template_string(TEMPLATE.read_text(), context, autoescape=False) + + +def test_template_file_is_shipped() -> None: + assert TEMPLATE.is_file(), TEMPLATE + + +# --------------------------------------------------------------------------- +# Feature sections +# --------------------------------------------------------------------------- + + +def test_feature_section_uses_the_features_link() -> None: + """The feature is read off the need, never guessed from the module id.""" + out = _render(features=["feat__something_else"]) + assert 'id == "feat__something_else"' in out + assert "feat__baselibs" not in out + + +def test_single_feature_keeps_the_plain_heading() -> None: + out = _render() + assert "Feature\n-------\n" in out + assert "Feature: " not in out + + +def test_one_section_per_feature_with_qualified_headings() -> None: + out = _render(features=["feat__demo_one", "feat__demo_two"]) + assert 'id == "feat__demo_one"' in out + assert 'id == "feat__demo_two"' in out + assert "Feature: Demo One\n" + "-" * len("Feature: Demo One") in out + assert "Feature: Demo Two\n" + "-" * len("Feature: Demo Two") in out + + +def test_feature_workproducts_match_on_the_feature_slug() -> None: + out = _render(features=["feat__baselibs"]) + assert '"baselibs" in id.replace("_", "").lower()' in out + # The feature table carries only the two feature-level work products. + feature_block = out[: out.index("Components\n----------")] + assert "wp__requirements_inspect" in feature_block + assert "wp__sw_arch_verification" in feature_block + assert "wp__sw_component_fmea" not in feature_block + + +# --------------------------------------------------------------------------- +# Component sections +# --------------------------------------------------------------------------- + + +def test_component_overview_lists_exactly_the_linked_components() -> None: + out = _render() + assert ( + ':filter: id in ["comp__baselibs_json", "comp__baselibs_bit_manipulation"]' + in out + ) + + +def test_component_title_and_anchor_derive_from_the_id() -> None: + out = _render() + assert ".. _comp-bit-manipulation:" in out + assert "Bit Manipulation\n" + "~" * len("Bit Manipulation") in out + assert ".. _comp-json:" in out + assert "Json\n~~~~" in out + + +def test_every_component_gets_the_full_set_of_workproducts() -> None: + out = _render(components=["comp__baselibs_json"]) + for wp in ( + "wp__requirements_inspect", + "wp__sw_arch_verification", + "wp__sw_implementation_inspection", + "wp__sw_component_dfa", + "wp__sw_component_fmea", + ): + assert f":need:`{wp}`" in out + + +def test_needpie_filters_guard_against_missing_verify_fields() -> None: + """Needs without ``*_verifies_back`` must not break the pie filters.""" + out = _render() + assert '"fully_verifies_back" in locals()' in out + assert '"partially_verifies_back" in locals()' in out + + +# --------------------------------------------------------------------------- +# Structure +# --------------------------------------------------------------------------- + + +@pytest.mark.parametrize( + "rubric", + [ + "Feature Requirements Statistics", + "Feature Architecture Statistics", + "Feature Inspection Statistics", + "Components", + "Component Overview", + "Component Requirements Statistics", + "Component Architecture Statistics", + "Requirements Traceability", + "Architectural Elements", + "Verification & Safety Analysis Documents", + ], +) +def test_all_report_sections_are_present(rubric: str) -> None: + """Every section must be a real heading — rubrics produce no TOC entries.""" + assert f"{rubric}\n" in _render() + + +def test_list_tables_have_a_consistent_number_of_fields_per_row() -> None: + """A short row silently corrupts a ``list-table``; catch it here.""" + lines = _render().splitlines() + checked = 0 + i = 0 + while i < len(lines): + if not lines[i].strip().startswith(".. list-table::"): + i += 1 + continue + indent = len(lines[i]) - len(lines[i].lstrip()) + j, per_row = i + 1, [] + while j < len(lines): + line = lines[j] + if line.strip() and (len(line) - len(line.lstrip())) <= indent: + break + stripped = line.strip() + # A cell may be empty ("- " with nothing after it), e.g. the + # branch-% column for a file with no branch data. + if stripped == "*" or stripped.startswith("* - ") or stripped == "* -": + per_row.append(1) + elif (stripped == "-" or stripped.startswith("- ")) and per_row: + per_row[-1] += 1 + j += 1 + assert len(set(per_row)) == 1, f"ragged list-table at line {i + 1}: {per_row}" + checked += 1 + i = j + # one feature work-product table + one per component + assert checked >= 3 + + +def test_no_unrendered_jinja_remains() -> None: + out = _render() + for marker in ("{{", "}}", "{%", "%}"): + assert marker not in out, marker + + +def test_no_rubrics_are_used() -> None: + """A rubric is not a section: it yields no TOC entry and no anchor.""" + assert ".. rubric::" not in _render() + + +def test_every_heading_underline_is_long_enough() -> None: + """A short underline makes docutils drop the section (and its TOC entry).""" + lines = _render().splitlines() + headings = 0 + for title, underline in zip(lines, lines[1:], strict=False): + if not underline or set(underline) - set("-~^") or not title.strip(): + continue + if len(set(underline)) != 1 or len(underline) < 3: + continue + assert len(underline) >= len(title.rstrip()), ( + f"underline too short for {title!r}" + ) + headings += 1 + # 3 feature + Components + Component Overview + 2x(title + 5 subsections) + assert headings >= 15, headings diff --git a/src/extensions/score_sphinx_bundle/BUILD b/src/extensions/score_sphinx_bundle/BUILD index 113803b97..0cb93f404 100644 --- a/src/extensions/score_sphinx_bundle/BUILD +++ b/src/extensions/score_sphinx_bundle/BUILD @@ -37,6 +37,7 @@ py_library( "@score_docs_as_code//src/extensions/score_mounts", "@score_docs_as_code//src/extensions/score_source_code_linker", "@score_docs_as_code//src/extensions/score_metrics", + "@score_docs_as_code//src/extensions/score_module_verification_report", "@score_docs_as_code//src/extensions/score_sync_toml", "@score_docs_as_code//src/helper_lib", ], diff --git a/src/extensions/score_sphinx_bundle/__init__.py b/src/extensions/score_sphinx_bundle/__init__.py index f3399e507..6c0546aae 100644 --- a/src/extensions/score_sphinx_bundle/__init__.py +++ b/src/extensions/score_sphinx_bundle/__init__.py @@ -41,6 +41,7 @@ "needs_config_writer", "score_sync_toml", "score_metrics", + "score_module_verification_report", "broken_link_fix", ] diff --git a/src/needs_templates/mod_ver_report.need b/src/needs_templates/mod_ver_report.need new file mode 100644 index 000000000..607d69921 --- /dev/null +++ b/src/needs_templates/mod_ver_report.need @@ -0,0 +1,324 @@ +{# + Content template for the ``mod_ver_report`` need type. + + Everything the report needs is read off the need itself, so the template is + self-sufficient: ``belongs_to`` names the module, ``components`` and + ``features`` name the architecture needs the report describes (all three are + mandatory links, see metamodel.yaml). Sphinx-Needs renders this template when + the need is created, which is *before* the needs graph exists — so the + template must never try to look other needs up. Component titles and slugs + are therefore derived from the ids by string manipulation, and everything + else is delegated to ``needtable`` / ``needpie``, which resolve at write + time. + + The template is applied as ``:post_template:``, not ``:template:``. A need's + *content* cannot open new sections ("Unexpected section title"), but + post-content is placed after the need at document level, where real headings + work — and real headings are what give the report its TOC entries and + per-component navigation. +#} +{% set module_id = belongs_to|first|default("") %} +{% set module_short = module_id|replace("mod__", "") %} +{% set component_prefix = "comp__" ~ module_short ~ "_" %} + +{% set component_workproducts = [ + ["wp__requirements_inspect", "Requirements Inspection"], + ["wp__sw_arch_verification", "Architecture Inspection"], + ["wp__sw_implementation_inspection", "Implementation Inspection"], + ["wp__sw_component_dfa", "DFA"], + ["wp__sw_component_fmea", "FMEA"], + ] %} +{% set feature_workproducts = [ + ["wp__requirements_inspect", "Requirements Inspection"], + ["wp__sw_arch_verification", "Architecture Inspection"], + ] %} + +{#- One work-product row: the need link, its kind, the realising document and + its status. Both cells are needtables over the same filter, differing only + in :columns:, so an empty match renders as an empty cell. -#} +{% macro workproduct_rows(slug_norm, workproducts) %} +{%- for wp in workproducts %} + * - :need:`{{ wp[0] }}` + - {{ wp[1] }} + - .. needtable:: + :filter: type == "document" and "{{ slug_norm }}" in id.replace("_", "").lower() and "{{ wp[0] }}" in realizes + :columns: id + :style: table + - .. needtable:: + :filter: type == "document" and "{{ slug_norm }}" in id.replace("_", "").lower() and "{{ wp[0] }}" in realizes + :columns: status + :style: table +{%- endfor %} +{% endmacro %} + +.. raw:: html + + + +{#- ===================================================================== -#} +{#- Feature sections — one per id in :features: -#} +{#- ===================================================================== -#} +{% for feature_id in features %} +{% set feature_slug = feature_id|replace("feat__", "") %} +{% set feature_slug_norm = feature_slug|replace("_", "")|lower %} + +{% set feature_heading = "Feature" if features|length == 1 else "Feature: " ~ feature_slug|replace("_", " ")|title %}{{ feature_heading }} +{{ "-" * (feature_heading|length) }} + +.. needtable:: + :filter: id == "{{ feature_id }}" + :columns: title as "Name";id as "Id";safety;security;status + :style: table + +Feature Requirements Statistics +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. grid:: 1 2 2 2 + :gutter: 3 + + .. grid-item:: + + .. needpie:: Feature Requirements Status + :labels: valid, invalid + :colors: #37a12d, #ca2828 + :legend: + + type == "feat_req" and "{{ feature_id }}" in satisfied_by and status == "valid" + type == "feat_req" and "{{ feature_id }}" in satisfied_by and status == "invalid" + + .. grid-item:: + + .. needpie:: Feature Requirements Test Coverage + :labels: fully covered, partially covered, not covered + :colors: #37a12d, #f0a500, #ca2828 + :legend: + + type == "feat_req" and "{{ feature_id }}" in satisfied_by and ("fully_verifies_back" in locals() and len(fully_verifies_back) > 0) + type == "feat_req" and "{{ feature_id }}" in satisfied_by and ("partially_verifies_back" in locals() and len(partially_verifies_back) > 0) and not ("fully_verifies_back" in locals() and len(fully_verifies_back) > 0) + type == "feat_req" and "{{ feature_id }}" in satisfied_by and not ("fully_verifies_back" in locals() and len(fully_verifies_back) > 0) and not ("partially_verifies_back" in locals() and len(partially_verifies_back) > 0) + +.. dropdown:: Show requirements table + :animate: fade-in + + .. needtable:: + :filter: type == "feat_req" and "{{ feature_id }}" in satisfied_by + :style: table + :columns: id;title;safety;status;fully_verifies_back;partially_verifies_back + :colwidths: 13,22,8,10,23,24 + :sort: id + +Feature Architecture Statistics +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. grid:: 1 2 2 2 + :gutter: 3 + + .. grid-item:: + + .. needpie:: Feature Architecture Elements Status + :labels: valid, invalid + :colors: #37a12d, #ca2828 + :legend: + + type in ["feat_arc_sta", "feat_arc_dyn"] and "{{ feature_id }}" in belongs_to and status == "valid" + type in ["feat_arc_sta", "feat_arc_dyn"] and "{{ feature_id }}" in belongs_to and status == "invalid" + + .. grid-item:: + + .. needpie:: Feature Architecture Elements Inspection Status + :labels: inspected, not inspected + :colors: #37a12d, #ca2828 + :legend: + + type in ["feat_arc_sta", "feat_arc_dyn"] and "{{ feature_id }}" in belongs_to and "inspected" in tags + type in ["feat_arc_sta", "feat_arc_dyn"] and "{{ feature_id }}" in belongs_to and "inspected" not in tags + +.. dropdown:: Show architectural elements table + :animate: fade-in + + .. needtable:: + :filter: type in ["feat_arc_sta", "feat_arc_dyn"] and "{{ feature_id }}" in belongs_to + :style: table + :columns: id;title;safety;status;tags + :colwidths: 25,30,10,15,20 + :sort: id + +Feature Inspection Statistics +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Presence of the feature-level inspection work products. + +.. dropdown:: Show work products table + :animate: fade-in + + .. list-table:: + :header-rows: 1 + :widths: 30 25 25 20 + :class: wp-doc-table + + * - Work Product + - Kind + - Realized by + - Status +{{- workproduct_rows(feature_slug_norm, feature_workproducts) }} +{% endfor %} + +{#- ===================================================================== -#} +{#- Components -#} +{#- ===================================================================== -#} + +Components +---------- + +Component Overview +~~~~~~~~~~~~~~~~~~ + +.. needtable:: + :filter: id in [{% for c in components %}"{{ c }}"{% if not loop.last %}, {% endif %}{% endfor %}] + :columns: id as "Component";safety;security;status + :style: table + :sort: id + +{% for component_id in components %} +{% set component_slug = component_id|replace(component_prefix, "") %} +{% set component_slug_norm = component_slug|replace("_", "")|lower %} +{% set component_title = component_slug|replace("_", " ")|title %} + +.. _comp-{{ component_slug|replace("_", "-")|lower }}: + +{{ component_title }} +{{ "~" * (component_title|length) }} + +.. raw:: html + +
+ +Component Requirements Statistics +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. grid:: 1 2 2 2 + :gutter: 3 + + .. grid-item:: + + .. needpie:: {{ component_title }} Requirements Status + :labels: valid, invalid + :colors: #37a12d, #ca2828 + :legend: + + type == "comp_req" and "{{ component_id }}" in satisfied_by and status == "valid" + type == "comp_req" and "{{ component_id }}" in satisfied_by and status == "invalid" + + .. grid-item:: + + .. needpie:: {{ component_title }} Requirements Test Coverage + :labels: fully covered, partially covered, not covered + :colors: #37a12d, #f0a500, #ca2828 + :legend: + + type == "comp_req" and "{{ component_id }}" in satisfied_by and ("fully_verifies_back" in locals() and len(fully_verifies_back) > 0) + type == "comp_req" and "{{ component_id }}" in satisfied_by and ("partially_verifies_back" in locals() and len(partially_verifies_back) > 0) and not ("fully_verifies_back" in locals() and len(fully_verifies_back) > 0) + type == "comp_req" and "{{ component_id }}" in satisfied_by and not ("fully_verifies_back" in locals() and len(fully_verifies_back) > 0) and not ("partially_verifies_back" in locals() and len(partially_verifies_back) > 0) + +Component Architecture Statistics +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. grid:: 1 2 2 2 + :gutter: 3 + + .. grid-item:: + + .. needpie:: {{ component_title }} Architecture Elements Status + :labels: valid, invalid + :colors: #37a12d, #ca2828 + :legend: + + type in ["comp_arc_sta", "comp_arc_dyn"] and "{{ component_id }}" in belongs_to and status == "valid" + type in ["comp_arc_sta", "comp_arc_dyn"] and "{{ component_id }}" in belongs_to and status == "invalid" + + .. grid-item:: + + .. needpie:: {{ component_title }} Architecture Elements Inspection Status + :labels: inspected, not inspected + :colors: #37a12d, #ca2828 + :legend: + + type in ["comp_arc_sta", "comp_arc_dyn"] and "{{ component_id }}" in belongs_to and "inspected" in tags + type in ["comp_arc_sta", "comp_arc_dyn"] and "{{ component_id }}" in belongs_to and "inspected" not in tags + +Requirements Traceability +^^^^^^^^^^^^^^^^^^^^^^^^^ + +The following table lists all requirements of this component together with their +verification status and the tests that (fully or partially) verify them: + +.. dropdown:: Show requirements table + :animate: fade-in + + .. needtable:: + :filter: type == "comp_req" and "{{ component_id }}" in satisfied_by + :style: table + :columns: id;title;safety;status;fully_verifies_back;partially_verifies_back + :colwidths: 13,22,8,10,23,24 + :sort: id + +Architectural Elements +^^^^^^^^^^^^^^^^^^^^^^ + +The following table lists the architectural elements of this component +together with their inspection status. Elements that have been formally +inspected carry the ``inspected`` tag; elements without that tag have not +yet been inspected. + +.. dropdown:: Show architectural elements table + :animate: fade-in + + .. needtable:: + :filter: type in ["comp_arc_sta", "comp_arc_dyn"] and "{{ component_id }}" in belongs_to + :style: table + :columns: id;title;safety;status;tags + :colwidths: 25,30,10,15,20 + :sort: id + +Verification & Safety Analysis Documents +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Presence of the standard verification and safety analysis work products for +this component. A dash (``—``) means the corresponding document is missing. + +.. dropdown:: Show work products table + :animate: fade-in + + .. list-table:: + :header-rows: 1 + :widths: 30 25 25 20 + :class: wp-doc-table + + * - Work Product + - Kind + - Realized by + - Status +{{- workproduct_rows(component_slug_norm, component_workproducts) }} +{% endfor %}