feat: allow canonical regulatory approvals

docs/configuration/table.md

@@ -158,7 +158,7 @@ Defines subject-predicate-object relationships.
 | `predicate` | String | No | Biolink predicate. Defaults to `"related_to"`. |
 | `object` | NodeEncoding | Yes | Object entity configuration |
 | `qualifiers` | List[Qualifier] | No | Edge qualifiers (context) |
-| `category_override` | Map[Categories, EdgeCategories] | No | Pin the association class per resolved object category (bare names, no `biolink:` prefix), replacing the derived (subject, object) pair lookup for those rows. Rows whose object category is absent from the map derive as before. Pinned classes are still reconciled against the section predicate (a class whose `predicate` slot rejects it is walked up the association hierarchy, with a warning at config time). Use it when one section mixes object categories the pair lookup merges — e.g. `Disease` and `PhenotypicFeature` both derive to `ChemicalEntityToDiseaseOrPhenotypicFeatureAssociation`, but only `EntityToDiseaseAssociation` / `EntityToPhenotypicFeatureAssociation` declare slots like `FDA_regulatory_approvals` and `number_of_cases`. Rows pinned to either of those classes may additionally carry `disease_context_qualifier` under a deliberate Tablassert policy grant, ahead of the pinned Biolink model. |
+| `category_override` | Map[Categories, EdgeCategories] | No | Pin the association class per resolved object category (bare names, no `biolink:` prefix), replacing the derived (subject, object) pair lookup for those rows. Rows whose object category is absent from the map derive as before. Pinned classes are still reconciled against the section predicate (a class whose `predicate` slot rejects it is walked up the association hierarchy, with a warning at config time). Use it when one section mixes object categories the pair lookup merges — e.g. `Disease` and `PhenotypicFeature` both derive to `ChemicalEntityToDiseaseOrPhenotypicFeatureAssociation`, but only `EntityToDiseaseAssociation` / `EntityToPhenotypicFeatureAssociation` receive the class-scoped `regulatory_approvals` grant and declare slots like `number_of_cases`. Rows pinned to either of those classes may additionally carry `disease_context_qualifier` under a deliberate Tablassert policy grant. |
 
 **Example:**
 ```yaml
@@ -245,6 +245,8 @@ annotations:
 
 The separator is a property of the data, not of the slot. Inspect the table's cells and set `split_by` to the separator the cells actually use: `","` for comma-joined ids like `"EFO:0001,EFO:0002"` above, `";"` for `"EFO:0001;EFO:0002"`, `"|"` only if the cells happen to be pipe-joined.
 
+For the canonical `regulatory_approvals` annotation, a pipe-delimited source column uses `split_by: "|"` and emits a real per-row array such as `["011111", "022222"]`; combine it with a class-scoped `category_override` when only the intended association classes receive that grant.
+
 `split_by` is the one multivalued encoding: every row's cell becomes its own JSON array, so an array that differs per row, the shape a literal can never express, is declared directly. Values are trimmed and blanks dropped; a null cell stays null.
 
 Single-value cells need no `split_by` at all: `prune_to_class` wraps a scalar bound for a uniformly multivalued slot into a one-element list, so a lone `EFO:0001` cell already emits as `has_evidence: ["EFO:0001"]`. Reach for it when the cells actually join multiple values. Leave such a column without `split_by` and the joined cell stays a scalar: the same wrapping yields a one-element list holding the whole string (`has_evidence: ["EFO:0001;EFO:0002"]`), structurally valid Biolink that hands consumers one unusable blob instead of two ids.
@@ -536,7 +538,7 @@ Optional edge attributes (statistical metadata, notes, etc.).
 | `split_by` | String | No | Separator splitting each cell of a `method: column` encoding into a real JSON array. See [`split_by`](#split_by). |
 | (inherits Encoding) | | | All Encoding fields available (method, encoding, regex, etc.) |
 
-Multivalued Biolink slots such as `has_evidence` or `FDA_regulatory_approvals`, whose consumers iterate the value, must emit a real JSON array rather than a scalar. [`split_by`](#split_by) is the multivalued encoding: point it at a column whose cells join multiple values, set it to the separator the cells actually use, and each cell's delimited text splits into a per-row array. A single-value column needs no `split_by`; the scalar is wrapped into a one-element array.
+Multivalued Biolink slots such as `has_evidence` or `regulatory_approvals`, whose consumers iterate the value, must emit a real JSON array rather than a scalar. [`split_by`](#split_by) is the multivalued encoding: point it at a column whose cells join multiple values, set it to the separator the cells actually use, and each cell's delimited text splits into a per-row array. A single-value column needs no `split_by`; the scalar is wrapped into a one-element array.
 
 **Example:**
 ```yaml
@@ -557,7 +559,7 @@ annotations:
 
 Annotation names fall into three groups at build time:
 
-- **Allowed edge fields:** names on the edge allow-list: [Biolink Association](https://biolink.github.io/biolink-model/) slots, qualifier slots, and curated KGX/Tablassert edge fields (e.g. `p_value`, `adjusted_p_value`, `knowledge_level`, `supporting_text`, `publications`, `effect_size`, `effect_type`, `statistical_significance_qualifier`, qualifier slots like `severity_qualifier` / `disease_context_qualifier`) are written to edges verbatim. `effect_size` and `effect_type` became real Association slots in biolink-model 4.4.4 (PR #1774) and emit as real JSON numbers and enum tokens. Annotation names are matched case-insensitively against the allow-list and emitted under the canonical slot spelling: Biolink's mixed-case `FDA_regulatory_approvals` slot (declared on `EntityToDiseaseAssociation` / `EntityToPhenotypicFeatureAssociation`) can be authored in any casing and reaches the final edge verbatim as `FDA_regulatory_approvals`; it is a multivalued slot, so declare [`split_by`](#split_by) for pipe-joined cells like `011111|022222`.
+- **Allowed edge fields:** names on the edge allow-list: [Biolink Association](https://biolink.github.io/biolink-model/) slots, qualifier slots, and curated KGX/Tablassert edge fields (e.g. `p_value`, `adjusted_p_value`, `knowledge_level`, `supporting_text`, `publications`, `effect_size`, `effect_type`, `statistical_significance_qualifier`, qualifier slots like `severity_qualifier` / `disease_context_qualifier`) are written to edges verbatim. `effect_size` and `effect_type` became real Association slots in biolink-model 4.4.4 (PR #1774) and emit as real JSON numbers and enum tokens. Annotation names are matched case-insensitively against the allow-list and emitted under the canonical slot spelling. The class-scoped `regulatory_approvals` grant is available on `EntityToDiseaseAssociation` and `EntityToPhenotypicFeatureAssociation`; it is a multivalued slot, so declare [`split_by`](#split_by) for pipe-joined cells like `011111|022222`.
 - **Study metadata:** `study_size`, `study_cohort`, `study_context`, `study_date_range`, `study_method_description`, and `study_method_types` are current Biolink `Study` node properties ([biolink-model PR #1770](https://github.com/biolink/biolink-model/pull/1770)). They describe the study itself, not an association, so their values are carried on the edge's **inlined supporting Study** (`has_supporting_studies` to `Study`, the COHD/ICEES pattern) rather than emitted as edge fields. Deprecated `supporting_study_*` spellings and `sample_size` are accepted as aliases and renamed onto canonical `study_*` names by coercion. They never appear in final JSON. Declaring any of them is legal and emits a `BiolinkRelocationWarning` naming where the value went.
 - **Unsatisfiable slots:** names the Biolink LinkML schema declares but attaches to **no** Pydantic class, derived from the installed `biolink-model`. A record carrying one could never validate, so its value is preserved in the inlined supporting study's `StudyResult.description`. A slot leaves the set automatically once a release attaches it.
 

src/tablassert/biolink.py

@@ -257,9 +257,9 @@ def _association_model_fields() -> set[str]:
 
     Unions ``model_fields`` across the entire association family rather than only
     walking ``Association.__mro__``. Walking the base MRO alone silently excludes
-    subclass-only evidence slots -- ``clinical_approval_status``,
-    ``number_of_cases``, ``FDA_regulatory_approvals`` -- which then get demoted
-    into ``supporting_text`` by :func:`lib.fold_unknown_to_supporting_text`.
+    subclass-only evidence slots -- ``clinical_approval_status`` and
+    ``number_of_cases`` -- which then get demoted into ``supporting_text`` by
+    :func:`lib.fold_unknown_to_supporting_text`.
     """
     fields: set[str] = set()
     for klass in _association_classes():
@@ -529,24 +529,23 @@ class EffectTypes(str, Enum):
 
 
 CLASS_FIELD_OVERRIDES: dict[str, frozenset[str]] = {
-    "EntityToDiseaseAssociation": frozenset({"disease_context_qualifier"}),
-    "EntityToPhenotypicFeatureAssociation": frozenset({"disease_context_qualifier"}),
+    "EntityToDiseaseAssociation": frozenset({"disease_context_qualifier", "regulatory_approvals"}),
+    "EntityToPhenotypicFeatureAssociation": frozenset({"disease_context_qualifier", "regulatory_approvals"}),
 }
 """Per-class grants of edge fields the resolved association class does not declare.
 
 Keys are bare association class names (``association_class(cat).__name__``), values
 the slots ``lib.prune_to_class`` keeps on rows resolved to that class even though the
 installed model attaches them elsewhere.
 
-The motivating case is a DAKP contraindication edge: ``FDA_regulatory_approvals`` is
-declared only on the ``EntityToDisease`` / ``EntityToPhenotypicFeature`` classes the
-edge is pinned to, while ``disease_context_qualifier`` is declared only on the
+The motivating case is a DAKP contraindication edge: ``regulatory_approvals`` is a
+canonical slot not yet attached by the installed model, while
+``disease_context_qualifier`` is declared only on the
 ``ChemicalEntityToDiseaseOrPhenotypicFeatureAssociation`` lineage -- so one edge can
-natively carry one slot or the other, never both. Tablassert deliberately emits the
-qualifier on the pinned classes ahead of the pinned model (pending an upstream Biolink
-widening), exactly as :data:`KNOWN_PENDING_EDGE_FIELDS` emits KGX carryovers ahead of
-it. ``_validation_record`` strips granted fields before record validation so the
-deliberate gap is not reported as ``extra_forbidden``.
+natively carry both only through these explicit class-scoped grants. Tablassert
+deliberately emits the granted fields on the pinned classes ahead of the pinned model
+(pending an upstream Biolink widening). ``_validation_record`` strips granted fields
+before record validation so the deliberate gap is not reported as ``extra_forbidden``.
 
 A tripwire test asserts every granted field is still absent from its class: the moment
 a biolink-model release attaches the slot, the suite fails and the stale grant is
@@ -582,7 +581,12 @@ class EffectTypes(str, Enum):
 
 
 ALLOWED_EDGE_FIELDS: frozenset[str] = (
-    (frozenset(_association_model_fields()) | {q.value for q in Qualifiers} | TABLASERT_EDGE_EXTRAS)
+    (
+        frozenset(_association_model_fields())
+        | {q.value for q in Qualifiers}
+        | TABLASERT_EDGE_EXTRAS
+        | frozenset().union(*CLASS_FIELD_OVERRIDES.values())
+    )
     - UNSATISFIABLE_EDGE_FIELDS
     - DISABLED_EDGE_FIELDS
 )
@@ -591,9 +595,9 @@ class EffectTypes(str, Enum):
 Any column on an edge frame that is not in this set is folded into the
 ``supporting_text`` ``list[str]`` field by ``lib.fold_unknown_to_supporting_text()``
 as a ``"column: value"`` string. Composed of the fields declared by *any* Biolink
-association class, the derived qualifier slot names, and the curated
-``TABLASERT_EDGE_EXTRAS`` -- less the slots that no Pydantic class can hold and the
-fields disabled by Tablassert policy.
+association class, the derived qualifier slot names, the curated
+``TABLASERT_EDGE_EXTRAS``, and explicit :data:`CLASS_FIELD_OVERRIDES` grants -- less the
+slots that no Pydantic class can hold and the fields disabled by Tablassert policy.
 
 Note this is a per-*family* allow-list: a field being permitted here does not mean the
 specific association class chosen for a given edge accepts it. Per-record pruning

src/tablassert/models.py

@@ -710,10 +710,9 @@ class Annotation(Encoding):
     def clean_annotation(cls, annotation: str) -> str:
         cleaned: str = annotation.strip()
         lowered: str = cleaned.lower()
-        # Allow-listed slots may carry uppercase (Biolink's ``FDA_regulatory_approvals``):
-        # any casing the author declares canonicalizes onto the allow-listed spelling
-        # verbatim, so the emitted edge field preserves its case exactly instead of being
-        # lowercased into an unknown name.
+        # Allow-listed slots may carry uppercase: any casing the author declares
+        # canonicalizes onto the allow-listed spelling verbatim, so the emitted edge
+        # field preserves its case instead of being lowercased into an unknown name.
         return next((field for field in ALLOWED_EDGE_FIELDS if field.lower() == lowered), lowered)
 
     @model_validator(mode="after")

tests/test_biolink.py

@@ -337,16 +337,16 @@ def test_allowed_edge_fields_excludes_approval_ids() -> None:
     assert "approval_ids" not in ALLOWED_EDGE_FIELDS
 
 
-def test_allowed_edge_fields_keeps_fda_regulatory_approvals_strict() -> None:
-    """``FDA_regulatory_approvals`` is allowed AND strictly valid -- no pending exemption.
+def test_allowed_edge_fields_includes_class_override_grants_without_global_extras() -> None:
+    """Canonical class grants reach the family allow-list without becoming global extras.
 
-    Unlike the retired ``approval_ids`` carve-out, this is a real Biolink slot (declared by
-    ``EntityToDiseaseAssociation`` / ``EntityToPhenotypicFeatureAssociation``), so it never
-    rides ``KNOWN_PENDING_EDGE_FIELDS``: an edge carrying it validates outright.
+    This protects the compatibility boundary: ``regulatory_approvals`` is emitted as a
+    real edge column for the two pinned classes, but is neither an arbitrary global
+    allow-list entry nor a ``TABLASERT_EDGE_EXTRAS`` carryover.
     """
-    assert "FDA_regulatory_approvals" not in TABLASERT_EDGE_EXTRAS
-    assert "FDA_regulatory_approvals" in ALLOWED_EDGE_FIELDS
-    assert "FDA_regulatory_approvals" not in KNOWN_PENDING_EDGE_FIELDS
+    assert "regulatory_approvals" in ALLOWED_EDGE_FIELDS
+    assert "regulatory_approvals" not in TABLASERT_EDGE_EXTRAS
+    assert "regulatory_approvals" not in KNOWN_PENDING_EDGE_FIELDS
 
 
 def test_supporting_case_ids_is_an_allow_listed_pending_extra() -> None:
@@ -369,7 +369,16 @@ def test_allowed_edge_fields_includes_subclass_only_slots() -> None:
     Deriving the allow-list from the base ``Association`` MRO alone silently demotes
     evidence slots such as ``clinical_approval_status`` into ``supporting_text``.
     """
-    for slot in ("clinical_approval_status", "number_of_cases", "FDA_regulatory_approvals"):
+    subclass_only: set[str] = set()
+    for cls in vars(bm).values():
+        if inspect.isclass(cls) and issubclass(cls, bm.Association):
+            subclass_only |= set(cls.model_fields) - set(bm.Association.model_fields)
+    expected_subclass_only = {"clinical_approval_status", "number_of_cases"}
+    assert expected_subclass_only <= subclass_only
+    assert expected_subclass_only <= set(ALLOWED_EDGE_FIELDS)
+    allowed_subclass_only: set[str] = subclass_only & set(ALLOWED_EDGE_FIELDS)
+    assert allowed_subclass_only
+    for slot in allowed_subclass_only:
         assert slot not in bm.Association.model_fields, slot
         assert slot in ALLOWED_EDGE_FIELDS, slot
 
@@ -386,8 +395,14 @@ def test_class_field_overrides_track_the_installed_model() -> None:
     Tripwire: the moment a biolink-model release attaches a granted slot to the class,
     this fails and the stale grant is removed from ``CLASS_FIELD_OVERRIDES`` (same
     philosophy as the ``UNSATISFIABLE_EDGE_FIELDS`` derivation guard). A field the
-    family allow-list would strip anyway must never be granted.
+    family allow-list would strip anyway must never be granted. The canonical
+    ``regulatory_approvals`` grant is intentionally present on exactly the two
+    association classes that need it while the installed model catches up.
     """
+    assert {
+        "EntityToDiseaseAssociation": frozenset({"disease_context_qualifier", "regulatory_approvals"}),
+        "EntityToPhenotypicFeatureAssociation": frozenset({"disease_context_qualifier", "regulatory_approvals"}),
+    } == CLASS_FIELD_OVERRIDES
     for class_name, fields in CLASS_FIELD_OVERRIDES.items():
         cls: type[Any] = association_class(f"biolink:{class_name}")
         assert issubclass(cls, bm.Association), class_name
@@ -397,16 +412,28 @@ def test_class_field_overrides_track_the_installed_model() -> None:
 
 
 def test_validate_record_tolerates_class_field_override_grants() -> None:
-    """A granted field on its granted class is not reported; on any other class it is.
+    """Canonical grants validate on both targets but remain forbidden on an unrelated class.
 
     The grant is a deliberate, class-scoped step ahead of the pinned model, so its
-    ``extra_forbidden`` must not surface in validation -- while the same slot on an
-    ungranted class stays a real defect.
+    ``extra_forbidden`` must not surface on either intended target -- while the same
+    field on an ungranted class stays a real defect through the installed-model boundary.
     """
-    record: dict[str, Any] = {"category": ["biolink:EntityToDiseaseAssociation"], "disease_context_qualifier": "MONDO:0005148"}
-    assert "disease_context_qualifier: extra_forbidden" not in validate_record(record, edge=True)
-    control: dict[str, Any] = {**record, "category": ["biolink:GeneToDiseaseAssociation"]}
-    assert "disease_context_qualifier: extra_forbidden" in validate_record(control, edge=True)
+    base: dict[str, Any] = {
+        "id": "e1",
+        "subject": "CHEBI:1",
+        "predicate": "biolink:associated_with",
+        "object": "MONDO:0005148",
+        "knowledge_level": "statistical_association",
+        "agent_type": "data_analysis_pipeline",
+        "regulatory_approvals": ["FDA:1"],
+    }
+    for category in ("EntityToDiseaseAssociation", "EntityToPhenotypicFeatureAssociation"):
+        record: dict[str, Any] = {**base, "category": [f"biolink:{category}"]}
+        assert validate_record(record, edge=True) == []
+    control: dict[str, Any] = {**base, "category": ["biolink:GeneToDiseaseAssociation"]}
+    assert "regulatory_approvals: extra_forbidden" in validate_record(control, edge=True)
+    unknown: dict[str, Any] = {**base, "category": ["biolink:EntityToDiseaseAssociation"], "not_a_biolink_field": "x"}
+    assert "not_a_biolink_field: extra_forbidden" in validate_record(unknown, edge=True)
 
 
 def test_allowed_edge_fields_excludes_unattached_qualifiers() -> None:
@@ -484,8 +511,7 @@ def test_is_pending_problem_only_exempts_extra_forbidden_pending_fields() -> Non
     assert not is_pending_problem("approval_ids: extra_forbidden")
     # Same field, a REAL failure -> not exempt either way.
     assert not is_pending_problem("approval_ids: missing")
-    # ``FDA_regulatory_approvals`` is a real model slot -> extra_forbidden is a real failure.
-    assert not is_pending_problem("FDA_regulatory_approvals: extra_forbidden")
+    # A real installed-model slot -> extra_forbidden is a real failure.
     # ``effect_size`` is a real Association slot since biolink-model 4.4.4 -> never exempt.
     assert not is_pending_problem("effect_size: extra_forbidden")
     # A genuinely malformed value on a real slot -> never exempt.