docs: move the Learn page onto the Astro site and generate its statistics

[parthrohit22]

Summary

Moves the Learn page onto the Astro site as a generated route and refreshes the repository-derived statistics in it and in the README.

Why this PR exists

The hosted Learn page was connected to my fork rather than the upstream repository, so changes merged into OWASP/openshield did not appear on that deployment. The page also carried hand-maintained counts that went stale every time a rule landed — which is exactly what the two earlier review rounds on this PR caught.

Changes

• docs/learn/index.html (the hand-maintained page) is removed; the Learn route now lives at website/src/pages/learn.astro and deploys with the rest of the site.
• .github/scripts/update_learn_page.py recomputes every count from scanner/rules/ and playbooks/cli/ and rewrites both the Learn route and the README in place, so the numbers cannot drift again. It fails loudly when a pattern matches zero times rather than silently leaving a file unchanged.
• The README feature-table patterns are anchored on the row label and the unit that follows the number, not on the full sentence. Pinning the whole prose made the script fail on an unrelated reword (the category list gaining "Kubernetes workloads", the playbook row becoming "review-gated remediation script") while the counts themselves went stale. Losing the row, or the Runs N Azure security rules / (N playbooks) shape, still fails loudly.
• website/scripts/verify-site.mjs covers the generated route.

Current statistics, regenerated against the merge target

Statistic	Value
Azure security rules	143
Matching remediation playbooks	143
CRITICAL / HIGH / MEDIUM / LOW	5 / 94 / 40 / 4 (sums to 143)
Largest category (chart 100% baseline)	Network, 35

The "4 Compliance frameworks" metric stays at 4 deliberately — CIS, NIST CSF, ISO 27001 and SOC 2, the general compliance mapper narrative. The two PQC-specific framework files are not part of it.

Validation

• python .github/scripts/update_learn_page.py — regenerates cleanly, and a second run produces no diff (idempotent, so CI commits only on a real change).
• pytest tests/test_update_learn_page.py — 4 passed, including two new tests covering that a reworded README row still updates and that a removed row still fails.
• ruff check / ruff format --check on the script and its tests — clean.
• npm run check in website/ — build, configure-cms and verify pass for 15 HTML pages.
• Documentation and tooling only: no scanner, API, infrastructure, credential, governance, or policy-document changes.

[ritiksah141]

The updates are internally consistent and the mechanics are clean (docs-only, no leftover stale counts, DCO signed, CI green), but the numbers were computed against a dev snapshot from before #277 and #320 merged. Since your fork sync, #277 added 10 new perimeter rules (az_net_018 through az_net_027) plus 10 playbooks, so every headline number is wrong at the merge target:

Stat	This PR	Actual at dev tip
Azure security rules	96	106
Remediation playbooks	96	106 per-rule (107 .sh files including the review playbook)
HIGH checks	58	67
MEDIUM checks	32	33
Network category bar	23 (untouched)	33

Compute 5 is the only value that still matches. Related issues that come with the same root cause:

1. The category bar widths are scaled to Network 23 as the max, so they need rescaling to 33 = 100% after the rebase.
2. The severity boxes do not sum to the headline: 58 + 32 + 4 = 94 vs a headline of 96, because there is no CRITICAL box. The repo has 2 CRITICAL rules. At dev tip the real split is HIGH 67, MEDIUM 33, LOW 4, CRITICAL 2.
3. The "4 Compliance frameworks" metric: compliance/frameworks/ now holds 6 JSON files (CIS, NIST CSF, ISO 27001, SOC 2, ENISA PQC, NCSC PQC). If 4 is deliberate (core mapper frameworks only), fine as-is; otherwise update to 6.

This is timing, not process: your sync landed before those merges. The fix is to rebase onto current dev and regenerate: rules and playbooks to 106, HIGH 67, MEDIUM 33, LOW 4, add a CRITICAL 2 severity box, Network bar to 33 with rescaled widths, and the README feature table plus mermaid diagram to 106. Happy to re-review once that lands.

[parthrohit22]

@ritiksah141 Rebased onto current dev and recomputed everything against the real tip: rules/playbooks 106, HIGH 67, MEDIUM 33, added the missing CRITICAL box (2), Network bar rescaled to 33.

On the "4 vs 6" compliance frameworks question — good catch flagging it rather than guessing. Turns out 4 is deliberate: .github/scripts/update_learn_page.py hardcodes COMPLIANCE_FRAMEWORK_COUNT = 4 # CIS, NIST, ISO 27001, SOC 2 on purpose, separately from the 2 PQC framework files. I actually got this wrong on my first pass (changed it to 6 since FRAMEWORK_FILE_MAP in the API has 6 entries) — the repo's own auto-update workflow caught it and reverted that one field back to 4 on push. Left as-is now, matches the existing convention.

CI's green on the current head.

[ritiksah141]

All good from my side. approving it

[TFT444]

Docs-only change, clean diff, DCO signed, CI green. The internal consistency of the PR is good: the category bars sum to 106, bar widths are correctly scaled to Network=33 as 100%, and the severity boxes (CRITICAL 2 + HIGH 67 + MEDIUM 33 + LOW 4) sum exactly to 106. The addition of the CRITICAL severity box and the rescaling of bar widths are correct.

However, the headline number is stale. The PR was generated from a fork snapshot that matches dev after #277 (az_net_018..027, 10 new network rules) but before #279 (az_cache_001, az_cosmos_001, az_cosmos_002, az_db_005..007, az_idn_016..025, az_stor_006..009 20 rules across four categories). Current dev tip has 126 rules, not 106. The delta breaks down as:

Category	This PR	dev tip	Delta
Network	33	35	+2
Identity	15	25	+10 (AZ-IDN-016..025)
Database	4	8	+4 (cosmos_001, db_005..007)
Storage	5	9	+4 (stor_006..009)
Total	106	126	+20

Severity at dev tip: CRITICAL 3, HIGH 82, MEDIUM 37, LOW 4 (total 126). The PR's severity split is correct for its 106-rule snapshot but wrong relative to the actual merge target.

What needs updating before merge:

1. README and docs/learn/index.html headline rule/playbook count: 106 → 126
2. Category bars: Network 33→35, Identity 15→25, Database 4→8, Storage 5→9, with bar widths rescaled to Network=35 as 100%
3. Severity boxes: CRITICAL 3, HIGH 82, MEDIUM 37, LOW 4
4. HIGH-severity checks metric card: 67 → 82

The "4 Compliance frameworks" metric is fine as a deliberate choice (CIS, NIST CSF, ISO 27001, SOC 2 the two PQC-specific frameworks are not part of the general compliance mapper narrative).

Happy to re-review once rebased to current dev tip and statistics regenerated.

[parthrohit22]

@TFT444 rebased onto current dev and re-ran .github/scripts/update_learn_page.py against the tip. It reports "already current; nothing to do" — the earlier regeneration commits (c6a8ea6, 5aaab48) already brought every count to the current dev state, and #278 (the only commit merged since) was rule corrections, not new rules, so no count moved.

Verified against the repo at dev tip:

• 126 Azure security rules / 126 per-rule playbooks (127 .sh files incl. review_enterprise_resilience.sh)
• Severity boxes: CRITICAL 3, HIGH 82, MEDIUM 37, LOW 4 (sum 126)
• Category bars: Network 35, Identity 25, Security Operations 10, Storage 9, Database 8, Supply Chain 8, Kubernetes 6, KeyVault 6, Serverless 5, Compute 5, Backup 4, PostQuantum 3, Data Link 2
• README feature table and mermaid diagram: 126

Net diff vs dev is README.md + docs/learn/index.html only. CI green. Re-requesting review.

[parthrohit22]

@TFT444 re-review please. All four items from your Sep 4 review are on 89c6a24, and I've re-verified them against the current dev tip (b7e9a40) rather than re-running the generator and trusting its "nothing to do".

The branch is 0 commits behind dev, so these are the merge-target numbers, not a snapshot's.

Ground truth, via the repo's own collect_rule_stats():

severity:   {'CRITICAL': 3, 'HIGH': 82, 'MEDIUM': 37, 'LOW': 4, 'INFO': 0}
categories: Network 35, Identity 25, Security Operations 10, Storage 9, Database 8,
            Supply Chain 8, Kubernetes 6, KeyVault 6, Serverless 5, Compute 5,
            Backup 4, PostQuantum 3, Data Link 2
count_rules: 126

What's rendered, item by item:

Your item	State on 89c6a24
1. Headline rule/playbook count 106 → 126	126 Azure security rules; metric cards 126 scan rules / 126 CLI remediation playbooks; README feature table and mermaid diagram both 126
2. Network 33→35, Identity 15→25, Database 4→8, Storage 5→9, rescaled to Network=35	All four match; Network is the 100% bar
3. Severity boxes CRITICAL 3, HIGH 82, MEDIUM 37, LOW 4	Match
4. HIGH-severity metric card 67 → 82	<strong>82</strong><span>High-severity checks</span>

Internal consistency, checked rather than assumed — I parsed the rendered bars back out of docs/learn/index.html and recomputed the widths:

categories: 13   sum: 126   width mismatches: []

Severity boxes sum to 3 + 82 + 37 + 4 = 126, and the category bars sum to 126 independently. Every bar width equals round(count / 35 * 100).

The 4 Compliance frameworks metric is unchanged, per your note that it's a deliberate choice — update_learn_page.py hardcodes COMPLIANCE_FRAMEWORK_COUNT = 4 separately from the two PQC framework files.

Net diff against dev is still README.md + docs/learn/index.html only. All 21 checks green, DCO signed.

[parthrohit22]

@TFT444 @ritiksah141 statistics regenerated against the current dev tip. The branch is already level with upstream/dev (0 behind), so no rebase was needed — the numbers were simply stale.

Rather than hand-editing them again, I ran .github/scripts/update_learn_page.py, which recomputes everything from scanner/rules/ and playbooks/cli/:

Statistic	Was	Now
Azure security rules	126	143
Matching playbooks	126	143
CRITICAL / HIGH / MEDIUM / LOW	3 / 82 / 37 / 4	5 / 94 / 40 / 4 (sums to 143)
Largest category (100% baseline)	Network 35	Network 35

The category chart rescaled and re-sorted itself — Kubernetes moved to 21 and Compute to 7, so both jumped several rows. README feature table and both Mermaid nodes updated to 143.

Worth flagging, since it is the root cause of this PR going stale twice: the script failed before it could update anything. The two README feature rows had been reworded upstream (the category list gained "Kubernetes workloads", the playbook row became "review-gated remediation script"), and the patterns pinned the entire sentence, so both matched zero times and the script exited 1 by design. That is the failure mode where a count silently goes stale while the automation looks healthy. Both patterns are now anchored on the row label and the unit that follows the number instead of the surrounding prose — losing the row itself, or the Runs N Azure security rules / (N playbooks) shape, still fails loudly. Two tests cover both directions.

Verification: the script is idempotent (second run produces no diff, so CI commits only on a real change); pytest tests/test_update_learn_page.py 4 passed; ruff check and ruff format --check clean; npm run check in website/ passes for 15 HTML pages.

On the "4 Compliance frameworks" metric — keeping it at 4 as you both agreed: CIS, NIST CSF, ISO 27001, SOC 2, with the two PQC framework files deliberately outside that narrative.

The PR description was still describing the old 96-rule docs-only change, so it has been rewritten to match what this diff actually is. Re-requesting review.