diff --git a/flatpak/README.md b/flatpak/README.md index e3f5505..d16587e 100644 --- a/flatpak/README.md +++ b/flatpak/README.md @@ -4,14 +4,49 @@ This directory contains files for submitting PDFApps to [Flathub](https://flathu ## ⚠️ Status: dormant — needs regeneration before next submission -The files in this directory are out of date relative to the root `requirements.txt`: +`requirements-pinned.txt` and `python-modules.yml` are out of date relative to +the root `requirements.txt`. The canonical list of what is absent lives in +`FLATPAK_OMITTED` in `tests/test_flatpak_dependency_pins.py`; keep this section +and that set in step. The eight entries split into four kinds: -- `requirements-pinned.txt` is missing **cryptography, python-pptx, openpyxl, beautifulsoup4, ebooklib**. -- `python-modules.yml` does not contain wheel URLs / hashes for those packages. +**Deliberate** (never needed in the Flatpak): -Effect: a Flatpak built today would have **PDF→PPTX, PDF→XLSX, import HTML and import EPUB broken**. Other conversions and the core viewer/editor still work. +- `pyinstaller`: only builds the Windows executable. -The divergence happened because new Python deps were added to `requirements.txt` (notably in v1.12.0) without regenerating the Flatpak pin file. Dependabot keeps the shared packages in sync (e.g. it bumps `pypdf` here too) but it does not add new packages. +**Build blocker** (the Flatpak build fails outright, before any feature runs): + +- `lxml`: a hard, unconditional dependency of `python-docx`, which *is* pinned + (`lxml>=3.1.0` in its metadata). The manifest installs with `--no-index + --find-links` and **without** `--no-deps`, so pip has to resolve `lxml` from + the local wheel directory and there is no wheel there. + +**Floor-only pins** (no Flatpak package requires them today): + +- `urllib3`, `idna`: present at the root only to raise the floor on transitive + advisories. They must come back here the moment a package that pulls + `requests` is added to the Flatpak pin set. + +`lxml` is not the only unmet hard dependency of the current pin set: `cffi` +(from `cryptography`), `typing_extensions` (from `python-docx`), `packaging` +(from `pytesseract`), `qtpy` (from `qtawesome`) and the `PySide6_Essentials` / +`PySide6_Addons` meta-packages are missing too. Those are tracked separately; +regenerating with `req2flatpak` (below) is what fixes all of them at once. + +**Feature gaps** (would build, but tools are dead): + +- `python-pptx`, `openpyxl`, `beautifulsoup4`, `ebooklib`: a Flatpak built + from these pins would have **PDF→PPTX, PDF→XLSX, import HTML and import + EPUB broken**. Other conversions and the core viewer/editor still work. + +(`cryptography` was added when `pypdf` was bumped to 6.16.2; it is an optional +`pypdf` extra that AES-encrypted PDFs require, and it is pinned.) + +The divergence happened because new Python deps were added to `requirements.txt` +(notably in v1.12.0) without regenerating the Flatpak pin file. **Dependabot does +not cover this directory**: `.github/dependabot.yml` declares the `pip` ecosystem +only for `directory: "/"`, so it neither bumps nor adds anything under +`flatpak/`. The tests in `tests/test_flatpak_dependency_pins.py` are the net that +now catches that drift. **Before submitting to Flathub**, regenerate everything from the root `requirements.txt`: @@ -50,7 +85,7 @@ A previous AI-assisted PR (#8301) was closed by a maintainer. Repeat offenses ca ## Files - `io.github.nelsonduarte.PDFApps.yml` — Flatpak manifest (KDE Platform 6.8) -- `python-modules.yml` — Python deps with SHA256 (generated by `req2flatpak`) +- `python-modules.yml` — Python deps with SHA256 (generated by `req2flatpak`; regenerate it, do not hand-edit, because a manual edit silently desynchronises the wheel list from `requirements-pinned.txt`) - `io.github.nelsonduarte.PDFApps.metainfo.xml` — AppStream metadata - `io.github.nelsonduarte.PDFApps.desktop` — Desktop entry - `flathub.json` — Restrict to `x86_64` (PySide6 wheels limitation) diff --git a/flatpak/python-modules.yml b/flatpak/python-modules.yml index e44de13..3392210 100644 --- a/flatpak/python-modules.yml +++ b/flatpak/python-modules.yml @@ -3,14 +3,19 @@ name: python3-package-installation buildsystem: simple build-commands: - pip3 install --verbose --exists-action=i --no-index --find-links="file://${PWD}" - --prefix=${FLATPAK_DEST} --no-build-isolation PySide6 shiboken6 pypdf qtawesome - pillow pymupdf pytesseract python-docx + --prefix=${FLATPAK_DEST} --no-build-isolation PySide6 shiboken6 pypdf cryptography + qtawesome pillow pymupdf pytesseract python-docx sources: - type: file url: https://files.pythonhosted.org/packages/d8/de/af89d71410c83b10654d86ff9aff2a4f87c30163658f1cc145242e222526/pyside6-6.11.1-cp310-abi3-manylinux_2_34_x86_64.whl sha256: b1fc521ba2bb5109425ab8add06bddbdd524abcad06cfa012cc39a22a189feb2 only-arches: - x86_64 +- type: file + url: https://files.pythonhosted.org/packages/85/66/6ccca4722987ddedaa7fc9c3f4708af7431f5535666c174350830888c6b7/cryptography-50.0.1-cp311-abi3-manylinux_2_34_x86_64.whl + sha256: 51afcfceb15597cf2635068e4ac9a56b2abde622edde17f37d85fd7b5306497a + only-arches: + - x86_64 - type: file url: https://files.pythonhosted.org/packages/84/21/a35af28dcc61f37ed850a2d64c65c701321dfbf25085e469d5559360cbbf/pillow-12.3.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl sha256: 78cb2c6865a35ab8ff8b75fd122f6033b92a62c82801110e48ddd6c936a45d91 @@ -22,8 +27,8 @@ sources: only-arches: - x86_64 - type: file - url: https://files.pythonhosted.org/packages/49/e6/136aa8993a2ae7214e0b0ef2edaa0d2e08d1d4e4982635b08a835ff31ec8/pypdf-6.14.2-py3-none-any.whl - sha256: 3f07891af76dc002657e04993ab9b4de81de29f9013b9761d0b7968bff12e946 + url: https://files.pythonhosted.org/packages/13/f1/a2da3b55acd4ab737bf728c97edaaed5ec1d3c1236acb639dcdfa97e42c7/pypdf-6.16.2-py3-none-any.whl + sha256: c8b09a59399062fb45a1b8156c18a787a10a3dae03ac9674397a226712c94604 - type: file url: https://files.pythonhosted.org/packages/7a/33/8312d7ce74670c9d39a532b2c246a853861120486be9443eebf048043637/pytesseract-0.3.13-py3-none-any.whl sha256: 7a99c6c2ac598360693d83a416e36e0b33a67638bb9d77fdcac094a3589d4b34 diff --git a/flatpak/requirements-pinned.txt b/flatpak/requirements-pinned.txt index b7670dc..a07bb3e 100644 --- a/flatpak/requirements-pinned.txt +++ b/flatpak/requirements-pinned.txt @@ -1,6 +1,7 @@ PySide6==6.11.1 shiboken6==6.11.1 -pypdf==6.14.2 +pypdf==6.16.2 +cryptography==50.0.1 qtawesome==1.4.2 pillow==12.3.0 pymupdf==1.28.0 diff --git a/requirements.txt b/requirements.txt index 256c5d6..43c85ea 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,8 +1,19 @@ PySide6>=6.11.1 -pypdf>=6.14.2 +# PYSEC-2026-3655 / PYSEC-2026-3656 (fixed in 6.15.0), CVE-2026-82398 +# (6.15.0), CVE-2026-84309 (6.16.0) and CVE-2026-84310 / CVE-2026-84311 +# (6.16.1). 6.16.1 is the first release clear of all six, so the floor +# is the current 6.16.2 rather than the 6.15.0 the earliest advisory +# alone would suggest. +pypdf>=6.16.2 # GHSA-537c-gmf6-5ccf: signature verification bypass in legacy OpenSSL -# providers fixed in 48.0.1. -cryptography>=49.0.0 +# providers fixed in 48.0.1, and PYSEC-2026-3552 / CVE-2026-69247 +# (Bleichenbacher oracle in PKCS#7 EnvelopedData decryption, introduced +# in 44.0.0) fixed in 50.0.0. 49.0.0 is the only 49.x release and it is +# vulnerable, so the floor must stay at 50 or above: do not lower it. +# Also the AES backend pypdf resolves at import time (pypdf declares it +# only as the optional `crypto` extra), so without it AES-encrypted PDFs +# raise DependencyError. +cryptography>=50.0.1 qtawesome>=1.4.2 pillow>=12.3.0 pymupdf>=1.28.0 diff --git a/tests/test_flatpak_dependency_pins.py b/tests/test_flatpak_dependency_pins.py new file mode 100644 index 0000000..debb276 --- /dev/null +++ b/tests/test_flatpak_dependency_pins.py @@ -0,0 +1,233 @@ +"""Consistency tests for the three files that pin Python dependencies. + +The Flatpak build is driven by ``flatpak/python-modules.yml``, which has +**zero references anywhere in .github/**: no workflow audits it and no +workflow regenerates it. ``security-deps.yml`` audits +``flatpak/requirements-pinned.txt`` instead, so the wheel list can drift +away from the pin file and CI stays green while the shipped Flatpak +installs a different (possibly vulnerable) version. That is exactly how +``pypdf`` stayed at 6.14.2 in the Flatpak with six known CVEs while the +root ``requirements.txt`` had already moved on. + +These tests close that loop without needing network access or pip-audit: +they assert the three files agree with each other. +""" + +from __future__ import annotations + +import re +from pathlib import Path + +import pytest + +try: + import yaml +except ImportError: # pragma: no cover - PyYAML is a dev dependency + yaml = None + +ROOT = Path(__file__).resolve().parent.parent +REQUIREMENTS = ROOT / "requirements.txt" +PINNED = ROOT / "flatpak" / "requirements-pinned.txt" +MODULES = ROOT / "flatpak" / "python-modules.yml" +FLATPAK_README = ROOT / "flatpak" / "README.md" +DEPENDABOT = ROOT / ".github" / "dependabot.yml" + +# Packages the Flatpak deliberately does not ship: PyInstaller only +# builds the Windows executable, and the rest are the still-missing +# converters documented in flatpak/README.md. Listing them here keeps +# the parity test honest about what is knowingly absent instead of +# silently ignoring every mismatch. +FLATPAK_OMITTED = { + "pyinstaller", + "python-pptx", + "openpyxl", + "beautifulsoup4", + "ebooklib", + "lxml", + "urllib3", + "idna", +} + + +def _normalise(name: str) -> str: + """PEP 503 normalisation, so ``PySide6``/``pyside6`` compare equal.""" + return re.sub(r"[-_.]+", "-", name).strip().lower() + + +def _parse_requirements(path: Path) -> dict[str, str]: + """Map normalised package name -> version specifier text.""" + out: dict[str, str] = {} + for line in path.read_text(encoding="utf-8").splitlines(): + s = line.strip() + if not s or s.startswith("#"): + continue + m = re.match(r"^([A-Za-z0-9][A-Za-z0-9_.\-]*)\s*(.*)$", s) + assert m, f"unparsable requirement line in {path.name}: {line!r}" + out[_normalise(m.group(1))] = m.group(2) + return out + + +def _load_modules() -> dict: + if yaml is None: + pytest.skip("PyYAML not available") + with MODULES.open(encoding="utf-8") as f: + return yaml.safe_load(f) + + +def _wheel_versions() -> dict[str, str]: + """Map normalised package name -> version, read from wheel filenames. + + Wheel names are ``{distribution}-{version}(-{build})?-{tags}.whl`` + per PEP 427, so the first two dash-separated fields are all we need. + """ + data = _load_modules() + out: dict[str, str] = {} + for source in data["sources"]: + filename = source["url"].rsplit("/", 1)[-1] + assert filename.endswith(".whl"), f"non-wheel source: {filename}" + dist, version = filename[: -len(".whl")].split("-")[:2] + out[_normalise(dist)] = version + return out + + +def test_pinned_file_and_wheel_list_cover_the_same_packages(): + pinned = set(_parse_requirements(PINNED)) + wheels = set(_wheel_versions()) + assert pinned == wheels, ( + "flatpak/requirements-pinned.txt and flatpak/python-modules.yml " + "list different packages; regenerate the wheel list with " + "req2flatpak (see flatpak/README.md).\n" + f"only in pin file: {sorted(pinned - wheels)}\n" + f"only in wheel list: {sorted(wheels - pinned)}" + ) + + +def test_wheel_versions_match_the_pinned_versions(): + pinned = _parse_requirements(PINNED) + wheels = _wheel_versions() + mismatches = [] + for name, spec in pinned.items(): + m = re.fullmatch(r"==\s*(.+)", spec) + assert m, f"{name} must be pinned with '==' in {PINNED.name}, got {spec!r}" + want = m.group(1).strip() + got = wheels.get(name) + if got != want: + mismatches.append(f"{name}: pin={want} wheel={got}") + assert not mismatches, ( + "flatpak/python-modules.yml ships versions that differ from " + "flatpak/requirements-pinned.txt, so pip-audit on the pin file " + "does not describe what the Flatpak actually installs:\n" + + "\n".join(mismatches) + ) + + +def test_build_command_installs_every_pinned_package(): + data = _load_modules() + command = " ".join(data["build-commands"]) + listed = { + _normalise(tok) + for tok in command.split("--no-build-isolation", 1)[1].split() + } + assert listed == set(_parse_requirements(PINNED)), ( + "the pip3 install command in python-modules.yml does not name the " + "same packages as requirements-pinned.txt; a wheel present in " + "sources but absent from the command is downloaded and never " + "installed." + ) + + +def test_flatpak_pins_satisfy_the_root_requirements_floor(): + """Every Flatpak pin must be >= the floor in requirements.txt. + + Without this, a security bump to requirements.txt can land while the + Flatpak keeps installing the vulnerable version. + """ + root = _parse_requirements(REQUIREMENTS) + pinned = _parse_requirements(PINNED) + + def parts(v: str) -> tuple[int, ...]: + return tuple(int(x) for x in re.findall(r"\d+", v)[:4]) + + stale = [] + for name, spec in pinned.items(): + floor_spec = root.get(name) + if floor_spec is None: + continue # e.g. shiboken6, a PySide6 transitive not pinned at root + m = re.fullmatch(r">=\s*(.+)", floor_spec) + if not m: + continue + want, got = parts(m.group(1)), parts(spec.lstrip("= ")) + if got < want: + stale.append(f"{name}: flatpak pin {spec.lstrip('= ')} < requirements floor {m.group(1)}") + assert not stale, "\n".join(stale) + + +def test_no_root_dependency_is_silently_dropped_from_the_flatpak(): + """Guard the *known* gap in flatpak/README.md against growing. + + New runtime deps must either be added to the Flatpak pins or listed + explicitly in FLATPAK_OMITTED, so an omission is a deliberate, + reviewed decision rather than an oversight nobody noticed. + """ + root = set(_parse_requirements(REQUIREMENTS)) + pinned = set(_parse_requirements(PINNED)) + unaccounted = root - pinned - {_normalise(n) for n in FLATPAK_OMITTED} + assert not unaccounted, ( + "these requirements.txt packages are neither pinned for the " + "Flatpak nor listed as deliberately omitted: " + f"{sorted(unaccounted)}" + ) + + +def test_cryptography_is_pinned_for_the_flatpak(): + """pypdf needs cryptography for AES; it is only an optional extra. + + pypdf declares cryptography under the ``crypto``/``full`` extras, so + a --no-index install of the bare ``pypdf`` wheel resolves + ``pypdf._crypt_providers`` to the pure-Python fallback, whose + CryptAES raises DependencyError. The app requests AES-256 in + app/tools/encrypt.py and app/editor/tab.py, so without this pin + encrypting and opening AES PDFs is broken inside the Flatpak only. + """ + assert "cryptography" in _parse_requirements(PINNED) + assert "cryptography" in _wheel_versions() + + +def test_flatpak_readme_documents_every_omitted_package(): + """FLATPAK_OMITTED is canonical; the README must not drift from it. + + The README previously listed only four of the omitted packages, which + hid the fact that ``lxml`` is a *build* blocker (a hard dependency of + the pinned ``python-docx``) and not just another missing converter. + """ + status = FLATPAK_README.read_text(encoding="utf-8").split( + "**Before submitting to Flathub**", 1 + )[0] + undocumented = [n for n in sorted(FLATPAK_OMITTED) if f"`{n}`" not in status] + assert not undocumented, ( + "flatpak/README.md does not mention these deliberately omitted " + f"packages: {undocumented}" + ) + + +def test_dependabot_does_not_cover_the_flatpak_directory(): + """Pin the assumption the README now states. + + The README used to claim Dependabot bumped shared packages inside + ``flatpak/``. It does not: the ``pip`` ecosystem is declared only for + ``directory: "/"``. If a ``/flatpak`` entry is ever added, this test + fails so the README stops being wrong in the other direction. + """ + if yaml is None: + pytest.skip("PyYAML not available") + with DEPENDABOT.open(encoding="utf-8") as f: + cfg = yaml.safe_load(f) + pip_dirs = { + u.get("directory") + for u in cfg["updates"] + if u.get("package-ecosystem") == "pip" + } + assert pip_dirs == {"/"}, ( + "dependabot pip coverage changed; update the claim in " + f"flatpak/README.md. Directories: {sorted(pip_dirs)}" + )