From 199dbd956bc4c76ae979be6270261ecca8f28ee7 Mon Sep 17 00:00:00 2001 From: nelsonduarte Date: Thu, 3 Sep 2026 13:10:03 +0100 Subject: [PATCH] fix: pin cryptography for the Flatpak and bump pypdf past six CVEs The Flatpak manifest never listed cryptography, which breaks AES encryption and AES-encrypted PDF opening in the Flatpak build. pypdf declares cryptography only as the optional `crypto` extra, and the manifest installs with `--no-index --find-links`, so the extra is never resolved from PyPI and the wheel is simply absent from the sandbox. The failure is asymmetric, which is why it went unnoticed: the editor opens encrypted PDFs through PyMuPDF, which carries its own crypto and kept working, while the Encrypt tool goes through pypdf and raised DependencyError. Only the pypdf path was dead. Dependency floors: - pypdf 6.14.2 was exposed to six advisories: PYSEC-2026-3655 and PYSEC-2026-3656 (fixed in 6.15.0), CVE-2026-82398 (6.15.0), CVE-2026-84309 (6.16.0), and CVE-2026-84310 / CVE-2026-84311 (6.16.1). 6.15.0 still carried three of them, so the floor is 6.16.2 rather than the 6.15.0 the earliest advisory alone would suggest. - cryptography is floored at 50.0.1 for PYSEC-2026-3552 / CVE-2026-69247, a Bleichenbacher oracle in PKCS#7 EnvelopedData decryption introduced in 44.0.0 and fixed in 50.0.0. 49.0.0 is the only release in the 49 series and it is vulnerable, so the previous `cryptography>=49.0.0` floor could never resolve to a safe version. Adds tests/test_flatpak_dependency_pins.py with eight consistency tests tying requirements.txt, flatpak/requirements-pinned.txt and flatpak/python-modules.yml together, since flatpak/python-modules.yml has no reference anywhere in .github/ and so nothing in CI reads it. Also corrects flatpak/README.md, which claimed Dependabot keeps the shared packages in sync. It does not: .github/dependabot.yml declares the pip ecosystem only for `directory: "/"`, so nothing under flatpak/ is ever bumped or added. Co-Authored-By: Claude Opus 4.8 --- flatpak/README.md | 47 +++++- flatpak/python-modules.yml | 13 +- flatpak/requirements-pinned.txt | 3 +- requirements.txt | 17 +- tests/test_flatpak_dependency_pins.py | 233 ++++++++++++++++++++++++++ 5 files changed, 299 insertions(+), 14 deletions(-) create mode 100644 tests/test_flatpak_dependency_pins.py diff --git a/flatpak/README.md b/flatpak/README.md index e3f5505..d16587e 100644 --- a/flatpak/README.md +++ b/flatpak/README.md @@ -4,14 +4,49 @@ This directory contains files for submitting PDFApps to [Flathub](https://flathu ## ⚠️ Status: dormant — needs regeneration before next submission -The files in this directory are out of date relative to the root `requirements.txt`: +`requirements-pinned.txt` and `python-modules.yml` are out of date relative to +the root `requirements.txt`. The canonical list of what is absent lives in +`FLATPAK_OMITTED` in `tests/test_flatpak_dependency_pins.py`; keep this section +and that set in step. The eight entries split into four kinds: -- `requirements-pinned.txt` is missing **cryptography, python-pptx, openpyxl, beautifulsoup4, ebooklib**. -- `python-modules.yml` does not contain wheel URLs / hashes for those packages. +**Deliberate** (never needed in the Flatpak): -Effect: a Flatpak built today would have **PDF→PPTX, PDF→XLSX, import HTML and import EPUB broken**. Other conversions and the core viewer/editor still work. +- `pyinstaller`: only builds the Windows executable. -The divergence happened because new Python deps were added to `requirements.txt` (notably in v1.12.0) without regenerating the Flatpak pin file. Dependabot keeps the shared packages in sync (e.g. it bumps `pypdf` here too) but it does not add new packages. +**Build blocker** (the Flatpak build fails outright, before any feature runs): + +- `lxml`: a hard, unconditional dependency of `python-docx`, which *is* pinned + (`lxml>=3.1.0` in its metadata). The manifest installs with `--no-index + --find-links` and **without** `--no-deps`, so pip has to resolve `lxml` from + the local wheel directory and there is no wheel there. + +**Floor-only pins** (no Flatpak package requires them today): + +- `urllib3`, `idna`: present at the root only to raise the floor on transitive + advisories. They must come back here the moment a package that pulls + `requests` is added to the Flatpak pin set. + +`lxml` is not the only unmet hard dependency of the current pin set: `cffi` +(from `cryptography`), `typing_extensions` (from `python-docx`), `packaging` +(from `pytesseract`), `qtpy` (from `qtawesome`) and the `PySide6_Essentials` / +`PySide6_Addons` meta-packages are missing too. Those are tracked separately; +regenerating with `req2flatpak` (below) is what fixes all of them at once. + +**Feature gaps** (would build, but tools are dead): + +- `python-pptx`, `openpyxl`, `beautifulsoup4`, `ebooklib`: a Flatpak built + from these pins would have **PDF→PPTX, PDF→XLSX, import HTML and import + EPUB broken**. Other conversions and the core viewer/editor still work. + +(`cryptography` was added when `pypdf` was bumped to 6.16.2; it is an optional +`pypdf` extra that AES-encrypted PDFs require, and it is pinned.) + +The divergence happened because new Python deps were added to `requirements.txt` +(notably in v1.12.0) without regenerating the Flatpak pin file. **Dependabot does +not cover this directory**: `.github/dependabot.yml` declares the `pip` ecosystem +only for `directory: "/"`, so it neither bumps nor adds anything under +`flatpak/`. The tests in `tests/test_flatpak_dependency_pins.py` are the net that +now catches that drift. **Before submitting to Flathub**, regenerate everything from the root `requirements.txt`: @@ -50,7 +85,7 @@ A previous AI-assisted PR (#8301) was closed by a maintainer. Repeat offenses ca ## Files - `io.github.nelsonduarte.PDFApps.yml` — Flatpak manifest (KDE Platform 6.8) -- `python-modules.yml` — Python deps with SHA256 (generated by `req2flatpak`) +- `python-modules.yml` — Python deps with SHA256 (generated by `req2flatpak`; regenerate it, do not hand-edit, because a manual edit silently desynchronises the wheel list from `requirements-pinned.txt`) - `io.github.nelsonduarte.PDFApps.metainfo.xml` — AppStream metadata - `io.github.nelsonduarte.PDFApps.desktop` — Desktop entry - `flathub.json` — Restrict to `x86_64` (PySide6 wheels limitation) diff --git a/flatpak/python-modules.yml b/flatpak/python-modules.yml index e44de13..3392210 100644 --- a/flatpak/python-modules.yml +++ b/flatpak/python-modules.yml @@ -3,14 +3,19 @@ name: python3-package-installation buildsystem: simple build-commands: - pip3 install --verbose --exists-action=i --no-index --find-links="file://${PWD}" - --prefix=${FLATPAK_DEST} --no-build-isolation PySide6 shiboken6 pypdf qtawesome - pillow pymupdf pytesseract python-docx + --prefix=${FLATPAK_DEST} --no-build-isolation PySide6 shiboken6 pypdf cryptography + qtawesome pillow pymupdf pytesseract python-docx sources: - type: file url: https://files.pythonhosted.org/packages/d8/de/af89d71410c83b10654d86ff9aff2a4f87c30163658f1cc145242e222526/pyside6-6.11.1-cp310-abi3-manylinux_2_34_x86_64.whl sha256: b1fc521ba2bb5109425ab8add06bddbdd524abcad06cfa012cc39a22a189feb2 only-arches: - x86_64 +- type: file + url: https://files.pythonhosted.org/packages/85/66/6ccca4722987ddedaa7fc9c3f4708af7431f5535666c174350830888c6b7/cryptography-50.0.1-cp311-abi3-manylinux_2_34_x86_64.whl + sha256: 51afcfceb15597cf2635068e4ac9a56b2abde622edde17f37d85fd7b5306497a + only-arches: + - x86_64 - type: file url: https://files.pythonhosted.org/packages/84/21/a35af28dcc61f37ed850a2d64c65c701321dfbf25085e469d5559360cbbf/pillow-12.3.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl sha256: 78cb2c6865a35ab8ff8b75fd122f6033b92a62c82801110e48ddd6c936a45d91 @@ -22,8 +27,8 @@ sources: only-arches: - x86_64 - type: file - url: https://files.pythonhosted.org/packages/49/e6/136aa8993a2ae7214e0b0ef2edaa0d2e08d1d4e4982635b08a835ff31ec8/pypdf-6.14.2-py3-none-any.whl - sha256: 3f07891af76dc002657e04993ab9b4de81de29f9013b9761d0b7968bff12e946 + url: https://files.pythonhosted.org/packages/13/f1/a2da3b55acd4ab737bf728c97edaaed5ec1d3c1236acb639dcdfa97e42c7/pypdf-6.16.2-py3-none-any.whl + sha256: c8b09a59399062fb45a1b8156c18a787a10a3dae03ac9674397a226712c94604 - type: file url: https://files.pythonhosted.org/packages/7a/33/8312d7ce74670c9d39a532b2c246a853861120486be9443eebf048043637/pytesseract-0.3.13-py3-none-any.whl sha256: 7a99c6c2ac598360693d83a416e36e0b33a67638bb9d77fdcac094a3589d4b34 diff --git a/flatpak/requirements-pinned.txt b/flatpak/requirements-pinned.txt index b7670dc..a07bb3e 100644 --- a/flatpak/requirements-pinned.txt +++ b/flatpak/requirements-pinned.txt @@ -1,6 +1,7 @@ PySide6==6.11.1 shiboken6==6.11.1 -pypdf==6.14.2 +pypdf==6.16.2 +cryptography==50.0.1 qtawesome==1.4.2 pillow==12.3.0 pymupdf==1.28.0 diff --git a/requirements.txt b/requirements.txt index 256c5d6..43c85ea 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,8 +1,19 @@ PySide6>=6.11.1 -pypdf>=6.14.2 +# PYSEC-2026-3655 / PYSEC-2026-3656 (fixed in 6.15.0), CVE-2026-82398 +# (6.15.0), CVE-2026-84309 (6.16.0) and CVE-2026-84310 / CVE-2026-84311 +# (6.16.1). 6.16.1 is the first release clear of all six, so the floor +# is the current 6.16.2 rather than the 6.15.0 the earliest advisory +# alone would suggest. +pypdf>=6.16.2 # GHSA-537c-gmf6-5ccf: signature verification bypass in legacy OpenSSL -# providers fixed in 48.0.1. -cryptography>=49.0.0 +# providers fixed in 48.0.1, and PYSEC-2026-3552 / CVE-2026-69247 +# (Bleichenbacher oracle in PKCS#7 EnvelopedData decryption, introduced +# in 44.0.0) fixed in 50.0.0. 49.0.0 is the only 49.x release and it is +# vulnerable, so the floor must stay at 50 or above: do not lower it. +# Also the AES backend pypdf resolves at import time (pypdf declares it +# only as the optional `crypto` extra), so without it AES-encrypted PDFs +# raise DependencyError. +cryptography>=50.0.1 qtawesome>=1.4.2 pillow>=12.3.0 pymupdf>=1.28.0 diff --git a/tests/test_flatpak_dependency_pins.py b/tests/test_flatpak_dependency_pins.py new file mode 100644 index 0000000..debb276 --- /dev/null +++ b/tests/test_flatpak_dependency_pins.py @@ -0,0 +1,233 @@ +"""Consistency tests for the three files that pin Python dependencies. + +The Flatpak build is driven by ``flatpak/python-modules.yml``, which has +**zero references anywhere in .github/**: no workflow audits it and no +workflow regenerates it. ``security-deps.yml`` audits +``flatpak/requirements-pinned.txt`` instead, so the wheel list can drift +away from the pin file and CI stays green while the shipped Flatpak +installs a different (possibly vulnerable) version. That is exactly how +``pypdf`` stayed at 6.14.2 in the Flatpak with six known CVEs while the +root ``requirements.txt`` had already moved on. + +These tests close that loop without needing network access or pip-audit: +they assert the three files agree with each other. +""" + +from __future__ import annotations + +import re +from pathlib import Path + +import pytest + +try: + import yaml +except ImportError: # pragma: no cover - PyYAML is a dev dependency + yaml = None + +ROOT = Path(__file__).resolve().parent.parent +REQUIREMENTS = ROOT / "requirements.txt" +PINNED = ROOT / "flatpak" / "requirements-pinned.txt" +MODULES = ROOT / "flatpak" / "python-modules.yml" +FLATPAK_README = ROOT / "flatpak" / "README.md" +DEPENDABOT = ROOT / ".github" / "dependabot.yml" + +# Packages the Flatpak deliberately does not ship: PyInstaller only +# builds the Windows executable, and the rest are the still-missing +# converters documented in flatpak/README.md. Listing them here keeps +# the parity test honest about what is knowingly absent instead of +# silently ignoring every mismatch. +FLATPAK_OMITTED = { + "pyinstaller", + "python-pptx", + "openpyxl", + "beautifulsoup4", + "ebooklib", + "lxml", + "urllib3", + "idna", +} + + +def _normalise(name: str) -> str: + """PEP 503 normalisation, so ``PySide6``/``pyside6`` compare equal.""" + return re.sub(r"[-_.]+", "-", name).strip().lower() + + +def _parse_requirements(path: Path) -> dict[str, str]: + """Map normalised package name -> version specifier text.""" + out: dict[str, str] = {} + for line in path.read_text(encoding="utf-8").splitlines(): + s = line.strip() + if not s or s.startswith("#"): + continue + m = re.match(r"^([A-Za-z0-9][A-Za-z0-9_.\-]*)\s*(.*)$", s) + assert m, f"unparsable requirement line in {path.name}: {line!r}" + out[_normalise(m.group(1))] = m.group(2) + return out + + +def _load_modules() -> dict: + if yaml is None: + pytest.skip("PyYAML not available") + with MODULES.open(encoding="utf-8") as f: + return yaml.safe_load(f) + + +def _wheel_versions() -> dict[str, str]: + """Map normalised package name -> version, read from wheel filenames. + + Wheel names are ``{distribution}-{version}(-{build})?-{tags}.whl`` + per PEP 427, so the first two dash-separated fields are all we need. + """ + data = _load_modules() + out: dict[str, str] = {} + for source in data["sources"]: + filename = source["url"].rsplit("/", 1)[-1] + assert filename.endswith(".whl"), f"non-wheel source: {filename}" + dist, version = filename[: -len(".whl")].split("-")[:2] + out[_normalise(dist)] = version + return out + + +def test_pinned_file_and_wheel_list_cover_the_same_packages(): + pinned = set(_parse_requirements(PINNED)) + wheels = set(_wheel_versions()) + assert pinned == wheels, ( + "flatpak/requirements-pinned.txt and flatpak/python-modules.yml " + "list different packages; regenerate the wheel list with " + "req2flatpak (see flatpak/README.md).\n" + f"only in pin file: {sorted(pinned - wheels)}\n" + f"only in wheel list: {sorted(wheels - pinned)}" + ) + + +def test_wheel_versions_match_the_pinned_versions(): + pinned = _parse_requirements(PINNED) + wheels = _wheel_versions() + mismatches = [] + for name, spec in pinned.items(): + m = re.fullmatch(r"==\s*(.+)", spec) + assert m, f"{name} must be pinned with '==' in {PINNED.name}, got {spec!r}" + want = m.group(1).strip() + got = wheels.get(name) + if got != want: + mismatches.append(f"{name}: pin={want} wheel={got}") + assert not mismatches, ( + "flatpak/python-modules.yml ships versions that differ from " + "flatpak/requirements-pinned.txt, so pip-audit on the pin file " + "does not describe what the Flatpak actually installs:\n" + + "\n".join(mismatches) + ) + + +def test_build_command_installs_every_pinned_package(): + data = _load_modules() + command = " ".join(data["build-commands"]) + listed = { + _normalise(tok) + for tok in command.split("--no-build-isolation", 1)[1].split() + } + assert listed == set(_parse_requirements(PINNED)), ( + "the pip3 install command in python-modules.yml does not name the " + "same packages as requirements-pinned.txt; a wheel present in " + "sources but absent from the command is downloaded and never " + "installed." + ) + + +def test_flatpak_pins_satisfy_the_root_requirements_floor(): + """Every Flatpak pin must be >= the floor in requirements.txt. + + Without this, a security bump to requirements.txt can land while the + Flatpak keeps installing the vulnerable version. + """ + root = _parse_requirements(REQUIREMENTS) + pinned = _parse_requirements(PINNED) + + def parts(v: str) -> tuple[int, ...]: + return tuple(int(x) for x in re.findall(r"\d+", v)[:4]) + + stale = [] + for name, spec in pinned.items(): + floor_spec = root.get(name) + if floor_spec is None: + continue # e.g. shiboken6, a PySide6 transitive not pinned at root + m = re.fullmatch(r">=\s*(.+)", floor_spec) + if not m: + continue + want, got = parts(m.group(1)), parts(spec.lstrip("= ")) + if got < want: + stale.append(f"{name}: flatpak pin {spec.lstrip('= ')} < requirements floor {m.group(1)}") + assert not stale, "\n".join(stale) + + +def test_no_root_dependency_is_silently_dropped_from_the_flatpak(): + """Guard the *known* gap in flatpak/README.md against growing. + + New runtime deps must either be added to the Flatpak pins or listed + explicitly in FLATPAK_OMITTED, so an omission is a deliberate, + reviewed decision rather than an oversight nobody noticed. + """ + root = set(_parse_requirements(REQUIREMENTS)) + pinned = set(_parse_requirements(PINNED)) + unaccounted = root - pinned - {_normalise(n) for n in FLATPAK_OMITTED} + assert not unaccounted, ( + "these requirements.txt packages are neither pinned for the " + "Flatpak nor listed as deliberately omitted: " + f"{sorted(unaccounted)}" + ) + + +def test_cryptography_is_pinned_for_the_flatpak(): + """pypdf needs cryptography for AES; it is only an optional extra. + + pypdf declares cryptography under the ``crypto``/``full`` extras, so + a --no-index install of the bare ``pypdf`` wheel resolves + ``pypdf._crypt_providers`` to the pure-Python fallback, whose + CryptAES raises DependencyError. The app requests AES-256 in + app/tools/encrypt.py and app/editor/tab.py, so without this pin + encrypting and opening AES PDFs is broken inside the Flatpak only. + """ + assert "cryptography" in _parse_requirements(PINNED) + assert "cryptography" in _wheel_versions() + + +def test_flatpak_readme_documents_every_omitted_package(): + """FLATPAK_OMITTED is canonical; the README must not drift from it. + + The README previously listed only four of the omitted packages, which + hid the fact that ``lxml`` is a *build* blocker (a hard dependency of + the pinned ``python-docx``) and not just another missing converter. + """ + status = FLATPAK_README.read_text(encoding="utf-8").split( + "**Before submitting to Flathub**", 1 + )[0] + undocumented = [n for n in sorted(FLATPAK_OMITTED) if f"`{n}`" not in status] + assert not undocumented, ( + "flatpak/README.md does not mention these deliberately omitted " + f"packages: {undocumented}" + ) + + +def test_dependabot_does_not_cover_the_flatpak_directory(): + """Pin the assumption the README now states. + + The README used to claim Dependabot bumped shared packages inside + ``flatpak/``. It does not: the ``pip`` ecosystem is declared only for + ``directory: "/"``. If a ``/flatpak`` entry is ever added, this test + fails so the README stops being wrong in the other direction. + """ + if yaml is None: + pytest.skip("PyYAML not available") + with DEPENDABOT.open(encoding="utf-8") as f: + cfg = yaml.safe_load(f) + pip_dirs = { + u.get("directory") + for u in cfg["updates"] + if u.get("package-ecosystem") == "pip" + } + assert pip_dirs == {"/"}, ( + "dependabot pip coverage changed; update the claim in " + f"flatpak/README.md. Directories: {sorted(pip_dirs)}" + )