[26.04_linux-nvidia] arm_ffa failing to load after kexec by clsotog · Pull Request #563 · NVIDIA/NV-Kernels

clsotog · 2026-09-02T16:44:02Z

On NVIDIA arm64 Vera systems, kexec can leave the current kernel's FF-A state active during shutdown. The next kernel can then fail FF-A initialization with:

ARM FF-A: failed to register FFA RxTx buffers

Upstream fixes this by tearing down the FF-A driver from the platform shutdown path:

https://lore.kernel.org/all/20260901131112.3437516-1-sudeep.holla@kernel.org/
https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/id=3bb3e80faf21e432f6e6d89c55fb587c323b8813

This PR backports that fix for the NVIDIA kernel.

Backport notes:
The upstream shutdown fix is built on top of the newer FF-A platform driver/device structure. The NVIDIA branch did not have all of that infrastructure, so this PR also backports the FF-A platform-driver preparation series to keep the fix aligned with upstream:
https://lore.kernel.org/all/20260508-b4-ffa_plat_dev-v1-0-c5a30f8cf7b8@kernel.org/

TPM CRB-over-FF-A / IMA notes:
While validating the FF-A shutdown fix on Vera, we also hit a shutdown/kexec crash in the TPM CRB-over-FF-A path. To address that, this PR includes the upstream TPM patch that removes the old built-in tpm_crb_ffa
forced-registration workaround:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/id=677042afb97ac5057e1d2900139f123bb15ba6e6

After adding that patch, the TPM CRB probe can again defer until the FF-A TPM device is available. On NVIDIA arm64, FF-A, CRB-over-FF-A, and IMA are built in, so IMA must not run before the deferred TPM probe is retried. Without the
IMA late-sync support, boot can report:

ima: No TPM chip found, activating TPM-bypass!

Therefore this PR also backports the upstream IMA late-sync prerequisite series and enables CONFIG_IMA_INIT_LATE_SYNC=y for NVIDIA arm64 only:
https://lore.kernel.org/all/20260605144325.434436-1-yeoreum.yun@arm.com/

Tested at SJ, GB200:

kexec no longer fails with ARM FF-A: failed to register FFA RxTx buffers
Vera:

kexec no longer fails with ARM FF-A: failed to register FFA RxTx buffers
IMA no longer reports No TPM chip found, activating TPM-bypass!
shutdown/kexec completes without the tpm_crb_ffa_remove() crash

Nvbug: https://nvbugspro.nvidia.com/bug/5693529
LP: https://bugs.launchpad.net/ubuntu/+source/linux-nvidia-bos/+bug/2166208

…tfs_initcall" This reverts commit 0e0546e, which was added to address ordering issues with the IMA LSM initialisation where the TPM would not be fully ready by the time IMA wanted it. This has been resolved within IMA by retrying setup during late_initcall_sync if the TPM is not available at first. Signed-off-by: Yeoreum Yun <yeoreum.yun@arm.com> Link: https://patch.msgid.link/20260508-b4-ffa_plat_dev-v1-1-c5a30f8cf7b8@kernel.org Signed-off-by: Sudeep Holla <sudeep.holla@kernel.org> (cherry picked from commit cc7e8f2) Signed-off-by: Carol L Soto <csoto@nvidia.com>

Move the FF-A core bring-up and teardown paths into platform driver probe and remove callbacks, and register a synthetic arm-ffa platform device to bind the driver. This makes the FF-A core lifetime follow the driver model while keeping the device creation internal to the FF-A core. Use normal platform driver registration so the probe path has standard driver-core semantics. The synthetic platform device is a temporary bridge until ACPI and devicetree describe the FF-A core device or object. Once those firmware description paths are defined, the internal platform device creation can be dropped and the driver can bind to the firmware-described device directly. Since the transport selection now happens from the platform probe path, drop the __init annotation from ffa_transport_init(). Reviewed-by: Yeoreum Yun <yeoreum.yun@arm.com> Link: https://patch.msgid.link/20260508-b4-ffa_plat_dev-v1-2-c5a30f8cf7b8@kernel.org Signed-off-by: Sudeep Holla <sudeep.holla@kernel.org> (cherry picked from commit e659fc8) Signed-off-by: Carol L Soto <csoto@nvidia.com>

Pass a parent device into ffa_device_register() and use the synthetic arm-ffa platform device as the parent for each registered FF-A device. This keeps the enumerated FF-A partition devices anchored below the FF-A core device in the driver model, matching the platform-driver conversion of the core transport. Suggested-by: Yeoreum Yun <yeoreum.yun@arm.com> Reviewed-by: Yeoreum Yun <yeoreum.yun@arm.com> Link: https://patch.msgid.link/20260508-b4-ffa_plat_dev-v1-3-c5a30f8cf7b8@kernel.org Signed-off-by: Sudeep Holla <sudeep.holla@kernel.org> (cherry picked from commit 7fe2ec9) Signed-off-by: Carol L Soto <csoto@nvidia.com>

When protected KVM is enabled, the kernel includes a pKVM FF-A proxy that sits in front of the normal FF-A driver. The proxy has to perform its own FF-A version negotiation and setup first, so that it can mediate subsequent FF-A traffic correctly. Defer FF-A core probing until pKVM has completed initialization. This keeps the normal driver from negotiating the FF-A version or performing other transport setup before the pKVM proxy is ready, and lets the driver model retry probing once the protected KVM state required by the FF-A transport is available. Suggested-by: Yeoreum Yun <yeoreum.yun@arm.com> Reviewed-by: Yeoreum Yun <yeoreum.yun@arm.com> Link: https://patch.msgid.link/20260508-b4-ffa_plat_dev-v1-4-c5a30f8cf7b8@kernel.org Signed-off-by: Sudeep Holla <sudeep.holla@kernel.org> (cherry picked from commit 3acc80a) Signed-off-by: Carol L Soto <csoto@nvidia.com>

nirmoy · 2026-09-02T16:53:19Z

BaseOS Kernel Review

Warning

⚠️ Review needs attention

Default configurations can initialize IMA before the deferred FF-A TPM appears, leaving IMA without a TPM. FF-A teardown is also skipped during normal kexec without a shutdown callback.

Findings: Critical 0 · High 0 · Medium 2 · Low 4

🔍 Review artifacts

🟠 Boro review: 6 findings
🔵 Test plan: ready for validation

📦 Kernel deb builds — 🟢 2/2 passed

🟢 amd64: build report and 4 debs
🟢 arm64: build report and 4 debs

Note

Build reports and debs are retained for 10 days after the PR closes.

Review metadata

Reviewed head: 4f2225d246d8
Overall status: attention needed
Architectures: 2/2 successful

This comment is maintained by BaseOS Reviewer and updated when the GitHub watcher publishes a newer review.

github-actions · 2026-09-02T16:56:15Z

PR Validation Report

Patchscan ✅ No Missing Fixes

All cherry-picked commits checked — no missing upstream fixes found.

PR Lint ⚠️ Warnings

Details

Checking 10 commits...

Cherry-pick digest:
┌──────────────┬──────────────────────────────────────────────────────────────────┬────────────┬─────────┬───────────────────────────┐
│ Local        │ Referenced upstream / Patch subject                              │ Patch-ID   │ Subject │ SoB chain                 │
├──────────────┼──────────────────────────────────────────────────────────────────┼────────────┼─────────┼───────────────────────────┤
│ 4f2225d246d8 │ [UBUNTU] [Config] Enable IMA_INIT_LATE_SYNC                      │ N/A        │ N/A     │ csoto                     │
├──────────────┼──────────────────────────────────────────────────────────────────┼────────────┼─────────┼───────────────────────────┤
│ de57eb27b304 │ firmware: arm_ffa: tear down driver during shutdown              │ match      │ found   │ ok, backporter: csoto     │
├──────────────┼──────────────────────────────────────────────────────────────────┼────────────┼─────────┼───────────────────────────┤
│ 80560be66aeb │ 677042afb97a tpm: tpm_crb_ffa: revert defered_probed when tpm_cr │ match      │ match   │ preserved + csoto added   │
├──────────────┼──────────────────────────────────────────────────────────────────┼────────────┼─────────┼───────────────────────────┤
│ 8bbe1219e6d6 │ c0fc127a9819 security: ima: rename boot_aggregate when ima is in │ match      │ match   │ preserved + csoto added   │
├──────────────┼──────────────────────────────────────────────────────────────────┼────────────┼─────────┼───────────────────────────┤
│ 6dc046eb9aa6 │ 1ebe7997b53b security: ima: introduce IMA_INIT_LATE_SYNC option  │ context    │ match   │ preserved + csoto added   │
├──────────────┼──────────────────────────────────────────────────────────────────┼────────────┼─────────┼───────────────────────────┤
│ 8f498264a034 │ fd88b5cb8b58 security: lsm: allow LSMs to register for late_init │ match      │ match   │ preserved + csoto added   │
├──────────────┼──────────────────────────────────────────────────────────────────┼────────────┼─────────┼───────────────────────────┤
│ 769e617ffb53 │ 3acc80a78e45 firmware: arm_ffa: Defer probe until pKVM is initia │ context    │ match   │ preserved + csoto added   │
├──────────────┼──────────────────────────────────────────────────────────────────┼────────────┼─────────┼───────────────────────────┤
│ c4f69e1d6c46 │ 7fe2ec9fb8e9 firmware: arm_ffa: Set the core device as FF-A devi │ match      │ match   │ preserved + csoto added   │
├──────────────┼──────────────────────────────────────────────────────────────────┼────────────┼─────────┼───────────────────────────┤
│ 16ffd7d5061c │ e659fc8e537c firmware: arm_ffa: Register core as a platform driv │ match      │ match   │ preserved + csoto added   │
├──────────────┼──────────────────────────────────────────────────────────────────┼────────────┼─────────┼───────────────────────────┤
│ f579b7e44e97 │ cc7e8f21b9f0 Revert "firmware: arm_ffa: Change initcall level of │ match      │ match   │ preserved + csoto added   │
└──────────────┴──────────────────────────────────────────────────────────────────┴────────────┴─────────┴───────────────────────────┘

Lint results:
W: 8bbe1219e6d6 ("security: ima: rename boot_aggregate when ima is i"): subject 73 chars (>72)
W: f579b7e44e97 ("Revert "firmware: arm_ffa: Change initcall level o"): subject 82 chars (>72)

There are situations where LSMs have dependencies that might mean they want to be initialised later in the boot process, to ensure those dependencies are available. In particular there are some TPM setups (Arm FF-A devices, SPI attached TPMs) required by IMA which are not guaranteed to be initialised for regular initcall_late. Add an initcall_late_sync option that can be used in these situations. Signed-off-by: Yeoreum Yun <yeoreum.yun@arm.com> Cc: Paul Moore <paul@paul-moore.com> Acked-by: Paul Moore <paul@paul-moore.com> Signed-off-by: Mimi Zohar <zohar@linux.ibm.com> (cherry picked from commit fd88b5c) Signed-off-by: Carol L Soto <csoto@nvidia.com>

To generate the boot_aggregate log in the IMA subsystem with TPM PCR values, the TPM driver must be built as built-in and must be probed before the IMA subsystem is initialized. However, when the TPM device operates over the FF-A protocol using the CRB interface, probing fails and returns -EPROBE_DEFER if the tpm_crb_ffa device — an FF-A device that provides the communication interface to the tpm_crb driver — has not yet been probed. To ensure the TPM device operating over the FF-A protocol with the CRB interface is probed before IMA initialization, the following conditions must be met: 1. The corresponding ffa_device must be registered, which is done via ffa_init(). 2. The tpm_crb_driver must successfully probe this device via tpm_crb_ffa_init(). 3. The tpm_crb driver using CRB over FF-A can then be probed successfully. (See crb_acpi_add() and tpm_crb_ffa_init() for reference.) Unfortunately, ffa_init(), tpm_crb_ffa_init(), and crb_acpi_driver_init() are all registered with device_initcall, which means crb_acpi_driver_init() may be invoked before ffa_init() and tpm_crb_ffa_init() are completed. When this occurs, probing the TPM device is deferred. However, the deferred probe can happen after the IMA subsystem has already been initialized, since IMA initialization is performed during late_initcall, and deferred_probe_initcall() is performed at the same level. And the similar situation is reported on TPM devices attached on SPI bus[0]. To resolve this, introduce IMA_INIT_LATE_SYNC option to initialise IMA at late_inicall_sync so that IMA is initialized with the TPM device probed deferred. When this option is enabled, modules that access files in the initramfs through usermode helper calls such as request_module() during initcall must not be built-in. Otherwise, IMA may miss measuring those files [1]. Link: https://lore.kernel.org/all/aYXEepLhUouN5f99@earth.li/ [0] Link: https://lore.kernel.org/all/2b3782398cc17ce9d355490a0c42ebce9120a9ae.camel@linux.ibm.com/ [1] Suggested-by: Mimi Zohar <zohar@linux.ibm.com> Signed-off-by: Yeoreum Yun <yeoreum.yun@arm.com> [zohar@linux.ibm.com: Fixed Kconfig merge conflict] Signed-off-by: Mimi Zohar <zohar@linux.ibm.com> (cherry picked from commit 1ebe799) [csoto: minor conflict at Kconfig due to missing IMA_STAGING] Signed-off-by: Carol L Soto <csoto@nvidia.com>

…sync The Linux IMA (Integrity Measurement Architecture) subsystem used for secure boot, file integrity, or remote attestation cannot be a loadable module for few reasons listed below: o Boot-Time Integrity: IMA’s main role is to measure and appraise files before they are used. This includes measuring critical system files during early boot (e.g., init, init scripts, login binaries). If IMA were a module, it would be loaded too late to cover those. o TPM Dependency: IMA integrates tightly with the TPM to record measurements into PCRs. The TPM must be initialized early (ideally before init_ima()), which aligns with IMA being built-in. o Security Model: IMA is part of a Trusted Computing Base (TCB). Making it a module would weaken the security model, as a potentially compromised system could delay or tamper with its initialization. IMA must be built-in to ensure it starts measuring from the earliest possible point in boot which inturn implies TPM must be initialised and ready to use before IMA. Unfortunately some TPM drivers (such as Arm FF-A, or SPI attached TPM devices) are not reliably available during the initcall_late stage, resulting in a log error: ima: No TPM chip found, activating TPM-bypass! To address this issue, IMA_INIT_LATE_SYNC is introduced. However, a remote attestation service cannot determine when IMA has been initialized because the boot_aggregate measurement name remains unchanged, even though IMA is initialized later at late_initcall_sync when IMA_INIT_LATE_SYNC is enabled. Therefore, use a distinct boot_aggregate name when IMA_INIT_LATE_SYNC is enabled, allowing the remote attestation service to identify when IMA has been initialized. Signed-off-by: Jonathan McDowell <noodles@meta.com> [yeoreum.yun@arm.com: modified to align with the IMA_INIT_LATE_SYNC change] Signed-off-by: Yeoreum Yun <yeoreum.yun@arm.com> Signed-off-by: Mimi Zohar <zohar@linux.ibm.com> (cherry picked from commit c0fc127) Signed-off-by: Carol L Soto <csoto@nvidia.com>

commit 746d9e9 ("tpm: tpm_crb_ffa: try to probe tpm_crb_ffa when it's built-in") probe tpm_crb_ffa forcefully when it's built-in to integrate with IMA. However, IMA now provides the IMA_INIT_LATE_SYNC build option, which initialises IMA at the late_initcall_sync level, so this change is no longer required. Signed-off-by: Yeoreum Yun <yeoreum.yun@arm.com> Link: https://git.kernel.org/pub/scm/linux/kernel/git/sudeep.holla/linux.git/commit/?h=for-next/ffa/updates&id=cc7e8f21b9f0c229d68cf19a837cba82b5ac2d87 [0] Link: https://git.kernel.org/pub/scm/linux/kernel/git/sudeep.holla/linux.git/commit/?h=for-next/ffa/updates&id=e659fc8e537c7a21d5d693d6f30d8852f2fa8d91 [1] Link: https://lore.kernel.org/r/20260605144325.434436-5-yeoreum.yun@arm.com Signed-off-by: Jarkko Sakkinen <jarkko@kernel.org> (cherry picked from commit 677042a) Signed-off-by: Carol L Soto <csoto@nvidia.com>

The platform core invokes a driver's shutdown callback, rather than its remove callback, while preparing devices for a normal kexec. Without a shutdown callback, the FF-A driver leaves notifications, partition devices, and the RX/TX mapping active before the replacement kernel is booted. Use ffa_remove() for shutdown so the existing cleanup runs before a normal kexec and other orderly system shutdowns. Reported-by: Nat Gurumoorthy <natg@google.com> Closes: https://lore.kernel.org/all/20260729162731.1383875-1-natg@google.com/ Reported-by: Carol L Soto <csoto@nvidia.com> Closes: https://lore.kernel.org/all/20260818224404.3694580-1-csoto@nvidia.com Reported-by: Maxi Saparov <masaparov@coreweave.com> Closes: https://lore.kernel.org/all/20260826222337.73480-1-maxi.saparov@gmail.com Link: https://patch.msgid.link/20260901131112.3437516-1-sudeep.holla@kernel.org Tested-by: Carol L Soto <csoto@nvidia.com> Signed-off-by: Sudeep Holla <sudeep.holla@kernel.org> (cherry picked from commit 3bb3e80faf21e432f6e6d89c55fb587c323b8813 linux-next) Signed-off-by: Carol L Soto <csoto@nvidia.com>

jamieNguyenNVIDIA · 2026-09-03T15:12:49Z

Acked-by: Jamie Nguyen <jamien@nvidia.com>

nvmochs · 2026-09-03T15:38:59Z

@clsotog

Two nits for "UBUNTU: [Config] Enable IMA_INIT_LATE_SYNC":

Please fix the commit message wrap to around 74 chars (it's currently 89 chars)
Please locate the CONFIG_IMA_INIT_LATE_SYNC ahead of the CONFIG_IOMMU* stanzas (otherwise the next time someone runs updateconfigs it will show it as being out or order)

clsotog · 2026-09-03T16:19:06Z

@clsotog

Two nits for "UBUNTU: [Config] Enable IMA_INIT_LATE_SYNC":

Please fix the commit message wrap to around 74 chars (it's currently 89 chars)

Please locate the CONFIG_IMA_INIT_LATE_SYNC ahead of the CONFIG_IOMMU* stanzas (otherwise the next time someone runs updateconfigs it will show it as being out or order)

Thanks, Fixed.

nirmoy · 2026-09-03T17:10:09Z

Acked-by: Nirmoy Das <nirmoyd@nvidia.com>

BugLink: https://bugs.launchpad.net/bugs/2166208 Enable IMA_INIT_LATE_SYNC for arm64 so IMA runs after deferred TPM CRB-over-FF-A probing. Signed-off-by: Carol L Soto <csoto@nvidia.com>

nvmochs · 2026-09-03T18:24:55Z

Thanks for addressing my comments, no further issues from me.

Acked-by: Matthew R. Ochs <mochs@nvidia.com>

nvmochs · 2026-09-03T19:07:59Z

Merged, closing PR.

b5ac6097a561 (nresolute/main-next) UBUNTU: [Config] Enable IMA_INIT_LATE_SYNC
523877d6f59a firmware: arm_ffa: Tear down driver during shutdown
b350035911ae tpm: tpm_crb_ffa: revert defered_probed when tpm_crb_ffa is built-in
eb64874421de security: ima: rename boot_aggregate when ima is initialised at late_sync
15147923dcf3 security: ima: introduce IMA_INIT_LATE_SYNC option
f034b97cccab security: lsm: allow LSMs to register for late_initcall_sync init
673cbc9d2bda firmware: arm_ffa: Defer probe until pKVM is initialized
4733a5c16174 firmware: arm_ffa: Set the core device as FF-A device parent
86c4dc6a9f1e firmware: arm_ffa: Register core as a platform driver
f3a07f7b5377 Revert "firmware: arm_ffa: Change initcall level of ffa_init() to rootfs_initcall"

LeviYeoReum and others added 4 commits September 2, 2026 08:09

nirmoy added the help wanted Extra attention is needed label Sep 2, 2026

LeviYeoReum and others added 5 commits September 2, 2026 13:00

clsotog force-pushed the clsotog/kexec-ffa-2604 branch 3 times, most recently from 51deb28 to df4c16d Compare September 3, 2026 06:19

nirmoy added the has_1_ack label Sep 3, 2026

clsotog force-pushed the clsotog/kexec-ffa-2604 branch from df4c16d to 5379f55 Compare September 3, 2026 16:17

nirmoy added has_2_acks and removed help wanted Extra attention is needed has_1_ack labels Sep 3, 2026

UBUNTU: [Config] Enable IMA_INIT_LATE_SYNC

4f2225d

BugLink: https://bugs.launchpad.net/bugs/2166208 Enable IMA_INIT_LATE_SYNC for arm64 so IMA runs after deferred TPM CRB-over-FF-A probing. Signed-off-by: Carol L Soto <csoto@nvidia.com>

clsotog force-pushed the clsotog/kexec-ffa-2604 branch from 5379f55 to 4f2225d Compare September 3, 2026 18:20

nvmochs closed this Sep 3, 2026

Navigation Menu

Uh oh!

[26.04_linux-nvidia] arm_ffa failing to load after kexec - #563

[26.04_linux-nvidia] arm_ffa failing to load after kexec#563
clsotog wants to merge 10 commits into
NVIDIA:26.04_linux-nvidiafrom
clsotog:clsotog/kexec-ffa-2604

clsotog commented Sep 2, 2026 •

edited

Loading

Uh oh!

nirmoy commented Sep 2, 2026 •

edited

Loading

⚠️ Review needs attention

Uh oh!

github-actions Bot commented Sep 2, 2026 •

edited

Loading

Uh oh!

jamieNguyenNVIDIA commented Sep 3, 2026

Uh oh!

nvmochs commented Sep 3, 2026

Uh oh!

clsotog commented Sep 3, 2026

Uh oh!

nirmoy commented Sep 3, 2026

Uh oh!

nvmochs commented Sep 3, 2026

Uh oh!

nvmochs commented Sep 3, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Uh oh!

Conversation

clsotog commented Sep 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nirmoy commented Sep 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

BaseOS Kernel Review

⚠️ Review needs attention

🔍 Review artifacts

📦 Kernel deb builds — 🟢 2/2 passed

Uh oh!

github-actions Bot commented Sep 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Validation Report

Patchscan ✅ No Missing Fixes

PR Lint ⚠️ Warnings

Uh oh!

jamieNguyenNVIDIA commented Sep 3, 2026

Uh oh!

nvmochs commented Sep 3, 2026

Uh oh!

clsotog commented Sep 3, 2026

Uh oh!

nirmoy commented Sep 3, 2026

Uh oh!

nvmochs commented Sep 3, 2026

Uh oh!

nvmochs commented Sep 3, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

clsotog commented Sep 2, 2026 •

edited

Loading

nirmoy commented Sep 2, 2026 •

edited

Loading

github-actions Bot commented Sep 2, 2026 •

edited

Loading