audio: steamaudio: add Steam Audio spatial audio offload module and LLEXT

[lgirdwood]

Summary

This pull request adds an offload module and dynamically loadable ELF extension (LLEXT) for Valve Steam Audio spatial audio rendering engine in Sound Open Firmware (SOF).

Target platform: Intel ACE 3.0 (Panther Lake / PTL) and compatible Xtensa DSPs.

Architecture Highlights

• Core 3D Spatial Audio Engine:
  • Direct HRTF binaural convolution with azimuth/elevation interpolation and ITD/ILD calculation.
  • Distance attenuation (1/r inverse law with configurable min/max clamp).
  • Frequency-dependent air absorption filtering (high-frequency damping modeling humidity/temperature).
  • Sound source directivity modeling (dipole/cardioid patterns).
  • Occlusion filtering with frequency-dependent transmission coefficients.
  • Ray-traced Bounding Volume Hierarchy (BVH) reflection rendering directly on the DSP.
• Continuous 64-Channel Audio Architecture:
  • The host streams a continuous 64-channel 48 kHz 16/32-bit PCM CBR audio stream into the DSP pipeline.
  • The DSP component dynamically culls silent/inactive voices using a 64-bit active mask (active_sources_mask) within the audio processing loop, avoiding unnecessary processing overhead.
• Hardware-Timestamped VBR Metadata Stream (snd_compr):
  • Variable Bitrate (VBR) 3D spatial positioning metadata is delivered via a dedicated DMA-driven compressed ALSA PCM device (snd_compr) using magic protocol 0x534D5441 (SMTA).
  • Metadata packets are hardware-timestamped to align sample-accurately with the PCM stream.
• Modular LLEXT Support:
  • Component can be statically compiled into the firmware image or built as a dynamically loadable LLEXT (steamaudio.llext) using CONFIG_COMP_STEAMAUDIO=m.
  • Registered UUID 53746561-6d61-7564-696f737465616d31 in uuid-registry.txt.

Performance & Host Offload Metrics (Intel PTL ACE 3.0 @ 800 MHz)

Measured across real game scene phases:

Scene Phase	Active Sources	DSP MCPS	DSP Core Load	Host CPU (Offloaded)	Host CPU (Native Host)	Host Savings
1. Exploration	4 / 64	20.9 MCPS	2.61%	0.00%	1.84%	+1.84% CPU
2. Combat Skirmish	16 / 64	64.1 MCPS	8.01%	0.00%	7.91%	+7.91% CPU
3. Heavy Battlefield	64 / 64	236.9 MCPS	29.61%	0.00%	31.41%	+31.41% CPU
4. Ambience Return	8 / 64	35.3 MCPS	4.41%	0.00%	3.92%	+3.92% CPU

• Latency: DSP end-to-end processing latency is 5.33 ms (vs 12.50 ms host audio buffer latency, -57.3% reduction).

Verification & Testing

• Comprehensive unit tests executed with mathematical verification of HRTF convolution, distance falloff, directivity, occlusion, and BVH ray-tracing.
• Continuous 64-channel PCM streaming and VBR compressed metadata packet parsing tested and verified on Intel Panther Lake hardware (Aphid DUT).
• Built and tested sof-ptl.ri firmware image with CONFIG_COMP_STEAMAUDIO=m and verified dynamic LLEXT module load.

Signed-off-by: Liam Girdwood liam.r.girdwood@linux.intel.com

[Copilot]

🟡 Changes recommended

Critical IPC3 build failures and audio-buffer and format-handling issues remain unresolved.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Adds a Steam Audio spatial-audio offload component, DSP processing, IPC4 controls, BVH helpers, PTL integration, and LLEXT packaging.

Changes:

• Registers the component UUID and PTL module configuration.
• Adds spatial-audio, reverb, metadata, and BVH processing.
• Integrates static/LLEXT builds through Kconfig and CMake.

File summaries

File	Reviewed changes and findings
uuid-registry.txt	Registers the Steam Audio component UUID.
tools/rimage/config/ptl.toml.h	Adds the module to PTL image configuration.
src/audio/steamaudio/steamaudio.toml	Defines module capabilities. critical (3 votes): Output advertises all channel configurations, but processing only writes stereo; restrict output or implement channel-aware processing.
src/audio/steamaudio/steamaudio.h	Defines protocol, state, and APIs. moderate (1 vote): Documented SMTA magic differs from the STEA constant used by the component. nit (2 votes): Add Doxygen parameter and return documentation.
src/audio/steamaudio/steamaudio.c	Implements lifecycle and adapter registration. moderate (1 vote): Discards source_to_sink_copy() errors. moderate (1 vote): Reset does not clear delay buffers or direct filter state.
src/audio/steamaudio/steamaudio-ipc4.c	Implements IPC4 controls. moderate (3 votes): Directivity, air absorption, and transmission are accepted but unused. moderate (3 votes): Reverb fields other than wet gain are unused. moderate (1 vote): BVH query control is unhandled. moderate (3 votes): Getter does not update data_offset_size. moderate (3 votes): Ambisonics configuration is stored but unused. moderate (1 vote): Direct getter leaves response fields uninitialized. moderate (1 vote): Binaural getter omits interpolation and HRTF slot fields.
src/audio/steamaudio/steamaudio-generic.c	Implements audio processing. critical (3 votes): Circular-buffer pointers are processed as contiguous arrays, risking overrun at wrap boundaries. moderate (1 vote): S16 source lock is not released when sink acquisition fails. moderate (3 votes): Only channel 0 is processed. critical (3 votes): PCM data is interpreted as metadata without dedicated-stream and length validation. moderate (2 votes): FDN delay buffers are not cleared on reset. moderate (3 votes): Elevation and HRIR data are unused. moderate (1 vote): S32 source lock is not released on sink failure. moderate (1 vote): S32 processing also drops all but channel 0. moderate (1 vote): Room geometry is not used for tracing or occlusion. moderate (1 vote): S24 is incorrectly routed through S32 conversion.
src/audio/steamaudio/steamaudio_bvh.c	Adds BVH helpers. moderate (1 vote): The single-node BVH has invalid child indices and should either build valid nodes or remain explicitly linear.
src/audio/steamaudio/llext/llext.toml.h	Defines the LLEXT manifest configuration.
src/audio/steamaudio/llext/CMakeLists.txt	Builds the LLEXT module. critical (1 vote): IPC4-only sources are compiled for IPC3 LLEXT configurations.
src/audio/steamaudio/Kconfig	Adds component configuration. critical (2 votes): The component is enabled for every IPC variant although IPC3 lacks the referenced configuration symbols.
src/audio/steamaudio/CMakeLists.txt	Integrates component sources. critical (2 votes): IPC3 builds omit the IPC4 implementation while the interface still references its symbols.
src/audio/Kconfig	Registers the component Kconfig.
src/audio/CMakeLists.txt	Adds the component directory to the build.
app/boards/intel_adsp_ace30_ptl.conf	Enables PTL LLEXT configuration.

Review details

Suppressed comments (14)

src/audio/steamaudio/steamaudio-generic.c:330

• The source read lock is still held when sink_get_buffer_s16() fails. Because source_get_data_s16() records an outstanding request, the next process call will return -EBUSY and can stall the pipeline. Release the source fragment with a zero free size before returning the error.

	if (ret)
		return ret;

src/audio/steamaudio/steamaudio-generic.c:377

• The 32-bit path obtains circular source/sink fragments but then treats both pointers as contiguous arrays. A wrapped ring position will read or write outside the corresponding buffer instead of processing all requested samples. Handle the two circular segments using the returned buffer starts/sizes.

	ret = source_get_data_s32(source, in_bytes, &src, &x_start, &x_size);
	if (ret)
		return ret;

	ret = sink_get_buffer_s32(sink, out_bytes, &dst, &y_start, &y_size);

src/audio/steamaudio/steamaudio-generic.c:379

• The source read lock is still held when sink_get_buffer_s32() fails. Since source_get_data_s32() records an outstanding request, the next process call can return -EBUSY and stall the pipeline. Release the source fragment with a zero free size before returning the error.

	if (ret)
		return ret;

src/audio/steamaudio/steamaudio-generic.c:384

• The S32 implementation also discards every input channel except channel 0, so multichannel sources lose voices even when the module is configured for them. Keep the channel handling consistent with the declared input contract rather than silently ignoring cd->channels - 1 channels.

	for (uint32_t i = 0; i < frames; i++)
		cd->in_scratch[i] = (float)src[i * cd->channels] * (1.0f / 2147483648.0f);

src/audio/steamaudio/steamaudio-generic.c:169

• The room scene is initialized but no processing path calls steamaudio_dsp_trace_closest_hit() or steamaudio_dsp_test_occlusion(). Consequently this geometry cannot affect the audio, so the advertised DSP BVH/room occlusion rendering is not active.

	/* Initialize BVH scene with standard test room (8m x 10m x 3.5m) */
	steamaudio_dsp_scene_init_box_room(&cd->scene, 8.0f, 10.0f, 3.5f);

src/audio/steamaudio/steamaudio-generic.c:300

• A malformed synchronized header with num_samples == 0 reaches this division and produces an invalid gain step. Validate the header fields, including a nonzero sample count, before using them.

	cd->direct.gain_step = (cd->direct.target_gain - cd->direct.current_gain) / (float)hdr->num_samples;

src/audio/steamaudio/steamaudio-generic.c:414

• This S24 path is routed to the S32 handler, which converts input with Q1.31 scaling and emits Q1.31 output. SOF S24_4LE is Q1.23; other processors sign-extend it and shift by 8, so this path attenuates S24 input by roughly 256 and writes an invalid S24 representation. Use a format-specific conversion or pass the source format into the handler.

	case SOF_IPC_FRAME_S24_4LE:
		return steamaudio_process_s32;

src/audio/steamaudio/steamaudio-ipc4.c:106

• STEAMAUDIO_PARAM_BVH_QUERY is defined as a public control but falls through to the unhandled-parameter error. The host cannot provide a scene or query through this API, and the renderer therefore remains tied to the fixed test room initialized in steamaudio_dsp_init(). Implement the control or remove the advertised interface.

	default:
		comp_warn(dev, "steamaudio: unhandled param_id 0x%x", param_id);
		return -EINVAL;
	}

src/audio/steamaudio/steamaudio-ipc4.c:125

• The direct getter writes only comp_type, distance, and occlusion; the remaining fields in the packed response (flags, transmission type, directivity, and both EQ arrays) are left as whatever was in the caller's buffer. Populate the full state or at least clear unsupported fields before returning a configuration.

		struct sof_steamaudio_direct_config *cfg = (struct sof_steamaudio_direct_config *)fragment;
		cfg->comp_type = STEAMAUDIO_PARAM_DIRECT_CONFIG;
		cfg->distance_attenuation = cd->direct.current_gain;
		cfg->occlusion = 0.0f;

src/audio/steamaudio/steamaudio-ipc4.c:138

• The binaural getter similarly omits interpolation and hrtf_slot_id, leaving those response fields uninitialized. Clear the response and fill every field, or reject/omit a getter that cannot represent the current configuration.

		cfg->comp_type = STEAMAUDIO_PARAM_BINAURAL_CONFIG;
		cfg->direction[0] = cd->binaural.direction[0];
		cfg->direction[1] = cd->binaural.direction[1];
		cfg->direction[2] = cd->binaural.direction[2];
		cfg->spatial_blend = cd->binaural.spatial_blend;

src/audio/steamaudio/steamaudio.c:60

• The return value is discarded here. source_to_sink_copy() can return -EFBIG or -ENOSPC without consuming/committing data, but returning zero tells the adapter that processing succeeded and can leave the pipeline stalled. Return the copy result.

	source_to_sink_copy(source, sink, true, frames * cd->frame_bytes);

src/audio/steamaudio/steamaudio.c:103

• The reset path calls steamaudio_dsp_init, but that function resets reverb indices without clearing delay_buffers (and does not clear the direct filter states). After a pipeline reset, pre-reset reverb/filter history can leak into the new stream instead of starting from silence.

		steamaudio_dsp_init(cd, cd->sample_rate);

src/audio/steamaudio/steamaudio.h:19

• The PR describes the compressed metadata protocol as SMTA (0x534D5441), but this component defines and checks STEA (0x53544541). Packets produced with the documented SMTA magic will be ignored by check_inband_bitstream; the producer and DSP must use one protocol constant.

#define STEAMAUDIO_SOF_SYNC_WORD            0x53544541  /* 'STEA' */
#define STEAMAUDIO_SOF_PROTOCOL_VERSION     0x00010000

src/audio/steamaudio/steamaudio_bvh.c:177

• Only one BVH node is created, but its child indices are set to 0 and 11 even though no child nodes exist; steamaudio_dsp_trace_closest_hit() then ignores these fields and linearly scans every triangle. This is not a valid BVH, and a traversal using these indices would access outside nodes. Build valid child nodes or keep the scene representation explicitly linear.

	scene->num_nodes = 1;
	scene->nodes[0].bounds.min = (struct dsp_vec3){ x0, y0, z0 };
	scene->nodes[0].bounds.max = (struct dsp_vec3){ x1, y1, z1 };
	scene->nodes[0].left_child = 0;
	scene->nodes[0].right_child = 11;

• Files reviewed: 15/15 changed files
• Comments generated: 14
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.