FCE-3838: React Native camera effects on Fishjam's own camera

[MiloszFilimowski]

Description

The React Native docs now send users to camera effects on the camera Fishjam already manages, instead of VisionCamera. VisionCamera stays as an integration.

An effect is a camera track middleware. Peers keep receiving it as peer.cameraTrack:

const blur = createCameraEffectMiddleware(createBackgroundBlurEffect(() => ({ segmentation, radius: 24 })));
setCameraTrackMiddleware(isBlurOn ? null : blur);

Menu changes

• Tutorials: new "Add background blur to your camera". "Draw WebGPU effects into your published camera" is rewritten on createCameraFrameProcessorSession, with no VisionCamera.
• How-to → Camera effects (new): overview, background effects (blur and image), WebGPU effects, frame processing in a worklet, and stream middleware (moved here, now covering React Native too).
• How-to → Custom sources: keeps non-camera content (web, React Native, low-level frame API).
• Integrations → Vision Camera (new): the VisionCamera tutorial, how-to, and WebGPU-on-VisionCamera guide, moved from the pages above.
• Concepts: new "How camera effects work". "How custom sources work" links to it.
• Redirects for the four moved pages.

Build and API reference

• web-client-sdk submodule bumped to fff4e570 (web-client-sdk#606).
• New submodules video-effects (v0.1.5) and react-native-worklets (v0.12.1), with TypeDoc pages.
• typegpu pinned to 0.12.4 (the version video-effects uses). The VisionCamera WebGPU snippet maps typegpu to the SDK's 0.11 through hidden Twoslash flags, so every snippet still type-checks.

Tested

• All snippets type-check, yarn build passes with no broken links, and the pre-commit checks (format, spellcheck, typecheck) pass.
• Ran the tutorial code end to end in a test app: iOS simulator (SimCam) and Galaxy S23, debug and release builds, staging room. Blur, grayscale + watermark and the frame monitor work, and each device receives the other's effect.

Testing found these, now fixed in the docs:

• Android needs minSdkVersion: 26 (react-native-webgpu). The docs add it with expo-build-properties.
• In Android release builds, expo-asset keeps bundled images as drawable resources, so reading them with File fails. The docs include a workaround, checked in a release build.
• tgpu.resolve({ externals }) returns empty WGSL in TypeGPU 0.12. The examples use the array form.

Performance on the Galaxy S23 (release build, 720x1280): blur runs at 30 fps with no dropped frames at about 110% CPU of 800% and 33% GPU; the WebGPU tutorial effect at about 111% CPU and 12% GPU. Graphics memory showed a 200 MB to 1 GB saw-tooth with any effect on react-native-webgpu 0.5.15. The cause was in react-native-webgpu (GPUExternalTexture.destroy() did not release the imported frame); fixed upstream in wcandillon/react-native-webgpu#478, released in 0.10.1, memory flat at about 145 MB after it.

Fixes opened in our repos along the way:

• fix(background-image): load the image on React Native video-effects#6: background images load on React Native (released in 0.1.4)
• fix(react-native): release camera imports for frames the kernel skips video-effects#7: frames that skip render() no longer hold GPU memory (released in 0.1.4)
• fix(react-native): use own WebGPU flag constants, release 0.1.5 video-effects#9: own WebGPU flag constants, because react-native-webgpu 0.8.4 and newer export them as undefined (released in 0.1.5). The WebGPU tutorial uses the GPUTextureUsage / GPUShaderStage globals for the same reason.
• Set Android minSdkVersion 26 and react-native-webgpu 0.10.1 in fishjam-chat web-client-sdk#610: fishjam-chat sets Android minSdkVersion 26 and moves to react-native-webgpu 0.10.1

The docs require @fishjam-cloud/react-native-client 0.30.2 or newer, @fishjam-cloud/video-effects 0.1.5 or newer and react-native-webgpu 0.10.1 or newer.

[linear]

FCE-3838