English | 日本語

🎬 Outcasts AppPromoVideo

A desktop tool that takes an existing app repository and UI snapshots and outputs scene-by-scene prompts and reference images ready to paste into video-generation AIs

promovideo.mp4

"Ready-to-paste output."
Not clever copywriting — prompts with the right granularity to paste directly into video-generation UIs, and reference images that stay faithful to your snapshots.
It doesn't create the video itself. It crafts the ultimate "brief" for video-generation AIs.

---

💡 Why AppPromoVideo?

Trying to create an app promo video with video-generation AIs (Google Veo / OpenAI Sora / MiniMax / Runway / Luma, etc.) quickly hits two walls:

1. Prompt trial-and-error: Scene breakdown, camera work, duration allocation, and English prompt assembly take enormous effort.
2. Drift from the real app: Generic prompts produce UI, tone, and style that look nothing like the actual app.

AppPromoVideo solves both at once — just feed it a repository and UI screenshots.

Three Core Values

• 🎯 Ready-to-Paste Output
  Optimized English prompts for Veo / Sora / generic models. One-click copy to clipboard and paste straight into any video-generation UI.
• 🎨 Visually Faithful Reference Images
  Extracts a color palette and style anchors from UI snapshots. Generates reference images faithful to the real app by compositing AI-generated backdrops with actual screenshots (rounded corners, drop shadows, headline overlays).
• 🛡️ Secure Local CLI Execution (Secure & Sandboxed)
  Instead of calling LLM HTTP APIs directly, launches a locally authenticated CLI (claude -p / Aider / custom, etc.) as a safe subprocess. Thorough sandboxing with Windows Job Objects / Unix pgids for reliable termination of descendant processes and read-only filesystem boundaries.

---

🔄 Pipeline Overview

The LLM writes, Rust inspects, the CLI scans, the model paints the stage, and Rust composites the screen.

flowchart LR
    subgraph Input ["📥 1. Input"]
        Repo["📂 App Repository<br/>(code, README, config)"]
        Snap["🖼️ UI Snapshots<br/>(real screen captures)"]
    end

    subgraph Pipeline ["⚙️ 2. Pipeline (Rust / CLI)"]
        Brief["RepoBrief Extraction<br/>(codebase compression)"]
        LLM["🤖 Local LLM CLI<br/>(Claude / Aider / custom)<br/>*secure subprocess*"]
        Inspect["Rust Inspection Loop<br/>(JSON Schema / duration / English check)"]
        Brief --> LLM --> Inspect
    end

    subgraph ImageGen ["🎨 3. Reference Image Compositing"]
        Palette["Palette & Style Anchor Extraction"]
        Compose["Backdrop Compositing & Headline Overlay<br/>(ComfyUI / Gemini / OpenAI)"]
        Palette --> Compose
    end

    subgraph Output ["📦 4. Output Package"]
        Scene["🎬 Scene-by-Scene Prompts<br/>(Veo / Sora / Generic)"]
        RefImg["🖼️ Style Reference Images<br/>(backdrop + real UI composite)"]
        Pkg["📄 promo.json + Markdown"]
    end

    Repo --> Brief
    Snap --> Palette
    Snap --> Compose
    Inspect --> Scene
    Compose --> RefImg
    Scene --> Pkg
    RefImg --> Pkg

Loading

---

✨ Key Features

• Automatic scene-by-scene prompt generation: Supports durations of 15s / 30s / 60s and aspect ratios of 16:9 / 9:16 / 1:1.
• One-click clipboard copy: Copy in the exact format for Veo / Sora (pure prompts without ratio/duration) or generic models.
• Multi-provider image generation: ComfyUI (local, no API key required), Gemini, and OpenAI (images/edits) supported.
• Headline / caption overlay: Layout catchphrases onto images with configurable font, position, and style.
• History (Runs): Past generations are safely isolated under runs/<run_id>/ and recallable at any time.
• GUI & CLI: Ships with both a Tauri 2 desktop app and a promo CLI for terminal / scripted automation.

---

🚀 Quick Start

Requirements

• OS: Windows / macOS / Linux (cross-platform)
• Rust: 2024 edition / rust-version 1.85 or later
• Node.js: Stable (LTS recommended; for frontend build)
• Local LLM CLI: Authenticated claude (Claude Code CLI) or Aider available on PATH

Build & Launch (GUI)

# 1. Clone the repository
git clone https://github.com/betyourluck/AppPromoVideo.git
cd AppPromoVideo

# 2. Run Rust workspace tests
cargo test --workspace

# 3. Launch GUI (Tauri 2 + Vue 3)
cd app
npm install
npm run tauri dev

Tip

If you hit a build error caused by sccache, run with RUSTC_WRAPPER= unset.

# Frontend unit tests / type check / build
cd app
npx vitest run
npm run build

# Backend unit tests / lint
cd app/src-tauri
cargo test
cargo clippy

---

🖥️ How to Use the GUI

+-----------------------------------------------------------------------------------------+
| [TopBar] AppPromoVideo                              [Runs] [Settings] [Theme] [Min/Max/X] |
+-----------------------+---------------------------------+-------------------------------+
| 📁 InputPane           | 🎬 ScenePanel                   | 📜 LogPanel                   |
|                       |                                 |                               |
| - Repository selection| - App summary & differentiators | - Real-time progress logs     |
| - UI snapshots        | - Scene list (Prompt & Image)   | - CLI subprocess tracing      |
| - Duration / ratio /  | - Copy (Veo / Sora / All)       | - Errors & warnings           |
|   language            | - Reference image trigger       |                               |
| [ ▶ Analyze → Build   | [ 💾 Export to folder ]         |                               |
|     Scenes ]          |                                 |                               |
+-----------------------+---------------------------------+-------------------------------+

1. Launch & connectivity check: On startup the app automatically checks LLM CLI connectivity/authentication (check_cli).
2. Configure inputs (InputPane):
   • Specify the target repository path.
   • Add UI screenshots (file picker / drag & drop / clipboard paste).
   • Set the video concept, duration (15 / 30 / 60s), aspect ratio (16:9 / 9:16 / 1:1), and copy language.
3. Analyze & build scenes:
   • Click "Analyze → Build Scenes" to run run_pipeline.
   • The right pane (LogPanel) streams subprocess progress in real time.
4. Review & copy prompts (ScenePanel):
   • View the app summary, differentiators, and per-scene prompts.
   • Copy individual scenes or all at once, formatted for the target AI (Veo / Sora / generic).
5. Generate reference images:
   • Optionally run "Generate Reference Images" to create visual reference images (OpenAI / Gemini / ComfyUI).
6. Export artifacts:
   • Use "Open Folder" or "Export to Folder" to save the complete output package.

---

⌨️ CLI (promo) Usage

A promo CLI binary is included for terminal and scripted automation.

# Run analyze → scene composition → image generation → package export in one go
cargo run -p pipeline --bin promo -- run <path/to/repo> --concept "An innovative task management tool"

# Output only the RepoBrief (compressed summary manifest) for a repository (no LLM)
cargo run -p pipeline --bin promo -- brief <path/to/repo>

# Generate additional reference / scene images for an existing promo.json
cargo run -p pipeline --bin promo -- images promo_out/promo.json --images comfy

# Test headline overlay (visual check, no LLM)
cargo run -p pipeline --bin promo -- caption in.png "Noto Sans JP" 0 "Sync instantly." out.png

# Test backdrop + screenshot compositing (visual check, no LLM)
cargo run -p pipeline --bin promo -- compose backdrop.png screenshot.png out.png 16:9

# List available fonts
cargo run -p pipeline --bin promo -- fonts

---

📦 Output Package Layout

Each run is fully isolated under a unique runs/<run_id>/ directory.

promo_out/
└── runs/
    └── 20260913_001234_abc1/
        ├── promo.json             # Complete JSON with scene structure, prompts, and metadata
        ├── scenes.md              # Human-readable Markdown prompt collection
        ├── repobrief.txt          # Compressed codebase context fed to the LLM
        └── images/
            ├── ref_001.png        # Style reference image harmonized with app UI
            ├── scene_001_cut.png  # Composited cut (backdrop + real screenshot with headline)
            └── ...

---

🏗️ Workspace Architecture

Built as a Rust workspace (4 crates) plus a Tauri 2 desktop application.

Crate / Directory	Role	Responsibilities
crates/promo_core	Pure-function core domain	No process/HTTP side effects. Type definitions for RepoBrief / ScenePlan, mechanical JSON Schema generation, rescue of malformed LLM JSON (fenced_json), prompt text generation.
crates/cli_runner	Secure CLI subprocess execution	Launches a local LLM CLI as a subprocess. Safe argv construction (stdin/temp-file transport), scrubbing of auth env vars, reliable forced termination of descendants via Windows Job Objects / Unix pgids.
crates/image_gen	Image generation & compositing engine	Reference image generation via ComfyUI (polling), Gemini, and OpenAI. Dominant-color extraction from UI snapshots (Palette), rounded-corner / drop-shadow compositing, headline caption overlay.
crates/pipeline	Orchestration & CLI	I/O harness wiring the crates together. File collection, LLM interaction, Rust-side inspection loop (auto-regeneration on violation), package export, and the promo CLI binary itself.
app/	Desktop GUI	Tauri 2 + Vue 3 + TypeScript. Calls the backend via #[tauri::command] and receives progress in real time via Tauri Events. Settings and history UI.

---

🎨 Image Generation Provider Support

Provider	Integration	API Key	Status & Notes
ComfyUI	Local HTTP polling (/prompt → /history → /view)	Not required	✅ Verified on real hardware. Fully local, free, and fast.
Gemini	Google GenAI REST API	Required	✅ Verified on real hardware. High fidelity and strong instruction following.
OpenAI	images/edits endpoint	Required	⚠️ Verified on real hardware (stable operation confirmed with a single reference image).

---

⚙️ Configuration & Security

• Separated config storage:
  • UI settings and CLI options: %APPDATA%/jp.outcasts.apppromovideo/settings.json
  • Image-generation API keys: .env in the same directory
• Credential leak prevention:
  • Raw API keys are never exposed to the WebView (frontend); they are held and used only on the backend (Rust side) — the frontend only receives a boolean indicating whether a key is set.
• LLM CLI authentication troubleshooting:
  • If the GUI repeatedly returns 401 (authentication_failed) for the LLM CLI on startup, enable "Use OAuth login" in Settings (suppresses propagation of ANTHROPIC_API_KEY to the child process).
• Known constraints & pitfalls:
  • Additional gotchas and workarounds are collected in failures.md (PowerShell redirect conflicts, gen becoming a reserved keyword in Rust 2024, etc.).

---

📚 Documentation

Detailed design principles and operational rules are maintained in the following ledgers:

• 📘 specs/01_promo_pipeline.md — Spec, phase plan, and grounding limits
• 📐 data_contract.yaml — Types, limits, enums, and invariants (canonical source)
• 🧭 CLAUDE.md — North star, architecture, and development rules
• ⚠️ failures.md — Known pitfalls, workarounds, and incident ledger
• 📜 history.md — Implementation log and history

---

📄 License

Released under the MIT License.