Skip to content

PROJECT_LAYOUT.md

Latest commit

 

History

History
227 lines (166 loc) · 8.26 KB

PROJECT_LAYOUT.md

File metadata and controls

227 lines (166 loc) · 8.26 KB
Raw

Project layout — directory review

This document describes what each folder is for, who uses it (web user, CLI user, agent platform), and where new files should go.

Overview

excel-python/
├── scripts/          ← CLI entry points (thin wrappers, run by humans)
├── src/              ← Python library (imported by scripts, web, tests)
├── agents/           ← Agent roles & platform guides (multi-platform, committed)
├── specs/            ← Per-workbook deliverables (Markdown, committed)
├── extractions/      ← Per-workbook JSON input (local, gitignored)
├── web/              ← FastAPI app + static UI (browser users)
├── docs/             ← Methodology & reference (committed)
├── data/             ← Static reference data (formula locales, etc.)
├── tests/            ← pytest
├── sample/           ← Local test workbooks (gitignored)

Data flow

flowchart TB
  subgraph input [Input — local]
    XLSM[".xlsm workbook"]
    JSON["extractions/*_extraction.json"]
  end

  subgraph collect [Collection]
    WEB["web/ — 3-step UI"]
    CLI["scripts/extract.py"]
  end

  subgraph agents_layer [Agents]
    DEF["agents/definitions/ — roles"]
    PLAT["agents/platforms/ — Cursor, CLI, …"]
    INST["specs/*_AGENT_PROMPT.md — instance prompts"]
  end

  subgraph output [Output — committed]
    LOGIC["specs/*_LOGIC_ANALYSIS.md"]
    SPEC["specs/*_FUNCTIONAL_SPEC.md"]
  end

  XLSM --> WEB
  XLSM --> CLI
  CLI --> JSON
  WEB --> JSON
  JSON --> INST
  DEF --> INST
  PLAT --> INST
  INST --> LOGIC
  INST --> SPEC
Loading

scripts/ — Python CLI entry points

Purpose: Commands a technical user runs in a terminal. Each script should stay thin: parse args, call src/, write files.

Script Phase Writes to
extract.py 1 — Extract extractions/<name>_extraction.json
generate_logic_agent.py 2 — Logic specs/<name>_LOGIC_ANALYSIS.md, specs/<name>_LOGIC_AGENT_PROMPT.md
generate_spec_agent.py 2b — Functional specs/<name>_DRAFT_SPEC.md, specs/<name>_AGENT_PROMPT.md
generate_spec.py 2b — LLM specs/<name>_FUNCTIONAL_SPEC.md

Do not put: business logic (belongs in src/), agent role text (belongs in agents/definitions/), or generated per-workbook specs (belongs in specs/).

Run from repo root:

python scripts/extract.py sample/my_workbook.xlsm
python scripts/generate_logic_agent.py extractions/my_workbook_extraction.json

src/ — Python library

Purpose: Reusable code: extraction, sheet model, prompt rendering, LLM client, future migrated features (f01_*, …).

Area Modules Used by
Extraction workbook_extract.py, extraction.py, cell_labels.py scripts/extract.py, web/server.py
Annotation sheet_model.py, cell_annotations.py Web annotate API
Logic analysis logic_analyst.py scripts/generate_logic_agent.py, web API
Functional spec spec_generator.py, spec_skeleton.py, agent_spec.py Spec scripts, optional LLM
Onboarding business_context.py Web + spec context
LLM llm/ generate_spec.py (optional)
Legacy UI ui.py, context.py Streamlit web/app.py only

Do not put: CLI argparse main blocks, HTML, or committed Markdown outputs.

agents/ — Agent definitions (multi-platform)

Purpose: Describe who the agents are and how to run them on different platforms. This folder is the long-term home for agent configuration, independent of Cursor, OpenAI, or the web UI.

agents/
├── definitions/          # Platform-agnostic roles (source of truth for prompts)
│   ├── logic_analyst.md
│   └── spec_writer.md
└── platforms/            # How to invoke agents per environment
    ├── cursor.md
    ├── cli.md
    └── web.md
Subfolder Contents Committed?
definitions/ Role, mission, constraints — same meaning on every platform Yes
platforms/ Invocation steps (Cursor @file, CLI script, future API) Yes

Generated instance prompts (per workbook)

Workbook-specific agent prompts (with JSON paths, cell samples, task sections) are generated artefacts, not generic definitions. They currently live in:

  • specs/<name>_LOGIC_AGENT_PROMPT.md
  • specs/<name>_AGENT_PROMPT.md

Future option: specs/<name>/ subfolder per workbook if the file count grows.

Python today: src/agent_definitions.py loads roles from agents/definitions/*.md.
src/logic_analyst.py and src/agent_spec.py render instance prompts into specs/*_AGENT_PROMPT.md.

specs/ — Per-workbook deliverables

Purpose: Human-readable outputs for migration — validated or to be validated. Committed to git (unlike extractions/).

File pattern Audience Produced by
*_LOGIC_ANALYSIS.md Dev + owner Skeleton: CLI or web /review; completed: Cursor Agent
*_LOGIC_AGENT_PROMPT.md Cursor / future agents scripts/generate_logic_agent.py
*_DRAFT_SPEC.md Owner + agent scripts/generate_spec_agent.py
*_AGENT_PROMPT.md Cursor (functional spec) scripts/generate_spec_agent.py
*_FUNCTIONAL_SPEC.md Business + dev LLM or Cursor Agent

Still the right place for migration artefacts referenced in Phase 2–4 of MIGRATION_GUIDE.md.

extractions/ — Technical JSON (local)

Purpose: Machine-readable inventory from Excel. Regeneratable from the workbook.

Gitignored. CLI writes here; web keeps enriched JSON in browser session until saved manually or via a future export API.

web/ — Browser application

Purpose: FastAPI server + static HTML/JS for non-technical users.

Path Role
server.py API + routes
vocal-onboarding.html Step 1 — file + context
sheet-annotate.html Step 2 — describe sheet
review.html Step 3 — wishes + logic analysis
migration-session.js Browser session
workflow-ui.js Breadcrumb / steps
app.py Legacy Streamlit (not used in main flow)

Agents do not run inside the browser. The web prepares data; agents run in Cursor, CLI, or future cloud runtimes (see agents/platforms/).

docs/ — Methodology (3 files)

File Purpose
PROJECT_LAYOUT.md This file — folders, data flow, where to add code
MIGRATION_GUIDE.md 8-phase process (extraction → decommission); Phase 3 prioritization template
VBA_REFERENCE.md VBA → Python tables — use during Phase 5 coding

tests/, data/, sample/

Folder Purpose
tests/ pytest — mirror src/ and API
data/ Static JSON (e.g. French→English formula names)
sample/ Local .xlsm for manual testing (gitignored)

Two user profiles — where they work

Profile Primary folders Typical path
Web user web/ → download .md Upload → annotate → review
CLI user scripts/, extractions/, specs/, agents/platforms/cursor.md extract → generate_logic_agent → Cursor

Checklist — where to add something new

I want to add… Put it in…
New CLI command scripts/my_command.py → call src/
Extraction / parsing logic src/
New agent role agents/definitions/ + renderer in src/
Cursor / Copilot / API runbook agents/platforms/
Workbook logic analysis (final) specs/<name>_LOGIC_ANALYSIS.md
Workbook JSON snapshot extractions/ (local)
Web page or API route web/
Migration phase doc docs/
Migrated Python feature src/fXX_*.py + tests/

Future platforms (agents)

When adding a new runtime (e.g. GitHub Copilot, cloud agent, cursor-sdk):

  1. Add agents/platforms/<platform>.md with inputs, outputs, and file paths.
  2. Reuse agents/definitions/ — do not duplicate roles in platform docs.
  3. Instance prompts stay tied to workbook name under specs/.
  4. Optional: scripts/run_agent.py as a generic launcher once multiple backends exist.