Skip to content

Commit 6bdfcee

Browse files
amar-pythonamar-python
committed
Initialize repository and add CSV loader/T&E loader fixes with cleanup and safe duplicate hashing
0 parents  commit 6bdfcee

306 files changed

Lines changed: 35934 additions & 0 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.dockerignore‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
.git
2+
.gitignore
3+
**/.terraform
4+
**/.terraform.lock.hcl
5+
**/*.tfstate
6+
**/*.tfstate.*
7+
.terraform.tfstate.lock.info
8+
.remember
9+
.abacusai
10+
_norton_
11+
.github
12+
*.md

‎.gitattributes‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
# Pin line endings per file type.
2+
# This file overrides core.autocrlf — without it, Git for Windows would
3+
# silently translate LF → CRLF on checkout of shell scripts, which break
4+
# inside Linux containers with: syntax error near unexpected token $'do\r''
5+
#
6+
# After adding this file you may need to re-normalise existing files:
7+
# git add --renormalize .
8+
# git status # shows files Git will re-write
9+
# git commit -m "Normalise line endings via .gitattributes"
10+
11+
# Default: text files use LF (preferred for cross-platform)
12+
* text=auto eol=lf
13+
14+
# Shell scripts — MUST be LF (Linux/Mac/Cloud Shell execute these)
15+
*.sh text eol=lf
16+
*.bash text eol=lf
17+
18+
# PowerShell — keep CRLF (Windows-native)
19+
*.ps1 text eol=crlf
20+
*.psm1 text eol=crlf
21+
*.psd1 text eol=crlf
22+
23+
# Windows batch — CRLF
24+
*.bat text eol=crlf
25+
*.cmd text eol=crlf
26+
27+
# Web / config — LF
28+
*.py text eol=lf
29+
*.sql text eol=lf
30+
*.yml text eol=lf
31+
*.yaml text eol=lf
32+
*.json text eol=lf
33+
*.toml text eol=lf
34+
*.md text eol=lf
35+
*.txt text eol=lf
36+
*.tf text eol=lf
37+
*.hcl text eol=lf
38+
*.tfvars text eol=lf
39+
*.dockerignore text eol=lf
40+
*.gitignore text eol=lf
41+
Dockerfile text eol=lf
42+
Makefile text eol=lf
43+
44+
# Binary files — never translate
45+
*.png binary
46+
*.jpg binary
47+
*.jpeg binary
48+
*.gif binary
49+
*.ico binary
50+
*.pdf binary
51+
*.zip binary
52+
*.skill binary
53+
*.tar binary
54+
*.gz binary
55+
*.xlsx binary
56+
*.docx binary
57+
*.pptx binary

‎.github/dependabot.yml‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
version: 2
2+
updates:
3+
# Python dev dependencies (requirements-dev.txt at repo root)
4+
- package-ecosystem: "pip"
5+
directory: "/"
6+
schedule:
7+
interval: "weekly"
8+
open-pull-requests-limit: 5
9+
labels:
10+
- "dependencies"
11+
12+
# GitHub Actions used by the workflows
13+
- package-ecosystem: "github-actions"
14+
directory: "/"
15+
schedule:
16+
interval: "weekly"
17+
open-pull-requests-limit: 5
18+
labels:
19+
- "dependencies"

‎.github/workflows/python-validator-tests.yml‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
name: Python Validator Tests
2+
3+
on:
4+
push:
5+
branches:
6+
- main
7+
- master
8+
pull_request:
9+
10+
jobs:
11+
python-validator-tests:
12+
runs-on: windows-latest
13+
steps:
14+
- name: Checkout
15+
uses: actions/checkout@v4
16+
17+
- name: Setup Python
18+
uses: actions/setup-python@v7
19+
with:
20+
python-version: '3.11'
21+
22+
- name: Install dev dependencies
23+
# api/requirements.txt is needed too: tests/test_api.py imports FastAPI
24+
# at module level, so without it pytest collection fails.
25+
run: pip install -r requirements-dev.txt -r api/requirements.txt
26+
27+
# Database-backed tests (integration/e2e/parity) run in the Linux
28+
# integration-postgres job of quality-gate.yml, which has a PostgreSQL
29+
# service. Service containers are Linux-only, so this Windows job covers
30+
# the database-free markers. Missing prerequisites FAIL rather than skip,
31+
# so this job must not collect the database-backed markers.
32+
- name: Run validator tests (PowerShell)
33+
shell: pwsh
34+
run: powershell -NoProfile -ExecutionPolicy Bypass -File "tests/run_python_tests.ps1"
35+
36+
# Final block accounts for every test, including skips (with reasons)
37+
# and tests deselected by the marker filter. --strict fails on any skip.
38+
- name: Final result with skip accounting
39+
shell: pwsh
40+
run: |
41+
python scripts/test_report.py `
42+
--markers "unit or regression or security or snapshot" --strict

‎.github/workflows/quality-gate.yml‎

Lines changed: 155 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,155 @@
1+
name: Quality Gate
2+
3+
on:
4+
push:
5+
branches:
6+
- main
7+
- master
8+
pull_request:
9+
10+
jobs:
11+
# Fast, deterministic checks — no database, no network beyond pip.
12+
free-tier:
13+
name: free-tier (unit, lint, health, evals-p)
14+
runs-on: ubuntu-latest
15+
steps:
16+
- name: Checkout
17+
uses: actions/checkout@v4
18+
19+
- name: Setup Python
20+
uses: actions/setup-python@v7
21+
with:
22+
python-version: '3.11'
23+
24+
- name: Install dev dependencies
25+
# api/requirements.txt is needed too: tests/test_api.py imports FastAPI
26+
# at module level, so without it pytest collection fails.
27+
run: pip install -r requirements-dev.txt -r api/requirements.txt
28+
29+
- name: Unit / regression / security / snapshot tests — final result
30+
run: |
31+
python3 scripts/test_report.py \
32+
--markers "unit or regression or security or snapshot" --strict
33+
34+
- name: Lint (flake8 + bandit)
35+
run: make lint
36+
37+
- name: Component health check
38+
run: make health
39+
40+
- name: Evals — Tier P (offline validator scenarios)
41+
run: python3 evals/runner.py --tiers p --verbose
42+
43+
# Reports (incl. the VCRM gap report) are written to evals/reports/ and
44+
# would otherwise vanish with the runner — keep them for auditability.
45+
- name: Upload eval reports
46+
if: always()
47+
uses: actions/upload-artifact@v4
48+
with:
49+
name: eval-reports-free-tier
50+
path: evals/reports/
51+
if-no-files-found: ignore
52+
retention-days: 30
53+
54+
# Full integration surface: Tier I deploys env_dev.sql twice (idempotency),
55+
# Tier S runs the 142-assertion SQL suite, then the PG-gated pytest tiers run.
56+
integration-postgres:
57+
name: integration (postgres service)
58+
runs-on: ubuntu-latest
59+
services:
60+
postgres:
61+
image: postgres:16-alpine
62+
env:
63+
POSTGRES_PASSWORD: postgres
64+
ports:
65+
- 5432:5432
66+
options: >-
67+
--health-cmd "pg_isready -U postgres"
68+
--health-interval 5s
69+
--health-timeout 5s
70+
--health-retries 10
71+
env:
72+
PGHOST: localhost
73+
PGPORT: "5432"
74+
PGUSER: postgres
75+
PGPASSWORD: postgres
76+
PGDATABASE: postgres
77+
steps:
78+
- name: Checkout
79+
uses: actions/checkout@v4
80+
81+
- name: Setup Python
82+
uses: actions/setup-python@v7
83+
with:
84+
python-version: '3.11'
85+
86+
- name: Install dev dependencies
87+
# api/requirements.txt is needed too: tests/test_api.py imports FastAPI
88+
# at module level, so without it pytest collection fails.
89+
run: pip install -r requirements-dev.txt -r api/requirements.txt
90+
91+
- name: Install PostgreSQL client
92+
run: sudo apt-get update && sudo apt-get install -y --no-install-recommends postgresql-client
93+
94+
- name: Write CI database config
95+
run: |
96+
cat > build/config.local.env <<'EOF'
97+
DB_ENGINE="postgresql"
98+
PG_HOST="localhost"
99+
PG_PORT="5432"
100+
PG_SUPERUSER="postgres"
101+
PG_SUPERUSER_PASSWORD="postgres"
102+
PG_DB_DEV="te_mgmt_dev"
103+
PG_SCHEMA_DEV="te_dev"
104+
PG_DB_TEST="te_mgmt_test"
105+
PG_SCHEMA_TEST="te_test"
106+
PG_DB_STAGING="te_mgmt_staging"
107+
PG_SCHEMA_STAGING="te_staging"
108+
PG_DB_PROD="te_mgmt_prod"
109+
PG_SCHEMA_PROD="te_prod"
110+
EOF
111+
112+
# env_*.sql scripts do not create their databases — deploy_all.sh does that
113+
# before invoking them (see build/te_core_schema.sql:41). Mirror it here.
114+
# Only *.example.sql are committed (concrete env_<env>.sql are gitignored),
115+
# so materialise them before anything that deploys an environment.
116+
- name: Materialise environment launchers from templates
117+
run: |
118+
for env in dev test staging prod; do
119+
cp "build/environments/env_${env}.example.sql" \
120+
"build/environments/env_${env}.sql"
121+
done
122+
ls -1 build/environments/
123+
124+
- name: Create environment databases
125+
run: |
126+
for db in te_mgmt_dev te_mgmt_test te_mgmt_staging te_mgmt_prod; do
127+
if [ -z "$(psql -d postgres -tA -c "SELECT 1 FROM pg_database WHERE datname = '$db'")" ]; then
128+
psql -v ON_ERROR_STOP=1 -d postgres -c \
129+
"CREATE DATABASE \"$db\" WITH OWNER = postgres ENCODING = 'UTF8' TEMPLATE = template0 CONNECTION LIMIT = -1"
130+
fi
131+
done
132+
133+
- name: Evals — Tiers P, I, S
134+
run: python3 evals/runner.py --tiers p,i,s --verbose
135+
136+
- name: Deploy all environments (for cross-env parity)
137+
run: |
138+
for env in dev test staging prod; do
139+
bash build/deploy_all.sh "$env"
140+
done
141+
142+
# Prints a final block accounting for every test: PASSED / FAILED /
143+
# ERROR / SKIPPED (with reasons) / NOT RUN (deselected). --strict fails
144+
# the build on any skip.
145+
- name: Full test suite — final result with skip accounting
146+
run: python3 scripts/test_report.py --strict
147+
148+
- name: Upload eval reports
149+
if: always()
150+
uses: actions/upload-artifact@v4
151+
with:
152+
name: eval-reports-integration
153+
path: evals/reports/
154+
if-no-files-found: ignore
155+
retention-days: 30

‎.gitignore‎

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
# Secrets & environment-specific config (NEVER commit)
2+
config.local.env
3+
*.local.env
4+
build/environments/env_dev.sql
5+
build/environments/env_test.sql
6+
build/environments/env_staging.sql
7+
build/environments/env_prod.sql
8+
!build/environments/*.example.sql
9+
*.tfvars
10+
!*.tfvars.example
11+
*.pgpass
12+
.env
13+
.env.*
14+
!.env.example
15+
16+
# Terraform
17+
**/.terraform/
18+
*.tfstate
19+
*.tfstate.*
20+
terraform.tfvars
21+
22+
# Eval / test runtime output
23+
evals/reports/
24+
**/csv/logs/
25+
26+
# Tool / session / editor folders
27+
.abacusai/
28+
.claude/
29+
.cursor/
30+
.vscode/
31+
.idea/
32+
33+
# Python
34+
__pycache__/
35+
*.py[cod]
36+
.pytest_cache/
37+
.venv/
38+
venv/
39+
40+
# OS
41+
.DS_Store
42+
Thumbs.db
43+
44+
# Exploratory / not-yet-wired folders
45+
_norton_/
46+
extend to Oracle dbs/
47+
48+
# Runtime test artifacts (regenerate with scripts/test_report.py)
49+
tests/snapshots/
50+
tfplan
51+
*.tfplan
52+
terraform-provider-*.log
53+
54+
*.zip

‎.markdownlint.json‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
{
2+
"default": true,
3+
"MD013": false,
4+
"MD060": false,
5+
"MD024": { "siblings_only": true }
6+
}

‎.markdownlintignore‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
node_modules/
2+
evals/reports/

0 commit comments

Comments
 (0)