Added validation configuration for EPA_airqualityindex

[shourya116]

Added validation configuration for EPA_AirQualityIndex. Job ran successfully.

Files updated are : manifest.json, validation_config.json, EPA_AQI.mcf

GCS Bucket

Cloud Batch Job

[google-cla]

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

[gemini-code-assist]

Code Review

This pull request introduces golden data validation for the US EPA Air Quality Index script. It adds golden observation and summary report CSV files, configures a new validation_config.json with validation rules (such as checking deleted records percentage and verifying output data against the golden datasets), and updates the manifest to include these new files and validation configurations while adjusting resource limits. I have no feedback to provide as there are no review comments.

[abhishekjaisw]

@shourya116 Please reply to comments suggeted by pulkeet . also take reference from #2193 and add details like that .

[abhishekjaisw]

Code Review Summary (LGTM with 1 PR Description Hygiene Note)

Reviewed HEAD commit 20f38776c7ea193a04176dc7114cd7ac863eec97:

• Code Changes (EPA_AQI.mcf, manifest.json, validation_config.json): LGTM.
  • check_deleted_records_percent (0.1% threshold) appropriately handles minor upstream historical monitor retractions (0.000209%).
  • check_max_date_consistent and dynamic check_max_date_freshness (max_year >= (EXTRACT(YEAR FROM CURRENT_DATE) - 1)) verified via DuckDB against summary_report.csv (MaxDate: 2025-12-31, 0 failing rows).
  • AirQualityIndex_AirPollutant schema node properly defined in EPA_AQI.mcf and linked via "node_mcf": "EPA_AQI.mcf".
  • Cloud Batch disk resource limit right-sized to 300 GB.

Action Items Before Merge:

1. [P2] PR Description Hygiene: Please edit the PR description above to remove the internal https://pantheon.corp.google.com/... URLs and replace them with gs://datcom-import-test/... URIs (or https://console.cloud.google.com/... links without internal corp parameters).
2. Google CLA: Please resolve the failing cla/google check so the PR can be merged.

[abhishekjaisw]

LGTM

[shourya116]

comment addressed

[shourya116]

resolved comments

[rohitkumarbhagat]

/gcbrun