Update CDC500_states query - #2149
Open
shvngisingh wants to merge 14 commits into
Open
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request refactors the SQL query in scripts/us_cdc/cdc500_state/process.py by introducing Common Table Expressions (CTEs) and updating the source table references to spanner_dc_graph_prod_DEFAULT. The review identifies an opportunity to improve query performance by removing a redundant join in the svo_count CTE and recommends using SAFE_DIVIDE to handle potential division-by-zero errors during the percentage calculation.
shvngisingh
force-pushed
the
cdc500_states_query
branch
from
August 10, 2026 11:37
72c94a9 to
44d036e
Compare
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
…streamline aggregation in CDC500 state query
Contributor
|
Add validation_config.json |
… ACS 5Yr facet join, explicit cohort CASE mapping, modular entrypoint, and add unit test suite
Contributor
Author
Added |
pulkeet-wq
reviewed
Sep 1, 2026
…ME docs, remove default validation rules, use logging.error, and clean up query helpers
…ame error test, and fix senior cohort statvars
- Exclude age-bracketed cancer screening indicators lacking composite Census ACS cohorts (Mammography, Cervical screening, Pap smear, Colorectal screening). - Add QUALIFY deduplication in svo_count for 1:1 population weighting. - Raise RuntimeError on empty query results and write CSV atomically. - Add scoped SQL_VALIDATOR check for cohort vintage freshness in validation_config.json. - Remove duplicate source_files in manifest.json and ensure trailing newlines. - Expand unit tests to verify query constants, error handling, and CLI flags. - Update README.md with rationale, prerequisites, testing, and automation details.
- Add check_deleted_records_percent override (threshold 100) in validation_config.json for TMCF unit/scalingFactor migration. - Remove redundant try/except re-raise logging in run_process to eliminate double tracebacks. - Add unit tests for demographic cohort regex mappings and population-weighted aggregation. - Fix demographic cohort examples in README.md to reflect excluded cancer screening indicators.
… validation_config.json
…plication in process.py and use TRY_CAST in validation_config.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR is for updating query in process.py from using the old resolved StatVarObservation table to the new TimeSeries and Observation schemas tables of latest spanner .