sfn-entity-resolution-dynamodb-cdk: Zero-Lambda entity matching with Step Functions SDK integration - #3278
Open
NithinChandranR-AWS wants to merge 2 commits into
Conversation
…ing pipeline AWS Step Functions orchestrates Entity Resolution matching jobs triggered by S3 uploads via EventBridge. Uses native SDK integration (CallAwsService) to start and poll matching jobs, storing results in DynamoDB. Includes Glue database/table for schema mapping and ML-based entity matching.
… and DependsOn for Entity Resolution role Entity Resolution MatchingWorkflow was created in parallel with the IAM role policy, causing a race condition where the service could not read Glue table metadata. Added explicit DependsOn and expanded Glue permissions (GetTableVersion, GetPartitions, BatchGetPartition).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
AWS Step Functions orchestrates AWS Entity Resolution ML matching jobs using native SDK integrations (zero Lambda). Amazon S3 uploads trigger Amazon EventBridge, which starts the state machine to match customer records and store results in Amazon DynamoDB.
Architecture
Amazon S3 (upload) -> Amazon EventBridge -> AWS Step Functions (CallAwsService SDK integration) -> AWS Entity Resolution (ML matching) -> Amazon DynamoDB (results)
Zero Lambda functions. Step Functions uses native CallAwsService to:
Deployed and Tested
Stack deploys in us-east-1 with 27 resources (108s). Verified: