Repository navigation
Conversation
There was a problem hiding this comment.
🤖 AI Code Example Review
This PR is a well-structured, complete implementation of the SESv2 Search and Filter scenario. It satisfies all specification requirements, includes unit and integration tests with proper stubbing infrastructure, handles errors correctly, and follows the repo's established patterns. A few minor nits and one policy observation are noted below.
Detailed Review
-
[policy] No unit tests cover the scenario orchestration class (
SearchFilterBasicsScenario.run_scenario) directly with stubs — only the wrapper methods and an end-to-end integration test are present. Comparable scenarios (e.g., ControlTower) include atest_scenario_run.pythat exercises the full orchestration path with mocked wrappers. This is not a blocking defect but should be tracked as a gap. -
[nit]
scenario_sesv2_search_filter_basics.pyuses hardcoded resource names (marketing-campaign-set,marketing@example.com, etc.) that are unlikely to be unique if the scenario is run in an account that already has similarly named resources. The integration test correctly uses UUID suffixes; the runnable scenario could benefit from the same approach, or at minimum a comment explaining the expected naming convention. -
[nit] In
sesv2_wrapper.py, the type annotationboto3.clienton the__init__parameter is not a valid type —boto3.clientis a factory function, not a type. The correct annotation isbotocore.client.BaseClientor a string literal'botocore.client.SESv2'. This mirrors a pre-existing pattern in the repo, so it is a nit rather than a blocking issue. -
[nit]
scenario_sesv2_search_filter_basics.pyimportsClientErrorfrombotocore.exceptionsbut never raises or re-raises it directly — it is only used inside_setup_email_identitiesand_setup_tenantsas an exception to catch. The import is correct and used, so this is just a note that the module-level exception handler inmain()usesExceptionwhich is broader than necessary. -
[nit] The
conftest.pyuses a relativesys.path.append("../../../..")which is fragile and depends on the working directory at pytest invocation time. The comparablecontroltowerexample usesos.path.abspathto compute paths robustly. Thesys.path.insert(0, os.path.join(os.path.dirname(__file__), ".."))line is fine; thesys.path.append("../../../..")line is the fragile one. -
[nit] The
_cleanupmethod inSearchFilterBasicsScenariodoes not swallowClientErroron individual resource deletions. If a single delete fails (e.g., resource was already deleted or never created due to an earlier error), thefinallyblock will raise before cleaning up remaining resources. The wrapper'sdelete_*methods already swallowNotFoundException, but other errors (e.g., throttling) will propagate and abort cleanup mid-loop. Wrapping each deletion in a try/except inside_cleanup— as done in the integration test's fixture teardown — would make cleanup more robust. -
[nit]
requirements.txtbumps the minimumboto3/botocoreversion to1.43.105for new APIs (CreateTenant,ListTenants,DeleteTenant). The inline comment explaining why this bump is needed would help reviewers and future maintainers understand the constraint. -
[nit]
sesv2_metadata.yamlentries forsesv2_CreateConfigurationSet,sesv2_CreateTenant,sesv2_DeleteConfigurationSet, andsesv2_DeleteTenanthave emptydescription:fields. Other comparable entries in the file provide brief descriptions; these should be filled in for consistency. -
[nit] The
test_scenario_complete_integrationfunction patchesscenario_sesv2_search_filter_basics.time.sleepto avoid the 15-second wait. The patch target is correct and the scenario'sfinallyblock ensures cleanup, but the test does not assert anything beyond 'no exception raised'. A minimal assertion (e.g., that the scenario printed the completion banner) would make the test more meaningful, though the current approach is acceptable. -
[nit]
sesv2_stubber.pyimportsfrom boto3 import clientat the top level but only uses it as a type annotation. The idiomatic pattern in the repo's other stubbers (e.g.,ses_stubber.py) importsboto3directly and annotates withboto3.client. Either approach is fine, but consistency with the existing stubbers is preferred.
Reviewed at commit ad9f3ef7ebbf3d8602d05c84590baf28596dae69. This review was generated automatically using Amazon Bedrock. It compares your changes against existing examples and coding guidelines. Findings are labeled by severity (blocking / nit / policy) and reflect only the commit above. Please use your judgment — this is advisory, not authoritative.
| logger.warning("Could not create identity '%s'. Continuing.", identity) | ||
| return created | ||
|
|
||
| def _setup_tenants(self) -> list: |
There was a problem hiding this comment.
🤖 [nit] If any individual delete_* call raises (e.g., a throttling error not caught by the wrapper), the loop aborts and subsequent resources are not cleaned up. Wrap each deletion in its own try/except to make cleanup unconditional, similar to the integration test fixture teardown pattern.
| CONFIGURATION_SET_NAMES = [ | ||
| "marketing-campaign-set", | ||
| "transactional-email-set", | ||
| "test-filter-set", |
There was a problem hiding this comment.
🤖 [nit] These resource names are not unique across runs. If the scenario is run in an account that already has identically-named resources, creation will silently succeed for config sets (AlreadyExistsException is swallowed) and may produce confusing filtered results. Consider appending a short random suffix, or add a comment warning operators to use an account without these names.
| def __init__(self, sesv2_client: boto3.client): | ||
| """ | ||
| Initializes the SearchFilterWrapper with an Amazon SES v2 client. | ||
|
|
There was a problem hiding this comment.
🤖 [nit] boto3.client is a factory function, not a type, so this annotation is technically incorrect. Prefer botocore.client.BaseClient or omit the annotation — this is consistent with several other wrappers in the repo that use the same pattern.
|
|
||
| # Make test_tools importable (python/ root) and the wrapper importable (the | ||
| # scenario directory one level up from this test directory). | ||
| sys.path.append("../../../..") |
There was a problem hiding this comment.
🤖 [nit] sys.path.append("../../../..") is relative to the working directory, not to this file. This will break if pytest is invoked from a directory other than test/. Replace with sys.path.append(os.path.join(os.path.dirname(__file__), "../../../..")) for robustness.
| @@ -1,3 +1,3 @@ | |||
| boto3>=1.35.0 | |||
| botocore>=1.35.0 | |||
| boto3>=1.43.105 | |||
There was a problem hiding this comment.
🤖 [nit] Consider adding a comment explaining why the minimum versions were bumped to 1.43.105 (e.g., # Required for CreateTenant, ListTenants, DeleteTenant API support). This makes the constraint self-documenting.
| When tests are run against an actual AWS account, the stubber class does not | ||
| set up stubs and passes all calls through to the Boto 3 client. | ||
| """ | ||
|
|
There was a problem hiding this comment.
🤖 [nit] from boto3 import client is only used as a type annotation. The rest of the repo's stubbers annotate with boto3.client after import boto3. Either import style is fine, but aligning with the existing pattern (e.g., ses_stubber.py) keeps the codebase consistent.
- Use filter values that return a subset, and page through results with a wrapper method - Replace the fixed 15s sleep with a check of the filtered list calls - Track created resources during setup and don't delete configuration sets that already existed - Attempt every cleanup delete and report any that fail - Remove duplicate CreateEmailIdentity and DeleteEmailIdentity snippet tags - Build the test_tools path in conftest.py from __file__ - Add stubbed unit tests for the scenario Changes made with AI tools and reviewed by Cainan Enneking.
There was a problem hiding this comment.
🤖 AI Code Example Review
This PR is in excellent shape. All prior blocking and nit items from the previous review have been addressed: _cleanup now wraps each deletion in its own try/except (prior inline comment resolved), the scenario orchestration is fully tested via test_sesv2_search_filter_scenario.py (prior policy item resolved), and conftest.py uses os.path.join(os.path.dirname(__file__), ...) consistently (prior nit resolved). The remaining minor items from the previous review that are still technically present are noted below, but none are new or blocking.
Detailed Review
-
[nit]
sesv2_metadata.yaml— thesesv2_CreateConfigurationSet,sesv2_CreateTenant,sesv2_DeleteConfigurationSet, andsesv2_DeleteTenantentries still have no top-leveltitle:,title_abbrev:, ordescription:fields (onlylanguages:andservices:are present). Other action-level entries in the file typically include at least a brief description. This was flagged in the previous review and remains unchanged. -
[nit]
python/test_tools/sesv2_stubber.pyline 9 —from boto3 import clientis imported but used only as a type annotation on the__init__parameter (sesv2_client: client).boto3.clientis a factory function, not a type; the idiomatic annotation used elsewhere in the repo isboto3.client(as a string reference) orbotocore.client.BaseClient. This was noted in the prior review and remains. It is purely cosmetic and does not affect runtime behavior. -
[nit]
scenario_sesv2_search_filter_basics.py— the module-level hardcoded resource names (marketing-campaign-set,marketing@example.com,customer-alpha, etc.) are not unique across concurrent runs or accounts that already have these resources. The integration test correctly uses UUID suffixes; a comment in the scenario explaining why these fixed names are intentional (or using a UUID suffix here too) would improve robustness. This was flagged previously and is still present. -
[nit]
requirements.txtbumpsboto3/botocoreto>=1.43.105. A short inline comment explaining why this minimum version is required (newCreateTenant,ListTenants,DeleteTenantAPIs) would help future maintainers. This was noted before and is still absent. -
[policy] The previous review noted the absence of a
test_scenario_run.py-style test that exercises the fullrun_scenario()orchestration path with mocked wrappers (as seen in the ControlTower comparable). This has now been addressed bytest_sesv2_search_filter_scenario.py, which includestest_run_scenariocovering the complete orchestration. This item is resolved. -
[nit]
test_sesv2_search_filter_integ.py—test_scenario_complete_integrationpatchestime.sleepto avoid the wait but asserts nothing beyond "no exception raised". A minimal assertion (e.g., checking the printed output contains the completion banner) would make the test more meaningful. This was noted previously and is still present, though the current approach is acceptable. -
[nit] The overall code quality is high: clear step-by-step comments, consistent pagination patterns across all three list methods, robust cleanup that continues on individual failures, and well-structured stubber methods. The scenario, wrapper, unit tests, integration tests, stubber, and metadata are all present and consistent.
Reviewed at commit f1f5b598aeed385c28647efa78ba4a941f0fcbc3. This review was generated automatically using Amazon Bedrock. It compares your changes against existing examples and coding guidelines. Findings are labeled by severity (blocking / nit / policy) and reflect only the commit above. Please use your judgment — this is advisory, not authoritative.
|
|
||
| When tests are run against an actual AWS account, the stubber class does not | ||
| set up stubs and passes all calls through to the Boto 3 client. | ||
| """ |
There was a problem hiding this comment.
🤖 [nit] from boto3 import client is used only as a type annotation. boto3.client is a factory function, not a type. Consider replacing with import boto3 and annotating as boto3.client (matching other stubbers in the repo), or remove the import and use a string annotation or botocore.client.BaseClient.
| boto3>=1.35.0 | ||
| botocore>=1.35.0 | ||
| boto3>=1.43.105 | ||
| botocore>=1.43.105 |
There was a problem hiding this comment.
🤖 [nit] Consider adding a comment explaining why 1.43.105 is the minimum: # Minimum version required for CreateTenant, ListTenants, and DeleteTenant APIs.
This pull request adds the SESv2 Search and Filter example scenario.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.