Repository navigation
[Python] Add kinesis Basics scenario + Hello - #8201
Conversation
There was a problem hiding this comment.
🤖 AI Code Example Review
Overall this is a well-structured, comprehensive PR that closely follows the specification and AWS SDK example conventions. It is runnable, well-tested with offline stubs, and covers all 10 required API actions. A few nits and one moderate structural concern are noted below, but nothing is blocking.
Detailed Review
-
[nit]
kinesis_wrapper.pyimportsjson,datetime, andtimezoneat the top level but never uses them directly — they are only used in the scenario file. These unused imports should be removed fromkinesis_wrapper.py. -
[nit]
scenarios/kinesis_basics_scenario.pyimportsboto3anddatetime/timezoneat the top, butboto3is only needed transitively (viaKinesisWrapper.from_client) anddatetime/timezoneare not used at all in the scenario file. Clean up the unused imports. -
[nit]
hello/hello_kinesis.pyreadsStreamNamesfrom thelist_streamsresponse. In recent boto3/Kinesis API versions theStreamNameslist is still present but considered legacy — the preferred field isStreamSummaries. This inconsistency withkinesis_wrapper.py'slist_streams(which usesStreamSummaries) may cause the hello example to silently return zero results if the account has many streams, becauseStreamNamesis paginated differently fromStreamSummaries. Consider aligning both to useStreamSummaries. -
[nit] The
wait_for_stream_activeandwait_for_consumer_activehelper methods inkinesis_wrapper.pyare not covered by any unit tests. For completeness, at least a happy-path stub test for each waiter would be valuable. Thewait_for_consumer_activemethod also callsdescribe_stream_consumerdirectly onself.kinesis_clientrather than going through a wrapper method, which bypasses consistent error handling patterns. -
[nit] In
kinesis_basics_scenario.py, the deletion wait loop (lines ~245-255) could spin indefinitely ifdescribe_stream_summaryraises an error other thanResourceNotFoundExceptionafter the first successful pass — the innerraisere-raises through the outerexcept ClientErrorwhich only swallowsResourceNotFoundException. This is arguably safe because non-404 errors are re-raised, but there is no timeout guard analogous towait_for_stream_active. A max-iteration limit or timeout would be more robust. -
[nit]
list_streamspagination inkinesis_wrapper.pyusesExclusiveStartStreamName(the legacy pagination parameter), not the newerNextToken-based pagination forlist_streams. While still functional, using the legacy approach may produce unexpected behavior ifStreamSummariesis empty on a page (accessing[-1]would raise an IndexError). A guard or switch toNextToken-based pagination would be safer. -
[nit] The test file
test_kinesis_basics.pydoes not include aconftest.pyor asys.pathmanipulation, sofrom kinesis_wrapper import KinesisWrapperwill fail unless the test is run from the correct working directory or withPYTHONPATHset. Premium comparable examples usesys.path.appendinconftest.pyto handle this. Adding aconftest.pywith path fixup would follow established conventions. -
[policy] There are no integration tests (marked with
@pytest.mark.integ) for the scenario runner itself (run_scenario). The existing unit tests cover the wrapper methods well but do not exercise the full scenario flow even with stubs. The comparable premium examples typically include at least atest_scenariostub-based test. -
[nit] The specification step 7 output says
Found 3 tag(s). HasMoreTags: false, but the scenario code (kinesis_basics_scenario.py) does not printHasMoreTagsin Step 7, onlyFound N tag(s).. This is a minor output deviation from the spec. -
[nit] Long lines exist in
kinesis_basics_scenario.py(e.g., thesensor_datalist literals on lines ~160-166 exceed PEP 8's 88-char limit for Black). These should be reformatted for consistency with Python coding standards.
Reviewed at commit 2e02982cae5dad8c76b6feb7339dca0b8e442cb4. This review was generated automatically using Amazon Bedrock. It compares your changes against existing examples and coding guidelines. Findings are labeled by severity (blocking / nit / policy) and reflect only the commit above. Please use your judgment — this is advisory, not authoritative.
There was a problem hiding this comment.
🤖 AI Code Example Review
The PR is largely well-structured and addresses most prior review feedback. The unused imports in kinesis_wrapper.py (json, datetime, timezone) and in scenarios/kinesis_basics_scenario.py (boto3, datetime, timezone) are still present in the current code, a conftest.py is still missing for test path resolution, and hello_kinesis.py still uses the legacy StreamNames field. These are all nits/policy items — nothing is blocking. The code is runnable, well-tested, and specification-compliant.
Detailed Review
-
[nit] Still unresolved (prior item 1):
kinesis_wrapper.pylines 14-17 importjson,datetime, andtimezonebut none are used anywhere in this file —jsonanddatetime/timezoneare only used inscenarios/kinesis_basics_scenario.py. Remove these three unused imports fromkinesis_wrapper.pyto avoid linter warnings and keep the module clean. -
[nit] Still unresolved (prior item 2):
scenarios/kinesis_basics_scenario.pylines 23-26 importdatetime,timezone, andboto3.datetimeandtimezoneare never referenced in the scenario file (no calls todatetime(...)ortimezone.utc).boto3is also unused —KinesisWrapper.from_client()creates the client internally. Remove these three unused imports. -
[nit] Still unresolved (prior item 3):
hello/hello_kinesis.pyline 29 usesresponse.get("StreamNames", list())— the legacy field. Thekinesis_wrapper.pylist_streamsmethod correctly usesStreamSummaries. For consistency and to avoid silently returning zero results in accounts with many streams,hello_kinesis.pyshould useStreamSummariesand accesss['StreamName']from each summary dict, matching the wrapper's approach. -
[policy] Still unresolved (prior item 7): There is no
conftest.pyinpython/example_code/kinesis/test/. The test file importsfrom kinesis_wrapper import KinesisWrapperwhich will fail unless tests are run from the parent directory or withPYTHONPATHset. Premium comparable examples (e.g.,basics/controltower/test/conftest.py) usesys.path.appendto handle this. Aconftest.pywith path fixup would follow established conventions. -
[nit] Still unresolved (prior item 9): The specification Step 7 output specifies
Found 3 tag(s). HasMoreTags: falsebutscenarios/kinesis_basics_scenario.pyline ~207 only printsFound {len(tag_list)} tag(s).without theHasMoreTagsvalue. This is a minor deviation from the spec. Consider printing theHasMoreTagsvalue, or updating the spec — but currently they don't match. -
[nit] Still unresolved (prior item 10): Long lines remain in
scenarios/kinesis_basics_scenario.py— for example thesensor_datalist literals on lines ~160-164 exceed 88 characters (Black's default). These should be reformatted for PEP 8 / Black compliance. -
[nit] Prior item 6 (list_streams pagination) — partially addressed: The
list_streamspagination logic on line ~68 accessesresponse["StreamSummaries"][-1]["StreamName"]whenHasMoreStreamsis True. IfStreamSummarieshappened to be empty on a page whileHasMoreStreamsis True (an unlikely but theoretically possible edge case), this would raise anIndexError. A guardif response.get("StreamSummaries")before this access would be safer, though this is unlikely to occur in practice. -
[nit] Prior item 4 (waiter coverage) — still unresolved:
wait_for_stream_activeandwait_for_consumer_activeinkinesis_wrapper.pyhave no unit tests. Adding at least a happy-path stubbed test for each would improve coverage. This was noted in the previous review and remains unaddressed. -
[policy] Prior item 8 — still unresolved: There is still no integration test (e.g.,
@pytest.mark.integdecoratedtest_run_scenario) for the scenario runner. Premium comparable examples typically include at least a scenario-level test with stubs. -
[nit] The
README.mdlinks for "Hello Kinesis" and "Learn the basics" both point topython kinesis_wrapper.pyin the instructions section, but "Hello Kinesis" should point topython hello/hello_kinesis.pyand "Learn the basics" should point topython scenarios/kinesis_basics_scenario.py. The entry points are misidentified in the README run instructions.
Reviewed at commit 8d54cc90521810d4a7bfccb9048fb7c2aa8f6f8d. This review was generated automatically using Amazon Bedrock. It compares your changes against existing examples and coding guidelines. Findings are labeled by severity (blocking / nit / policy) and reflect only the commit above. Please use your judgment — this is advisory, not authoritative.
There was a problem hiding this comment.
🤖 AI Code Example Review
The PR is in good shape — all previously identified blocking and nit issues have been addressed or resolved in this commit. The remaining items are minor nits and one policy observation. The code is runnable, well-structured, specification-compliant, and properly tested with offline stubs.
Detailed Review
-
[nit] Prior issue resolved:
hello_kinesis.pynow usesStreamSummaries— The previous concern abouthello_kinesis.pyusing the legacyStreamNamesfield has been fully addressed. The current code at line 28 correctly usesresponse.get('StreamSummaries', list()), consistent withkinesis_wrapper.py. -
[nit] Prior issue resolved:
conftest.pypath fixup added — Theconftest.pynow includessys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))to allowkinesis_wrapperto be imported from the correct directory. This matches the established convention. -
[nit] Prior issue resolved: unused imports in
kinesis_wrapper.pyremoved — Confirmed thatjson,datetime, andtimezoneare no longer imported inkinesis_wrapper.py. The imports are now justlogging,time,boto3, andClientError. -
[nit] Prior issue resolved: unused imports in
kinesis_basics_scenario.pycleaned up — Confirmed thatboto3and unuseddatetime/timezoneare no longer in the scenario file. The current imports arejson,logging,time,ClientError, andKinesisWrapper. -
[nit] Prior
list_streamspagination IndexError risk still present — Inkinesis_wrapper.pyaround line 63-69, theHasMoreStreamsloop readssummaries[-1]['StreamName']for theExclusiveStartStreamNametoken. This uses the previous page's summaries, not the newly fetched ones. If theStreamSummariesfield in a non-first page's response happened to be empty (edge case),summaries[-1]would raiseIndexError. This is the same nit from the prior review. It's a defensive-coding nit, not blocking, since the guardif not summaries: breakruns before the[-1]access using the priorresponse's summaries — so it's actually safe on closer inspection. No action strictly required. -
[nit]
wait_for_consumer_activecallsdescribe_stream_consumerdirectly — Inkinesis_wrapper.py(lines ~328-343), thewait_for_consumer_activemethod callsself.kinesis_client.describe_stream_consumer(...)directly rather than through a wrapper method. This is a minor inconsistency with the class's own pattern of wrapping client calls with error handling. Sincewait_for_consumer_activeis not a snippet-tagged action method, it's only a style note. -
[nit] Step 7 output format minor spec deviation — The spec example shows
HasMoreTags: false(lowercase boolean as a string), while the Python code atkinesis_basics_scenario.pyline ~214 printsf"Found {len(tag_list)} tag(s). HasMoreTags: {has_more_tags}"which will emitHasMoreTags: False(Python capitalized bool). This is cosmetic but technically diverges from the spec output. Could usestr(has_more_tags).lower()if exact spec conformance is required. -
[policy] No integration test for
run_scenario— As noted in the previous review, there is no@pytest.mark.integ-marked test for the full scenario flow. All unit tests cover individual wrapper methods only. This is a policy observation — the team should decide whether a scenario-level stub test is required. -
[nit]
stub_describe_stream_summarymissingEncryptionTypefield — Inkinesis_stubber.py(lines ~170-185), the stub response fordescribe_stream_summarydoes not include anEncryptionTypekey. The scenario code atkinesis_basics_scenario.pycallssummary.get('EncryptionType', 'NONE')with a default, so this won't cause a failure, but the stub is slightly less realistic than it could be. -
[nit]
kinesis_basics_scenario.pyimports from parent directory withoutsys.pathmanipulation — The scenario file usesfrom kinesis_wrapper import KinesisWrapper(a sibling directory) without any path manipulation. This works when run aspython scenarios/kinesis_basics_scenario.pyfrom thekinesis/directory but may fail when imported from another context. Theconftest.pyhandles this for tests, but the scenario itself has nosys.pathguard. Consider addingsys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))at the top of the scenario file for robustness.
Reviewed at commit 9ae23db0988f5a44e0f1faf1385dcf310f1957d1. This review was generated automatically using Amazon Bedrock. It compares your changes against existing examples and coding guidelines. Findings are labeled by severity (blocking / nit / policy) and reflect only the commit above. Please use your judgment — this is advisory, not authoritative.
RuiqiGao-aws
left a comment
There was a problem hiding this comment.
I ran the Basics scenario for real in a sandbox account (us-west-2, 3 runs) and hit two blockers.
First, the documented command python scenarios/kinesis_basics_scenario.py fails with ModuleNotFoundError: No module named 'kinesis_wrapper' - the script imports from its parent directory, which isn't on sys.path when run from the README's location.
Second, with PYTHONPATH set as a workaround, all 3 runs fail at Step 3: ListShards raises ResourceInUseException ("instead in state CREATING") even though DescribeStreamSummary already reported the stream ACTIVE - the two control-plane views converge at different times for a fresh stream. On the plus side, the finally cleanup worked every run; no streams were leaked. Unit tests (20/20) and the metadata validator pass, but the stubber can't catch either of these.
RuiqiGao-aws
left a comment
There was a problem hiding this comment.
Re-verified at head 6d75b75: the README-documented command now runs end-to-end in a real account — rc=0, all 10 steps, 5 records written and read back, and no streams left afterwards.
All 43 unit tests pass, including the three new ones covering the list_shards retry path. Both blockers are resolved; thanks for the quick turnaround.
Fine as a follow-up if you prefer - approving.
Fix CI failures in the Kinesis Basics scenario + Hello PR: - Quote/rename the scenario title so the leading &AWSKinesis; entity is not parsed as a YAML anchor (resolves the parse error that broke Validate Doc Metadata, Lint Yaml, and Validate WRITEMEs). - Use the Basics convention: title 'Learn ... basics using an &AWS; SDK' and title_abbrev 'Learn the basics' (title must start uppercase). - Add 'category: Hello' to kinesis_Hello so it is treated as a Hello example rather than a generated API example (also clears the duplicate kinesis:ListStreams error). - Remove title/title_abbrev/synopsis from the four API-action entries (DescribeStreamSummary, ListShards, GetShardIterator, RemoveTagsFromStream) since generated API examples cannot define them. - Regenerate python/example_code/kinesis/README.md to match metadata.
Resolve the inline AI review comments on the Kinesis Basics PR: - Remove unused imports from kinesis_wrapper.py (json, datetime, timezone) and kinesis_basics_scenario.py (datetime, timezone, boto3). - hello_kinesis.py: use StreamSummaries instead of the legacy StreamNames field, matching the wrapper's list_streams. - Guard list_streams pagination against an empty StreamSummaries page. - Print HasMoreTags in Step 7; list_tags_for_stream now returns (tags, has_more). - Reformat the sensor_data list literal and drop a placeholder-less f-string for Black/PEP 8 compliance. - Add snippet tags to hello_kinesis.py and kinesis_basics_scenario.py and point the Hello/Scenario metadata at them so the README run commands reference the correct entry-point files. - Convert test_kinesis_basics.py to the standard make_stubber + KinesisStubber pattern used across python/example_code, extend KinesisStubber with the basics actions, and restore the shared conftest fixtures (fixes the earlier conftest regression).
Rename the generic kinesis_Scenario metadata key to the standard kinesis_Scenario_GettingStarted and merge the existing SAP ABAP version into it, so there is exactly one Basics scenario per service. The generic key produced a dangling cross-reference anchor (kinesis_example_kinesis_Scenario_section) that broke the ZonBook XSLT transform during the Code Library build. Regenerate the README so the custom-content markers match the renamed key.
Merging the Basics scenario under kinesis_Scenario_GettingStarted changed the shared synopsis, so the SAP ABAP service README was out of date. Regenerate it with writeme.py to satisfy Validate WRITEMEs.
Fix two issues found running the Basics scenario for real (us-west-2):
- Scenario import: `python scenarios/kinesis_basics_scenario.py` raised
ModuleNotFoundError for `kinesis_wrapper` because the parent directory
was not on sys.path. Add the parent dir to sys.path before importing,
matching the pattern test/conftest.py already uses.
- ListShards race: a freshly created stream can report ACTIVE via
DescribeStreamSummary before its shards are listable, so ListShards
transiently raised ResourceInUseException ("in state CREATING"). Add a
bounded retry loop on ResourceInUseException in list_shards, mirroring
the existing ACTIVE-wait polling pattern.
Add unit tests covering the retry-then-succeed and give-up-after-max-wait
paths. Verified with black --check and pytest (22/22) in the
sdk-examples-python container.
The list_shards retry change shifted line numbers in kinesis_wrapper.py, so the README's single-action snippet links were stale. Regenerated with writeme.py; only line-number anchors changed. Fixes the Validate WRITEMEs CI check.
6d75b75 to
02a1d43
Compare
Auto-generated Python code examples for kinesis