Skip to content

feat: apply FC-0118 ADRs to youtube transcripts APIs - #39100

Merged
Faraz32123 merged 5 commits into
masterfrom
feat/apply_fc_0118_ADRs_to_youtube_transcript_endpoints
Oct 9, 2026
Merged

Faraz32123 merged 5 commits into
masterfrom
feat/apply_fc_0118_ADRs_to_youtube_transcript_endpoints

Conversation

@Faraz32123

@Faraz32123 Faraz32123 commented Sep 11, 2026 •

Copy link
Copy Markdown
Contributor

Related Issue: #39061

Standardizes the two v0 YouTube transcript endpoints (YoutubeTranscriptCheckView, YoutubeTranscriptUploadView in cms/djangoapps/contentstore/rest_api/v0/views/transcripts.py) onto a new, additive v1 surface. v0 is untouched — every existing address still resolves to the same view with the same body; the two v0 operations are marked deprecated in the schema by a post-processing hook, not in code.

Only the two endpoints listed in #39061 are migrated. The sibling TranscriptView (/video_transcripts/...) is a different resource and is out of scope.

New addresses

Legacy (unchanged, now deprecated in the schema) New
GET /api/contentstore/v0/youtube_transcripts/{course_id}/check GET /api/authoring/v1/courses/{course_key}/youtube_transcript_checks/
GET /api/contentstore/v0/youtube_transcripts/{course_id}/upload POST /api/authoring/v1/courses/{course_key}/youtube_transcript_imports/

Two changes worth calling out

The import endpoint becomes a POST. The legacy GET .../upload performs four domain writes — an outbound YouTube download, create_external_video (not idempotent), create_or_update_video_transcript, and video.save(). That is the GET-mutates violation ADR 0030 targets. The new endpoint is a POST; the frozen v0 GET keeps all four writes exactly as today, so nothing is taken away from existing callers.

The legacy URL regexes truncate, and that is preserved. ^youtube_transcripts/{COURSE_ID_PATTERN}/check?$ makes the final character optional, so reverse() actually emits /chec, and all four spellings resolve today: /chec, /check, /uploa, /upload. This PR does not touch v0/urls.py at all, so all four keep resolving to their original views. /check/ keeps 404ing, as it does now. Verified by direct resolve() assertions and by a URL-snapshot diff of all 2181 base routes.

ADR compliance

ADR Status Notes
0025 Serializers Applied Five serializers, help_text on every field. Responses are serializer-produced; requests are validated before the legacy call. Two field/reality fixes adopted: current_item_subs is a string-or-null (v0 declared ListField), and edx_video_id is nullable — None is a legitimate success value for V2 libraries. Neither changes a wire value: the v0 serializers were declared but never instantiated.
0026 Permissions Applied permission_classes = (IsAuthenticated, HasCourseAuthorAccess) on each viewset, reusing the existing class. All three legacy authorization layers survive.
0027 Schema Applied @extend_schema per action, tags=["openedx-platform-sdk"] on both, every 4xx documented with ErrorResponseSerializer.
0028 Views Applied Plain viewsets.ViewSet — the data path is modulestore/contentstore/edxval plus an outbound HTTP call, not an ORM queryset, so ModelViewSet does not apply. Both actions delegate to the existing legacy functions unchanged, so no new N+1s; a query-count test pins this against the legacy address.
0029 Errors Applied StandardizedErrorMixin first in the MRO. Django-native PermissionDenied and Http404/ItemNotFoundError are translated to their DRF equivalents at the service boundary — without that, a denied caller would be reported as internal/500 rather than authz/403. No error types are registered: the registry is process-wide, so a 405 or a malformed body keeps the library's current classification until edx-drf-extensions covers them, as in #39102. A missing youtube entry in the import's videos is reported on videos. Error text is fixed and generic; the upstream exception message is logged server-side, never returned.
0030 GET side effects Applied See above. The new GET is proven write-free by runtime SQL capture, not by static inspection — a hygiene check cannot see a write one frame down.
0031 Merges Applied as a reasoned non-merge Same resource domain, but they differ in side effects, HTTP semantics and response shape, so the ADR's two-part test fails. Merging would force a safe read through a write-shaped entry point and re-commit the 0030 violation. They are consolidated onto one URLconf and one service module instead.
0032 Pagination Not applicable Neither endpoint returns a collection — each returns a single object (9 fields and 2 fields). There is no list to bound.
0033 Filtering Not applicable No list, so nothing to filter or order. The one input parameter identifies which block is addressed; it is an identifier payload, not a filter, and varying it cannot widen what the caller may see.
0034 Authentication Applied BearerAuthenticationAllowInactiveUser dropped. (JwtAuthentication, SessionAuthenticationAllowInactiveUser) is a documented deviation from the default pair, commented at the declaration: inactive Studio authors reach this from the authoring UI today, and the default session class would refuse callers v0 accepts.
0035 MFE config Not applicable Neither endpoint returns site or front-end configuration. Both are authenticated, per-course, per-block authoring operations.
0036 Nested JSON Not applicable Both responses are flat sets of thin scalars with no sub-objects or tree shape, which the ADR excludes by its own terms.
0037 Versioning Applied, except the DEPR link New version; v0 frozen; all four live spellings preserved. The v0 operations are marked deprecated: true in the schema by a post-processing hook; the v0 code is unchanged. The deprecation is being recorded as a row on the existing DEPR issue #39128 rather than a new one. Removal window: one named release after Verawood.
0038 URLs Applied Verbs removed from the path (check/upload become the noun collections youtube_transcript_checks/youtube_transcript_imports); one level of nesting under courses/; course key via path converter, so a malformed key is a routing-level 404; mandatory trailing slash; version-free URL names. api_name is authoring, not contentstore — the ADR's Rejected Alternatives names URLs mirroring the codebase as publishing an implementation detail as a public contract.
OEP-66 Scoping Not applicable as a mechanism Neither view has a get_queryset() for ScopedQuerysetMixin to wrap. The three layers are still present: endpoint access via permission classes, record visibility via the course key plus the item's own course-key re-check, and user input applied strictly after both.

Compatibility evidence

  • URL gate: 0 FAIL / 0 WARN. Head 2183 routes vs base 2181 — the only difference is the two new routes. Every base address still resolves.
  • All four live v0 spellings assert-tested to resolve to their original views; the legacy URL names still reverse.
  • Legacy tests pass unmodified: cms/djangoapps/contentstore/views/tests/test_transcripts.py, 44 passed, file untouched in this diff.
  • v0/urls.py and cms/djangoapps/contentstore/views/transcripts_ajax.py are not modified. The latter matters because the one real frontend consumer (frontend-app-authoring) calls the legacy non-REST Studio routes that share it.
  • Consumer search across the openedx org found no external caller of either REST path.
  • 80 new tests, including a full deny matrix (anonymous → 401 authn; student and other-course instructor → 403 authz; instructor and staff → 200), a cross-course locator refusal on both endpoints asserting the slug is authz and not internal, and tests proving the course-level permission fires before the item is loaded.

Files changed outside the v1 package, and why

File Why
cms/urls.py Mounts api/authoring/v1/ after the existing contentstore include, and calls register_url_converters() before the first pattern using a converter. Additive only.
cms/lib/spectacular.py The filter admits only ^/api/contentstore/v\d+/, so the new endpoints would route and test green but be silently absent from the published schema. Widened to ^/api/(contentstore|authoring)/v\d+/, plus a post-processing hook marking the v0 paths deprecated — done here rather than in the v0 files, which are frozen.
cms/envs/common.py, cms/envs/devstack.py, cms/envs/production.py Register the post-processing hook (restating postprocess_schema_enums, since setting the key replaces drf-spectacular's defaults), and remove SCHEMA_PATH_PREFIX_TRIM so paths publish at their full address instead of mixing prefixes.

Gate results

versions        PASS   (allowlisted files justified above)
hygiene         PASS   0 FAIL / 0 WARN, 13 files
urls            PASS   0 FAIL / 0 WARN, 2181 base routes preserved
schema present  PASS   both endpoints published, no mixed-prefix warning
old tests       PASS   44 legacy tests green, file unmodified
new tests       PASS   80
ruff            PASS
schema compare  FAIL   — expected; see below

On the schema-compare failure. Removing SCHEMA_PATH_PREFIX_TRIM re-keys every existing path from its trimmed form (/v0/...) to its full address (/api/contentstore/v0/...). The differ sees 56 removals and 56 additions as unrelated and reports them as breaking. Reconciled mechanically instead:

  • base 57 paths → head 59 paths;
  • 0 removed paths fail to reappear at their full address;
  • 2 genuinely new paths — the two /api/authoring/v1/ endpoints;
  • exactly 2 operations differ in any key across the entire schema — the two v0 YouTube operations, each gaining deprecated: true. No other operation changed.

Reverting the trim removal would turn this gate green and leave the schema publishing a mix of trimmed and untrimmed paths that no single server URL serves.

Known gaps

@Faraz32123
Faraz32123 force-pushed the feat/apply_fc_0118_ADRs_to_youtube_transcript_endpoints branch 2 times, most recently from 2863071 to 8906fb7 Compare September 24, 2026 07:31
Comment thread cms/djangoapps/contentstore/rest_api/v0/views/transcripts.py Outdated
Add a conforming v1 surface for the two YouTube transcript endpoints at
/api/authoring/v1/courses/{course_key}/youtube_transcript_{checks,imports}/.
The import operation becomes a POST because the legacy GET performs four
domain writes. Responses are serializer-produced and errors use the
standardized envelope. The v0 routes are untouched and keep resolving at
all four of their live spellings; they are marked deprecated in the schema.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@Faraz32123
Faraz32123 force-pushed the feat/apply_fc_0118_ADRs_to_youtube_transcript_endpoints branch from 8906fb7 to 67256cd Compare September 26, 2026 09:55
@Faraz32123 Faraz32123 linked an issue Sep 26, 2026 that may be closed by this pull request
@Faraz32123
Faraz32123 marked this pull request as ready for review September 26, 2026 10:09
Comment thread cms/envs/production.py
Comment thread cms/envs/production.py
…ADRs_to_youtube_transcript_endpoints

# Conflicts:
#	cms/envs/devstack.py
#	cms/envs/production.py
#	cms/urls.py
- Publish full paths from cms/envs/common.py as well. cms.envs.development
  inherits these settings to generate the committed docs/cms-openapi.yaml,
  which otherwise mixed trimmed /v0/... paths with untrimmed
  /api/authoring/v1/... ones and marked nothing deprecated. Drop
  SCHEMA_PATH_PREFIX_TRIM, widen SCHEMA_PATH_PREFIX to both mounts, and add
  the hook that marks the superseded v0 paths deprecated.
- Require a "youtube" entry in the import's videos, so a missing one is
  reported on videos rather than as a failure on locator.
- Test a locator that is neither a course nor a V2 library block, which the
  handlers reject with TranscriptsRequestValidationException.
The registry in edx_rest_framework_extensions.errors is process-wide, so
registering MethodNotAllowed and ParseError here changed the error type of
every standardized Studio endpoint, not only these two. Left to an
edx-drf-extensions follow-up, as in #39102; the 405 tests assert the status
only.
A failure on YouTube's side is reported as a 400, matching v0. It can
become a 502 once edx-drf-extensions registers an error type for
upstream failures.
@Faraz32123
Faraz32123 merged commit 806e495 into master Oct 9, 2026
46 checks passed
@Faraz32123
Faraz32123 deleted the feat/apply_fc_0118_ADRs_to_youtube_transcript_endpoints branch October 9, 2026 09:33
Abdul-Muqadim-Arbisoft added a commit to edly-io/edx-platform that referenced this pull request Oct 9, 2026
Brings in openedx#39100 (YouTube transcripts) and openedx#39157 (course content), which
also add conforming routes under /api/authoring/:

- v1/authoring_urls.py holds both APIs' routes under the one authoring_v1
  namespace, and cms/urls.py mounts v1 and v2 from master beside v3 and v4.
- The URL converters are registered once, where master registers them.
- cms/lib/spectacular.py keeps master's cms_mark_superseded_paths beside
  cms_mark_migrated_paths and CmsAutoSchema, and the settings list both
  hooks. The SCHEMA_PATH_PREFIX comment says it also derives every
  operationId, not just the tags.
- cms/lib/tests/test_spectacular.py merges both test modules; master's
  CmsApiFilterTest replaces this branch's narrower one.
- The YouTube URL-name test checks that the shared namespace's names are
  unique and include its own, rather than that they are its own alone.
blarghmatey added a commit to mitodl/lehrer that referenced this pull request Oct 13, 2026
…ls (#325)

* fix(settings): regenerate the master release's aqueduct settings models

edx-platform master changed SPECTACULAR_SETTINGS in common.py twice.
openedx/openedx-platform#39100 gave the CMS a
POSTPROCESSING_HOOKS list, dropped its SCHEMA_PATH_PREFIX_TRIM, and widened
its SCHEMA_PATH_PREFIX to cover /api/authoring.
openedx/openedx-platform#39078 did the same for the LMS
hooks and trim flag and added a third CMS hook.

The committed models went stale, so the Settings Verification drift check
fails on main for every master cell in mit-ol and generic, and every PR that
touches a build manifest inherits the failure.

Claude-Session: https://claude.ai/code/session_01YRDRrCw7tdqWA4uE8QhGPe

* fix(settings): drop spectacular hooks the running release does not define

One generated model, rendered from master, serves every release in a group.
Verawood's cms/envs/common.py has no SPECTACULAR_SETTINGS, so the model's
default is the live setting there. After the regeneration that default names
cms_mark_superseded_paths and cms_mark_migrated_paths, which verawood's
cms/lib/spectacular.py does not define. drf-spectacular imports hooks when a
schema is requested, so Studio would boot and then fail /authoring-api/schema/.

spectacular_with_servers now leaves out hooks whose module does not bind the
name. It reads the module's source instead of importing it, because the
validators run while settings are being built and edx-platform's spectacular
modules import Django REST Framework, which reads settings on import.

verify-settings now resolves the hooks after `manage.py check`, which passes
with a dangling hook. With the pruning in place that step guards the pruner:
it fails when a hook the source appears to bind cannot be imported.

Claude-Session: https://claude.ai/code/session_01YRDRrCw7tdqWA4uE8QhGPe

* fix(settings): resolve spectacular hook modules without importing parents

importlib.util.find_spec imports each parent package to find the next
segment, so an operator hook such as vendor.hooks.fn ran vendor/__init__.py
while Django settings were still being built. Walk the dotted path through
PathFinder instead, which only reads the filesystem. The test fixture's
package __init__ now raises on import to cover it.

Claude-Session: https://claude.ai/code/session_01YRDRrCw7tdqWA4uE8QhGPe
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[API] YouTube Transcripts

3 participants