feat(pathways): implement Pathway content

[Agrendalath]

This also includes the changes from #819, which I opened from the fork.

This is just a draft, as I'm still wrapping up the APIs.

[openedx-webhooks]

Thanks for the pull request, @Agrendalath!

This repository is currently maintained by @axim-engineering.

Once you've gone through the following steps feel free to tag them in a comment and let them know that your changes are ready for engineering review.

🔘 Get product approval

If you haven't already, check this list to see if your contribution needs to go through the product review process.

• If it does, you'll need to submit a product proposal for your contribution, and have it reviewed by the Product Working Group.
  • This process (including the steps you'll need to take) is documented here.
• If it doesn't, simply proceed with the next step.

🔘 Provide context

To help your reviewers and other members of the community understand the purpose and larger context of your changes, feel free to add as much of the following information to the PR description as you can:

• Dependencies

  This PR must be merged before / after / at the same time as ...

• Blockers

  This PR is waiting for OEP-1234 to be accepted.

• Timeline information

  This PR must be merged by XX date because ...

• Partner information

  This is for a course on edx.org.

• Supporting documentation
• Relevant Open edX discussion forum threads

🔘 Get a green build

If one or more checks are failing, continue working on your changes until this is no longer the case and your build turns green.

🔘 Update the status of your PR

Your PR is currently marked as a draft. After completing the steps above, update its status by clicking "Ready for Review", or removing "WIP" from the title, as appropriate.

---

Where can I find more information?

If you'd like to get more details on all aspects of the review process for open source pull requests (OSPRs), check out the following resources:

• Overview of Review Process for Community Contributions
• Pull Request Status Guide
• Making changes to your pull request

When can I expect my changes to be merged?

Our goal is to get community contributions seen and reviewed as efficiently as possible.

However, the amount of time that it takes to review and merge a PR can vary significantly based on factors such as:

• The size and impact of the changes that it introduces
• The need for product review
• Maintenance status of the parent repository

💡 As a result it may take up to several weeks or months to complete a review and merge your PR.

[kdmccormick]

Hey @Agrendalath , I know this is still a draft, but I wanted to raise something as early as possible.

We need more indirection between the PathwayItems and CourseRuns. As currently implemented, it seems very likely that the data and business logic layers will assume that only CourseRuns can ever fulfill PathwayItems. In particular, having course_runs and preferred_course_run_row directly on the PathwayItemVersion model is a red flag for me. Generally speaking, I would really like to make sure that we are not just building towards an over-engineered version of course-based programs (if that's all we wanted, then the data model could be so much simpler).

I recommend introducing a data model called something like FulfillmentOption, which represents the generic notion of "a way to fulfill a PathwayItem". Sub-classes of this would represent the concrete avenues for fulfilment (CourseRun for Willow; things like Section and sub-Pathway in the future).

class PathwayItemVersion(PublishableEntityVersionMixin):
    pathway_item = FK(
        PathwayItem,
        related_name="versions",
    )
    # Note: No direct relationship to CourseRun on this model

class FulfillmentOption(Model):
    Meta.abstract = True
    pathway_item_version = FK(
        PathwayItemVersion,
        related_name="fullfillment_options"
    )

# For Willow:
class CourseRunFulfillmentOption(FulfillmentOption):
    course_run = FK(CourseRun)

# Future releases:
class SectionFulfillmentOption(FulfillmentOption):
   ...
class NestedPathwayFulfillmentOption(FulfillmentOption):
   ...

Then, whichever model we use to track pathway item completion, we could track the option they took to fulfill the pathway without tying it to CourseRuns. Something like:

class PathwayItemPassed(Model):
    user = FK(User)
    pathway_item = FK(PathwayItem)
    fulfilled_by = FK(FulfillmentOption)

We would need some way to represent which fulfillment method is "preferred" for a given Item. I was thinking that could be a single field PathwayItemVersion.preferred_fulfillment_option, or an ordering like FulfillmentOption.priority.

Lastly, I'd like to get your thoughts on how we represent the scenario "an admin has manually marked a particular PathwayItem as passed for a particular user". It's not an explicit MVP requirement, but it's something we should certainly support, and I wonder if we could build the Willow data model in a way that just allows it out-of-the-box via Django admin.

[Agrendalath]

@kdmccormick, I intentionally left this out of this PR. Defining the fulfillment criteria was one item on the long list of "things removed from MVP" mentioned during today's meeting. I didn't add this layer to the MVP to keep more flexibility in designing it later. Calculating grades is fast using the grades API, so we don't need to track individual item status in the MVP (calculating them on the fly is cheaper than redesigning models in later iterations).

I may be missing something, though, so if introducing another layer in front of the CourseRuns in a migration in the next iteration would be more challenging, I can implement it in the current branch and add notes that these models may be redesigned in the future.

[kdmccormick]

Gotcha. It makes sense that you'd want to avoid building a more complex model which will end up being thrown away when we add more complex critieria. My concern is that, with the current model, it will be very natural for app code to assume that Items will always have CourseRuns, whether we intend that or not.

I think that building the layer between PathwayItems and CourseRuns now will help developers avoid that assumption and also will set us up decently well for the next iteration. The model holds well up to most of the future features I can imagine, like:

• An Item requires multiple things to be completed. Add a row PathwayItemVersion.num_fulfillments_needed, backfilled to 1.
• A custom grade threshold is needed. Add a row CourseRunFulfillmentOption.passing_grade_override, defaulting to NULL.
• An Item requires an sub-sequence of CourseRuns itself. Create a NestedPathwayFulfillmentOption, and nest the sub-sequence within its own Pathway.

cc @ormsbee

[rpenido]

Hi @Agrendalath! I'm trying to figure out how we will sort the data to fill our current UI/UX design.
From the learner's view (authenticated user), we need a list of the enrolled Pathways and the current progress (passed / total items):

My understanding from openedx_learning-ADR0006 is that we will have it somewhat attached to the PathwayEnrollment. Is that right?

[Agrendalath]

@rpenido, I suppose we can access these via get_pathway_enrollments now (the CatalogPathways are already prefetched, so it needs no extra queries). I will get into the progress part once we wrap up #819.