api_reference.md

API reference

AdaptiveRanker is the package-root API. It fits from completed learning attempts, ranks an application-approved candidate set, and accepts feedback. Supporting modules add durable audit records, catalog validation, and a reference controlled-pilot adapter.

from orchid_ranker import AdaptiveRanker

Use the production-serving guide for the ordinary request loop and the pilot integration contract when a next-item choice is part of an efficacy study.

At a glance

Need	API	Result
Fit a learning policy	AdaptiveRanker().fit(events, ...)	The fitted ranker
Rank safe exercises	ranker.recommend(user_id, ids)	AdaptiveLearningRecommendation values
Adapt after an attempt	ranker.observe(...)	Updates the learner state
Keep a reproducible serving record	ranker.recommend_and_log(...)	Ranked items and immutable LoggedDecision
Attach a delayed score safely	ranker.observe_decision(...)	Durable DecisionOutcome and normally a learner update
Validate authoring data	validate_learning_catalog(frame)	Normalized exercises and all diagnostics
Run a reference experiment adapter	AdaptivePracticePilot	Assignment, decisions, lifecycle evidence, analysis frames

All timestamps must be finite, non-negative numeric values in one consistent application-defined unit. Outcomes are binary 0 or 1; do not turn missing, abandoned, or unscored work into 0.

Orchid is an in-process Python library. It does not expose HTTP endpoints or manage authentication, authorization, rate limits, or learner-data access. The application must enforce those controls before calling Orchid. Keep stable user and item IDs across fit, serving, and feedback; durable decision IDs and event IDs must be unique in the application's namespace.

The base orchid-ranker install runs the empirical learner without PyTorch. Install orchid-ranker[kt] for neural knowledge tracing. When PyTorch is unavailable, the default fallback uses the empirical learner and records the reason in learning_readiness(). If fallback_to_empirical=False requests a neural tracer without PyTorch, fitting raises ImportError with installation guidance.

Failure	Exception	Caller action
Invalid schema, candidate set, timestamp, or conflicting retry	ValueError	Correct the request or reconcile the original immutable event.
Unknown decision or learner assignment during outcome import	KeyError	Restore the missing decision or assignment before retrying.
Operation attempted before fit or before a required lifecycle step	RuntimeError	Complete the prerequisite operation, then retry.
Wrong object type or unknown configuration field	TypeError	Correct the integration code.

These exception categories are part of the documented integration contract; message text is diagnostic and may improve between versions.

AdaptiveRanker

Construct and fit

ranker = AdaptiveRanker(
    kt_backbone="empirical",  # optional AdaptiveRankerConfig override
    decision_store=store,     # optional durable store
)

ranker.fit(
    events,
    user_col="user_id",
    item_col="item_id",
    outcome_col="outcome",
    timestamp_col="timestamp",
    category_col=None,
    difficulty_col=None,
    catalog=None,
    catalog_item_col="item_id",
    catalog_category_col="category_id",
    catalog_difficulty_col="difficulty",
    prerequisite_by_concept=None,
)

Pass either an AdaptiveRankerConfig as the first argument or its fields as keyword overrides; unknown names raise TypeError. fit returns the same ranker and expects a pandas DataFrame of completed attempts with learner, item, binary outcome, and timestamp columns.

A canonical catalog is the preferred source of item category and difficulty. It validates historical coverage and registers catalog-only items for future serving. Its default columns are item_id, category_id, and difficulty; use the three catalog_*_col arguments for a different source schema. prerequisite_by_concept constrains ranking, but it does not replace the application's item-level eligibility.

ranker = AdaptiveRanker().fit(
    completed_attempts,
    catalog=exercise_catalog,
    prerequisite_by_concept={"advanced": ["fundamentals"]},
)

Use ranker.is_fitted after a successful fit. ranker.learning_readiness() returns data support, outcome balance, metadata coverage, selected tracer, and recommended next steps; it is rollout evidence, not a model-quality guarantee. ranker.diagnostics() additionally reports adaptive, policy, semantic, and decision-store state.

Sparse learning data automatically uses Orchid's empirical learner. For a first frozen pilot, explicitly set kt_backbone="empirical".

recommend

recommendations = ranker.recommend(
    user_id,
    candidate_item_ids,
    top_k=3,
    concept_goal=None,
    context_hash=None,
)

Ranks the supplied candidate set, returning AdaptiveLearningRecommendation values in descending order.

Field	Meaning
item_id	Candidate in ranking order
score	Relative score within this request
outcome_probability	Estimated probability of a positive next attempt
difficulty, category_id, competence	Available learning context
item_support, concept_support, feedback_supported	Evidence/support indicators
prerequisites_met, recent_repetition	Constraint and sequencing signals

Scores are for ranking the current candidates, not globally calibrated learning values. outcome_probability estimates the next practice result, not retained mastery. An explicit empty sequence returns [] and never means “all known items.” candidate_item_ids=None only permits a configured candidate generator or explicit catalog fallback.

The application owns eligibility: remove unavailable, completed, accommodation-inappropriate, prerequisite-blocked, and assessment-only items before calling recommend. PilotCatalog enforces a published version of these controls for the reference pilot.

observe and register_items

ranker.observe(
    user_id=learner_id,
    item_id=exercise_id,
    outcome=1,
    timestamp=attempt_time,
    update_global=True,
)

ranker.register_items(catalog)

observe advances the live learner state from a completed attempt. Set update_global=False only where a learner may adapt while shared aggregate statistics must remain frozen, such as treatment traffic in a controlled pilot.

register_items adds catalog items absent from fitting history with an OOV prior, making them available for recommendation and feedback immediately. Refit later to learn item-specific parameters from their accumulated outcomes.

Durable decisions and delayed outcomes

Use a decision store whenever a served exercise must be reproducible or evaluated later.

from orchid_ranker.decision_store import SQLiteDecisionStore

store = SQLiteDecisionStore("orchid.sqlite")
ranker = AdaptiveRanker(decision_store=store).fit(completed_attempts)

LoggedDecision records the full candidate list, chosen item, score/probability evidence, policy identity, and immutable application metadata. DecisionOutcome links one later result to one decision. Its apply_state flag says whether that event may change adaptive state; update_global says whether a permitted update may change shared aggregate statistics.

recommend_and_log

ranked, decision = ranker.recommend_and_log(
    user_id=learner_id,
    candidate_item_ids=eligible_ids,
    timestamp=request_time,
    top_k=1,
    exploration=0.0,
    decision_id=request_id,
    policy_version="course-2026.1",
    decision_metadata={"catalog_version": "2026.1"},
)

This is the serving method for auditable decisions. It requires a non-empty explicit candidate set and returns the selected item first plus an immutable LoggedDecision.

Parameter	Operational meaning
decision_id	Stable application request ID. An exact retry returns the saved choice; reuse with different inputs raises ValueError.
exploration	Epsilon-uniform sampling among candidates that survive configured serving constraints. Start pilots at 0.0.
decision_metadata	JSON-compatible application context retained immutably and included in idempotency checks.
allow_unsupported_feedback	Use only if an external system supplies the complete feedback path for an unsupported item.
min_*, max_*, require_prerequisites	Support, predicted-outcome, difficulty, and prerequisite guards applied before sampling.

Persist the decision before rendering the item. Do not change the candidate list, model/catalog identity, or metadata on a delivery retry.

observe_decision, persistence, and recovery

linked = ranker.observe_decision(
    decision.decision_id,
    outcome=correct,
    timestamp=scored_at,
    outcome_event_id=lms_score_event_id,
)

observe_decision first durably attaches the outcome, then applies it to the live learner state. A decision accepts one immutable outcome. Repeating the same payload is safe; a conflicting payload raises ValueError. Pass an LMS-global outcome_event_id whenever it exists: Orchid enforces uniqueness across decisions, not merely within one decision.

persist_decision_outcome(...) accepts the same outcome, reward, timestamp, category, event ID, apply_state, and update_global arguments but does not change state. Use it when a lifecycle adapter must persist score evidence before projecting it into a learner model.

Method	Use
ranker.replay_pending_outcomes()	Continue from the same restored model baseline. Applies durable, uncheckpointed outcomes in learner-time order.
ranker.replay_all_outcomes_from_baseline()	Rebuild a newly fitted/restored baseline. Do not call it on a live projection or outcomes will be applied twice.
ranker.decision_log_frame(completed_only=False)	Export decisions joined with outcomes, event IDs, and state-update flags.

InMemoryDecisionStore is thread-safe but process-local. SQLiteDecisionStore is transactional, uses WAL mode, and is appropriate for a single-host service; close it explicitly or use it as a context manager. Both implement DecisionOutcomeStore; a custom backend must support immutable, idempotent decision/outcome creation plus outcome-application checkpoints.

Catalog validation

from orchid_ranker.learning_catalog import (
    LearningCatalogSchema,
    validate_learning_catalog,
)

validation = validate_learning_catalog(
    authoring_catalog,
    schema=LearningCatalogSchema(),
    require_complete_metadata=True,
)
validation.raise_for_errors()

validate_learning_catalog never mutates the DataFrame and returns all findings rather than failing at the first one. It validates canonical exercise identities (item_id, optionally content_version), metadata, duplicate/conflicting versions, prerequisites, dangling references, and cycles.

Input/result	Use
LearningCatalogSchema	Maps source columns. item_id_col is required; set an optional column to None only where it does not exist upstream.
require_complete_metadata=True	Makes absent curriculum, skill/category, difficulty, assessment, and prerequisite metadata blocking errors. Use for a pilot import gate.
allow_external_prerequisites=True	Permits prerequisite IDs owned by another catalog; otherwise dangling IDs are errors.
LearningCatalogValidation.exercises	Normalized immutable CanonicalExercise values.
.diagnostics, .errors, .warnings	Actionable findings, including source row indices.
.is_valid, .raise_for_errors()	Boolean gate or concise exception after inspection.

This is an import-quality gate, not a serving eligibility engine. Use PilotCatalog when delivery must be checked against a frozen catalog.

Controlled pilot API

orchid_ranker.pilot is a reference single-host LMS adapter. It creates an immutable experiment manifest, deterministic sticky assignment, append-only delivery evidence, and analysis exports. It is not an LMS connector and does not replace a study design, power calculation, or governance process.

Build a frozen pilot

from orchid_ranker.pilot import (
    AdaptivePracticePilot,
    PilotCatalog,
    PilotRequest,
    SQLiteExperimentAssignmentStore,
    SQLitePilotLifecycleStore,
)

catalog = PilotCatalog.from_frame(
    authoring_catalog,
    catalog_version="networking-2026.1",
)
database = "networking-pilot.sqlite"
pilot = AdaptivePracticePilot(
    ranker,
    catalog,
    experiment_id="networking-routing",
    model_artifact_id="empirical-2026-08-30",
    authored_policy_version="routing-static-v1",
    eligibility_rule_version="v1",
    treatment_fraction=0.5,
    randomization_salt="deployment-secret",
    assignment_store=SQLiteExperimentAssignmentStore(database),
    lifecycle_store=SQLitePilotLifecycleStore(database),
    initial_mode="aa",
)

PilotCatalog.from_frame requires one active content version per item_id. It validates content_version, course/module, a skill or category, difficulty, assessment flag, prerequisites, availability, required flag, and unique authored sequence position. eligible_items(...) rejects—not silently drops—an LMS candidate that is outside scope, unavailable, completed, assessment-only, prerequisite-blocked, or inconsistent with a required item.

AdaptivePracticePilot requires a fitted empirical baseline without a CQL overlay. It persists an ExperimentManifest with catalog digest/version, model artifact/config identity, authored policy, eligibility version, and allocation configuration. Reusing an experiment ID with a changed artifact or catalog fails rather than mixing evidence.

For durable use, put SQLiteDecisionStore, SQLiteExperimentAssignmentStore, and SQLitePilotLifecycleStore on the same database path. The in-memory variants are for tests and one-process prototypes.

Serve an LMS request

For an efficacy study, enroll the complete assigned cohort before any practice request. This creates a durable course-run record for learners who never request an exercise:

assignment = pilot.enroll(
    learner_id,
    course_run_id=cohort_id,
    timestamp=enrolled_at,
    stratum=baseline_band,
)
roster = pilot.enrollment_frame()

enroll is idempotent for the same immutable learner, run, timestamp, and stratum. The assignment is sticky across course runs. Add the assessment due date from the pre-declared course calendar to the exported roster. serve can assign on demand for ordinary usage, but an efficacy roster must include all learners assigned before practice, including those with no decision.

served = pilot.serve(PilotRequest(
    request_id=lms_request_id,
    user_id=learner_id,
    course_id="networking",
    module_id="routing",
    course_run_id=cohort_id,
    timestamp=request_time,
    completed_item_ids=tuple(completed_ids),
    candidate_item_ids=tuple(exact_lms_eligible_ids),
    stratum=baseline_band,
))

candidate_item_ids is required and must be the exact non-null LMS-approved set. request_id is namespaced by experiment and course run, preventing cross-experiment collisions. Assignment is stable per learner and stratified by the supplied stratum.

serve returns a PilotDecision:

Field	Meaning
decision	Immutable LoggedDecision; render decision.chosen_item_id.
arm	Sticky randomized assignment (control or treatment).
effective_arm	Actually delivered arm after applying the current mode.
mode	Mode recorded on this decision.
chosen_content_version	Exact revision the LMS must render.
reason_code, eligible_item_ids	Why it was chosen and the exact candidates.

Every decision automatically records an immutable explanation snapshot. Shadow decisions also record the Orchid proposal; halted decisions record a kill-switch fallback event.

Modes and operation events

pilot.set_mode(
    "shadow",
    event_id=change_ticket_id,
    timestamp=changed_at,
    reason="validated A/A delivery joins",
)

Mode	Actual delivery
"aa"	Authored control for both assignments; assignments and lifecycle evidence are still audited.
"shadow"	Authored control; Orchid proposal is computed and recorded but not rendered.
"active"	Frozen Orchid treatment only for assigned treatment learners; assigned controls remain authored.
"halted"	All future requests use authored control with KILL_SWITCH evidence. A halted experiment cannot be re-enabled.

set_mode is idempotent for the same event payload and records an operation event. A transition cannot move backward in time; use a new experiment_id to restart after a halt.

Record actual delivery and score it

pilot.record_rendered(
    served.decision.decision_id,
    event_id=lms_render_event_id,
    item_id=served.decision.chosen_item_id,
    content_version=served.chosen_content_version,
    timestamp=rendered_at,
)
pilot.record_submitted(
    served.decision.decision_id,
    event_id=lms_submission_event_id,
    item_id=served.decision.chosen_item_id,
    content_version=served.chosen_content_version,
    timestamp=submitted_at,
)
outcome = pilot.record_scored(
    served.decision.decision_id,
    outcome_event_id=lms_score_event_id,
    item_id=served.decision.chosen_item_id,
    content_version=served.chosen_content_version,
    outcome=correct,
    timestamp=scored_at,
)

These calls are append-only and idempotent for the same event ID/payload. The adapter enforces decision → rendered → submitted → scored, the selected item/version, and nondecreasing timestamps. record_scored first stores its score event, then persists/uses its outcome. It requires an LMS-global outcome_event_id.

Only active effective-treatment scores update learner-local state, and those updates leave aggregate empirical statistics frozen. Control, A/A, shadow, and halted scores remain durable audit evidence with apply_state=False.

Recovery method	Purpose
pilot.recover_scored_events()	Finish the outcome projection after a crash between score-event persistence and application.
pilot.rebuild_state_from_baseline()	Persist score evidence and rebuild treatment learner state from a freshly restored/fitted ranker baseline. Do not call on a live projection.
pilot.observe_decision(...)	Compatibility method for a score that already has a submission event. New integrations should call record_scored.

Import delayed assessments and export evidence

assessments = pilot.import_delayed_assessments(assessment_frame)
analysis = pilot.analysis_frame()

import_delayed_assessments requires a pandas DataFrame with assessment_event_id, user_id, course_run_id, assessment_form_version, timestamp, score, and independent. independent must be literal True; the method accepts a matching practice decision or an explicit course-run enrollment, including an enrolled learner who never practiced. It rejects unassigned learners and unrelated runs, and never feeds assessments into adaptive state.

Use enrollment_frame() for the complete randomized roster, assessment_frame() for imported assessments, decision_frame(completed_only=False) for joined delivery decisions/outcomes, and analysis_frame() for decisions plus rendered/submitted/scored IDs and timestamps, fallback/shadow/explanation evidence, and matching assessments. analysis_frame() has one row per decision and must not be the denominator for a learner-level efficacy estimate.

orchid_ranker.pilot_analysis.analyze_pilot_retention(...) is an experimental reference analysis for one course run. It joins the frozen enrollment roster to independent assessments, waits for every assessment window to close, and reports assigned-arm composite scores, assessment coverage, bounds on the treatment-minus-control difference under missing scores, and delivery-audit checks. It records the bootstrap seed and sample count so a report can be reproduced. Its exact report fields may change in a minor release; pre-register the outcome and analysis plan before using it for a real study. Unlike the general ranker, this day-based analysis requires Unix-second timestamps for enrollment due dates and assessments. See the one-course sample study.

Offline policy and evaluation APIs

The base adaptive policy is the recommended first deployment. Offline CQL is an opt-in promotion path, not a shortcut around a controlled learning study.

report = ranker.fit_policy(
    earlier_completed_decisions,
    evaluation_decisions=later_completed_decisions,
)

fit_policy trains tabular CQL from a training decision log and promotes it only if a strictly later, disjoint evaluation log passes the user-cluster-bootstrap rollout gate. By default it requires at least 30 evaluation events and 30 users. The evaluation window must also follow the base-model fitting data. Evaluation decisions must carry the current base-policy version; Orchid records it in policy_metadata during recommend_and_log. fit_policy_rolling(...) creates the same strict chronological split from a trailing source window.

Use ope_report(...) or bootstrap_ope_report(...) for logged-policy evaluation and shadow_report() to summarize traffic, outcome coverage, calibration/drift, and, when applicable, policy evidence. These reports require complete candidate lists, propensities, and linked outcomes; they do not prove a causal learning benefit on their own.