[kayaal34]

Closes #353 (the LLMDescriptionScorer part. TypeSafeDescriptionScorer isn't on dev yet).

Problem

• _init_event_loop took the loop from asyncio.get_event_loop(). When predict() is called from async code (a notebook, a FastAPI handler), that returns the running loop, and run_until_complete raises RuntimeError: This event loop is already running.
• On Python 3.14 get_event_loop() raises when no loop is set, so every instance fell back to new_event_loop(), and clear_cache removed it with delattr without closing it: one leaked loop per HPO trial.

Fix

• Each scorer owns a private loop (asyncio.new_event_loop()) and never touches the caller's loop. I kept one loop per instance on purpose rather than calling asyncio.run per batch: the generator's AsyncOpenAI client pools connections bound to the loop they were opened on, so running every batch on the same loop keeps that pool valid.
• _run_async(coro): with no running loop in the current thread, it runs the batch directly on the scorer's loop. Inside a running loop, it runs the batch on a one-off worker thread and blocks until it finishes. predict() stays synchronous.
• The loop is closed on refit (_init_event_loop) and in clear_cache.

Tests

• test_description_scorer_llm_predict_inside_running_loop: predict() called from a coroutine returns the same result as the sync call, and the scorer still works from sync code afterwards.
• test_description_scorer_llm_closes_its_event_loop: refit and clear_cache close the loop.
• Both new tests fail on dev and pass with this change. tests/modules and tests/pipeline filtered to description/LLM: 24 passed. ruff and mypy on the changed files pass.

This change touches the same lines as #361, so whichever merges second will need a one-line conflict resolved. I can rebase if needed.

[Copilot]

Copilot review overview

🟡 Changes recommended

Unresolved event-loop thread-safety, concurrency, and lifecycle concerns remain.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 1

Open (1)

• Prevent cross-thread event loop use and concurrent run_until_complete calls · New

What changed in this PR

Updates LLMDescriptionScorer to support synchronous prediction from running asyncio loops and manage private event-loop cleanup.

Changes:

• Adds worker-thread execution for running-loop callers.
• Closes loops during refit and cache clearing.
• Adds regression tests for async prediction and loop closure.

File	Summary
tests/​modules/​scoring/​test_description_llm.py	Tests running-loop prediction and loop closure.
src/​autointent/​modules/​scoring/​_description/​llm_encoder.py	Implements private loop management and worker-thread execution.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[kayaal34]

Good catch, fixed in the latest commit. The scorer's loop now runs run_forever() on one dedicated daemon thread started in _init_event_loop, and _run_async submits work with asyncio.run_coroutine_threadsafe(...).result(). So the loop (and the AsyncOpenAI client bound to it) is only ever driven from that thread, for sync callers, async callers and concurrent callers alike. _close_event_loop stops the loop, joins the thread and closes the loop. Added test_description_scorer_llm_predict_from_several_threads (8 concurrent predict() calls) and the close test now also checks the thread has stopped.

[kayaal34]

Pushed a follow-up addressing the Copilot review: the scorer's private event loop now lives on a single dedicated thread, so it's never driven from more than one thread (details in the review thread).

Locally: tests/modules/scoring/test_description_llm.py 8 passed, ruff check, ruff format --check and mypy clean on the changed files.

The CI runs for this PR ended as failed without starting any job (no logs), which looks like the fork workflows weren't allowed to start rather than a test failure. Could a maintainer approve or re-run them? Thanks!