Skip to content

refactor(narratives): decouple Gemini model configuration per role - #541

Merged
nick-nlb merged 1 commit into
datacommonsorg:mainfrom
nick-nlb:narratives-per-role-models
Oct 21, 2026
Merged

nick-nlb merged 1 commit into
datacommonsorg:mainfrom
nick-nlb:narratives-per-role-models

Conversation

@nick-nlb

Copy link
Copy Markdown
Collaborator

Overview

After #537 and #538, config_model.py defines seven GeminiRole members (CLASSIFY, DISCOVERY, SYNTHESIS, CHART_CONFIG, CHART_VALIDATION, FOLLOW_UP, and COMPACTION) and resolves each role's model, temperature, and thinking level through AgentConfig.resolve_role_config. However, GeminiConfig still carries only classify_model and discovery_model and routes six roles (DISCOVERY, SYNTHESIS, CHART_CONFIG, CHART_VALIDATION, FOLLOW_UP, and COMPACTION) to self.gemini.discovery_model, preventing operators from configuring a lighter or faster model for DISCOVERY, CHART_CONFIG, FOLLOW_UP, or COMPACTION without also changing the model used for SYNTHESIS.

This PR gives every GeminiRole its own <role>_model field on GeminiConfig (classify_model, discovery_model, synthesis_model, chart_config_model, chart_validation_model, follow_up_model, and compaction_model) and in schemas/agent-config.schema.json, and updates AgentConfig.resolve_role_config to resolve each role's model from its matching GeminiConfig attribute via _ROLE_MODEL_ATTRS. Because all workflow call sites already resolve their model via config.resolve_role_config(role).model, each LLM-calling role can now be configured independently without changing any workflow call sites.

Related Issues

Changes Made

  • Per-role model configuration and resolution (config_model.py): Adds synthesis_model, chart_config_model, chart_validation_model, follow_up_model, and compaction_model to GeminiConfig (each defaulting to DEFAULT_GEMINI_MODEL), parses all seven gemini.<role>_model keys in parse_agent_config via _read_model_name, and resolves each GeminiRole's model in AgentConfig.resolve_role_config via _ROLE_MODEL_ATTRS.
  • Schema, example configuration, and workflow docstrings (schemas/, workflows/): Adds the five new gemini.<role>_model properties to schemas/agent-config.schema.json and schemas/agent-config.example.json, and updates the config parameter docstrings in workflows/chart_config.py, workflows/follow_up.py, and workflows/transcript.py to reference their respective gemini.<role>_model keys.
  • Tests and documentation (config_model_test.py, docs/deployment.md): Parametrizes model-fallback and independent per-role model override tests across all seven GeminiRole members in config_model_test.py, and updates docs/deployment.md to document all seven gemini.<role>_model keys and their defaults.

Testing Done

  • pnpm test:agent (uv run pytest): 863 passed (826 on main at 16cc4e50, with 37 net new parametrized per-role model fallback and independent override test cases in config_model_test.py).
  • pnpm test:ui (vitest run): 26 files, 286 passed.
  • pnpm lint:ui (tsc --noEmit and biome check): passed across 48 checked files.
  • pnpm lint:agent (ruff format --check, ruff check, and mypy --strict): passed across all 64 source files with no overrides.
  • pnpm lint:deploy: terraform fmt -check, shellcheck, and hadolint passed (terraform validate has a pre-existing local Terraform 1.15 backend-config error unrelated to this PR; no deploy/ files are modified).
  • npx --yes -p ajv-cli@5.0.0 -p ajv-formats@3.0.1 ajv validate --spec=draft2020 --strict=false -c ajv-formats -s schemas/agent-config.schema.json -d defaults/agent-config.json and -d schemas/agent-config.example.json: both valid.
  • No UI files changed, so pnpm build was not run.

Checklist:

  • Unit tests passed
  • Integration tests passed (N/A)
  • Manual verification
  • Any updated goldens or fixtures were reviewed and are intentional (no goldens; test fixtures and assertions in config_model_test.py updated to cover all seven gemini.<role>_model fields)

Manual

Verified with uv run python against parse_agent_config and defaults/agent-config.json:

  • Default configuration: Loaded defaults/agent-config.json (which specifies only gemini.classify_model) through parse_agent_config. Verified that no warnings were logged, GeminiRole.CLASSIFY resolved to gemini-3.5-flash-lite at MINIMAL thinking, and DISCOVERY, SYNTHESIS, CHART_CONFIG, CHART_VALIDATION, FOLLOW_UP, and COMPACTION all resolved to gemini-3.8-flash (DEFAULT_GEMINI_MODEL).
  • Independent per-role override: Set {"gemini": {"synthesis_model": "gemini-3-pro", "follow_up_model": "gemini-3.5-flash-lite"}} and verified through resolve_role_config that SYNTHESIS resolved to gemini-3-pro and FOLLOW_UP resolved to gemini-3.5-flash-lite while DISCOVERY, CHART_CONFIG, CHART_VALIDATION, and COMPACTION remained on gemini-3.8-flash.
  • Blank model fallback: Set {"gemini": {"chart_config_model": " "}} and verified that parse_agent_config logged a single warning (Agent configuration key 'gemini.chart_config_model' is blank; using 'gemini-3.8-flash') and resolved GeminiRole.CHART_CONFIG to gemini-3.8-flash.

Risk & Rollback

All new gemini.<role>_model keys (synthesis_model, chart_config_model, chart_validation_model, follow_up_model, and compaction_model) are optional and default to gemini-3.8-flash (DEFAULT_GEMINI_MODEL), preserving identical model resolution when omitted (defaults/agent-config.json requires no changes). There are no active production deployments of Narratives. Rollback is reverting the PR.

Follow-ups

  • Continue with the agent loop updates, starting with the deterministic analytics library.

Checklist

  • I have read AGENTS.md and followed CODING_GUIDELINES.md, plus FRONTEND.md for UI changes.
  • I have run the app's lint, test, and build commands, as documented in that application's guide.
  • I have commented my code, particularly in hard-to-understand areas.
  • My changes generate no new warnings.

Note: Only Maintainers can approve and merge PRs. Expected initial review time: 3 business days.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces granular, per-role Gemini model configurations within the narratives agent. Instead of routing most roles through a single discovery_model, individual configuration fields have been added for synthesis_model, chart_config_model, chart_validation_model, follow_up_model, and compaction_model alongside the existing classify_model and discovery_model. The GeminiConfig class, parser, JSON schema, example configuration, and documentation have been updated to support these new fields, and the corresponding unit tests have been expanded to verify independent configuration and fallback behaviors. There are no review comments to evaluate, and I have no additional feedback to provide.

@nick-nlb
nick-nlb marked this pull request as ready for review October 21, 2026 19:17

@juliawu juliawu left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice! LGTM!

@nick-nlb
nick-nlb merged commit 21e6800 into datacommonsorg:main Oct 21, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants