You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Three-model AI pipeline: classify, rewrite and roundups on Foundry with Flex (#581) - #599
Every article and every weekly roundup went through one gpt-5.2 call per item or per step. That cost about €23.46 last month and gave inconsistent categorization, rewrites that sometimes dropped sections, and roundups that needed several extra AI passes. This PR splits the work over three models on Azure AI Foundry (Sweden Central, GlobalStandard) and runs background work on the Flex tier (50% cheaper). Estimated cost from measured tokens: about €15/month on Flex (about €30 if everything fell back to Standard).
Clean markdown, only for included items. Output limit scales with article length (8k to 60k reasoning plus output tokens, a safety limit instead of a fixed small cap), so long articles are no longer cut
3a. Roundup writer
RoundupNewsWriter
gpt-6-astra (medium)
One writer call per section per week, replacing the writer, enhancer, condenser and relevance filter
3b. Roundup metadata
RoundupMetadata
gpt-6.1-sol (low)
Title, tags and introduction
Timeline backfill
TimelineExtraction
gpt-6-luna (low)
Extracts only the dated timeline events from already stored articles. It has its own profile because it needs no include/exclude strictness and costs about 20x less than sol
AI client: v1 API with per-step profiles (AiCategorization:Profiles:*), patient Flex retries with fallback to Standard after 30 minutes, and per-call usage logging. Interactive calls (ad-hoc add, apply-transcript) use Standard
Timeline events: new timeline_events table (migration 049), filled during processing. A manual, resumable backfill job (POST /api/admin/timeline-backfill/trigger, dashboard button) fills it for existing articles
Roundups: lead stories get full paragraphs, other items a one-liner. Code builds the "Other X News" completeness block, "Also relevant this week" and "Dates to know" from the timeline events
Infra:openai.bicep deploys the three models next to gpt-5.2 (kept as rollback). applications.bicep sets the per-profile app settings
Prompts: classify prompt gets significance and product rules and excludes event announcements and promotional tip lists. The rewrite prompt leaves out related-post lists. The timeline prompt explains what a milestone is instead of capping the number of events
Evidence
An evaluation harness (tests/TechHub.Infrastructure.Tests/Evaluation, skipped unless TECHHUB_EVAL=1) ran the real models against a restore of production:
Classification of 100 included articles: 92-93% stay included. Of 73 articles production skipped, gpt-6.1-sol and gpt-5.2 keep 95.9% excluded, gpt-6-luna only 89.0% and lost on medium reasoning (82.2%), which is why classification uses sol
Rewrites: owner review found them better structured and better written. A mechanical fact check of all numbers, URLs and code terms against the original articles found nothing invented or altered
Roundups (7 sections, one week): 0 invalid links, 0 missing lead stories and 0 numbers missing from the sources. Production had 1-3 missing leads in four sections, one link matching no article and a few unsourced numbers
Timeline: bundled release notes no longer produce an event per bullet
Go-live and rollback
Merging and deploying switches production to the new models. After the deploy:
Run the timeline backfill once. It uses the TimelineExtraction profile (gpt-6-luna), so for all ~9,000 articles it costs about €1.4 on Flex (about €2.8 on Standard), a one-off
Watch cost, Flex versus fallback share and processed_urls failures for a month
The appsettings.json defaults are the new models and Flex as well, so nothing in the application still points at gpt-5.2; the gpt-5.2 deployment stays in Azure only as a rollback target.
Rollback: set a profile's deploymentName back to gpt-5.2 with service tier default in infra/applications.bicep and redeploy the application phase.
Review notes
The evaluation stages are skipped without TECHHUB_EVAL=1 (6 skipped tests). They cost money and need az login
No E2E test covers the new dashboard "Backfill Timeline" button (no existing dashboard E2E)
The E2E phase 1 database performance tests exceed their thresholds against a local database restored from production (about 9,000 items). They are unrelated to this change and I did not run the Playwright E2E on the final code
The gpt-6-luna deployment is kept as a spare and unused
Roundup article selection uses created_at, as before
… gpt-5.2 (#581)
Replace the single model deployment in openai.bicep with a modelDeployments array,
deployed one at a time, each using the content-categorization RAI policy.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
AiCategorization:Profiles (Classification, Rewrite, RoundupNewsWriter, RoundupMetadata)
replace the single DeploymentName/ApiVersion. Requests are built by AiRequestBodyBuilder
(model, reasoning_effort, service_tier; no temperature) and sent to /openai/v1/chat/completions.
All profiles still point at gpt-5.2 on the default tier, so behaviour is unchanged.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- migration 049: content_timeline_events and content_timeline_extractions
- TimelineEventRepository replaces an article's events in one transaction
- events are written for processed items and for apply-transcript; apply-transcript now writes ai_metadata v2
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…lt blocks (#581)
- split articles per section into lead (full write-up) and brief (one-line mention) from section_roles
- one writer call (key facts + last week's headings) and one strict-schema metadata call per section
- code builds the completeness check, 'Also relevant this week' and 'Dates to know' blocks and the table of contents
- remove narrative enhancer, condenser, relevance filter, GetArticlesForWeekAsync and CondensingEnabled
- fix previous-roundup lookup that skipped last week's roundup
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ndup backfill (#581)
- ModelEvaluationTests: classification, rewrite, timeline and roundup stages against a restored production database (skipped unless TECHHUB_EVAL=1)
- the roundup metadata backfill stores the timeline events its classification already extracted
- docs: how to run and read the evaluation
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Splits TechHub’s AI processing into configurable classification, rewrite and roundup profiles, with Flex retries to reduce background-processing costs.
Changes:
Adds profile-based AI requests, usage logging and deployment configuration.
Stores timeline events and introduces a resumable admin backfill.
Simplifies roundup generation and expands supporting tests and documentation.
…imeline backfill (#581)
Timeline extraction shared the Classification profile, which moved to gpt-6.1-sol.
The backfill only extracts dated facts, so it does not need that strictness and
costs about 20x less on luna.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
An HTTP timeout can surface as TaskCanceledException even when ct is not cancelled. AiCategorizationService.SendWithRetryAsync lets it escape on the final attempt, and this filter excludes it from per-article error handling. One timed-out article therefore stops the entire backfill and the background service labels it an admin abort. Propagate cancellation only when ct is cancelled; otherwise log/count the failure and continue.
The excerpt is not displayed above the body on internal detail pages: ContentItemDetail.razor:15-38 renders it only for externally linked items, and ContentRepository.RenderHtmlIfNeeded renders only Content. Following this instruction can therefore leave opening context out of the displayed article. Keep the rewritten body self-contained, or render the excerpt above the body for these items.
UTC publication dates shift Brussels-local timeline dates
FromUnixTimeSeconds produces a UTC value, which BuildTimelineExtractionPrompt formats directly as yyyy-MM-dd. Publication-sourced events, however, use the Brussels date. For an article published at 23:30 UTC on 14 March, the prompt says 14 March while the parser uses 15 March. This can shift model-resolved dates such as “tomorrow” by a day. Convert the timestamp to Europe/Brussels before passing it to extraction, and add a regression test around local midnight.
Month- and quarter-precision events use the first day of their period as a placeholder, not an exact event date. For example, a launch announced for November is stored as November 1 and excluded from a roundup ending November 1, even though it could still be upcoming. Compare the represented period's end against the week end for coarse dates, while retaining the exact comparison for day precision. Apply the same rule in RoundupBlocksBuilder.BuildDatesToKnow, which repeats this filter, and cover overlapping months and quarters with regression tests.
Invalidate extraction eligibility after edits without event replacement
An ordinary admin edit calls UpdateEditDataAsync without a timeline replacement. Changing a scanned article's content, excerpt or publication date therefore leaves its old events and current-version extraction marker intact. This eligibility check skips that article on subsequent non-forced backfills, so the dashboard's Backfill Timeline action cannot repair the stale events. Invalidate the extraction marker when an edit changes extraction inputs without replacing events, or persist and compare the extracted article revision. Add a regression test that edits an article after a completed scan and verifies that it becomes eligible again.
An explicitly empty ReasoningEffort means that AiRequestBodyBuilder should omit reasoning_effort. Skipping it here instead retains the low/medium value from appsettings.json, or an old user secret even with -Force, so local requests differ from the production profile. Distinguish absent settings from explicitly empty ones and allow an empty reasoning setting through both this loop and Set-Secret. Update the PowerShell test that currently requires this value to be skipped.
Rebase publication events when capping future publication dates
ContentProcessingService uses WithDateEpoch to cap future-dated articles to the processing date, but this copy keeps publication-sourced events on the original future date assigned during classification. Capping an article from 20 January to 15 January thus leaves its "available now" event on 20 January, so an already-live release can appear as upcoming. Rebase publication-sourced events to the new Europe/Brussels date while preserving explicit dates, and extend the future-date regression test.
Convert publication timestamps to Brussels time before extraction
FromUnixTimeSeconds returns a UTC-offset value, and BuildTimelineExtractionPrompt formats that value directly as DATE. A publication at 2026-03-14 23:30 UTC therefore reaches the model as March 14, while TimelineEventParser.GetBrusselsDate anchors publication-derived events to March 15. This gives relative dates such as "tomorrow" an inconsistent reference day. Convert the publication timestamp to Europe/Brussels before sending it to extraction, and add a midnight-boundary regression test.
Handle month and quarter periods in upcoming event filtering
Month and quarter dates represent periods, not exact days. An October article announcing GA "later in Q4 2026" is normalized to 2026-10-01, so a roundup ending 2026-10-25 drops that still-upcoming release. Determine whether the represented period extends beyond the roundup week rather than comparing only its first day. Apply the same rule in SectionRoundupRepository.GetUpcomingDateEventsAsync, whose SQL currently removes these events before the builder sees them. Keep the coarse display precision and add current-month and current-quarter regression cases.
Empty reasoning effort is incorrectly treated as missing
scripts/Setup-UserSecrets.ps1:295
An explicitly empty ReasoningEffort means “omit the parameter,” but this check treats it as a missing setting. For example, the empty Rewrite effort in the updated test leaves local configuration inheriting "low" from appsettings.json, while production omits the parameter. -Force cannot correct this because both this check and Set-Secret reject empty values. Preserve present-but-empty reasoning settings while still skipping missing settings, and update the regression test to verify that the local request omits reasoning_effort.
FromUnixTimeSeconds returns a UTC-offset value, and BuildTimelineExtractionPrompt formats PublishedAt directly as yyyy-MM-dd. Near midnight, the prompt therefore uses a different publication day from TimelineEventParser.GetBrusselsDate: 14 March at 23:30 UTC is 15 March in Brussels. Relative dates such as “tomorrow” can consequently be extracted one day early. Convert this value to Europe/Brussels before passing it to extraction, and add a regression test asserting the prompt date at this boundary.
An explicitly empty ReasoningEffort means “omit reasoning_effort”, but skipping it here leaves the low/medium value from appsettings.json active, or preserves an older user-secret value. A deployed profile that omits this parameter is therefore copied locally with it enabled, potentially causing rejected requests for a model that does not support it. Distinguish missing settings from explicitly empty reasoning settings and preserve the latter, including through Set-Secret. Update the test that currently expects an empty reasoning setting to be skipped.
The reason will be displayed to describe this comment to others. Learn more.
🟡 Changes recommended
Explicit empty profile overrides are lost during local synchronization, and publication-derived event dates remain stale when article dates are capped.
The reason will be displayed to describe this comment to others. Learn more.
🔵 Needs a closer look
The production model switch and cross-layer persistence changes require human validation, especially with an unresolved invalidation gap and no final-code E2E run.
No open issues 🎉
1 previously missed issue
In code that hasn't changed since last review
Invalidate extraction state after markdown review approval
This invalidation only covers edits through UpdateEditDataAsync. Approving a markdown review still writes content_items.content directly in ContentReviewRepository.ApplyChangeAsync (lines 288–308), including replacement text edited by an admin. For an already-scanned article, that leaves the old events and current-version extraction marker intact, so an ordinary timeline backfill skips the changed text. Invalidate the events and extraction marker in the review-approval transaction when the content changes, using the same parent-row locking as this edit path. Add a scan → markdown review approval → ordinary backfill regression test.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
Closes #581
Why
Every article and every weekly roundup went through one
gpt-5.2call per item or per step. That cost about €23.46 last month and gave inconsistent categorization, rewrites that sometimes dropped sections, and roundups that needed several extra AI passes. This PR splits the work over three models on Azure AI Foundry (Sweden Central, GlobalStandard) and runs background work on the Flex tier (50% cheaper). Estimated cost from measured tokens: about €15/month on Flex (about €30 if everything fell back to Standard).What changes
Classificationgpt-6.1-sol(low)Rewritegpt-6.1-sol(low)RoundupNewsWritergpt-6-astra(medium)RoundupMetadatagpt-6.1-sol(low)TimelineExtractiongpt-6-luna(low)AiCategorization:Profiles:*), patient Flex retries with fallback to Standard after 30 minutes, and per-call usage logging. Interactive calls (ad-hoc add, apply-transcript) use Standardtimeline_eventstable (migration 049), filled during processing. A manual, resumable backfill job (POST /api/admin/timeline-backfill/trigger, dashboard button) fills it for existing articlesopenai.bicepdeploys the three models next togpt-5.2(kept as rollback).applications.bicepsets the per-profile app settingsEvidence
An evaluation harness (
tests/TechHub.Infrastructure.Tests/Evaluation, skipped unlessTECHHUB_EVAL=1) ran the real models against a restore of production:gpt-6.1-solandgpt-5.2keep 95.9% excluded,gpt-6-lunaonly 89.0% and lost onmediumreasoning (82.2%), which is why classification uses solGo-live and rollback
Merging and deploying switches production to the new models. After the deploy:
TimelineExtractionprofile (gpt-6-luna), so for all ~9,000 articles it costs about €1.4 on Flex (about €2.8 on Standard), a one-offprocessed_urlsfailures for a monthThe
appsettings.jsondefaults are the new models and Flex as well, so nothing in the application still points atgpt-5.2; thegpt-5.2deployment stays in Azure only as a rollback target.Rollback: set a profile's
deploymentNameback togpt-5.2with service tierdefaultininfra/applications.bicepand redeploy the application phase.Review notes
TECHHUB_EVAL=1(6 skipped tests). They cost money and needaz logingpt-6-lunadeployment is kept as a spare and unusedcreated_at, as before