Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ts/packages/aiclient/src/decisionModels.ts
Original file line number Diff line number Diff line change
Expand Up @@ -518,7 +518,7 @@ function classificationCode(
: c.errorCategory;
}

class DecisionValidationError extends Error {
export class DecisionValidationError extends Error {
constructor(readonly code: DecisionFailureCode = "invalid_response") {
super(code);
}
Expand Down
26 changes: 24 additions & 2 deletions ts/packages/memory/service/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -841,10 +841,32 @@ have been removed. The `concurrency` and `frozenPreparation` programmatic settin
`TYPEAGENT_MEMORY_TOPIC_CONCURRENCY` and
`TYPEAGENT_MEMORY_TOPIC_FROZEN_PREPARATION` now fail explicitly.
Remove these controls instead of selecting another scheduling mode.
The `curated-lazy-v10:<hash>` recipe invalidates older-policy progress and receipts;
The default recipe is
`curated-lazy-v10:<hash>:generative-partition-v1:output-contract`.
It invalidates original-policy progress and receipts;
do not relabel saved artifacts. See the [topic curation guide](../../../tools/memory-maintenance-qualification/topic-curation-README.md)
for remaining settings, checkpoint reuse and replay migration.

Prepared-topic reconciliation now uses restricted-output generative decisions
by default. The configured chat model (Luna by default) chooses admission,
deferral and canonical group membership through typed `answers` with complete
probability distributions and confidence. This is not a native Decision 1 call.
Raw-only preparation, descriptive authoring, summaries, editorial validation
and organization remain generative. Authoring must preserve the decided partition.
Non-unit probabilities, inconsistent leaders, synthesis requests or changed
authored membership produce a logged failure and explicit original-generative
resolution; provider, admission and cancellation failures still propagate.
Context-aware packing includes the decision schema, original fallback and a
conservative bound on the authoring payload.

To roll back, set
`TYPEAGENT_MEMORY_TOPIC_GENERATIVE_DECISION_STAGE=original`, or pass
`topicCuration: { generativeDecisionStage: "original" }` to the service.
`output-contract` is the default; `decomposed` retains a choice-only diagnostic
control. All three are generative settings and conflict with native Decision
activation. Switching stages changes the policy and packet identity, so start
fresh work rather than relabeling receipts.

The LLM decides semantic equivalence, compatible scope and whether a concept
has a substantive, specific consumer purpose. Host rules validate supplied
identities, exact membership accounting, cited sources and bounded payloads;
Expand Down Expand Up @@ -894,7 +916,7 @@ catalog synchronization is still full-catalog work, not an incremental operation
Set `FileMemoryServiceOptions.decisionTopics: true`, or
`TYPEAGENT_MEMORY_DECISION_TOPICS=true` in the native/MCP/CLI/benchmark host,
to select the experimental `curated-decision-v2` pipeline. The default remains
the generative `curated-lazy-v10:<hash>` policy. Decision curation requires curated
the restricted-output generative policy described above. Decision curation requires curated
topics and explicitly rejects a conflicting custom `generateTopics` adapter;
ordinary custom-adapter receiver/cancellation behavior is unchanged.

Expand Down
24 changes: 24 additions & 0 deletions ts/packages/memory/service/src/topicCuration.ts
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,10 @@ import {
reconcileFrozenTopics,
runFrozenTopicWindows,
} from "./topicFrozenPreparation.js";
import {
withGenerativeTopicDecisions,
type TopicGenerativeDecisionReceipt,
} from "./topicGenerativeDecision.js";
import {
indexTopicDecisionEvidence,
topicClusterSignature,
Expand Down Expand Up @@ -89,6 +93,9 @@ export interface TopicCurationCall {
signal: AbortSignal;
}
export interface TopicCurationOptions extends TopicCurationSettings {
onGenerativeDecision?:
| ((receipt: TopicGenerativeDecisionReceipt) => void)
| undefined;
candidates?: Candidate[];
replayPackets?: TopicCurationPacket[];
onPacket?: (packet: TopicCurationPacket) => void;
Expand Down Expand Up @@ -441,6 +448,10 @@ function prepareTopicPacket(
payload,
schema,
batch.map((node) => [node.id, node.memberIds ?? [node.id]]),
...(options.generativeDecisionStage &&
options.generativeDecisionStage !== "original"
? ["generative-partition-v1", options.generativeDecisionStage]
: []),
]);
recordSearchTiming("topic.curation.packet", performance.now() - started, {
ordinal,
Expand Down Expand Up @@ -542,6 +553,19 @@ function createTopicCuration(
"Topic packet replay requires lazy curation with ordered installation",
);
options = { ...options };
if (
options.generativeDecisionStage &&
options.generativeDecisionStage !== "original"
) {
complete = withGenerativeTopicDecisions(
complete,
options.generativeDecisionStage,
options.onGenerativeDecision,
options.inspectRequest ?? complete.inspectRequest,
);
if (complete.inspectRequest)
options.inspectRequest = complete.inspectRequest;
}
if (options.compactIds) {
complete = compactTopicCompletion(complete, options.inspectRequest);
if (complete.inspectRequest)
Expand Down
17 changes: 16 additions & 1 deletion ts/packages/memory/service/src/topicCurationSettings.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,17 @@

import { viewHash } from "./viewMerge.js";

export type TopicGenerativeDecisionStage = "decomposed" | "output-contract";

export interface TopicCurationSettings {
inputTokenTarget?: number;
priorContextFraction?: number;
priorDeltas?: boolean;
compactIds?: boolean;
generativeDecisionStage?:
| TopicGenerativeDecisionStage
| "original"
| undefined;
}

export function validateTopicCurationSettings(
Expand All @@ -16,6 +22,10 @@ export function validateTopicCurationSettings(
if (
"concurrency" in options ||
"frozenPreparation" in options ||
(options.generativeDecisionStage !== undefined &&
!["original", "decomposed", "output-contract"].includes(
options.generativeDecisionStage,
)) ||
(options.inputTokenTarget !== undefined &&
(!Number.isSafeInteger(options.inputTokenTarget) ||
options.inputTokenTarget < 1)) ||
Expand All @@ -39,6 +49,8 @@ export function resolveTopicCurationSettings(
...settings,
priorDeltas: settings.priorDeltas ?? true,
compactIds: settings.compactIds ?? true,
generativeDecisionStage:
settings.generativeDecisionStage ?? "output-contract",
};
}

Expand All @@ -48,11 +60,14 @@ export function topicCurationPolicy(
): string {
if (!curated) return "raw-v1";
const resolved = resolveTopicCurationSettings(settings);
return `curated-lazy-v10:${viewHash([
const recipe = `curated-lazy-v10:${viewHash([
resolved.inputTokenTarget ?? null,
resolved.priorContextFraction ?? 0,
resolved.priorDeltas,
resolved.compactIds,
"frozen-two-packet-windows",
]).slice(0, 16)}`;
return resolved.generativeDecisionStage !== "original"
? `${recipe}:generative-partition-v1:${resolved.generativeDecisionStage}`
: recipe;
}
131 changes: 131 additions & 0 deletions ts/packages/memory/service/src/topicDecisionContract.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,131 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.

import {
normalizeDecisionAnswers,
type DecisionAnswer,
type DecisionQuestion,
type StructuredOutputJsonSchema,
} from "@typeagent/aiclient";

export type TopicChoiceQuestions = Record<
string,
Extract<DecisionQuestion, { type: "choice" }>
>;

export class TopicDecisionContractError extends Error {}

export function topicChoiceSchema(
questions: TopicChoiceQuestions,
): StructuredOutputJsonSchema {
if (!Object.keys(questions).length)
throw new TopicDecisionContractError("Empty topic decision task");
return {
name: "topic_decision_answers",
strict: true,
schema: {
type: "object",
additionalProperties: false,
required: ["answers"],
properties: {
answers: {
type: "object",
additionalProperties: false,
required: Object.keys(questions),
properties: Object.fromEntries(
Object.entries(questions).map(([id, question]) => {
const choices = Object.keys(question.criteria);
if (!choices.length)
throw new TopicDecisionContractError(
"Empty topic decision options",
);
return [
id,
{
type: "object",
additionalProperties: false,
required: [
"type",
"choice",
"probabilities",
"confidence",
],
properties: {
type: {
type: "string",
enum: ["choice"],
},
choice: {
type: "string",
enum: choices,
},
probabilities: {
type: "object",
additionalProperties: false,
required: choices,
properties: Object.fromEntries(
choices.map((choice) => [
choice,
{
type: "number",
minimum: 0,
maximum: 1,
},
]),
),
},
confidence: {
type: "number",
minimum: 0,
maximum: 1,
},
},
},
];
}),
),
},
},
},
};
}

export function topicDecisionRecord(value: unknown): Record<string, unknown> {
if (!value || typeof value !== "object" || Array.isArray(value))
throw new TopicDecisionContractError("Invalid topic decision object");
return value as Record<string, unknown>;
}

export function parseTopicChoiceAnswers(
response: unknown,
questions: TopicChoiceQuestions,
): Record<string, Extract<DecisionAnswer, { type: "choice" }>> {
const record = topicDecisionRecord(response);
if (Object.keys(record).length !== 1 || !("answers" in record))
throw new TopicDecisionContractError(
"Invalid topic decision response fields",
);
for (const value of Object.values(topicDecisionRecord(record.answers))) {
const answer = topicDecisionRecord(value);
const keys = ["type", "choice", "probabilities", "confidence"];
if (
Object.keys(answer).length !== keys.length ||
Object.keys(answer).some((key) => !keys.includes(key)) ||
!("confidence" in answer) ||
typeof answer.confidence !== "number"
)
throw new TopicDecisionContractError(
"Invalid topic decision answer fields",
);
}
const answers = normalizeDecisionAnswers(record, questions, "azure");
return Object.fromEntries(
Object.entries(answers).map(([id, answer]) => {
if (answer.type !== "choice")
throw new TopicDecisionContractError(
"Topic decision requires a choice answer",
);
return [id, answer];
}),
);
}
9 changes: 8 additions & 1 deletion ts/packages/memory/service/src/topicFrozenPreparation.ts
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,8 @@ export async function reconcileFrozenTopics(
};
delete inner.replayPackets;
delete inner.onPacket;
delete inner.generativeDecisionStage;
delete inner.onGenerativeDecision;
const prepared = await parallel(
batches.map(
(candidates) => (preparationSignal: AbortSignal) =>
Expand Down Expand Up @@ -154,7 +156,12 @@ export async function reconcileFrozenTopics(
true,
undefined,
current,
{ ...inner, candidates },
{
...inner,
candidates,
generativeDecisionStage: options.generativeDecisionStage,
onGenerativeDecision: options.onGenerativeDecision,
},
),
);
validate(reconciled);
Expand Down
1 change: 1 addition & 0 deletions ts/packages/memory/service/src/topicGeneration.ts
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,7 @@ function configuredGenerativeSettings(
"priorContextFraction",
"priorDeltas",
"compactIds",
"generativeDecisionStage",
].includes(key),
)
)
Expand Down
Loading
Loading