@jam-overture/loom/telemetry

CalibrationVerdicttype

What became of a proposal, once it is something a claim can be scored against.

type CalibrationVerdict = "survived" | "rejected";

UnjudgedReasontype

Why a proposal yielded no verdict. Kept apart from rejected on purpose: a commit that failed on the database says nothing about whether the model was right, and folding those into the rejections would make the runtime look overconfident every time infrastructure broke.

type UnjudgedReason = "awaiting-answer" | "failed" | "unsettled";

UNJUDGED_REASONSvalue

The buckets a fold has to open before the first proposal arrives, so a reason nobody hit still reads as zero rather than as missing.

const UNJUDGED_REASONS: readonly UnjudgedReason[]

verdictOffunction

A repair is scored as its own proposal, not merged into the one it replaced. Both graded themselves, and the refusal that prompted the repair is exactly the case where the grade was wrong — averaging it away would hide the datapoint calibration exists to collect.

const verdictOf: (proposal: ProposalEpisode) => CalibrationVerdict | UnjudgedReason

CALIBRATION_BUCKET_COUNTvalue

const CALIBRATION_BUCKET_COUNT = 10

CalibrationScoretype

type CalibrationScore = {
    readonly judged: number;
    readonly survived: number;
    /** null when nothing was judged — zero out of zero is not zero. */
    readonly observedRate: number | null;
    readonly meanConfidence: number | null;
    /**
     * `meanConfidence - observedRate`. Positive means the model claimed more than
     * it delivered. null when there is nothing to compare.
     */
    readonly gap: number | null;
};

ConfidenceBuckettype

Half-open [lower, upper), except the last bucket, which is closed so 1 has a home.

type ConfidenceBucket = CalibrationScore & {
    readonly lower: number;
    readonly upper: number;
};

PolicyCalibrationtype

A calibration reading confined to the claims one policy judged.

type PolicyCalibration = {
    /**
     * Which policy judged these claims, as its own dispositions named it.
     *
     * `null` means this window never saw the judgment — the page opened after the
     * disposition and before the commit, say. Distinct from the recorded string
     * `UNATTRIBUTED_POLICY_ID`, which is a judgment that really was made before the
     * Gate wrote down which policy made it. One is a gap in the reader, the other
     * is a gap in the record, and merging them would let a fixed record look like a
     * short page.
     */
    readonly policyId: string | null;
    readonly overall: CalibrationScore;
    readonly buckets: readonly ConfidenceBucket[];
    /**
     * The distinct fingerprints the judgments in this segment named, sorted.
     *
     * A name is host-declared, and hosts are asked to rename a policy they edit.
     * More than one fingerprint under one name means that did not happen, and this
     * row is pooling gates that differ by more than what they are called — the same
     * error the segments exist to correct, one level down. `rulesetContinuityOf`
     * reads this list; the list itself is carried so a host can match a digest to
     * the configuration it is looking at.
     */
    readonly fingerprints: readonly string[];
    /**
     * Judged claims in this segment whose disposition named no fingerprint. Kept
     * beside the list rather than folded into it: one fingerprint plus twelve
     * unfingerprinted judgments is not a segment shown to be constant, and a list
     * of length one would say it was.
     */
    readonly unfingerprinted: number;
};

CalibrationReporttype

type CalibrationReport = {
    readonly overall: CalibrationScore;
    readonly buckets: readonly ConfidenceBucket[];
    /**
     * The same claims again, split by the policy that judged each one. Every
     * judged proposal appears in exactly one segment, so the segments sum to
     * `overall` — the split is a partition, not a sample.
     *
     * Ordered by policy name, with the unrecorded segment last, so a reader
     * comparing two windows is comparing rows in the same order.
     */
    readonly byPolicy: readonly PolicyCalibration[];
    /** Proposals that produced no verdict, by why. Never in a denominator. */
    readonly unjudged: Readonly<Record<UnjudgedReason, number>>;
    /**
     * Records the fold could not attribute to a proposal, carried through rather
     * than dropped — the same reason `EpisodeFold` returns them. A rate computed
     * over a window that silently lost records is a rate over a denominator it
     * changed without saying so.
     */
    readonly unattributed: number;
    /**
     * Proposals the runtime authored rather than a model — the inverse behind a
     * revert, say. Segmented rather than dropped: they carry a confidence because
     * every proposal does, but nobody graded it, so scoring them would measure the
     * constant the runtime stamps. Reporting the count keeps that visible instead
     * of leaving a reader to wonder where the undos went.
     */
    readonly runtimeAuthored: number;
};

calibrationOffunction

Shown in use on What every ask leaves behind — Was the confidence worth anything?

const calibrationOf: (fold: EpisodeFold) => CalibrationReport

EpisodeAnswertype

type EpisodeAnswer = "confirmed" | "discarded";

FailureStagetype

type FailureStage = "interpretation" | "assessment" | "repair" | "application" | "custody" | "commit";

EpisodeFailuretype

type EpisodeFailure = TelemetryFailure & {
    readonly stage: FailureStage;
};

ProposalEpisodetype

type ProposalEpisode = {
    readonly proposalId: ProposalId;
    /** Set when this proposal replaced one the Gate refused. */
    readonly repairOf?: ProposalId;
    readonly provenance: Provenance;
    readonly rationale: string;
    readonly delta: TreeDelta;
    readonly proposedAt: string;
    readonly assessment?: AssessmentSummary;
    readonly disposition?: Disposition;
    /** The Gate held it and custody succeeded, so a human was asked. */
    readonly held: boolean;
    /** A refusal that was handed back for one more attempt. */
    readonly repairRequested: boolean;
    readonly answer?: EpisodeAnswer;
    /**
     * Who answered. Distinct from `provenance.actor`, which is who asked — a hold
     * exists to put a second person in the way of a change, and an episode that
     * showed only the asker would make the two look like one.
     */
    readonly answeredBy?: string;
    /** Applied in memory. Present without `committedRevision` means it did not persist. */
    readonly appliedRevision?: number;
    readonly committedRevision?: number;
    readonly failure?: EpisodeFailure;
    readonly settledAt?: string;
};

EpisodeResolutiontype

type EpisodeResolution = {
    readonly kind: "committed";
    readonly proposalId: ProposalId;
    readonly revision: number;
} | {
    readonly kind: "refused";
    readonly proposalId: ProposalId;
} | {
    readonly kind: "awaiting-answer";
    readonly proposalId: ProposalId;
} | {
    readonly kind: "discarded";
    readonly proposalId: ProposalId;
} | {
    readonly kind: "not-interpreted";
    readonly failure: EpisodeFailure;
} | {
    readonly kind: "not-writable";
    readonly failure: EpisodeFailure;
} | {
    readonly kind: "failed";
    readonly proposalId: ProposalId;
    readonly failure: EpisodeFailure;
}
/** Nothing in this window settled it — still in flight, or the page ends mid-episode. */
 | {
    readonly kind: "open";
};

EpisodeResolutionKindtype

type EpisodeResolutionKind = EpisodeResolution["kind"];

EPISODE_RESOLUTION_KINDSvalue

const EPISODE_RESOLUTION_KINDS: readonly EpisodeResolutionKind[]

IntentEpisodetype

type IntentEpisode = {
    readonly intentId: IntentId;
    readonly treeId: TreeId;
    /** Absent when the window opened after the intent was received. */
    readonly intent?: IntentSummary;
    /**
     * The policy this intent was last resolved to be judged under. Absent when
     * the window opened after resolution, or when the records predate it.
     *
     * "Last" matters for a proposal that was held and later confirmed: the Gate
     * resolves again on the second look, so a host that narrowed its policy in
     * between shows two resolutions and this keeps the one that decided the
     * outcome. Each proposal's disposition carries the policy that judged that
     * proposal, which is the finer-grained truth.
     */
    readonly policyId?: string;
    readonly startedAt: string;
    readonly proposals: readonly ProposalEpisode[];
    readonly resolution: EpisodeResolution;
};

EpisodeFoldtype

type EpisodeFold = {
    readonly episodes: readonly IntentEpisode[];
    /**
     * Records that name a proposal this window never saw proposed. Returned
     * rather than discarded: a fold that quietly dropped them would report a
     * refusal rate over a denominator it had silently changed.
     */
    readonly unattributed: readonly RecordedTelemetry[];
};

episodesOffunction

Shown in use on What every ask leaves behind — A record is not the story. An episode is.

const episodesOf: (records: readonly RecordedTelemetry[]) => EpisodeFold

EpisodeTallytype

What a window of episodes adds up to.

type EpisodeTally = {
    readonly episodes: number;
    readonly proposals: number;
    /** Proposals the Gate would not apply on its own. */
    readonly held: number;
    /** Proposals made to replace one the Gate refused. */
    readonly repairs: number;
    /**
     * Every kind, including the ones that did not happen. A resolution absent from
     * this map and one that occurred zero times are different claims, and only one
     * of them is true.
     */
    readonly byResolution: Readonly<Record<EpisodeResolutionKind, number>>;
};

tallyEpisodesfunction

const tallyEpisodes: (episodes: readonly IntentEpisode[]) => EpisodeTally

telemetryFailureSchemaschema

A failure, as telemetry stores it: the code, and a sentence the runtime wrote about it. code is an open string rather than a mirror of §1/§2/§5's error taxonomies on purpose — adding an error code somewhere else in the codebase must not make records written yesterday fail to parse today.

const telemetryFailureSchema: z.ZodObject<…>

TelemetryFailuretype

type TelemetryFailure = {
    readonly code: string;
    readonly detail: string;
};

intentSummarySchemaschema

The intent, minus what was said. Origin, scope and the revision it was aimed at are what an analysis needs; utteranceLength keeps "a one-word ask" and "three paragraphs" distinguishable without storing either.

const intentSummarySchema: z.ZodObject<…>

IntentSummarytype

type IntentSummary = {
    readonly intentId: IntentId;
    readonly origin: IntentOrigin;
    readonly actor?: string;
    readonly baseRevision: number;
    readonly scopeNodeId?: NodeId;
    readonly utteranceLength: number;
    readonly observedAt: string;
};

assessmentSummarySchemaschema

What the Gate was looking at, in the shape a query can group by. The disposition that follows says stakes, reversibility and confidence; this says how big the change was and what it touched, which is what makes "the Gate refuses this class of change" measurable rather than anecdotal.

const assessmentSummarySchema: z.ZodObject<…>

AssessmentSummarytype

type AssessmentSummary = {
    readonly proposalId: ProposalId;
    readonly stakes: StakeLevel;
    readonly reversible: boolean;
    readonly operationCount: number;
    readonly insertedNodeCount: number;
    readonly removedNodeCount: number;
    readonly movedNodeCount: number;
    readonly configuredNodeCount: number;
    readonly touchedPrimitiveTypes: readonly PrimitiveType[];
    /** Absent on a record written before the field existed, never defaulted. */
    readonly removedPrimitiveTypes?: readonly PrimitiveType[];
    readonly relocatedPrimitiveTypes?: readonly PrimitiveType[];
    readonly relocatedNodeCount?: number;
    readonly shallowestAffectedDepth: number;
    /** Absent on a record written before the field existed, never defaulted. */
    readonly affectedNodeCount?: number;
    /** Absent on a record written before the field existed, never defaulted. */
    readonly configuredPropKeys?: readonly string[];
    readonly retainedNodeCount: number;
    readonly irreversibilityReasons: readonly string[];
    /** Absent when no out-of-tree reason fired, and before the field existed. */
    readonly outOfTreeEffectTypes?: readonly PrimitiveType[];
    /** Absent on a record written before the field existed, never defaulted. */
    readonly stakeFactorCodes?: readonly StakeFactorCode[];
};

telemetryEventSchemaschema

const telemetryEventSchema: z.ZodDiscriminatedUnion<…>

TelemetryEventtype

type TelemetryEvent = {
    readonly type: "intent-received";
    readonly intent: IntentSummary;
} | {
    readonly type: "policy-resolved";
    readonly intentId: IntentId;
    readonly policyId: string;
    /** Absent on a record written before the Gate fingerprinted policies. */
    readonly policyFingerprint?: string;
} | {
    readonly type: "interpretation-failed";
    readonly intentId: IntentId;
    readonly failure: TelemetryFailure;
} | {
    readonly type: "change-proposed";
    readonly proposal: ProposedChange;
} | {
    readonly type: "assessment-failed";
    readonly proposalId: ProposalId;
    readonly failure: TelemetryFailure;
} | {
    readonly type: "change-assessed";
    readonly assessment: AssessmentSummary;
} | {
    readonly type: "disposition-decided";
    readonly proposalId: ProposalId;
    readonly disposition: Disposition;
} | {
    readonly type: "repair-requested";
    readonly refusedProposalId: ProposalId;
    readonly reason: DispositionReason;
} | {
    readonly type: "repair-failed";
    readonly refusedProposalId: ProposalId;
    readonly failure: TelemetryFailure;
} | {
    readonly type: "change-applied";
    readonly proposalId: ProposalId;
    readonly revision: number;
} | {
    readonly type: "application-failed";
    readonly proposalId: ProposalId;
    readonly failure: TelemetryFailure;
} | {
    readonly type: "intent-not-writable";
    readonly intentId: IntentId;
    readonly failure: TelemetryFailure;
} | {
    readonly type: "proposal-held";
    readonly proposalId: ProposalId;
} | {
    readonly type: "hold-failed";
    readonly proposalId: ProposalId;
    readonly failure: TelemetryFailure;
} | {
    readonly type: "hold-confirmed";
    readonly proposalId: ProposalId;
    readonly actor?: string;
} | {
    readonly type: "hold-discarded";
    readonly proposalId: ProposalId;
    readonly actor?: string;
} | {
    readonly type: "change-committed";
    readonly proposalId: ProposalId;
    readonly revision: number;
} | {
    readonly type: "commit-failed";
    readonly proposalId: ProposalId;
    readonly failure: TelemetryFailure;
};

TelemetryEventTypetype

type TelemetryEventType = TelemetryEvent["type"];

TELEMETRY_EVENT_TYPESvalue

Every event the journal can hold, in the order the runtime writes them.

const TELEMETRY_EVENT_TYPES: readonly TelemetryEventType[]

telemetryRecordSchemaschema

const telemetryRecordSchema: z.ZodObject<…>

TelemetryRecordtype

type TelemetryRecord = {
    readonly treeId: TreeId;
    readonly occurredAt: string;
    readonly event: TelemetryEvent;
};

recordOffunction

const recordOf: (envelope: RuntimeEventEnvelope) => TelemetryRecord

proposalIdOffunction

The proposal an event is about, when it is about one. Used by the fold rather than stored as a column: a query that needs an index on it is a change with a reason behind it, and duplicating a field into a column is how two copies of the same fact start disagreeing.

const proposalIdOf: (event: TelemetryEvent) => ProposalId | undefined

intentIdOffunction

The intent an event names directly. Everything else reaches one via its proposal.

const intentIdOf: (event: TelemetryEvent) => IntentId | undefined

TelemetryErrortype

The only way a journal fails. There is no not-found: reading a tree nothing has been recorded about is an empty page, not an error, because a journal makes no claim that a tree exists. There is no conflict either — appending an observation cannot collide with another observation.

type TelemetryError = {
    readonly code: "unavailable";
    readonly detail: string;
};

describeTelemetryErrorfunction

const describeTelemetryError: (error: TelemetryError) => string

RecordedTelemetrytype

A record as it comes back out, with the position the journal gave it.

type RecordedTelemetry = TelemetryRecord & {
    readonly seq: number;
    /**
     * When the journal learned of the record, which is not `occurredAt`.
     *
     * Both are here because they answer different questions and only one of them
     * the journal can vouch for. `occurredAt` is what the host said, and a host
     * that sets it wrongly — a skewed serverless clock, a replayed batch — is
     * describing its own timeline. `recordedAt` is stamped on arrival, so it is
     * the only timestamp retention may be measured against: an age a writer can
     * choose is an age a writer can dodge.
     */
    readonly recordedAt: string;
};

TelemetryReadRequesttype

type TelemetryReadRequest = {
    /** Absent reads every tree the handle can see, which is the scope rule. */
    readonly treeId?: TreeId;
    /** A cursor from a previous page's `older`/`newer`, passed back unread. */
    readonly cursor?: string;
    /**
     * Default `newer`, which with no cursor is the oldest page. `older` with no
     * cursor is the newest page — the read a reader almost always wants first.
     */
    readonly direction?: PageDirection;
    readonly limit?: number;
};

TelemetryPagetype

type TelemetryPage = PageEnds & {
    /**
     * Always ascending by `seq`, whichever end the page was taken from.
     *
     * The direction is a property of *which* records a page contains, never of
     * the order they arrive in. `episodesOf` folds a stream in arrival order, and
     * a page that sometimes came back reversed would make every consumer
     * responsible for knowing which — and silently wrong when it guessed.
     */
    readonly records: readonly RecordedTelemetry[];
};

DEFAULT_TELEMETRY_LIMITvalue

Larger than a tree listing's, because these are read to be folded rather than shown: an episode spans a dozen or so records, so a page of 200 is a handful of episodes and a page of 50 would routinely split one across a boundary.

const DEFAULT_TELEMETRY_LIMIT = 200

MAX_TELEMETRY_LIMITvalue

const MAX_TELEMETRY_LIMIT = 1000

clampTelemetryLimitfunction

const clampTelemetryLimit: (limit: number | undefined) => number

AssessmentLookuptype

What a consumer asks when it is holding a list of proposals and wants the Gate's reading of each one.

type AssessmentLookup = {
    readonly proposalIds: readonly ProposalId[];
    /** Absent looks across every tree the handle can see, as `read` does. */
    readonly treeId?: TreeId;
};

AssessmentLookupResulttype

The assessments that were found, and the ids this lookup did not reach.

type AssessmentLookupResult = {
    readonly assessments: ReadonlyMap<ProposalId, AssessmentSummary>;
    readonly unasked: readonly ProposalId[];
};

MAX_ASSESSMENT_LOOKUPvalue

How many proposals one lookup answers, and it is a cap rather than a clamp.

const MAX_ASSESSMENT_LOOKUP = 400

ForgetRequesttype

What a journal is asked to forget, expressed as a position rather than a rule.

type ForgetRequest = {
    readonly before: number;
};

ForgetOutcometype

type ForgetOutcome = {
    readonly removed: number;
};

TelemetryJournalinterface

Four operations, and the asymmetry is deliberate: writes arrive in batches because a request narrates several stages and should pay for one round trip, while reads are paged because a journal only grows.

interface TelemetryJournal {
    readonly record: (records: readonly TelemetryRecord[]) => Promise<Result<void, TelemetryError>>;
    readonly read: (request?: TelemetryReadRequest) => Promise<Result<TelemetryPage, TelemetryError>>;
    readonly assessments: (request: AssessmentLookup) => Promise<Result<AssessmentLookupResult, TelemetryError>>;
    readonly forget: (request: ForgetRequest) => Promise<Result<ForgetOutcome, TelemetryError>>;
}

TelemetryPrunertype

The halves of the journal its two in-package consumers actually use.

type TelemetryPruner = Pick<TelemetryJournal, "read" | "forget">;

TelemetryWritertype

type TelemetryWriter = Pick<TelemetryJournal, "record">;

memoryTelemetryJournalfunction

A journal in memory: the reference implementation of the contract, and the one pnpm dev runs on.

Shown in use on What every ask leaves behind — What this page is not showing you and Going to production — Three places state lives, and only one of them is your pages

const memoryTelemetryJournal: (clock?: Clock) => TelemetryJournal

RemeasuredFactortype

One rule's contribution, and whether it was run again or taken as it stood.

type RemeasuredFactor = {
    readonly code: StakeFactorCode;
    readonly level: StakeLevel;
    /**
     * `remeasured` — this rule was run against the policy given here.
     * `recorded` — no field of a policy can move it, so its code is its level.
     */
    readonly source: "remeasured" | "recorded";
};

RemeasuredStakestype

type RemeasuredStakes = {
    /** A floor, not an answer, whenever `unreadable` is not empty. */
    readonly level: StakeLevel;
    readonly factors: readonly RemeasuredFactor[];
    /**
     * Rules this record cannot answer under this policy, because a field they read
     * was not written down. Empty is the claim that `level` is exact.
     */
    readonly unreadable: readonly StakeFactorCode[];
};

remeasureStakesfunction

const remeasureStakes: (summary: AssessmentSummary, policy: GatePolicy) => RemeasuredStakes

MIN_RETENTION_MSvalue

The floor is not a style choice. A policy of zero would empty the journal on the next run, and the most likely way to write one is a units mistake — days where milliseconds were wanted. An hour is short enough to never obstruct a host that means it and long enough that maxAgeMs: 7 is refused rather than obeyed.

const MIN_RETENTION_MS: number

retentionPolicySchemaschema

const retentionPolicySchema: z.ZodObject<…>

RetentionPolicytype

type RetentionPolicy = z.infer<typeof retentionPolicySchema>;

DEFAULT_RETENTION_SCANvalue

How much of the journal one run will look at.

const DEFAULT_RETENTION_SCAN = 5000

MAX_RETENTION_SCANvalue

const MAX_RETENTION_SCAN = 50000

RetentionPlantype

type RetentionPlan = {
    /**
     * Forget every record below this position. `null` when nothing may go yet —
     * either the journal holds nothing old enough, or the oldest thing in it is
     * an episode still waiting on someone.
     */
    readonly before: number | null;
    readonly forgets: number;
    /** Older than the horizon, kept because their own episode is unsettled. */
    readonly keptUnsettled: number;
    /**
     * Older than the horizon and settled, kept only because an unsettled episode
     * sits in front of them. Reported separately because it is the price of the
     * prefix rule, and a host watching this number stay large has learned that
     * something very old is still waiting on an answer.
     */
    readonly keptBehind: number;
    readonly unsettledEpisodes: number;
};

horizonOffunction

const horizonOf: (policy: RetentionPolicy, now: string) => string | null

retentionPlanOffunction

What may be forgotten, as a pure function of records and an instant.

const retentionPlanOf: (records: readonly RecordedTelemetry[], horizon: string) => RetentionPlan

RetentionRequesttype

type RetentionRequest = {
    readonly policy: RetentionPolicy;
    readonly clock?: Clock;
    readonly scanLimit?: number;
};

RetentionOutcometype

What a run did, in the shape of SnapshotAudit: an outcome a host can log, show, or ignore, and never an exception. Nothing downstream depends on retention having happened, so a run that could not read the journal is a reported non-event rather than a fault anyone has to handle.

type RetentionOutcome = {
    readonly outcome: "forgot";
    readonly plan: RetentionPlan;
    readonly removed: number;
} | {
    readonly outcome: "nothing-to-forget";
    readonly plan: RetentionPlan;
}
/** The policy or the clock was not usable. Nothing was read and nothing was deleted. */
 | {
    readonly outcome: "refused";
    readonly reason: string;
} | {
    readonly outcome: "unavailable";
    readonly error: TelemetryError;
};

applyRetentionfunction

Reads the oldest end of a journal, decides what may go, and drops it.

Shown in use on What every ask leaves behind — Forgetting is a feature, and it has rules

const applyRetention: (journal: TelemetryPruner, request: RetentionRequest) => Promise<RetentionOutcome>

describeRetentionfunction

One line for an operator, because an outcome nobody can read is an outcome nobody acts on.

const describeRetention: (outcome: RetentionOutcome) => string

MAX_BUFFERED_RECORDSvalue

A ceiling on what one collector will hold. A host that forgets to flush leaks memory into the write path, and unbounded telemetry buffers are a worse failure than lost telemetry — so the buffer refuses to grow past this and says how much it dropped.

Shown in use on What every ask leaves behind — Turning it on is six lines and one decision

const MAX_BUFFERED_RECORDS = 1000

TelemetryCollectortype

type TelemetryCollector = {
    readonly sink: EventSink;
    /** Not yet written. Exposed so a host can decide a flush is worth the round trip. */
    readonly pending: () => readonly TelemetryRecord[];
    /** Records this collector could not hold or could not write. */
    readonly dropped: () => number;
    /**
     * Writes what has been collected and empties the buffer, whether or not the
     * write succeeded. Records are not retained for a retry: a journal that is
     * failing will fail the retry too, and a queue that grows while it drains
     * moves the outage into the write path.
     */
    readonly flush: () => Promise<Result<void, TelemetryError>>;
};

collectTelemetryfunction

Shown in use on What every ask leaves behind — Turning it on is six lines and one decision

const collectTelemetry: (journal: TelemetryWriter) => TelemetryCollector