{
  "title": "Alactic ACID workflow and manager evaluations",
  "date": "2026-10-25",
  "generatedAt": "2026-10-25T17:03:46.522433+00:00",
  "developer": "Alactic Inc",
  "scope": "Internal synthetic application evaluations. These evaluate ACID configurations and backend controls, not general foundation-model intelligence.",
  "catalog": [
    {
      "suite": "Workflow contracts",
      "variants": 1200,
      "families": 50
    },
    {
      "suite": "Workflow composition",
      "variants": 12000,
      "families": 24
    },
    {
      "suite": "Manager routing",
      "variants": 2336,
      "families": 48
    },
    {
      "suite": "Conversational corrections",
      "variants": 600,
      "families": 12
    },
    {
      "suite": "Complex workflow graphs",
      "variants": 192,
      "families": 2,
      "graphs": 96
    }
  ],
  "uniqueExecutedVariants": 15578,
  "completedModelCaseChecks": 32196,
  "recordedModelCaseAttempts": 32219,
  "completedBackendJourneys": 144,
  "results": [
    {
      "run": "production-development",
      "suite": "Workflow contracts",
      "phase": "Development baseline",
      "status": "complete",
      "configuration": "ACID production configuration",
      "completed": 150,
      "passed": 136,
      "providerUnavailable": 0,
      "transportUncertain": 0,
      "providerRefusals": 0,
      "expectedRefusals": 0,
      "failed": 14
    },
    {
      "run": "production-regression-v4",
      "suite": "Workflow contracts",
      "phase": "Matched development regression",
      "status": "complete",
      "configuration": "ACID production configuration",
      "completed": 150,
      "passed": 145,
      "providerUnavailable": 0,
      "transportUncertain": 0,
      "providerRefusals": 0,
      "expectedRefusals": 0,
      "failed": 5
    },
    {
      "run": "final-regression-v4",
      "suite": "Workflow contracts",
      "phase": "Previously used regression set",
      "status": "complete",
      "configuration": "ACID + OpenAI GPT-6 Astra",
      "completed": 300,
      "passed": 300,
      "providerUnavailable": 0,
      "transportUncertain": 0,
      "providerRefusals": 6,
      "expectedRefusals": 6,
      "failed": 0
    },
    {
      "run": "final-regression-v4",
      "suite": "Workflow contracts",
      "phase": "Previously used regression set",
      "status": "complete",
      "configuration": "ACID + OpenAI GPT-6 Luna",
      "completed": 300,
      "passed": 298,
      "providerUnavailable": 0,
      "transportUncertain": 0,
      "providerRefusals": 0,
      "expectedRefusals": 0,
      "failed": 2
    },
    {
      "run": "final-regression-v4",
      "suite": "Workflow contracts",
      "phase": "Previously used regression set",
      "status": "complete",
      "configuration": "ACID + OpenAI GPT-6.1 Sol",
      "completed": 300,
      "passed": 300,
      "providerUnavailable": 0,
      "transportUncertain": 0,
      "providerRefusals": 6,
      "expectedRefusals": 6,
      "failed": 0
    },
    {
      "run": "workflow-development-v1",
      "suite": "Workflow composition",
      "phase": "Development",
      "status": "stopped",
      "configuration": "ACID + OpenAI GPT-6 Astra",
      "completed": 5080,
      "passed": 5079,
      "providerUnavailable": 0,
      "transportUncertain": 1,
      "providerRefusals": 0,
      "expectedRefusals": 0,
      "failed": 1
    },
    {
      "run": "workflow-development-v1",
      "suite": "Workflow composition",
      "phase": "Development",
      "status": "stopped",
      "configuration": "ACID + OpenAI GPT-6.1 Sol",
      "completed": 9600,
      "passed": 9600,
      "providerUnavailable": 0,
      "transportUncertain": 0,
      "providerRefusals": 0,
      "expectedRefusals": 0,
      "failed": 0
    },
    {
      "run": "workflow-development-continuation-v2",
      "suite": "Workflow composition",
      "phase": "Unattempted development continuation after refinements",
      "status": "complete",
      "configuration": "ACID + OpenAI GPT-6 Astra",
      "completed": 4520,
      "passed": 4520,
      "providerUnavailable": 0,
      "transportUncertain": 0,
      "providerRefusals": 0,
      "expectedRefusals": 0,
      "failed": 0
    },
    {
      "run": "manager-development-v2",
      "suite": "Manager routing",
      "phase": "Development",
      "status": "stopped",
      "configuration": "ACID + OpenAI GPT-6 Astra",
      "completed": 1075,
      "passed": 1073,
      "providerUnavailable": 2,
      "transportUncertain": 0,
      "providerRefusals": 48,
      "expectedRefusals": 48,
      "failed": 2
    },
    {
      "run": "manager-development-v2",
      "suite": "Manager routing",
      "phase": "Development",
      "status": "stopped",
      "configuration": "ACID + OpenAI GPT-6.1 Sol",
      "completed": 1752,
      "passed": 1752,
      "providerUnavailable": 0,
      "transportUncertain": 0,
      "providerRefusals": 48,
      "expectedRefusals": 48,
      "failed": 0
    },
    {
      "run": "manager-development-continuation-v3",
      "suite": "Manager routing",
      "phase": "Development continuation after refinements",
      "status": "complete",
      "configuration": "ACID + OpenAI GPT-6 Astra",
      "completed": 677,
      "passed": 677,
      "providerUnavailable": 0,
      "transportUncertain": 0,
      "providerRefusals": 0,
      "expectedRefusals": 0,
      "failed": 0
    },
    {
      "run": "manager-production-baseline",
      "suite": "Manager routing",
      "phase": "Matched development baseline",
      "status": "complete",
      "configuration": "ACID production configuration",
      "completed": 96,
      "passed": 85,
      "providerUnavailable": 0,
      "transportUncertain": 0,
      "providerRefusals": 0,
      "expectedRefusals": 0,
      "failed": 11
    },
    {
      "run": "manager-production-repair-v1",
      "suite": "Manager routing",
      "phase": "Matched development regression",
      "status": "complete",
      "configuration": "ACID production configuration",
      "completed": 96,
      "passed": 95,
      "providerUnavailable": 0,
      "transportUncertain": 0,
      "providerRefusals": 0,
      "expectedRefusals": 0,
      "failed": 1
    },
    {
      "run": "workflow-heldout-v1",
      "suite": "Workflow composition",
      "phase": "Unseen variant holdout",
      "status": "stopped",
      "configuration": "ACID + OpenAI GPT-6 Astra",
      "completed": 15,
      "passed": 13,
      "providerUnavailable": 2,
      "transportUncertain": 0,
      "providerRefusals": 0,
      "expectedRefusals": 0,
      "failed": 2
    },
    {
      "run": "workflow-heldout-v1",
      "suite": "Workflow composition",
      "phase": "Unseen variant holdout",
      "status": "stopped",
      "configuration": "ACID + OpenAI GPT-6.1 Sol",
      "completed": 2400,
      "passed": 2400,
      "providerUnavailable": 0,
      "transportUncertain": 0,
      "providerRefusals": 0,
      "expectedRefusals": 0,
      "failed": 0
    },
    {
      "run": "workflow-heldout-continuation-v2",
      "suite": "Workflow composition",
      "phase": "Unattempted holdout continuation after refinements",
      "status": "complete",
      "configuration": "ACID + OpenAI GPT-6 Astra",
      "completed": 2385,
      "passed": 2385,
      "providerUnavailable": 0,
      "transportUncertain": 0,
      "providerRefusals": 0,
      "expectedRefusals": 0,
      "failed": 0
    },
    {
      "run": "manager-heldout-v1",
      "suite": "Manager routing",
      "phase": "Unseen variant holdout",
      "status": "complete",
      "configuration": "ACID + OpenAI GPT-6 Astra",
      "completed": 584,
      "passed": 584,
      "providerUnavailable": 0,
      "transportUncertain": 0,
      "providerRefusals": 16,
      "expectedRefusals": 16,
      "failed": 0
    },
    {
      "run": "manager-heldout-v1",
      "suite": "Manager routing",
      "phase": "Unseen variant holdout",
      "status": "complete",
      "configuration": "ACID + OpenAI GPT-6.1 Sol",
      "completed": 584,
      "passed": 584,
      "providerUnavailable": 0,
      "transportUncertain": 0,
      "providerRefusals": 16,
      "expectedRefusals": 16,
      "failed": 0
    },
    {
      "run": "dialogue-development-v1",
      "suite": "Conversational corrections",
      "phase": "Development",
      "status": "complete",
      "configuration": "ACID + OpenAI GPT-6 Astra",
      "completed": 480,
      "passed": 480,
      "providerUnavailable": 0,
      "transportUncertain": 0,
      "providerRefusals": 0,
      "expectedRefusals": 0,
      "failed": 0
    },
    {
      "run": "dialogue-development-v1",
      "suite": "Conversational corrections",
      "phase": "Development",
      "status": "complete",
      "configuration": "ACID + OpenAI GPT-6.1 Sol",
      "completed": 480,
      "passed": 480,
      "providerUnavailable": 0,
      "transportUncertain": 0,
      "providerRefusals": 0,
      "expectedRefusals": 0,
      "failed": 0
    },
    {
      "run": "dialogue-heldout-v1",
      "suite": "Conversational corrections",
      "phase": "Unseen variant holdout",
      "status": "complete",
      "configuration": "ACID + OpenAI GPT-6 Astra",
      "completed": 120,
      "passed": 120,
      "providerUnavailable": 0,
      "transportUncertain": 0,
      "providerRefusals": 0,
      "expectedRefusals": 0,
      "failed": 0
    },
    {
      "run": "dialogue-heldout-v1",
      "suite": "Conversational corrections",
      "phase": "Unseen variant holdout",
      "status": "complete",
      "configuration": "ACID + OpenAI GPT-6.1 Sol",
      "completed": 120,
      "passed": 120,
      "providerUnavailable": 0,
      "transportUncertain": 0,
      "providerRefusals": 0,
      "expectedRefusals": 0,
      "failed": 0
    },
    {
      "run": "complex-development-v1",
      "suite": "Complex workflow graphs",
      "phase": "Development after conversion fix",
      "status": "complete",
      "configuration": "ACID + OpenAI GPT-6 Astra",
      "completed": 96,
      "passed": 96,
      "providerUnavailable": 0,
      "transportUncertain": 0,
      "providerRefusals": 0,
      "expectedRefusals": 0,
      "failed": 0
    },
    {
      "run": "complex-development-v1",
      "suite": "Complex workflow graphs",
      "phase": "Development after conversion fix",
      "status": "complete",
      "configuration": "ACID + OpenAI GPT-6.1 Sol",
      "completed": 96,
      "passed": 96,
      "providerUnavailable": 0,
      "transportUncertain": 0,
      "providerRefusals": 0,
      "expectedRefusals": 0,
      "failed": 0
    },
    {
      "run": "complex-heldout-v1",
      "suite": "Complex workflow graphs",
      "phase": "Unseen variant holdout",
      "status": "complete",
      "configuration": "ACID + OpenAI GPT-6 Astra",
      "completed": 32,
      "passed": 31,
      "providerUnavailable": 0,
      "transportUncertain": 0,
      "providerRefusals": 0,
      "expectedRefusals": 0,
      "failed": 1
    },
    {
      "run": "complex-heldout-v1",
      "suite": "Complex workflow graphs",
      "phase": "Unseen variant holdout",
      "status": "complete",
      "configuration": "ACID + OpenAI GPT-6.1 Sol",
      "completed": 32,
      "passed": 30,
      "providerUnavailable": 0,
      "transportUncertain": 0,
      "providerRefusals": 0,
      "expectedRefusals": 0,
      "failed": 2
    },
    {
      "run": "complex-regression-v2",
      "suite": "Complex workflow graphs",
      "phase": "Previously inspected prompt regression",
      "status": "complete",
      "configuration": "ACID + OpenAI GPT-6 Astra",
      "completed": 128,
      "passed": 128,
      "providerUnavailable": 0,
      "transportUncertain": 0,
      "providerRefusals": 0,
      "expectedRefusals": 0,
      "failed": 0
    },
    {
      "run": "complex-regression-v2",
      "suite": "Complex workflow graphs",
      "phase": "Previously inspected prompt regression",
      "status": "complete",
      "configuration": "ACID + OpenAI GPT-6.1 Sol",
      "completed": 128,
      "passed": 128,
      "providerUnavailable": 0,
      "transportUncertain": 0,
      "providerRefusals": 0,
      "expectedRefusals": 0,
      "failed": 0
    },
    {
      "run": "complex-fresh-heldout-v2",
      "suite": "Complex workflow graphs",
      "phase": "Fresh graph holdout after prompt refinement",
      "status": "complete",
      "configuration": "ACID + OpenAI GPT-6 Astra",
      "completed": 64,
      "passed": 64,
      "providerUnavailable": 0,
      "transportUncertain": 0,
      "providerRefusals": 0,
      "expectedRefusals": 0,
      "failed": 0
    },
    {
      "run": "complex-fresh-heldout-v2",
      "suite": "Complex workflow graphs",
      "phase": "Fresh graph holdout after prompt refinement",
      "status": "complete",
      "configuration": "ACID + OpenAI GPT-6.1 Sol",
      "completed": 64,
      "passed": 64,
      "providerUnavailable": 0,
      "transportUncertain": 0,
      "providerRefusals": 0,
      "expectedRefusals": 0,
      "failed": 0
    },
    {
      "run": "manager-production-heldout-v1",
      "suite": "Manager routing",
      "phase": "Unseen variant holdout",
      "status": "stopped",
      "configuration": "ACID production configuration",
      "completed": 315,
      "passed": 289,
      "providerUnavailable": 18,
      "transportUncertain": 0,
      "providerRefusals": 0,
      "expectedRefusals": 0,
      "failed": 26
    },
    {
      "run": "journey-sol-v2",
      "suite": "Backend journeys",
      "phase": "Synthetic integration",
      "status": "complete",
      "configuration": "ACID + OpenAI GPT-6.1 Sol",
      "completed": 48,
      "passed": 48,
      "providerUnavailable": 0,
      "transportUncertain": 0,
      "providerRefusals": 0,
      "expectedRefusals": 0,
      "failed": 0
    },
    {
      "run": "journey-astra-v1",
      "suite": "Backend journeys",
      "phase": "Synthetic integration",
      "status": "complete",
      "configuration": "ACID + OpenAI GPT-6 Astra",
      "completed": 48,
      "passed": 48,
      "providerUnavailable": 0,
      "transportUncertain": 0,
      "providerRefusals": 0,
      "expectedRefusals": 0,
      "failed": 0
    },
    {
      "run": "journey-sol-runtime-regression",
      "suite": "Backend journeys",
      "phase": "Post-refinement integration regression",
      "status": "complete",
      "configuration": "ACID + OpenAI GPT-6.1 Sol",
      "completed": 24,
      "passed": 24,
      "providerUnavailable": 0,
      "transportUncertain": 0,
      "providerRefusals": 0,
      "expectedRefusals": 0,
      "failed": 0
    },
    {
      "run": "journey-astra-runtime-regression",
      "suite": "Backend journeys",
      "phase": "Post-refinement integration regression",
      "status": "complete",
      "configuration": "ACID + OpenAI GPT-6 Astra",
      "completed": 24,
      "passed": 24,
      "providerUnavailable": 0,
      "transportUncertain": 0,
      "providerRefusals": 0,
      "expectedRefusals": 0,
      "failed": 0
    }
  ],
  "runs": [
    {
      "id": "production-development",
      "status": "complete",
      "recordedThrough": "2026-10-25T14:11:57.182Z",
      "plannedChecks": 150,
      "recordedChecks": 150,
      "preflightStopped": 0
    },
    {
      "id": "production-regression-v4",
      "status": "complete",
      "recordedThrough": "2026-10-25T14:20:56.138Z",
      "plannedChecks": 150,
      "recordedChecks": 150,
      "preflightStopped": 0
    },
    {
      "id": "final-regression-v4",
      "status": "complete",
      "recordedThrough": "2026-10-25T14:17:46.586Z",
      "plannedChecks": 900,
      "recordedChecks": 900,
      "preflightStopped": 0
    },
    {
      "id": "workflow-development-v1",
      "status": "stopped",
      "recordedThrough": "2026-10-25T16:34:13.204Z",
      "plannedChecks": 19200,
      "recordedChecks": 14680,
      "preflightStopped": 0
    },
    {
      "id": "workflow-development-continuation-v2",
      "status": "complete",
      "recordedThrough": "2026-10-25T17:03:16.554Z",
      "plannedChecks": 4520,
      "recordedChecks": 4520,
      "preflightStopped": 0
    },
    {
      "id": "manager-development-v2",
      "status": "stopped",
      "recordedThrough": "2026-10-25T15:58:04.622Z",
      "plannedChecks": 3504,
      "recordedChecks": 2827,
      "preflightStopped": 0
    },
    {
      "id": "manager-development-continuation-v3",
      "status": "complete",
      "recordedThrough": "2026-10-25T16:14:56.020Z",
      "plannedChecks": 677,
      "recordedChecks": 677,
      "preflightStopped": 0
    },
    {
      "id": "manager-production-baseline",
      "status": "complete",
      "recordedThrough": "2026-10-25T15:31:32.188Z",
      "plannedChecks": 96,
      "recordedChecks": 96,
      "preflightStopped": 0
    },
    {
      "id": "manager-production-repair-v1",
      "status": "complete",
      "recordedThrough": "2026-10-25T15:37:24.706Z",
      "plannedChecks": 96,
      "recordedChecks": 96,
      "preflightStopped": 0
    },
    {
      "id": "workflow-heldout-v1",
      "status": "stopped",
      "recordedThrough": "2026-10-25T15:58:19.043Z",
      "plannedChecks": 4800,
      "recordedChecks": 2415,
      "preflightStopped": 0
    },
    {
      "id": "workflow-heldout-continuation-v2",
      "status": "complete",
      "recordedThrough": "2026-10-25T16:35:43.809Z",
      "plannedChecks": 2385,
      "recordedChecks": 2385,
      "preflightStopped": 0
    },
    {
      "id": "manager-heldout-v1",
      "status": "complete",
      "recordedThrough": "2026-10-25T15:53:16.494Z",
      "plannedChecks": 1168,
      "recordedChecks": 1168,
      "preflightStopped": 0
    },
    {
      "id": "dialogue-development-v1",
      "status": "complete",
      "recordedThrough": "2026-10-25T16:33:36.567Z",
      "plannedChecks": 960,
      "recordedChecks": 960,
      "preflightStopped": 0
    },
    {
      "id": "dialogue-heldout-v1",
      "status": "complete",
      "recordedThrough": "2026-10-25T16:31:50.342Z",
      "plannedChecks": 240,
      "recordedChecks": 240,
      "preflightStopped": 0
    },
    {
      "id": "complex-development-v1",
      "status": "complete",
      "recordedThrough": "2026-10-25T16:43:50.861Z",
      "plannedChecks": 192,
      "recordedChecks": 192,
      "preflightStopped": 0
    },
    {
      "id": "complex-heldout-v1",
      "status": "complete",
      "recordedThrough": "2026-10-25T16:45:09.973Z",
      "plannedChecks": 64,
      "recordedChecks": 64,
      "preflightStopped": 0
    },
    {
      "id": "complex-regression-v2",
      "status": "complete",
      "recordedThrough": "2026-10-25T16:50:14.890Z",
      "plannedChecks": 256,
      "recordedChecks": 256,
      "preflightStopped": 0
    },
    {
      "id": "complex-fresh-heldout-v2",
      "status": "complete",
      "recordedThrough": "2026-10-25T16:49:33.131Z",
      "plannedChecks": 128,
      "recordedChecks": 128,
      "preflightStopped": 0
    },
    {
      "id": "manager-production-heldout-v1",
      "status": "stopped",
      "recordedThrough": "2026-10-25T15:55:35.197Z",
      "plannedChecks": 584,
      "recordedChecks": 316,
      "preflightStopped": 1
    },
    {
      "id": "journey-sol-v2",
      "status": "complete",
      "recordedThrough": "2026-10-25T15:42:22.737Z",
      "plannedChecks": 48,
      "recordedChecks": 48,
      "preflightStopped": 0
    },
    {
      "id": "journey-astra-v1",
      "status": "complete",
      "recordedThrough": "2026-10-25T15:50:17.617Z",
      "plannedChecks": 48,
      "recordedChecks": 48,
      "preflightStopped": 0
    },
    {
      "id": "journey-sol-runtime-regression",
      "status": "complete",
      "recordedThrough": "2026-10-25T16:16:05.162Z",
      "plannedChecks": 24,
      "recordedChecks": 24,
      "preflightStopped": 0
    },
    {
      "id": "journey-astra-runtime-regression",
      "status": "complete",
      "recordedThrough": "2026-10-25T16:15:55.101Z",
      "plannedChecks": 24,
      "recordedChecks": 24,
      "preflightStopped": 0
    }
  ],
  "deterministicChecks": {
    "variants": 12000,
    "structuralFamilies": 24,
    "eventPaths": 446500,
    "passed": true,
    "method": "Nested specification interpreter compared with compiled graph, editable design roundtrip and production simulator; no model calls"
  },
  "recordedProposalReplay": {
    "kind": "Deterministic replay of recorded synthetic model proposals; no new inference",
    "checkedAt": "2026-10-25T16:14:37.044Z",
    "checked": 297,
    "beforePassed": 289,
    "afterPassed": 297
  },
  "runtimeConcurrency": {
    "passed": true,
    "checkedAt": "2026-10-25T16:26:09.631Z",
    "scope": "Local isolated MongoDB harness, synthetic records, no model inference or external delivery. Not production capacity.",
    "tenants": 8,
    "uniqueEvents": 400,
    "dispatchAttempts": 800,
    "concurrency": 24,
    "completedRuns": 400,
    "reminders": 400,
    "duplicateEffects": 0,
    "outboundAttempts": 0
  },
  "seededGraphChecks": {
    "passed": true,
    "seed": 2887589926,
    "variants": 5000,
    "eventPaths": 300000,
    "maxNodes": 35,
    "maxNestingDepth": 3,
    "scenarioHash": "49c0c6ada7299bda0fb7c5a82352b1d1a12ec5b366fb28929194b78ca70979c2",
    "scope": "Seeded synthetic compiler, editable-design roundtrip and simulator checks against nested specifications. No model calls or customer records."
  },
  "builderGraphChecks": {
    "passed": true,
    "seed": 3189777936,
    "graphs": 1000,
    "converted": 1000,
    "requiresGraphReference": 0,
    "eventPaths": 80000,
    "scope": "Synthetic visual-builder DAG conversion; no model or external actions."
  },
  "methodology": [
    "Fixtures are synthetic. No customer campaign was delivered and no customer database was used.",
    "A variant changes business context, literal wording, team identifiers, branches or requested edits. Many variants share a structural family; they are not independent real-world use cases.",
    "Workflow grading checks trigger behavior and action sequences on synthetic events. Compilation and simulation are also checked against a separate nested-step interpreter.",
    "Manager routing checks the proposed tool and required arguments. It does not measure the full quality of spoken audio, customer satisfaction or actual provider delivery.",
    "Deep OpenAI runs allow up to 8,192 output tokens for workflow planning and 2,048 for manager proposals. The production gateway allows up to 4,096 and 768 respectively. These are configuration tests with different operating limits, not controlled model rankings.",
    "Backend journeys use real inference with an isolated database and a mocked publication connector. They test cross-channel continuity, stale-review rejection, undo, simulation and idempotent approval.",
    "Conversational-correction fixtures test earlier discarded ideas, repeated corrections, Hindi and mixed-language instructions, ambiguous teammates, unsupported actions and requests that should leave a draft unchanged.",
    "The concurrency harness dispatches duplicate synthetic events against an isolated database with provider calls blocked. It checks duplicate local effects, not real delivery throughput or production capacity.",
    "Seeded graph checks compare 5,000 generated valid nested specifications with compiled graphs, editable plans and the simulator over 300,000 sampled event paths. Complex model cases use 96 frozen graphs from that corpus for creation and preservation edits: 48 development graphs, 16 initial holdout graphs and 32 fresh graphs after prompt refinement. The previously inspected prompt regression is not a blind holdout.",
    "Development samples informed improvements. Previously used regression sets are not fresh blind holdouts. Unseen variant holdouts share structural families with development.",
    "Runs preserve their own source and dataset snapshots. They span several implementation revisions; later fixes are checked in separately labelled regressions. Historical scores are not rewritten as results for the latest prompt.",
    "Expected provider refusals are reported separately. Total passes may include an expected refusal; they do not all represent completed workflow generation.",
    "Failures and provider-unavailable attempts remain in each row denominator. Stopped runs are partial and are labelled. A local expiry check that made no provider request is excluded from chart denominators and recorded separately.",
    "The production configuration holdout stopped when the Cloudflare account reached its daily free AI allocation. Availability failures are distinct from incorrect model answers. Its application-level test allowance did not provide a separate provider-level quota.",
    "Some larger-model runs stopped on temporary provider overload or uncertain transport outcomes. Continuation runs contain only previously unattempted cases; original errors and uncertain reservations are retained. Unavailable or uncertain requests are attempts, not inspectable model answers.",
    "Failed fixture pilots are excluded from this release comparison and retained in the internal audit."
  ],
  "limitations": [
    "Not an independent benchmark or comparison with competitors.",
    "No claim of original pretraining, changed foundation-model weights or guaranteed correctness.",
    "No live call-latency or audio-quality claim follows from these model tests.",
    "Passing synthetic tests cannot guarantee production delivery or immunity to prompt injection.",
    "The 7-case October 15 conversational smoke test is a separate historical sample."
  ]
}