Test the work.
Inspect the evidence.

Results, by configuration.

Each bar shows the fraction passing its stated checks, on a fixed 0–100% scale. Rows use different suites and phases; they are not a ranking. Expected provider refusals are included in passes only where refusal was the required outcome. Provider-unavailable and uncertain-transport requests remain in the denominator and are identified in the table. Partial runs do not cover their entire planned dataset.

What the tests changed.

• Workflow drafts are checked for branch behavior, bounded replies, valid teammates and supported triggers before they can be reviewed.
• Change summaries compare meaningful behavior. A manager can undo the last saved AI draft edit while leaving the published workflow unchanged.
• Call, WhatsApp and web planning share a versioned task. A changed draft invalidates its earlier activation review.
• Voice routing instructions distinguish cumulative campaign results from monthly reports. Long planning histories retain the latest corrections.
• Cancelled generation cannot overwrite a newer task when an old provider request finishes.
• Unsaved instructions survive coalesced jobs; undo excludes the entire reverted instruction batch. Oversized discussions stop with a clear explanation.
• Workflow creation and AI job admissions use shared transactional quotas, including simultaneous requests from the visual builder.
• Duplicate menu labels and reserved-looking node names retain their correct branch behavior when drafts are compared or edited.
• Provider quota failures stop repeated requests. Production and test deployments still share provider account limits unless they use separate accounts.
• Execution leases fence local workflow mutations. Old menu choices cannot continue a republished version, and inactive workspaces are excluded from enquiry scanning.
• Versioned bot enablement prevents an older pause request from overriding a newer publish request.
• Duplicate planning requests replay the same committed task, even at the hourly limit. Quoted customer text cannot accidentally create a new workflow.
• Workflow enquiry notifications are deferred until the database transaction commits, so a rolled-back attempt does not send a success notification.
• Nested menus with an early handover retain their shared later steps when converted back to an editable plan. Those steps no longer multiply across branches and exceed the size limit.

These are changes to Alactic’s instructions, backend and workflow controls. The evaluation does not demonstrate changes to underlying foundation-model weights.

Replaying a recorded routing failure.

The partial production-configuration holdout produced 297 inspectable proposals before the provider failures. The original checks passed 289. Replaying those same saved proposals through the corrected read-scope guard passes 297.

This is a deterministic regression using previously observed answers. It uses no new model calls, is not a blind holdout and does not replace a fresh production-model evaluation. The original failed scores remain in the table above.

Duplicate events, one local effect.

An isolated runtime check used 8 synthetic workspaces and 400 unique events, each dispatched twice through 24 concurrent workers. It recorded 400 completed runs, 400 internal reminders and 0 duplicate effects.

External provider calls were blocked; 0 outbound attempts were observed. This checks local idempotency in a synthetic database. It is not a measurement of production capacity or a promise of exactly-once external message delivery.

Stress the nested branches.

A separate seeded corpus checks 5,000 valid generated workflow specifications across 300,000 sampled event paths. The corpus reaches 35 graph nodes and 3 levels of nesting. It compares the specification with compilation, editable-plan conversion and the simulator.

This exposed a shared-continuation duplication defect around early handovers. The corrected conversion passes the corpus and a targeted regression. These are deterministic checks with no model requests; they are separate from the model-result table.

A further 1,000 generated visual-builder graphs passed 80,000 conversion checks on sampled paths. Those graphs are not restricted to the nested structure used by the planner fixtures.

Improving branch instructions.

The initial complex-graph development and holdout runs passed 253 of 256 model-case checks. Clearer instructions about shared continuations and branch-local exits passed 256 of 256 on the same cases, then 128 of 128 on a separate set of fresh graphs. The matched rerun is a regression, not an unseen benchmark. These scores do not establish the same result for the production inference configuration.

How to read this evidence.

1. Fixtures are synthetic. No customer campaign was delivered and no customer database was used.
2. A variant changes business context, literal wording, team identifiers, branches or requested edits. Many variants share a structural family; they are not independent real-world use cases.
3. Workflow grading checks trigger behavior and action sequences on synthetic events. Compilation and simulation are also checked against a separate nested-step interpreter.
4. Manager routing checks the proposed tool and required arguments. It does not measure the full quality of spoken audio, customer satisfaction or actual provider delivery.
5. Deep OpenAI runs allow up to 8,192 output tokens for workflow planning and 2,048 for manager proposals. The production gateway allows up to 4,096 and 768 respectively. These are configuration tests with different operating limits, not controlled model rankings.
6. Backend journeys use real inference with an isolated database and a mocked publication connector. They test cross-channel continuity, stale-review rejection, undo, simulation and idempotent approval.
7. Conversational-correction fixtures test earlier discarded ideas, repeated corrections, Hindi and mixed-language instructions, ambiguous teammates, unsupported actions and requests that should leave a draft unchanged.
8. The concurrency harness dispatches duplicate synthetic events against an isolated database with provider calls blocked. It checks duplicate local effects, not real delivery throughput or production capacity.
9. Seeded graph checks compare 5,000 generated valid nested specifications with compiled graphs, editable plans and the simulator over 300,000 sampled event paths. Complex model cases use 96 frozen graphs from that corpus for creation and preservation edits: 48 development graphs, 16 initial holdout graphs and 32 fresh graphs after prompt refinement. The previously inspected prompt regression is not a blind holdout.
10. Development samples informed improvements. Previously used regression sets are not fresh blind holdouts. Unseen variant holdouts share structural families with development.
11. Runs preserve their own source and dataset snapshots. They span several implementation revisions; later fixes are checked in separately labelled regressions. Historical scores are not rewritten as results for the latest prompt.
12. Expected provider refusals are reported separately. Total passes may include an expected refusal; they do not all represent completed workflow generation.
13. Failures and provider-unavailable attempts remain in each row denominator. Stopped runs are partial and are labelled. A local expiry check that made no provider request is excluded from chart denominators and recorded separately.
14. The production configuration holdout stopped when the Cloudflare account reached its daily free AI allocation. Availability failures are distinct from incorrect model answers. Its application-level test allowance did not provide a separate provider-level quota.
15. Some larger-model runs stopped on temporary provider overload or uncertain transport outcomes. Continuation runs contain only previously unattempted cases; original errors and uncertain reservations are retained. Unavailable or uncertain requests are attempts, not inspectable model answers.
16. Failed fixture pilots are excluded from this release comparison and retained in the internal audit.

The catalog contains 1,200 workflow-contract variants across 50 families, 12,000 compositional workflow variants across 24 structural families, 2,336 manager-routing variants across 48 families, 600 conversational-correction variants across 12 families, and 192 complex-workflow variants covering creation and edits of 96 larger graphs. Catalog size and executed test count are reported separately.

What remains unproven.

• Not an independent benchmark or comparison with competitors.
• No claim of original pretraining, changed foundation-model weights or guaranteed correctness.
• No live call-latency or audio-quality claim follows from these model tests.
• Passing synthetic tests cannot guarantee production delivery or immunity to prompt injection.
• The 7-case October 15 conversational smoke test is a separate historical sample.

Failed cases are visible above. A high score on these fixtures is not a guarantee for a new business, a new language, an unfamiliar instruction or an external service failure.

Earlier conversational sample.

The original seven recorded questions from 15 October remain available, with their own narrow criteria and request timings.