K12 Testing
Overview
K12 testing now uses two complementary layers:
- focused
vitestsuites for tools, routes, prompts, and workflow helpers - Bruno scenario assets for deployed-environment validation
Automated Test Layers
Unit and route tests
Use npm run test:run -- <file> for focused checks.
High-value K12 suites include:
| Path | Purpose |
|---|---|
packages/domain-k12/tests/tools/annex-editing.test.ts | Annex-editing tool and workflow-runner behavior |
packages/domain-k12/tests/workflows/k12-annex-steps.test.ts | Step-level prompt and routing behavior |
tests/shared/unit/routes/domain-tool-routes.test.ts | Legacy/domain route mapping and toggles |
tests/admin/unit/ui/dashboard-server.test.ts | Dashboard-side auth and APIM key handling |
packages/domain-k12/tests/workflows/k12-annex-steps.execute.test.ts | Annex execution path and role-context assembly |
packages/domain-k12/tests/workflows/k12-role-utils.test.ts | Role list parsing, effective role selection, and batching helpers |
Turn-based scenario catalog
The checked-in annex scenario catalog lives under:
packages/domain-k12/src/k12-turn-based/scenarios/annex-domain/packages/domain-k12/src/k12-turn-based/scenarios/annex-scenarios.ts
These scenarios drive:
- the workflow endpoints under
/k12/workflows/* - evaluator-based quality checks for single-turn and long multi-turn conversations
Primary scenario assets:
| Path | Purpose |
|---|---|
packages/domain-k12/src/k12-turn-based/scenarios/annex-domain/district-planning.ts | District planning scenario set |
packages/domain-k12/src/k12-turn-based/scenarios/annex-domain/facility-planning.ts | Facility planning scenario set |
packages/domain-k12/src/k12-turn-based/scenarios/annex-domain/functional-annex.ts | Functional annex scenario set |
packages/domain-k12/src/k12-turn-based/scenarios/annex-domain/hazard-annex.ts | Hazard annex scenario set |
packages/domain-k12/tests/workflows/k12-turn-based/scenarios.test.ts | Scenario catalog validation |
Bruno scenario flows
Bruno remains the deployed-environment runner for K12 scenario workflows.
Important folders:
| Path | Purpose |
|---|---|
bruno/30-tools-k12/annex-editing/annex-editing-kickoff/ | Dedicated kickoff endpoint probes |
bruno/60-flows-k12/annex-editing-scenarios/ | Manual annex-editing scenario payloads |
bruno/63-flows-k12/turn-based-workflows/ | Turn-based conversation, single-scenario, batch, and scenario-mining requests |
bruno/90-manual-k12-routing/ | Legacy annex routing verification |
Local Development Loop
When running the dashboard locally:
- start Mastra locally, typically on
http://localhost:4111 - start the dashboard server, typically on
http://localhost:8787 - open the direct harness URL
- use localhost scenario presets to run async multi-turn evaluations without APIM auth
The localhost harness path short-circuits through the local Mastra server instead of going back through APIM, which makes scenario iteration much faster than deployed testing loops.
Current Workflow-Level Coverage
The K12 turn-based workflow family currently covers:
| Workflow path | Purpose |
|---|---|
/k12/workflows/annex-turn-based-conversation | Run a raw multi-turn annex conversation and evaluate it |
/k12/workflows/annex-turn-based-scenario | Run a named checked-in annex scenario |
/k12/workflows/annex-turn-based-scenario-batch | Run a selected batch of named annex scenarios |
/k12/workflows/annex-turn-based-scenario-mining | Mine scenario drafts from existing interaction history |
The harness primarily exercises the single-scenario flow in deployed environments and the async local runner on localhost.