Reasoning Engine Orchestration

Overview

The Reasoning Engine orchestrates a multi-step analysis pipeline where each step builds upon the results of previous steps. This page provides detailed documentation on each step, its inputs, outputs, and how data flows through the system.

---

Pipeline Architecture

Complete Workflow Diagram

---

Step-by-Step Execution

Step 0: Preflight Intake

Purpose: Validate canonical input, filter usable datasets, and choose the execution plan.

Agent: Step Orchestrator Agent (optional)

Input:

Field	Type	Required	Description
objective	string	Yes	Analysis goal (e.g., "Analyze my stress patterns")
series	Series[]	No*	Time-series data with timestamps
datasets	NumericDataset[]	No*	Numeric arrays for correlation
hints	string[]	No	User-provided analysis guidance
compactedRows	Record[]	No*	Raw data for context

*At least one data input is required.

Output:

{
  taskId: string;
  workflowRunId: string;
  objective: string;
  series: Series[];
  datasets: NumericDataset[];
  hints: string[];
  plan: {
    stepsToExecute: string[];
    suggestedHints: string[];
  };
}

Database Logging:

• Creates parent record in log_ReasoningEngineTasks
• Creates step record in log_StepReasoningEngineTasks with StepOrder = 0
• Saves output to data_WorkflowStepOutputs

Execution Plan Priority:

1. Use a supplied stepPlan when present.
2. Build a plan from UI toggles when step toggles are supplied.
3. Call the Step Orchestrator Agent when enableStepOrchestration is enabled.
4. Fall back to all supported steps.

---

Step 1: Temporal Analysis

Purpose: Analyze time-series data to identify trends, compute statistics, and extract temporal insights.

Agent: Temporal Summary Agent (reasoning-engine-temporal-summary-agent)

Processing Logic:

1. Statistical Computation (algorithmic):

   • Mean, min, max for each series
   • Trend detection (increasing, decreasing, stable)
   • Volatility assessment

2. Agent Enhancement (if available):

   • Contextual interpretation
   • Pattern-aware summaries
   • Objective-aligned insights

Output Schema:

{
  summary: string;           // "Stress levels increased 40% over the week..."
  insights: string[];        // ["Peak stress on Wednesday", "Weekend recovery"]
  stats: Array<{
    id: string;              // Series identifier
    mean: number;
    min: number;
    max: number;
    trend: 'increasing' | 'decreasing' | 'stable';
  }>;
}

---

Step 2: Pattern Recognition

Purpose: Identify behavioral patterns, recurring themes, and anomalies in the data.

Agent: Pattern Recognition Agent (reasoning-engine-pattern-recognition-agent)

Algorithmic Patterns Detected:

Pattern	Trigger	Description
upward_trend	trend === 'increasing'	Metric showing sustained increase
downward_trend	trend === 'decreasing'	Metric showing sustained decrease
high_volatility	(max - min) / mean > threshold	High variability in values
low_engagement	checkInCount < threshold	Reduced data frequency
weekend_recovery	weekend values < weekday values	Weekend improvement pattern

Output Schema:

{
  matches: string[];         // Human-readable pattern descriptions
  confidence: number;        // 0-1 confidence score
  source: 'agent' | 'algorithmic' | 'mock';
  agentUsed: boolean;
  detectedPatterns: Array<{
    name: string;            // Pattern identifier
    series: string;          // Which series it applies to
    detail: string;          // Detailed description
  }>;
}

---

Step 3: Correlation Analysis

Purpose: Compute pairwise correlations between datasets to identify relationships.

Agent: Correlation Analysis Agent (reasoning-engine-correlation-analysis-agent)

Correlation Computation:

• Uses Pearson correlation coefficient
• Requires at least 2 datasets with matching length
• Skips step if insufficient data

Interpretation Thresholds:

Correlation (r)	Classification
r > 0.7	Strong positive
0.3 < r ≤ 0.7	Moderate positive
-0.3 ≤ r ≤ 0.3	Weak/None
-0.7 ≤ r < -0.3	Moderate negative
r < -0.7	Strong negative

Output Schema:

{
  pairs: Array<{
    a: string;               // First dataset ID
    b: string;               // Second dataset ID
    r: number;               // Correlation coefficient
  }>;
  summary: string;           // "Strong positive correlation between stress and sleep..."
}

---

Step 4: Domain Retrieval

Purpose: Retrieve relevant domain knowledge to enrich the analysis.

Agent: Domain Retrieval Agent (reasoning-engine-domain-retrieval-agent)

Current Behavior:

• Calls the reasoning-engine-domain-retrieval tool from the domain step
• Uses the Query Generator Agent to create focused searches when Perplexity is configured
• Executes Perplexity search for external context when credentials are available
• Falls back to local/stub context when external retrieval is unavailable

The committed implementation is not an empty placeholder. It supports live retrieval with a deterministic fallback so the pipeline can still complete in local or unconfigured environments.

Output Schema:

{
  knowledge: string[];       // Domain facts or fallback context relevant to analysis
  source: string;            // 'perplexity' | 'stub' | fallback source
}

---

Step 5: Explanation Generation

Purpose: Synthesize all prior analysis into a compassionate, user-facing narrative.

Agent: Explanation Agent (reasoning-engine-explanation-agent)

Key Responsibilities:

1. Load all prior step outputs from database
2. Synthesize temporal, pattern, and correlation findings
3. Generate compassionate, non-clinical narrative
4. Provide actionable (but non-prescriptive) recommendations

Tone Guidelines:

• Use "you/your" language
• Avoid clinical or alarming terminology
• Acknowledge data limitations
• Provide gentle, supportive suggestions

Output Schema:

{
  narrative: string;         // Full user-facing explanation
  recommendations: string[]; // Gentle suggestions
  sources: string[];         // References used
}

---

Step 6: Validation

Purpose: Verify that the generated narrative is consistent with the underlying data.

Agent: Validation Agent (reasoning-engine-validation-agent)

Validation Checks:

Check	Description
Trend Accuracy	Claimed trends match temporal.stats.trend
Correlation Strength	Correlation descriptions match actual r values
Pattern Presence	Mentioned patterns exist in pattern.detectedPatterns
Data Coverage	Claims don't exceed available data range
Confidence Alignment	Stated confidence matches data density

Output Schema:

{
  isValid: boolean;
  errors: string[];          // Blocking issues (contradictions)
  warnings: string[];        // Non-blocking concerns
  factCheck: Array<{
    claim: string;
    verdict: 'confirmed' | 'unconfirmed' | 'contradicted';
    note: string;
  }>;
}

---

Workflow Result Persistence

Purpose: Persist the assembled workflow state after validation.

Agent: None. Validation assembles the result and saves a workflow-result row through stepDataService.

Final Output Schema:

{
  taskId: string;
  workflowRunId: string;
  objective: string;
  outputProfile: string;

  temporal: TemporalOutput;
  pattern: PatternOutput;
  correlation: CorrelationOutput;
  domain: DomainOutput;
  explanation: ExplanationOutput;
  validation: ValidationOutput;

  result: {
    summary: string;
    insights: string[];
    recommendations: string[];
    confidence: number;
  };

  formattedOutput?: string;  // If an output profile formatter produced one
  errors: WorkflowError[];

  timing: {
    startedAt: string;
    completedAt: string;
    durationMs: number;
  };
}

note

The Phrasing Agent exists as a helper for outer application flows, but it is not a committed step in reasoningEngineWorkflow. The workflow chain ends at validation.

---

State Management

Minimal State Pattern

The workflow uses a minimal state pattern to prevent payload growth:

State Fields Passed Between Steps:

{
  taskId: string;           // Always passed
  workflowRunId: string;    // Always passed
  objective: string;        // Always passed
  errors: WorkflowError[];  // Accumulated errors
  // Step-specific output (minimal)
}

Full Outputs Retrieved From:

• data_WorkflowStepOutputs table
• Queried by (taskId, stepName)

Database Interaction Pattern

---

Error Handling

Error Classification

Severity	Behavior	Example
ERROR	Blocks workflow continuation	Invalid input, agent failure
WARNING	Logged but workflow continues	Missing optional data
INFO	Informational logging	Step skipped due to insufficient data

Error Schema

{
  step: string;              // Step where error occurred
  message: string;           // Error description
  type: string;              // Error classification
  severity: 'ERROR' | 'WARNING' | 'INFO';
}

Recovery Behavior

1. Step-Level Errors: Logged to log_Error, added to errors[] array
2. Recoverable Errors: Workflow continues with degraded output
3. Fatal Errors: Workflow stops, status set to 'failed'

---

Performance Considerations

Execution Timing

Step	Typical Duration	Notes
Preflight	< 100ms	Validation only
Temporal	100-500ms	Depends on series count
Pattern	200-1000ms	Agent enhancement adds latency
Correlation	100-300ms	O(n²) for n datasets
Domain	Variable	Perplexity retrieval adds network latency when configured; fallback is local
Explanation	500-2000ms	LLM generation
Validation	200-500ms	Fact-checking

Optimization Tips

1. Pre-aggregate data before sending to reduce series and dataset size
2. Use stepPlan or UI toggles to skip unnecessary temporal, pattern, correlation, or domain work
3. Use output profiles intentionally when formatted output is needed
4. Disable domain retrieval in plans that do not need external or contextual knowledge

---

Debugging Workflows

Tracing a Workflow Run

-- 1. Find the task
SELECT * FROM REASONING.log_ReasoningEngineTasks
WHERE TaskId = @taskId;

-- 2. View step progression
SELECT StepName, StepOrder, Status, AgentUsed, DurationMs, ErrorMessage
FROM REASONING.log_StepReasoningEngineTasks
WHERE TaskId = @taskId
ORDER BY StepOrder;

-- 3. Inspect step outputs
SELECT StepName, LEFT(OutputJson, 500) AS OutputPreview
FROM REASONING.data_WorkflowStepOutputs
WHERE TaskId = @taskId
ORDER BY StepOrder;

-- 4. Check for errors
SELECT StepName, ErrorType, Severity, Message
FROM REASONING.log_Error
WHERE TaskId = @taskId
ORDER BY StepOrder;

Common Issues

Symptom	Investigation	Resolution
Step stuck at running	Check step logs for timeout	Increase timeout, reduce input size
Empty pattern matches	Review temporal stats	Ensure data has variability
Correlation skipped	Check dataset count	Provide at least 2 datasets
Validation errors	Compare narrative to data	Review explanation agent output

---

Code References

Component	Location
Core Workflow	reasoning-engine.workflow.ts (src/mastra/workflows/reasoning-engine/reasoning-engine.workflow.ts)
Step Definitions	steps/*.step.ts (src/mastra/workflows/reasoning-engine/steps/)
Step Configuration	config.ts (src/mastra/workflows/reasoning-engine/steps/config.ts)
Step Orchestrator Agent	core/step-orchestrator.ts (src/mastra/agents/reasoning-engine/core/step-orchestrator.ts)
Temporal Agent	core/temporal.ts (src/mastra/agents/reasoning-engine/core/temporal.ts)
Pattern Agent	core/pattern.ts (src/mastra/agents/reasoning-engine/core/pattern.ts)
Domain Retrieval Agent	core/domain-retrieval.ts (src/mastra/agents/reasoning-engine/core/domain-retrieval.ts)
Explanation Agent	core/explanation.ts (src/mastra/agents/reasoning-engine/core/explanation.ts)
Validation Agent	core/validation.ts (src/mastra/agents/reasoning-engine/core/validation.ts)
Phrasing Helper	core/phrasing.ts (src/mastra/agents/reasoning-engine/core/phrasing.ts)

---

Next Steps

Reasoning Engine Overview

Architecture and design philosophy

Reasoning Engine Agents

Agent specifications and prompts

Agent Prompt Overrides

Database-driven configuration

Database: Core

Core schema and agent profiles