Domain Retrieval

Overview

The Domain Retrieval step retrieves relevant domain knowledge from the web using AI-enhanced search. It uses a query generator agent to create focused search queries from reasoning context, then executes web searches via Perplexity to gather current, synthesized information with citations.

File: src/mastra/tools/reasoning-engine/domain-retrieval/domain-retrieval.tool.ts

note

This step uses Perplexity API as the primary search solution with AI-powered synthesis and automatic citation tracking. The implementation includes parallel query execution, diversity checking, and smart query optimization.

Architecture

The domain retrieval flow involves four components:

1. Query Generator Agent - Analyzes reasoning context to generate 2-5 focused search queries with priorities, model selection, and search modes
2. Query Validation & Diversity Checking - Validates parameters and warns on similar queries (>70% Jaccard similarity)
3. Perplexity Search Tool - Executes parallel web searches (top 2 queries) with AI synthesis and citation tracking
4. Knowledge Aggregator - Merges results, deduplicates citations, and filters empty/generic responses

Current Implementation

// 1. Generate focused queries with Perplexity params
const queryGenAgent = await getQueryGeneratorAgent();
const contextSummary = `Objective: ${objective}\n\nFindings:\n${findingsSummary}`;
const queryGenResponse = await queryGenAgent.generate([{
  role: 'user',
  content: contextSummary
}]);

// 2. Extract and validate queries
const queries = extractQueriesFromResponse(queryGenResponse);
queries.forEach(q => {
  q.model = validModels.has(q.model) ? q.model : 'sonar';
  q.searchMode = validModes.has(q.searchMode) ? q.searchMode : 'internet';
});

// 3. Check query diversity
checkQueryDiversity(queries, logger);

// 4. Execute top 2 queries in parallel
const topQueries = queries.sort((a, b) => b.priority - a.priority).slice(0, 2);
const results = await Promise.all(
  topQueries.map(q => {
    let effectiveQuery = q.query;
    if (shouldUseRecencyFilter(q.query)) {
      effectiveQuery += ' (recent research 2024-2025)';
    }
    return perplexitySearchTool.execute({
      query: effectiveQuery,
      model: q.model || 'sonar',
      searchMode: q.searchMode || 'internet'
    });
  })
);

// 5. Filter empty/generic responses
const validResults = results.filter(r => 
  'success' in r && r.success && !isEmptyOrGenericResponse(r.answer)
);

// 6. Deduplicate citations and merge answers
const allCitations = new Set<string>();
validResults.forEach(r => r.citations.forEach(c => allCitations.add(c)));

const knowledge = [
  validResults[0].answer,  // Primary answer
  ...validResults.slice(1)
    .filter(r => r.answer.length > 200)
    .map(r => r.answer),    // Secondary answers
  ...Array.from(allCitations)  // All unique citations
];

Output Structure

{
  domain: {
    knowledge: Array<{
      content: string,
      title?: string,
      url?: string,
      score?: number,
      sourceType?: string
    }>,                           // Synthesized answers, citations, metadata, or fallback context
    source: 'perplexity-ai-synthesis' | 'stub' | 'stub-error-fallback',
    searchMetadata: {
      queriesExecuted: number,    // Number of parallel queries
      totalResponseTime: number,  // Combined search time in ms
      primaryQuery: {
        query: string,            // Top priority query
        model: string,            // sonar or sonar-pro
        searchMode: string,       // internet/academic/news
        rationale: string,        // Why this query
        priority: number          // Query priority (1-10)
      },
      citationCount: number,      // Total unique citations
      diversityWarnings?: string[] // Query similarity warnings
    }
  }
}

Example Execution

Input Context

{
  "objective": "Analyze quantum computing error correction trends",
  "temporal": {
    "insights": ["Error rates declining 15% quarterly"]
  },
  "correlation": {
    "insights": ["Strong correlation between qubit coherence and error rates"]
  }
}

Generated Queries

[
  {
    "query": "latest quantum error correction breakthroughs topological qubits 2025-2026",
    "rationale": "Targets time-sensitive progress in topological qubit error correction",
    "priority": 10,
    "model": "sonar-pro",
    "searchMode": "academic"
  },
  {
    "query": "quantum coherence time improvements error rates 2025",
    "rationale": "Explores correlation between coherence and error rates",
    "priority": 9,
    "model": "sonar",
    "searchMode": "internet"
  }
]

Output

{
  "domain": {
    "knowledge": [
      "Quantum error correction achieved significant milestones in 2026 with topological qubits demonstrating unprecedented stability...",
      "Recent advances in quantum coherence show 15% quarterly improvement in error rates, primarily driven by better qubit isolation techniques...",
      "https://riverlane.com/quantum-error-correction-2026",
      "https://spectrum.ieee.org/quantum-computing-coherence",
      "https://nature.com/articles/quantum-topology-advances"
    ],
      "source": "perplexity-ai-synthesis",
    "searchMetadata": {
      "queriesExecuted": 2,
      "totalResponseTime": 3420,
      "primaryQuery": {
        "query": "latest quantum error correction breakthroughs topological qubits 2025-2026",
        "model": "sonar-pro",
        "searchMode": "academic",
        "rationale": "Targets time-sensitive progress in topological qubit error correction",
        "priority": 10
      },
      "citationCount": 3
    }
  }
}

Components

Query Generator Agent

Registry ID: reasoning-engine-query-generator-agent
Profile ID: reasoning-query-generator
Code: src/mastra/agents/reasoning-engine/core/query-generator.ts

• Analyzes temporal insights, patterns, and correlations
• Generates 2-5 focused search queries
• Assigns rationale and priority (1-10) to each query
• Returns JSON with queries array

See Query Generator Agent documentation →

Perplexity Search Tool

Tool ID: perplexity-search (runtime registry key: base-perplexity-search) Code: src/mastra/tools/base/perplexity-search.tool.ts

• Platform-level AI-powered web search tool
• Supports multiple search modes (internet, academic, news, youtube, reddit)
• Returns synthesized answers with automatic citation tracking
• URL extraction fallback for reliable citation capture
• Default system prompt optimized for inline citations

See Perplexity Search Tool documentation →

Configuration

Required Environment Variables

PERPLEXITY_API_KEY=your-api-key

Optional Tuning

PERPLEXITY_DEFAULT_MODEL=sonar              # or sonar-pro
PERPLEXITY_TIMEOUT_MS=60000                 # Request timeout
PERPLEXITY_DEBUG_TIMINGS=true               # Enable performance logging
PERPLEXITY_DEBUG_CITATIONS=true             # Debug citation extraction

Skip Conditions

This step is skipped if:

• stepPlan exists and doesn't include 'domain'
• Query generator fails to produce valid queries
• All generated queries fail validation

When Perplexity is not configured or search fails, the tool returns local fallback knowledge with source: "stub" or source: "stub-error-fallback" so the workflow can still complete.

Implemented Features

✅ Parallel Query Execution

• Top 2 queries executed concurrently via Promise.all()
• Citation deduplication across all results
• Secondary answer merging (>200 chars, score 0.85)

✅ Query Validation & Diversity

• Model validation (sonar/sonar-pro)
• Search mode validation (internet/academic/news/youtube/reddit)
• Jaccard similarity checking (warns on >70% overlap)

✅ Smart Query Enhancement

• Automatic recency filter for health/wellness queries
• Appends "(recent research 2024-2025)" to relevant queries
• Empty/generic response detection with fallback

✅ Enhanced Query Generation

• LLM generates model selection per query
• LLM generates searchMode per query
• Rationale and priority scoring (1-10)

Future Enhancements

Vector Search Integration

• Query internal wellbeing knowledge base
• Combine Perplexity search with vector search results
• Add clinical guidelines and research citations

Advanced Caching

• Query result caching to avoid duplicate API calls
• Semantic query similarity matching
• Time-based cache invalidation