Skip to main content

Test LLM Model

Overview​

The admin-test-llm-model tool allows you to test different LLM configurations, parameters, and prompts. Use this to validate model deployments, tune generation parameters, and debug model behavior.

Tool ID: admin-test-llm-model

The examples use the direct runtime/operator route. Replit/Admin UI browser code should call the Admin UI BFF instead: local /admin-api/*, testing /dashboard/admin-api/*.

Input Schema​

{
  messages: Array<{              // Chat messages
    role: 'system' | 'user' | 'assistant';
    content: string;
  }>;
  modelName?: string;            // Model name override
  deploymentName?: string;       // Deployment name override
  temperature?: number;          // 0-2, controls randomness
  topP?: number;                 // 0-1, nucleus sampling
  frequencyPenalty?: number;     // -2 to 2, reduce repetition
  presencePenalty?: number;      // -2 to 2, encourage new topics
  maxOutputTokens?: number;      // Max tokens in response
}

Example Usage​

curl -s -X POST "${MASTRA_BASE_URL}/admin/tools/test-llm-model" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $BEARER_TOKEN" \
  -d '{
    "data": {
      "messages": [
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Explain quantum computing in simple terms."}
      ],
      "temperature": 0.7,
      "maxOutputTokens": 500
    }
  }'
const response = await fetch(`${MASTRA_BASE_URL}/admin/tools/test-llm-model`, {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'Authorization': `Bearer ${process.env.BEARER_TOKEN}`,
  },
  body: JSON.stringify({
    data: {
      messages: [
        { role: 'system', content: 'You are a helpful assistant.' },
        { role: 'user', content: 'What is machine learning?' }
      ],
      temperature: 0.3,
      maxOutputTokens: 200
    }
  })
});

const result = await response.json();
console.log('Response:', result.content);
console.log('Tokens used:', result.usage);

Output​

Returns the model response and usage statistics:

{
  content: string;              // Generated response
  usage: {
    promptTokens: number;
    completionTokens: number;
    totalTokens: number;
  };
  model: string;                // Model that was used
  deployment: string;           // Deployment name
  finishReason: string;         // 'stop' | 'length' | 'content_filter'
  responseTime: number;         // Time in ms
}

Use Cases​

  • Deployment Validation: Verify new model deployments work correctly
  • Parameter Tuning: Test different temperature/penalty values
  • Prompt Engineering: Compare responses with different system prompts
  • Performance Monitoring: Track response times and token usage