Skip to main content

Azure AI Inference provider

Overview​

Design goals: The Azure AI Inference provider bridges models deployed in the Azure tenant (typically Azure AI Foundry deployments) into Mastra's LLM surface. Its goal is to make tenant-hosted authentication feel native to Mastra (Managed Identity preferred; API keys optional) while preserving DB-driven defaults and parameter guardrails.

Docs guideline: This page assumes readers operate tenant-hosted models. Use tenant-first phrasing and emphasize Managed Identity usage as the default; include short examples showing both Managed Identity (no key) and API key fallbacks when relevant. Cross-link to the Azure introduction "Documentation guideline" for the global doc tone and examples.

src/mastra/providers/azure-ai-inference.ts is the adapter between Mastra and Azure AI Inference. It uses the @azure-rest/ai-inference client to call Azure AI Foundry deployments, and centralizes auth selection, config resolution, and optional DB-backed knob overrides.

Endpoint and deployment resolution​

  • The provider reads the endpoint exclusively from Core.LLMModelCapabilities in the database (via resolveModelConfigFromDb()). There is no environment variable fallback for the endpoint — if the DB is not seeded with an enabled capability row, the provider throws at initialization.
  • If the endpoint ends with /openai, the provider strips that suffix automatically.
  • The deployment name resolves from the DB capability row. If the capability row has no deployment name, it falls back to the model name.
  • DB-driven resolution keeps the runtime aligned with the admin/ops defaults and environment-specific capability catalogs. To change the endpoint or deployment, update Core.LLMModelCapabilities (not env vars).

Credentials and instrumentation​

  • The provider prefers Managed Identity (via DefaultAzureCredential).
  • If AZURE_INFERENCE_API_KEY is set and AZURE_FOUNDRY_USE_MANAGED_IDENTITY is not true, the provider uses key-based auth via AzureKeyCredential.
  • AZURE_INFERENCE_DEBUG_TIMINGS=true enables debug timing logs from this provider.

DB-backed knob overrides​

  • The provider can merge optional defaults from the database (llmParamsService) into request options.
  • Overrides are cached for LLM_PARAMS_CACHE_TTL_MS (default 60s) and can be disabled entirely via LLM_PARAMS_DB_OVERRIDES_DISABLED=true.
  • Overrides are applied only when the deployment's model capability advertises support for the parameter (via getModelCapability). This prevents sending unsupported knobs to stricter deployments.
  • Merge precedence is: DB overrides (lowest), agent/profile preferred settings (middle), explicit per-request options (highest).

Model capability validation​

  • The provider uses capability metadata to validate model parameters and to honor DB-driven flags like useManagedIdentity when present.

Operational notes​

  • The provider logs warnings when DB reads fail. There is no env var fallback for the endpoint; a missing or disabled DB capability row causes a hard initialization error.
  • The module lives beside other provider helpers (Content Safety, search providers, etc.), reinforcing that request pipelining happens through shared entry points in src/mastra/providers.