# Microsoft Foundry connectors

This guide shows you how to connect Microsoft Foundry agents to Twilio communication channels using the [`twilio-agent-connect-microsoft`](https://github.com/twilio/twilio-agent-connect-microsoft) package. The package provides two connectors — Agent Framework (recommended) and Voice Live — that handle channel routing, memory context, and session management.

When you're ready to deploy to production, see [Deploy to Azure Container Apps](/docs/conversations/agent-connect/integrations/microsoft-foundry/deploy).

## Prerequisites

Before you begin, make sure you have:

* TAC SDK installed and configured with your Twilio credentials and a Conversation Configuration (see [Add TAC to your agent prerequisites](/docs/conversations/agent-connect/build-with-tac#prerequisites))
* Python 3.10+
* A [Microsoft Foundry](https://azure.microsoft.com/en-us/products/ai-studio/) project with a deployed model (for example, GPT-4o)
* An Azure OpenAI API key for your deployed model

If you plan to use the Voice Live connector, you need a Microsoft Foundry Voice Live resource and a separate API key. See [Voice Live connector](#voice-live-connector) for setup.

## Choose a connector

The `twilio-agent-connect-microsoft` package provides connectors for different Microsoft Foundry services.

| Connector                 | Use case                                                                                                                                                                                    | Install extra     |
| ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------- |
| `AgentFrameworkConnector` | Build agents with [Microsoft Agent Framework](https://github.com/microsoft/agent-framework). Supports voice and SMS, session persistence, and custom tools. Recommended for most use cases. | `agent-framework` |
| `VoiceLiveConnector`      | Use [Microsoft Foundry Voice Live](https://learn.microsoft.com/en-us/azure/ai-services/speech-service/voice-live) for low-latency streaming inference over WebSocket. Voice only.           | `voice-live`      |

## Agent Framework connector

The Agent Framework connector is the recommended way to connect a Microsoft Foundry agent to Twilio channels. It supports both voice and SMS, with configurable session persistence.

### Install dependencies

```bash
pip install twilio-agent-connect-microsoft[agent-framework,server]
```

### Configure environment variables

Add the following to your `.env` file alongside your existing [TAC credentials](/docs/conversations/agent-connect/build-with-tac#configure-environment-variables).

```text
AZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com/
AZURE_AI_API_KEY=your_azure_openai_api_key
AZURE_AI_DEPLOYMENT_NAME=gpt-4o
```

### Set up the server

The `AgentFrameworkConnector` takes a function that creates an agent for each voice call or SMS message. The connector handles memory retrieval, channel routing, and conversation cleanup.

```python
# Fix SSL certificate verification (must be before other imports)
import truststore
truststore.inject_into_ssl()

import os

from agent_framework.openai import OpenAIChatClient
from dotenv import load_dotenv
from pathlib import Path

from tac_microsoft import (
    TAC,
    TACConfig,
    SMSChannelConfig,
    TACFastAPIServer,
    AgentFrameworkConnector,
    ConversationSession,
)

load_dotenv(Path(__file__).resolve().parent.parent / ".env")

# Azure AI client
client = OpenAIChatClient(
    azure_endpoint=os.environ["AZURE_OPENAI_ENDPOINT"],
    api_key=os.environ["AZURE_AI_API_KEY"],
    model=os.environ.get("AZURE_AI_DEPLOYMENT_NAME"),
)

# TAC setup
tac = TAC(config=TACConfig.from_env())

SYSTEM_PROMPT = "You are a helpful customer service agent. Be concise and friendly."


def create_agent(session: ConversationSession):
    """Return an Agent Framework agent for this conversation."""
    return client.as_agent(
        name="MyAgent",
        instructions=SYSTEM_PROMPT,
    )


connector = AgentFrameworkConnector(
    tac=tac,
    create_agent=create_agent,
    sms_config=SMSChannelConfig(memory_mode="always"),
)

server = TACFastAPIServer(
    tac=tac,
    voice_channel=connector.voice_channel,
    messaging_channels=[connector.sms_channel],
)

if __name__ == "__main__":
    server.start()
```

The connector automatically:

* Retrieves memory context from Conversation Memory and includes it in the first message (if memory retrieval is enabled)
* Routes responses to the correct channel (Voice or SMS)
* Cleans up agent resources when the conversation ends

### Add channel-aware prompts, tools, and hooks

You can customize the agent per channel, add tools inside your `create_agent` function, and pass hooks like `on_message` and `on_error` to the connector. This example adds:

* **Channel-aware prompts**: Different system prompts for voice and SMS.
* **Custom tools**: A `look_up_outage_tool` alongside built-in TAC tools (`create_memory_tool`, `create_knowledge_tool`).
* **`on_message` hook**: Prepends customer phone number and memory context to each message.
* **`on_error` hook**: Returns channel-appropriate error messages.
* **Session persistence**: `FileAgentSessionStore` for local development, with a `CosmosDBAgentSessionStore` option for production.
* **Conversation cleanup**: Deletes session data when a conversation ends.

```python
# Fix SSL certificate verification (must be before other imports)
import truststore
truststore.inject_into_ssl()

import asyncio
import logging
import os
from pathlib import Path

from agent_framework.openai import OpenAIChatClient
from azure.identity.aio import DefaultAzureCredential
from dotenv import load_dotenv

from tac_microsoft import (
    TAC,
    TACConfig,
    TACFastAPIServer,
    AgentFrameworkConnector,
    FileAgentSessionStore,
    ConversationSession,
    VoiceChannelConfig,
    SMSChannelConfig,
    format_memory_context,
)
from tac_microsoft.agent_framework_tools import create_knowledge_tool, create_memory_tool

load_dotenv(Path(__file__).resolve().parent.parent / ".env")

logger = logging.getLogger(__name__)

credential = DefaultAzureCredential()
client = OpenAIChatClient(
    credential=credential,
    azure_endpoint=os.environ["AZURE_OPENAI_ENDPOINT"],
    model=os.environ.get("AZURE_AI_DEPLOYMENT_NAME", "gpt-4o"),
)

tac = TAC(config=TACConfig.from_env())
knowledge_base_id = os.environ.get("TWILIO_KNOWLEDGE_BASE_ID")

# Build the knowledge tool once at startup — it doesn't depend on session state.
knowledge_tool = (
    asyncio.run(create_knowledge_tool(
        tac,
        knowledge_base_id,
        description="Search the knowledge base for relevant information.",
    ))
    if knowledge_base_id
    else None
)

# Channel-aware system prompts
VOICE_SYSTEM_PROMPT = """You are a customer service assistant on a phone call.
Keep responses clear, concise, and conversational.
Use plain text only — no markdown, no lists, no special formatting."""

SMS_SYSTEM_PROMPT = """You are a customer service assistant over SMS.
Keep responses concise and formatted for text messaging.
Use short paragraphs. Bullet points are OK."""


# Custom tool
def look_up_outage_tool(zip_code: str) -> str:
    """Check if there is a recent internet outage in a specific zip code."""
    return f"No reported outages in {zip_code}. Service is operating normally."


# Agent factory — called once per voice call, once per SMS message
def create_agent(session: ConversationSession):
    prompt = VOICE_SYSTEM_PROMPT if session.channel == "voice" else SMS_SYSTEM_PROMPT

    tools = [create_memory_tool(tac, session), look_up_outage_tool, knowledge_tool]

    return client.as_agent(
        name="OwlAgent",
        instructions=prompt,
        tools=[t for t in tools if t is not None],
    )


# Session persistence (FileAgentSessionStore for local development)
session_store = FileAgentSessionStore()

# To use CosmosDB instead (for horizontal scaling), uncomment below:
# from tac_microsoft import CosmosDBAgentSessionStore
# session_store = CosmosDBAgentSessionStore(
#     endpoint=os.environ["AZURE_COSMOS_ENDPOINT"],
#     credential=os.environ["AZURE_COSMOS_KEY"],
# )


def on_message(user_message, context, memory_response):
    """Customize the user message with context before sending it to the agent."""
    prefix = f"[Customer: {context.author_info.address if context.author_info else 'unknown'}]\n"
    return prefix + format_memory_context(memory_response, user_message)


def on_error(error, context):
    """Return a channel-appropriate error message."""
    logger.error("Agent error", extra={"conversation_id": context.conversation_id}, exc_info=error)
    if context.channel == "voice":
        return "I'm sorry, I'm having trouble right now. Please try again."
    return "Sorry, something went wrong. Please try again or call us for help."


async def handle_conversation_ended(context: ConversationSession) -> None:
    """Clean up sessions when a conversation closes."""
    await session_store.delete(context.conversation_id)
    logger.info("Session cleaned up", extra={"conversation_id": context.conversation_id})


tac.on_conversation_ended(handle_conversation_ended)

connector = AgentFrameworkConnector(
    tac=tac,
    create_agent=create_agent,
    on_message=on_message,
    on_error=on_error,
    voice_config=VoiceChannelConfig(memory_mode="never"),
    sms_config=SMSChannelConfig(memory_mode="always"),
    session_store=session_store,
)

server = TACFastAPIServer(
    tac=tac,
    voice_channel=connector.voice_channel,
    messaging_channels=[connector.sms_channel],
)

if __name__ == "__main__":
    server.start()
```

## Voice Live connector

[Microsoft Foundry Voice Live](https://learn.microsoft.com/en-us/azure/ai-services/speech-service/voice-live) provides low-latency streaming inference over WebSocket. The `VoiceLiveConnector` supports voice only (no SMS).

### Install dependencies

```bash
pip install twilio-agent-connect-microsoft[voice-live,server]
```

### Configure environment variables

Add the following to your `.env` file alongside your existing [TAC credentials](/docs/conversations/agent-connect/build-with-tac#configure-environment-variables). The endpoint is the hostname only — no `https://` prefix.

```text
AZURE_VOICE_LIVE_ENDPOINT=your-resource.services.ai.azure.com
AZURE_VOICE_LIVE_API_KEY=your_voice_live_api_key
AZURE_VOICE_LIVE_MODEL=gpt-4o
```

### Set up the server

```python
# Fix SSL certificate verification (must be before other imports)
import truststore
truststore.inject_into_ssl()

import os

from dotenv import load_dotenv
from tac.tools.base import function_tool

from tac_microsoft import (
    TAC,
    TACConfig,
    TACFastAPIServer,
    VoiceLiveConnector,
    VoiceLiveConfig,
)

load_dotenv()

# Custom tools — use @function_tool to create TACTool instances
@function_tool()
def look_up_outage(zip_code: str) -> str:
    """Check if there is a recent internet outage in a specific zip code.

    Args:
        zip_code: The zip code to check for outages.
    """
    return f"No reported outages in {zip_code}. Service is operating normally."


tac = TAC(config=TACConfig.from_env())

config = VoiceLiveConfig(
    endpoint=os.environ["AZURE_VOICE_LIVE_ENDPOINT"],
    model=os.environ.get("AZURE_VOICE_LIVE_MODEL", "gpt-4o"),
    api_key=os.environ.get("AZURE_VOICE_LIVE_API_KEY"),
    instructions="You are a helpful customer service agent. Be concise and friendly.",
    tools=[look_up_outage],
)

connector = VoiceLiveConnector(
    tac=tac,
    config=config,
)

server = TACFastAPIServer(
    tac=tac,
    voice_channel=connector.voice_channel,
)

if __name__ == "__main__":
    server.start()
```

Voice Live manages conversation state server-side through its WebSocket API, so you don't need local session management. When the model requests a tool call, the connector executes it and sends the result back through the WebSocket automatically.

## Run the example

To start from a working example instead of writing the server code yourself, clone the `twilio-agent-connect-microsoft` repository and run one of the included examples.

1. Clone the repository and install dependencies:

   ```bash
   git clone https://github.com/twilio/twilio-agent-connect-microsoft.git
   cd twilio-agent-connect-microsoft
   uv sync --all-extras
   ```
2. Create the shared `.env` file from the template. All examples read from this file:

   ```bash
   cp getting_started/examples/.env.example getting_started/examples/.env
   ```
3. Fill in your [TAC credentials](/docs/conversations/agent-connect/build-with-tac#configure-environment-variables) and the environment variables for the example you want to run:

   ## Agent Framework (basic)

   Basic example uses Azure OpenAI with API-key authentication.

   ```text
   AZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com/
   AZURE_AI_API_KEY=your_azure_openai_api_key
   AZURE_AI_DEPLOYMENT_NAME=gpt-4o
   ```

   ## Agent Framework (advanced)

   Advanced example uses Microsoft Foundry with Entra ID authentication through `DefaultAzureCredential`. Run `az login` before starting the server.

   ```text
   AZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com/
   AZURE_AI_DEPLOYMENT_NAME=gpt-4o
   ```

   ## Voice Live

   The Voice Live endpoint is the hostname only — no `https://` prefix.

   ```text
   AZURE_VOICE_LIVE_ENDPOINT=your-resource.services.ai.azure.com
   AZURE_VOICE_LIVE_API_KEY=your_voice_live_api_key
   AZURE_VOICE_LIVE_MODEL=gpt-4o
   ```
4. From the repository root, start the example server:

   ## Agent Framework (basic)

   ```bash
   uv run getting_started/examples/agent_framework/basic.py
   ```

   ## Agent Framework (advanced)

   ```bash
   uv run getting_started/examples/agent_framework/advanced.py
   ```

   ## Voice Live

   ```bash
   uv run getting_started/examples/voice_live/basic.py
   ```
5. Expose your local server with [ngrok](https://ngrok.com/) so Twilio can reach it, then configure your Twilio webhooks to point at the ngrok domain. For detailed steps, see [Expose your local server with ngrok](/docs/conversations/agent-connect/quickstart#expose-your-local-server-with-ngrok) and [Configure Twilio webhooks](/docs/conversations/agent-connect/quickstart#configure-twilio-webhooks).

## Test the integration

1. Start your server and make sure ngrok is running.
2. Call your Twilio phone number and verify that the voice channel works.
3. If you used the Agent Framework connector, send an SMS to your Twilio phone number and verify that you receive a response.
4. If you configured Conversation Memory, verify that responses include personalized context from the customer's profile.

## Next steps

* [Deploy to Azure Container Apps](/docs/conversations/agent-connect/integrations/microsoft-foundry/deploy): Deploy your TAC application to production on Azure.
* [Add TAC to your agent](/docs/conversations/agent-connect/build-with-tac): Add custom tools and knowledge search.
* [Troubleshooting](/docs/conversations/agent-connect/troubleshooting): Common issues and solutions.