TwiML™ Voice: <VirtualAgent> with Dialogflow CX

Before using <VirtualAgent>, complete the Dialogflow CX one-click integration.

<VirtualAgent> is a TwiML noun that is nested within the <Connect> verb.

After the <Connect><VirtualAgent> session ends, you might execute a new set of TwiML instructions for the call. To execute a new set of instructions, use the action and method attributes of the <Connect> noun.

The action attribute takes an absolute or relative URL as a value. After a <Connect><VirtualAgent> session ends, Twilio sends an HTTP request to this action URL. It expects a response that contains a new set of TwiML instructions. Lacking an action attribute, Twilio requests TwiML instructions from the current TwiML document.

1
const VoiceResponse = require('twilio').twiml.VoiceResponse;
2

3
const response = new VoiceResponse();
4
const connect = response.connect({
5
    action: 'https://myactionurl.com/twiml'
6
});
7
connect.virtualAgent({
8
    connectorName: 'project',
9
    statusCallback: 'https://mycallbackurl.com'
10
});
11

12
console.log(response.toString());

1
<?xml version="1.0" encoding="UTF-8"?>
2
<Response>
3
  <Connect action="https://myactionurl.com/twiml" >
4
    <VirtualAgent connectorName="project" statusCallback="https://mycallbackurl.com"/>
5
  </Connect>
6
</Response>

For Dialogflow CX, this HTTP request's body contains additional information about the result of the <Connect><VirtualAgent> session, along with Twilio's standard request parameters. A <Connect><VirtualAgent> session ends (and thereby triggers Twilio's HTTP request to the action URL) in the following ways:

• The Dialogflow CX agent session ends successfully and the caller did not hang up.
• The conversation with the Dialogflow CX agent requires a live agent handoff.
• The caller hangs up.
• The conversation with the virtual agent pauses.
• An error occurred during the VirtualAgent session.

The Twilio request body to the action URL contains parameters about the <Connect><VirtualAgent> session, along with Twilio's standard request parameters. To configure the call behavior after a <Connect><VirtualAgent> session or to log data for analytics purposes, write your app or service to consume this data from the request.

This property describes why the VirtualAgent session ended. Possible values include:

• completed: VirtualAgent terminated the session in one of the following ways:
  • The Dialogflow CX agent indicated end of conversation (by matching the end of conversation intent)
  • The caller hung up
• live-agent-handoff: VirtualAgent returned a live agent handoff response
• failed: An error occurred during VirtualAgent processing
• paused: VirtualAgent session was paused with the intention of the session being resumed later

This parameter contains Dialogflow-specific data from the agent interaction expressed in a serialized JSON object with key-value pairs.

To continue a call after a <Connect><VirtualAgent> session ends, your application should provide new TwiML instructions based on that session.

This optional <Connect> attribute indicates the HTTP request method to be used when Twilio sends a request to the action URL. If you don't specify a method attribute in <Connect>, POST is used by default.

The allowed values are POST and GET.

The following table lists all <VirtualAgent> attributes with their allowed values. The Dialogflow CX integration allows additional customization of the call flow using <Parameter> and <Configuration> TwiML nouns.

The Dialogflow CX connector instance configures this attribute in the Unique Name field. You can review your Dialogflow CX connector instances in the Twilio Console.

The statusCallback attribute points to an absolute URL where Twilio sends HTTP requests. During the Dialogflow agent interaction, Twilio sends HTTP requests to this URL each time an intent matches in the Dialogflow agent's conversation.

The Twilio request bodies to your statusCallback URL contains information about intents, transcripts (caller text vs. virtual agent text), sentiment analysis (if enabled), and custom parameters that your app sent to Dialogflow to build custom logic.

The Twilio request body to your statusCallback URL contains all of the following parameters:

This parameter in the status callback contains the Dialogflow CX-specific intent event data formatted as a serialized JSON object. The JSON object can contain the following key-value pairs.

The HTTP method your agent should use when requesting the statusCallback URL. This attribute accepts GET or POST. The value defaults to POST.

To pass custom parameters to your Dialogflow CX agent, nest the <Parameter> TwiML noun within <VirtualAgent>. The <Parameter> noun allows you to send custom session parameters to your Dialogflow CX virtual agent. Your agent can reference them as $session.params.parameter-name.

<Parameter> has two attributes: name and value. Use both every time you use <Parameter>.

1
const VoiceResponse = require('twilio').twiml.VoiceResponse;
2

3
const response = new VoiceResponse();
4
const connect = response.connect();
5
const virtualagent = connect.virtualAgent({connectorName: 'uniqueName'});
6
virtualagent.parameter({name: 'customer_name', value: 'Burton Guster'});
7

8
console.log(response.toString());

1
<Response>
2
  <Connect>
3
    <VirtualAgent connectorName="uniqueName">
4
      <Parameter name="customer_name" value="Burton Guster"/>
5
    </VirtualAgent>
6
  </Connect>
7
</Response>

You can pause and resume a Dialogflow CX agent conversation session at any time. This allows you to pass a call back and forth between Dialogflow and Twilio while keeping the session context intact. As a result, the caller doesn't need to repeat themselves or start from the beginning when Twilio passes the call back to Dialogflow.

Sessions stay active for a 30 minute time-to-live (TTL) period between pausing and resuming.

To pause a session, your Dialogflow CX agent must return a custom payload serialized JSON object that contains the key TWILIO_ACTION with the value PAUSE. This signals to Twilio that the session should be paused. Twilio then sends a request to the action URL you've specified. You can add additional parameters in the custom payload from your Dialogflow agent. Twilio includes the parameters in the request to the action URL.

When a session pauses, the Twilio request to your action URL performs two actions:

1. Sets VirtualAgentStatus to paused.
2. Returns VirtualAgentProviderData with a PauseParameters property. This property contains any custom parameters from the custom payload.

1
{
2
  "TWILIO_ACTION": "PAUSE",
3
  "memberId": "$session.params.member_id",
4
  "paymentAmount": "$122.34"
5
}

1
{
2
  "memberId": "abc123", 
3
  "paymentAmount": "$122.34"
4
}

To resume a session, send a request to Twilio with the TwiML <Connect><VirtualAgent> containing a nested <Config> noun. The <Config> noun can include the following parameters as name and value pairs.

You can only resume a session of the same Call SID which initiated the conversation.

To learn more about <Config>, consult the dynamic connector configuration section.

1
<?xml version="1.0" encoding="UTF-8"?>
2
<Response>
3
	<Connect>
4
		<VirtualAgent connectorName="uniqueName">
5
			<Config name="resumeEndUserId" value="projects/ccai-test-customer/locations/us-east1/conversations/119qL_EbVa6QqascUqOqi0taw/participants/40aMiI3SQZaEbby-EIyhrw"/>
6
			<Config name="resumeEventName" value="customEventName"/>
7
		</VirtualAgent>
8
	</Connect>
9
</Response>

With the TwiML <Config> noun, you can change the behavior of your virtual agent. This includes changing your underlying Dialogflow CX Connector's configuration or passing additional parameters. The <Config> noun nests inside of <VirtualAgent>. <Config> has two attributes: name and value, both of which must be used every time you use <Config>.

For each configuration option you want to override, include a new <Config> noun. The name attribute corresponds to one of your Dialogflow CX Connector's configuration options, such as language, sentimentAnalysis, voiceName, and welcomeIntent.

You can set additional attributes not present in your Dialogflow CX Connector configuration using this TwiML noun. These include voiceModel, speechModel, and speechModelVariant.

The TwiML <Config> noun accepts the following attributes:

1
const VoiceResponse = require('twilio').twiml.VoiceResponse;
2

3
const response = new VoiceResponse();
4
const connect = response.connect();
5
const virtualagent = connect.virtualAgent({connectorName: 'uniqueName'});
6
virtualagent.config({name: 'language', value: 'en-us'});
7
virtualagent.config({name: 'voiceName', value: 'en-US-Wavenet-C'});
8

9
console.log(response.toString());

1
<Response>
2
  <Connect>
3
    <VirtualAgent connectorName="uniqueName">
4
      <Config name="language" value="en-us"/>
5
      <Config name="voiceName" value="en-US-Wavenet-C"/>
6
    </VirtualAgent>
7
  </Connect>
8
</Response>