Voice Intelligence - Transcript Resource

Voice Intelligence supports various audio formats, each suited for different needs:

• Mono: A single audio channel, suitable for straightforward recordings where speaker differentiation isn't crucial.
• Stereo: Two channels providing spatial sound, but not specifically separating speakers.
• Dual-Channel: Two distinct audio tracks in the same file, ideal for differentiating speakers such as agents and customers in call recordings. This format enhances transcription accuracy and participant differentiation.

We recommend using dual-channel recordings to improve transcription accuracy, especially in scenarios requiring speaker differentiation.

The Channel parameter object contains information about the recording. When creating a new Transcript resource, specify the recording you want to transcribe via the Channel parameter's media_properties object. You can also customize participant/channel labeling via the Channel object's participants array. See the "Specify participant information" section below for more information.

The table below describes the properties of the Channel parameter object. Click Show child properties to show the media_properties and participants fields.

You can optionally provide a CustomerKey parameter to map a Transcript to an internal identifier known within your system. This unique identifier helps track the Transcript, and it's included in webhook callback when the results for Transcripts and Operators are available. Note that CustomerKey doesn't replace the Transcript SID in Voice Intelligence API calls.

To transcribe Recordings made via Twilio and stored within Twilio's infrastructure, provide the Recording SID in the Channel object's media_properties.source_sid property as shown below. REXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX represents a Recording SID.

In this scenario, the Channel information appears as follows:

1
{
2
     "media_properties":{
3
        "source_sid": "REXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
4
     }
5
}

1
// Download the helper library from https://www.twilio.com/docs/node/install
2
const twilio = require("twilio"); // Or, for ESM: import twilio from "twilio";
3

4
// Find your Account SID and Auth Token at twilio.com/console
5
// and set the environment variables. See http://twil.io/secure
6
const accountSid = process.env.TWILIO_ACCOUNT_SID;
7
const authToken = process.env.TWILIO_AUTH_TOKEN;
8
const client = twilio(accountSid, authToken);
9

10
async function createTranscript() {
11
  const transcript = await client.intelligence.v2.transcripts.create({
12
    channel: {
13
      media_properties: {
14
        source_sid: "REXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
15
      },
16
    },
17
    serviceSid: "GAaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
18
  });
19

20
  console.log(transcript.accountSid);
21
}
22

23
createTranscript();

1
{
2
  "account_sid": "ACaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
3
  "service_sid": "GAaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
4
  "sid": "GTaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
5
  "date_created": "2010-08-31T20:36:28Z",
6
  "date_updated": "2010-08-31T20:36:28Z",
7
  "status": "queued",
8
  "channel": {
9
    "media_properties": {
10
      "media_url": "http://foobar.test/ClusterTests/call1.wav"
11
    }
12
  },
13
  "data_logging": false,
14
  "language_code": "en-US",
15
  "media_start_time": null,
16
  "duration": 0,
17
  "customer_key": "aaaaaaaa",
18
  "url": "https://intelligence.twilio.com/v2/Transcripts/GTaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
19
  "redaction": true,
20
  "links": {
21
    "sentences": "https://intelligence.twilio.com/v2/Transcripts/GTaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/Sentences",
22
    "media": "https://intelligence.twilio.com/v2/Transcripts/GTaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/Media",
23
    "operator_results": "https://intelligence.twilio.com/v2/Transcripts/GTaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/OperatorResults"
24
  }
25
}

• The recording file size must not exceed 3GB.
• Audio duration can't exceed eight hours.
• Recordings shorter than two seconds aren't transcribed.
• Transcripts are indexed and available for search for 90 days.
• You can create only one Voice Intelligence Transcript resource for a given Recording resource. To re-transcribe a Recording, delete the original Voice Intelligence Transcript resource and create a new one.
• To create Transcripts for Twilio Recordings in external storage, use the MediaUrl parameter. The SourceSid parameter isn't supported for externally-stored Twilio Recordings.
• You can't create Voice Intelligence Transcripts from encrypted Voice Recordings, because Voice Intelligence can't decrypt those resources. You should move those recordings to your own external storage and generate pre-signed URLs for the decrypted recordings. Once you've done that, you can follow the instructions in the "Transcribe an external recording" section below.

To transcribe a recording stored externally, for example, a recording stored in your own S3 bucket, provide the recording's URL in the Channel object's media_properties.media_url property.

The following limitations apply when transcribing an external recording (specified by a MediaUrl):

• You must make external recordings stored in Twilio Assets public.
• Basic authentication on MediaUrls isn't supported for external recordings. If you store the recordings on S3, use a presigned URL. And when storing them on Azure Blob Storage, use a Shared Access Signature (SAS).
• MediaUrls that respond with a non-200 HTTP status code will result in a failed request.
• Requests to access external recordings are performed once. There is currently no retry behavior.

To transcribe the audio of a Twilio Video recording, it needs additional processing to become compatible with Voice Intelligence.

First, create a dual-channel audio recording by transcoding a separate audio-only composition for each participant in the Video Room.

1
curl -X POST "https://video.twilio.com/v1/Compositions" \ --data-urlencode "AudioSources=PAXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
2
\ --data-urlencode "StatusCallback=https://www.example.com/callbacks"
3
\ --data-urlencode "Format=mp4"
4
\ --data-urlencode "RoomSid=RMXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
5
\ -u $TWILIO_ACCOUNT_SID:$TWILIO_AUTH_TOKEN

Next, download the media from these compositions and merge them into a single stereo audio file.

ffmpeg -i speaker1.mp4 -i speaker2.mp4 -filter_complex "[0:a][1:a]amerge=inputs=2[a]" -map "[a]" -f flac -bits_per_raw_smaple 16 -ar 441000 output.flac

If the recording duration for each participant differs, you can avoid overlapping audio tracks. Use ffmpeg to create a single-stereo audio track with delay to cover the difference in track length. For example, if one audio track lasts 63 seconds and the other 67 seconds, use ffmpeg to create a stereo file with the first track, with four seconds of delay to match the length of the second track.

ffmpeg -i speaker1.wav -i speaker2.wav -filter_complex "aevalsrc=0:d=${second_to_delay}[s1];[s1][1:a]concat=n=2:v=0:a=1[ac2];[0:a]apad[ac1];[ac1][ac2]amerge=2[a]" -map "[a]" -f flac -bits_per_raw_sample 16 -ar 441000 output.flac

Finally, use the Create a new Voice Intelligence Transcript endpoint with the Channel parameter's media_properties.media_url property set to a publicly accessible URL of the audio file.

Recordings must be publicly accessible during transcription. The recordings can be hosted or used on a time-limited pre-signed URL. To share a recording on an existing AWS S3 bucket, read the "Sharing objects with pre-signed URLs" guide from AWS.

Twilio attempts to download an external recording for up to 10 minutes. After 10 minutes, the transcription fails.

You can't transcribe encrypted recordings.

Voice Intelligence doesn't perform speaker diarization on recordings, meaning it doesn't differentiate between different speakers. Additionally, using mono recordings can lead to reduced transcription accuracy. For improved transcription accuracy and participant differentiation, use dual-channel recordings.

Voice Intelligence supports both mono and stereo audio formats for the following media formats:

• WAV (PCM-encoded)
• MP3
• FLAC

The following limits apply to the media files:

• The maximum file size allowed is 3GB.
• The maximum audio length is eight hours.
• The minimum sample rate required is 8kHz (telephony grade). For best results, use 16KHz.

In this scenario, the Channel information appears as follows:

1
{
2
     "media_properties":{
3
        "media_url": "http://www.example.com/recording/call.wav"
4
     }
5
}

1
// Download the helper library from https://www.twilio.com/docs/node/install
2
const twilio = require("twilio"); // Or, for ESM: import twilio from "twilio";
3

4
// Find your Account SID and Auth Token at twilio.com/console
5
// and set the environment variables. See http://twil.io/secure
6
const accountSid = process.env.TWILIO_ACCOUNT_SID;
7
const authToken = process.env.TWILIO_AUTH_TOKEN;
8
const client = twilio(accountSid, authToken);
9

10
async function createTranscript() {
11
  const transcript = await client.intelligence.v2.transcripts.create({
12
    channel: {
13
      media_properties: {
14
        media_url: "https://example.com/your-recording.wav",
15
      },
16
    },
17
    serviceSid: "GAaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
18
  });
19

20
  console.log(transcript.accountSid);
21
}
22

23
createTranscript();

1
{
2
  "account_sid": "ACaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
3
  "service_sid": "GAaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
4
  "sid": "GTaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
5
  "date_created": "2010-08-31T20:36:28Z",
6
  "date_updated": "2010-08-31T20:36:28Z",
7
  "status": "queued",
8
  "channel": {
9
    "media_properties": {
10
      "media_url": "http://foobar.test/ClusterTests/call1.wav"
11
    }
12
  },
13
  "data_logging": false,
14
  "language_code": "en-US",
15
  "media_start_time": null,
16
  "duration": 0,
17
  "customer_key": "aaaaaaaa",
18
  "url": "https://intelligence.twilio.com/v2/Transcripts/GTaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
19
  "redaction": true,
20
  "links": {
21
    "sentences": "https://intelligence.twilio.com/v2/Transcripts/GTaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/Sentences",
22
    "media": "https://intelligence.twilio.com/v2/Transcripts/GTaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/Media",
23
    "operator_results": "https://intelligence.twilio.com/v2/Transcripts/GTaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/OperatorResults"
24
  }
25
}

By default, Voice Intelligence labels the left channel (channel one) as Agent and the right channel (channel two) as Customer. Depending on your call flow and the recorded call leg, this may not accurately reflect the participant/channel relationships on your recording. If needed, specify which participant is on a given channel via the Channel parameter's participants array.

Only two participants can be overridden in the Channel object of the Transcript resource.

The code sample below demonstrates an example request that overrides the default Voice Intelligence labels.

1
// Download the helper library from https://www.twilio.com/docs/node/install
2
const twilio = require("twilio"); // Or, for ESM: import twilio from "twilio";
3

4
// Find your Account SID and Auth Token at twilio.com/console
5
// and set the environment variables. See http://twil.io/secure
6
const accountSid = process.env.TWILIO_ACCOUNT_SID;
7
const authToken = process.env.TWILIO_AUTH_TOKEN;
8
const client = twilio(accountSid, authToken);
9

10
async function createTranscript() {
11
  const transcript = await client.intelligence.v2.transcripts.create({
12
    channel: {
13
      media_properties: {
14
        media_url: "https://example.com/your-recording",
15
      },
16
      participants: [
17
        {
18
          user_id: "id1",
19
          channel_participant: 1,
20
          media_participant_id: "+1555959545",
21
          email: "veronica.meyer@example.com",
22
          full_name: "Veronica Meyer",
23
          image_url:
24
            "https://images.unsplash.com/photo-1438761681033-6461ffad8d80",
25
          role: "Customer",
26
        },
27
        {
28
          user_id: "id2",
29
          channel_participant: 2,
30
          media_participant_id: "+1555959505",
31
          email: "lauryn.trujillo@example.com",
32
          full_name: "Lauryn Trujillo",
33
          image_url:
34
            "https://images.unsplash.com/photo-1554384645-13eab165c24b",
35
          role: "Agent",
36
        },
37
      ],
38
    },
39
    serviceSid: "GAaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
40
  });
41

42
  console.log(transcript.accountSid);
43
}
44

45
createTranscript();

1
{
2
  "account_sid": "ACaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
3
  "service_sid": "GAaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
4
  "sid": "GTaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
5
  "date_created": "2010-08-31T20:36:28Z",
6
  "date_updated": "2010-08-31T20:36:28Z",
7
  "status": "queued",
8
  "channel": {
9
    "media_properties": {
10
      "media_url": "http://foobar.test/ClusterTests/call1.wav"
11
    }
12
  },
13
  "data_logging": false,
14
  "language_code": "en-US",
15
  "media_start_time": null,
16
  "duration": 0,
17
  "customer_key": "aaaaaaaa",
18
  "url": "https://intelligence.twilio.com/v2/Transcripts/GTaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
19
  "redaction": true,
20
  "links": {
21
    "sentences": "https://intelligence.twilio.com/v2/Transcripts/GTaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/Sentences",
22
    "media": "https://intelligence.twilio.com/v2/Transcripts/GTaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/Media",
23
    "operator_results": "https://intelligence.twilio.com/v2/Transcripts/GTaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/OperatorResults"
24
  }
25
}