Google Cloud Pub/Sub destination
Beta
Data Destinations is currently available as a Private Beta product and the information contained in this document is subject to change. Data Destinations is not a HIPAA Eligible Service or PCI compliant and should not be enabled in workflows that are subject to HIPAA or PCI. To learn more, see Private beta legal information.
Google Cloud Pub/Sub is a fully managed messaging service. Sending event data to Pub/Sub lets you publish Twilio events to a topic you own, so any subscriber in your Google Cloud project can consume them.
- Category: Raw data
- Release status: Beta
- Authentication: Service account key
- Regional availability: US1
Get early access to native destinations for data warehouses, observability tools, and event pipelines.
You can send all event types supported by Event Streams to Google Cloud Pub/Sub. This destination has no user-identifier restriction.
Before you set up a Google Cloud Pub/Sub destination, make sure you have:
- A Google Cloud project with an existing Pub/Sub topic. Twilio delivers to an existing topic and doesn't create one for you.
- A Google service account with the
roles/pubsub.publisherrole granted on that topic, and a JSON key for that service account.
Info
See Set up a destination for the full UI and Destinations API setup flow common to every destination type.
To create a Google Cloud Pub/Sub destination in the Twilio Console:
- From your Google Cloud account, gather the settings listed in Destination settings.
- In the Twilio Console, go to Data > Data Destinations > Destinations > Add destination subscription.
- Complete the Add destination subscription wizard.
To create a Google Cloud Pub/Sub destination through the Destinations API, send a POST request to the Destinations endpoint and set destinationType to gcp-pubsub. For authentication and the other request body fields, see Create a destination.
POST "https://destinations.twilio.com/v1/ControlPlane/Destinations"
Set the config object in the request body as follows:
| Name | Type | Description |
|---|---|---|
credentials.credentialType (Required) | string | Set to SERVICE_ACCOUNT_KEY. |
credentials.credentialsJSON (Required) | string | Your Google service account JSON key, which authenticates to Pub/Sub. |
project (Required) | string | Google Cloud project ID that owns the target topic. |
topic (Required) | string | Target Pub/Sub topic name. Must exist before events can be sent. |
The following example shows a request body. Replace the uppercase placeholder values with your own.
1{2"destinationType": "gcp-pubsub",3"name": "My Google Cloud Pub/Sub destination",4"config": {5"credentials": {6"credentialType": "SERVICE_ACCOUNT_KEY",7"credentialsJSON": "CREDENTIALS_JSON"8},9"project": "PROJECT",10"topic": "TOPIC"11}12}
Use gcp-pubsub as the destinationType.
| Field | Description | Required | Default or choices |
|---|---|---|---|
| GCP Project | Google Cloud project ID that owns the target topic. | Yes | N/A |
| Topic | Target Pub/Sub topic name. Must already exist. Twilio doesn't create it for you. | Yes | N/A |
| Service Account JSON Key | Google service account JSON key used to authenticate to Pub/Sub. Sensitive, stored encrypted. | Yes | N/A |
Twilio authenticates to Pub/Sub by using the service account key you provide. The account needs only roles/pubsub.publisher on the target topic. Twilio never requests broader project-level access, such as roles/pubsub.viewer.
| Field | Pub/Sub field |
|---|---|
| Event payload | Message data |
| Event type | Message attribute: eventType |
| Event ID | Message attribute: eventId |
| Event time | Message attribute: eventTime |
| Event source | Message attribute: eventSource |
When you save a Google Cloud Pub/Sub destination, Twilio validates your credentials by checking whether the service account holds publish permission on the configured topic. This check has a limitation: the Google permission-check API can report success even when the topic doesn't exist. Twilio might not catch a missing topic when you save, and it can surface later as a delivery failure in the Debugger.
Twilio doesn't currently offer a way to manually trigger a test event for this destination, from either the Console or the CLI. To confirm delivery works, wait for a real event of a subscribed type to occur.
| Cause | What you'll see | Fix |
|---|---|---|
| Invalid or expired service account key | Validation fails with an authentication error | Re-check the Service Account JSON Key and update the destination |
Service account missing roles/pubsub.publisher on the topic | Validation may pass, then delivery fails with a permission error reported in the Debugger | Grant roles/pubsub.publisher on the topic to the service account |
| Topic does not exist | Validation may pass, then delivery fails with a not-found error reported in the Debugger | Create the topic in your Google Cloud project. Twilio doesn't create it |
| GCP Project or Topic name incorrect | Validation fails, or delivery fails | Confirm the project ID and topic name |
| Publish rejected as a conflicting operation (409) | Delivery errors reported in the Debugger; not retried | Rare, usually resolves on its own. Contact Twilio support if it persists. |
If Twilio can't deliver events to your topic, Twilio sends an error notification to the Debugger (available in US1 region only). If Twilio continues to encounter delivery errors, Twilio notifies you every 20 minutes after the initial error.