---
"@context": https://schema.org
"@type": TechArticle
"@id": https://www.twilio.com/docs/data-destinations/destination/kafka#article
headline: Apache Kafka destination
description: Learn how to send event data to Apache Kafka using a Twilio-managed destination.
url: https://www.twilio.com/docs/data-destinations/destination/kafka
inLanguage: en
dateModified: 2026-11-04T17:24:28.000Z
author:
  "@type": Organization
  name: Twilio Developer Education Team
publisher:
  "@type": Organization
  name: Twilio
---

# Apache Kafka destination

> \[!IMPORTANT]
>
> Data Destinations is currently available as a Private Beta product and the information contained in this document is subject to change. Data Destinations is not a HIPAA Eligible Service or PCI compliant and should not be enabled in workflows that are subject to HIPAA or PCI. To learn more, see [Private beta legal information](/docs/data-destinations#private-beta-legal-information).

[Apache Kafka](https://kafka.apache.org/) is a distributed event-streaming platform used for high-throughput, fault-tolerant message delivery. This destination publishes your events directly to a Kafka topic you manage on a publicly reachable provider, such as Confluent Cloud or Amazon MSK. Self-hosted clusters on a private network aren't supported.

### Destination overview

* **Category**: Raw data
* **Release status**: Beta
* **Authentication**: SASL
* **Regional availability**: US1

## Join the Data Destinations private beta

[Sign up](https://airtable.com/appaJfyV6LiIFMxuP/pagIaOAtKsQVxTfdW/form)

Get early access to native destinations for data warehouses, observability tools, and event pipelines.

## Eligible events

You can send all [event types](/docs/events/event-types-overview) supported by Event
Streams to Apache Kafka. This destination has no user-identifier
restriction.

## Get started

### Prerequisites

Before you set up a Kafka destination, make sure you have:

* A publicly reachable Kafka cluster (self-hosted clusters on a private network aren't supported) with a **topic already created**. Twilio doesn't create topics automatically, so delivery to a missing topic fails.
* The broker addresses for your cluster (for example, `broker1:9092,broker2:9092`).
* A **SASL username and password** for a SASL mechanism your cluster supports (`PLAIN`, `SCRAM-SHA-256`, or `SCRAM-SHA-512`). This destination requires SASL authentication, and you can't currently configure it without credentials.
* `Describe` and `Write` permissions on the target topic for that SASL principal, if your cluster enforces ACLs. Twilio uses `Describe` for the connectivity check it runs when you save and `Write` to publish events.
* TLS turned on for the broker listeners you're connecting to, if your cluster requires TLS.

## Create the destination

> \[!NOTE]
>
> See [Set up a destination](/docs/data-destinations/destination-setup) for the full UI and Destinations API setup flow common to every destination type.

### Twilio Console

To create an Apache Kafka destination in the Twilio Console:

1. From your Kafka cluster, gather the settings listed in [Destination settings](#destination-settings).
2. In the Twilio Console, go to **Data** > **Data Destinations** > **Destinations** > **Add destination subscription**.
3. Complete the **Add destination subscription** wizard.

### Destinations API

To create an Apache Kafka destination through the Destinations API, send a `POST` request to the Destinations endpoint and set `destinationType` to `kafka`. For authentication and the other request body fields, see [Create a destination](/docs/data-destinations/api-quickstart#create-a-destination).

```text title="Create a destination"
POST "https://destinations.twilio.com/v1/ControlPlane/Destinations"
```

Set the `config` object in the request body as follows:

| Name                                              | Type           | Description                                                                                 |
| ------------------------------------------------- | -------------- | ------------------------------------------------------------------------------------------- |
| `brokers` <br />(**Required**)                    | string         | A comma-separated list of broker addresses, for example `broker1:9092,broker2:9092`.        |
| `credentials.credentialType` <br />(**Required**) | string         | Set to `SASL`.                                                                              |
| `credentials.password` <br />(**Required**)       | string         | SASL password.                                                                              |
| `credentials.saslMechanism` <br />(**Required**)  | enum`<string>` | The SASL authentication mechanism. Valid values: `PLAIN`, `SCRAM-SHA-256`, `SCRAM-SHA-512`. |
| `credentials.username` <br />(**Required**)       | string         | SASL username.                                                                              |
| `tlsEnabled`                                      | boolean        | Whether to use TLS for broker connections.                                                  |
| `tlsSkipVerify`                                   | boolean        | Whether to skip TLS certificate verification. Not recommended for production.               |
| `topic` <br />(**Required**)                      | string         | Target Kafka topic name.                                                                    |

The following example shows a request body. Replace the uppercase placeholder values with your own.

```json title="Example request body"
{
  "destinationType": "kafka",
  "name": "My Apache Kafka destination",
  "config": {
    "brokers": "BROKERS",
    "credentials": {
      "credentialType": "SASL",
      "password": "PASSWORD",
      "saslMechanism": "PLAIN",
      "username": "USERNAME"
    },
    "topic": "TOPIC"
  }
}
```

## Destination settings

Use `kafka` as the `destinationType`.

| Field           | Description                                                                          | Required | Default or choices                        |
| --------------- | ------------------------------------------------------------------------------------ | -------- | ----------------------------------------- |
| Brokers         | Comma-separated broker addresses (for example, `broker1:9092,broker2:9092`)          | Yes      | None                                      |
| Topic           | Target Kafka topic name. Must already exist, since Twilio doesn't create it for you. | Yes      | None                                      |
| SASL Mechanism  | SASL authentication mechanism                                                        | Yes      | `PLAIN`, `SCRAM-SHA-256`, `SCRAM-SHA-512` |
| Username        | SASL username                                                                        | Yes      | None                                      |
| Password        | SASL password (sensitive, stored encrypted)                                          | Yes      | None                                      |
| TLS Enabled     | Encrypt the connection to your brokers                                               | No       | `false`                                   |
| TLS Skip Verify | Skip TLS certificate verification (not recommended for production)                   | No       | `false`                                   |

Twilio authenticates to your Kafka cluster by using SASL (`PLAIN`, `SCRAM-SHA-256`, or `SCRAM-SHA-512`), optionally combined with TLS encryption. **SASL Mechanism**, **Username**, and **Password** are all required, since you can't currently configure this destination for a cluster that doesn't require credentials.

## Default mappings

| Field         | Kafka field          |
| ------------- | -------------------- |
| Event source  | Partition key        |
| Event payload | Message value (JSON) |

## Verification and troubleshooting

When you save a Kafka destination, Twilio validates connectivity by issuing a non-destructive metadata request to your brokers. This request proves TCP reachability and SASL authentication without publishing any data. If validation fails, check that:

* Your **Brokers** address is correct and reachable from the public internet (or your configured network path).
* Your **SASL Mechanism**, **Username**, and **Password** are correct.
* **TLS Enabled** matches what your cluster listener actually requires.

Twilio doesn't currently offer a way to manually trigger a test event for this destination, from either the Console or the CLI. To confirm delivery works, wait for a real event of a subscribed type to occur.

### Common errors

| Cause                                                  | What you'll see                                                                                                            | Fix                                                                                 |
| ------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------- |
| Broker unreachable                                     | Validation fails with a connection error                                                                                   | Confirm the broker address and network path (firewall, security group, VPC peering) |
| Invalid SASL credentials                               | Validation fails with an authentication error                                                                              | Re-check your username, password, and SASL Mechanism                                |
| SASL principal lacks `Describe` ACL on the topic       | Validation fails with an authorization error, even with correct credentials                                                | Grant the SASL principal `Describe` on the topic                                    |
| SASL principal lacks `Write` ACL on the topic          | Passes validation, then delivery errors reported in the [Debugger](/docs/usage/troubleshooting/debugging-your-application) | Grant the SASL principal `Write` on the topic                                       |
| Topic does not exist                                   | Delivery errors reported in the [Debugger](/docs/usage/troubleshooting/debugging-your-application)                         | Create the topic on your cluster. Twilio doesn't create topics automatically        |
| TLS mismatch                                           | Connection or handshake errors                                                                                             | Confirm **TLS Enabled** matches your broker listener's configuration                |
| Event payload exceeds the broker's `message.max.bytes` | Delivery errors reported in the [Debugger](/docs/usage/troubleshooting/debugging-your-application); not retried            | Reduce the event payload size, or raise `message.max.bytes` on your broker/topic    |

If Twilio can't deliver events to your destination, Twilio sends an error notification to the [Debugger](/docs/usage/troubleshooting/debugging-your-application) (available in US1 region only). If Twilio continues to encounter delivery errors, Twilio notifies you every 20 minutes after the initial error.

## Additional resources

* [Destinations overview](/docs/data-destinations/destinations-overview)
* [Set up a destination](/docs/data-destinations/destination-setup)
* [Event types reference](/docs/events/event-types-overview)