---
"@context": https://schema.org
"@type": TechArticle
"@id": https://www.twilio.com/docs/events/eventstreams-quickstart/kinesis-scripts#article
headline: Kinesis scripts
description: Use the following scripts to automate the creation of a Kinesis Stream and retrieve messages.
url: https://www.twilio.com/docs/events/eventstreams-quickstart/kinesis-scripts
inLanguage: en
dateModified: 2026-11-04T17:24:28.000Z
author:
  "@type": Organization
  name: Twilio Developer Education Team
publisher:
  "@type": Organization
  name: Twilio
---

# Kinesis scripts

Use the following Bash scripts to automate the creation of a Kinesis Stream and retrieve a test message. Copy the code below and save it to your computer, for example as `create_kinesis_stream.sh` or `retrieve_test_message.sh`.

## Create a Kinesis stream

Run `chmod +x create_kinesis_stream.sh` to make it executable.

You also need `jq`, a command line JSON processor the script depends on. For installation instructions for your OS, see [the jq download page](https://jqlang.org/download/).

The script also depends upon the AWS CLI, which you will need to install and [configure](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-quickstart.html) before executing the script.

The script takes two arguments: your chosen AWS Kinesis Stream name and a shard count.

```bash
#!/bin/bash

JQ_CHECK=$(which jq)
if [ -z "$JQ_CHECK" ]; then
  echo
  echo "This script requires the jq JSON processor. Please install for your OS from https://jqlang.org/download/"
  echo
  exit 1
fi

if [ $# -ne 2 ]; then
  echo
  echo "usage: $0 <stream_name> <shard_count>"
  echo
  exit 1
fi

# Set the stream name
STREAM_NAME=${1:-twilio-events}
SHARD_COUNT=${2:-1}

# Create the initial stream
aws kinesis create-stream --stream-name $STREAM_NAME --shard-count $SHARD_COUNT
if [ $? -ne 0 ]; then
  echo "Kinesis create failed"
  exit 1
fi

# Get the ARN for the Kinesis Stream
KINESIS_ARN=$(aws kinesis describe-stream --stream-name $STREAM_NAME | jq -r .StreamDescription.StreamARN)

# Create the policy for the Kinesis Stream
POLICY_ARN=$(aws iam create-policy --policy-name twilio-events-kinesis-write --policy-document '{
   "Version": "2012-10-17",
   "Statement": [
       {
           "Sid": "Quickstart0",
           "Effect": "Allow",
           "Action": [
               "kinesis:PutRecord",
               "kinesis:PutRecords"
           ],
           "Resource": "'$KINESIS_ARN'"
       },
       {
           "Sid": "Quickstart1",
           "Effect": "Allow",
           "Action": [
               "kinesis:ListShards",
               "kinesis:DescribeLimits"
           ],
           "Resource": "*"
       }
   ]
}' | jq -r .Policy.Arn)

if [ -z "$POLICY_ARN" ]; then
  echo "Failed to create IAM policy"
  exit 1
fi

# Generate a random external ID
EXTERNAL_ID=$(openssl rand -hex 40)
if [ -z "$EXTERNAL_ID" ]; then
  echo "Failed to generate external ID"
  exit 1
fi

# This is Twilio's intermediary role, which needs permission to assume the
# role created below (Twilio assumes your role via a two-hop chain, so the
# trust policy must name this specific role, not just Twilio's account)
TWILIO_ASSUME_ROLE_ACCOUNT=${TWILIO_ASSUME_ROLE_ACCOUNT:-arn:aws:iam::177261743968:role/egress-domain-prod-isolated}

# Add the random external ID to the role ARN
# More information can be found here: https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-user_externalid.html
ROLE_ARN=$(aws iam create-role --role-name twilio-events-kinesis-write --assume-role-policy-document '{
 "Version": "2012-10-17",
 "Statement": [
   {
     "Effect": "Allow",
     "Principal": {
       "AWS": "'$TWILIO_ASSUME_ROLE_ACCOUNT'"
     },
     "Action": "sts:AssumeRole",
     "Condition": {
       "StringEquals": {
         "sts:ExternalId": "'$EXTERNAL_ID'"
       }
     }
   }
 ]
}' | jq -r .Role.Arn)

if [ -z "$ROLE_ARN" ]; then
  echo "Failed to create IAM role"
  exit 1
fi

# Finally attach the policy and the role
aws iam attach-role-policy --role-name twilio-events-kinesis-write --policy-arn $POLICY_ARN

if [ $? -ne 0 ]; then
  echo "Attaching policy to role failed"
  exit 1
fi

# Print out the values needed for creating the sink in nice JSON
echo "{"
echo '"arn":"'$KINESIS_ARN'",'
echo '"role_arn":"'$ROLE_ARN'",'
echo '"external_id":"'$EXTERNAL_ID'"'
echo "}"
```

## Retreive a test message

You can use the following Bash script to automate the retreival of a test message. Copy the code below and save it to your computer, for example as `retrieve_test_message.sh`.

```bash
#!/bin/bash

JQ_CHECK=$(which jq)
if [ -z "$JQ_CHECK" ]; then
  echo
  echo "This script requires the jq JSON processor. Please install for your OS from https://jqlang.org/download/"
  echo
  exit 1
fi

if [ $# -ne 1 ]; then
  echo
  echo "usage: $0 <stream_name>"
  echo
  exit 1
fi

# Set the stream name
STREAM_NAME=$1

# Choose the iterator type:
# TRIM HORIZON is for starting at the beginning of the Kinesis Stream.
# This can take a while if you have a lot of records.
# To use TRIM HORIZON, uncomment the following line:
# TYPE=TRIM_HORIZON

# AT_TIMESTAMP allows you to go back to a point in time. This is set for going back one hour
# To use AT_TIMESTAMP, uncomment the following two lines:
# TIMESTAMP=$(($(date +%s) - 3600)).000
# TYPE="AT_TIMESTAMP --timestamp $TIMESTAMP"

# LATEST means start at the most current point in the stream and read forward
TYPE=LATEST

# Get a list of shards
SHARDS=$(aws kinesis list-shards --stream-name $STREAM_NAME | jq -r .Shards[].ShardId)

# Get all the starting points
SHARD_ITERATOR=()
i=0
for shard in $SHARDS ; do
  SHARD_ITERATOR[$i]=$(aws kinesis get-shard-iterator --shard-id $shard --shard-iterator-type $TYPE --stream-name $STREAM_NAME --query 'ShardIterator')
  i=$((i+1))
done

# Start getting events from all shards and display them
while [ 1 ] ; do
  len=${#SHARD_ITERATOR[@]}
  for (( i=0; i < $len; i++ )); do
    DATA=$(aws kinesis get-records --limit 50 --shard-iterator ${SHARD_ITERATOR[$i]})
    SHARD_ITERATOR[$i]=$(echo $DATA | jq -r .NextShardIterator)
    ROWS=$(echo $DATA | jq -r .Records[].Data?)
    for row in $ROWS; do
      echo $row | base64 -d | jq .
    done
  done
done
```