Managing Indexes And Streams Of DynamoDB

Adding Indexes

• As someone new to DynamoDB, secondary indexes were tricky for me to grasp at first. Coming from a relational database background, I was used to just creating tables with different columns and joining them.
• With DynamoDB’s NoSQL model, you can only query or scan the main table by its primary key attributes. This is great for speed, but limiting.
• Adding global secondary indexes in DynamoDB opens up a lot more flexibility in how you query data. An index lets you organize data by alternative attributes besides the main table key. For example, in my gaming scorekeeping app, my main DynamoDB table uses “UserId” as the primary hash key to look up scores for each player.
• But I also wanted to be able to look up high scores by game title across all players. By adding a global secondary index on the “GameTitle” attribute, I can now query scores organized by game in addition to player. This kind of flexibility with indexing is powerful for modeling complex data relationships in a fast NoSQL paradigm like DynamoDB. Terraform makes it simple to add and manage indexes programmatically.
• DynamoDB supports global secondary indexes to enable alternative query paths. These can be added in Terraform:

resource "aws_dynamodb_table" "table" {
hash_key = "UserId"
index {
name = "GameTitleIndex"
hash_key = "GameTitle"
}
}

• This creates a GameTitleIndex to look up items by GameTitle instead of the main UserId key. Multiple indexes can be added.

Enabling Streams

When I first started with DynamoDB, I just viewed it as a fast NoSQL data store for my applications. But enabling DynamoDB streams opened my eyes to the possibilities of integrating it with other services. Streams provide a time-ordered log of item changes in a table. When enabled on a table, any creates, updates or deletes get captured and sent to a stream.

I can tap into this stream for some powerful workflows:

• Send data to Elasticsearch clusters to enable faster searching of table data.
• Push updates to downstream Lambda functions to trigger additional workflows.
• Sync a backup copy of the table to another region for disaster recovery.
• Analyze change patterns for usage metrics and insights.

The stream Integration patterns are endless!

My favorite use case so far is syncing my critical production tables to a different region every hour using streams and Lambda. This gives me peace of mind that my data is durable. Streams turn DynamoDB into a hub that can integrate with many different AWS services. The stream view options let you tune what data to send downstream. I’m excited to enable streams on more tables and build out sophisticated architectures around them. The ease of use in Terraform is just icing on the cake!

DynamoDB streams can track data changes and send to other services. Turn on streams with:

resource "aws_dynamodb_table" "table" {
stream_enabled = true
stream_view_type = "NEW_IMAGE"
}

This captures table changes and sends new item data.

I’m going to show how to use Terraform to create a DynamoDB table in AWS. Terraform lets you define infrastructure like databases as code. This makes it easy to version control and share with others. In this article, I’ll walk through the steps to set up a Terraform file and define a DynamoDB table in it. Then I’ll apply the plan to create the real table in AWS. Following along will show you a hands-on example of using Terraform to manage infrastructure as code. The end result will be a DynamoDB table defined in a Terraform config that can be reused and shared.