Using Powerpipe with Tailpipe

Powerpipe is the engine for visualizing Tailpipe detections. Let's see how that works.

Prerequisites

To get started, you will need to install Powerpipe, Tailpipe, and the AWS plugin for Tailpipe.

Powerpipe does not require Tailpipe; it can work with any PostgreSQL, MySQL, SQLite, or DuckDB database. Mods are written for a specific SQL syntax and database schema, however, and the examples in this article require a Tailpipe database with the AWS plugin.

First, install Powerpipe.

brew install turbot/tap/powerpipe

sudo /bin/sh -c "$(curl -fsSL https://powerpipe.io/install/powerpipe.sh)"

Next, install Tailpipe.

brew install turbot/tap/tailpipe

sudo /bin/sh -c "$(curl -fsSL https://tailpipe.io/install/tailpipe.sh)"

Now that Tailpipe is installed, install the AWS plugin for Tailpipe

tailpipe plugin install aws

Out of the box, Tailpipe will use the default AWS credentials from your credential file and/or environment variables; if you can run aws ec2 describe-vpcs, for example, then you should be able to run the examples.

The AWS plugin documentation provides additional examples to configure your credentials, and you can even configure Tailpipe to query multiple accounts and multiple regions.

Collect log data

Powerpipe mods for Tailpipe work with tables built from log data collected by Tailpipe. The Tailpipe docs show you how to configure the AWS plugin for Tailpipe and then collect log data. Follow those steps create the table aws_cloudtrail_log, and verify that you can run the sample queries shown there.

Run a benchmark

Powerpipe benchmarks provide a mechanism for defining and running log detections to evaluate threat and error patterns, system performance, and user behavior. Benchmarks are written in simple HCL, and packaged in mods. It is simple to create your own, but there are also many benchmarks available on the Powerpipe Hub.

Powerpipe always runs in the context of a mod. A Powerpipe mod is a portable, versioned collection of related Powerpipe resources (dashboards, benchmarks, detections) defined in HCL and distributed as simple text files. Powerpipe loads the mod from the mod location which defaults to the current directory.

Let's create a new directory for our mod:

mkdir learn_powerpipe_tailpipe
cd learn_powerpipe_tailpipe

Now install the Tailpipe AWS Detections mod.

powerpipe mod install github.com/turbot/tailpipe-mod-aws-detections

Start the server:

powerpipe server

Visit localhost:9033 in a browser.

The Tailpipe mod has detected 14 potential issues, of which 5 are detections related to updates to KMS keys. If you know that ExampleUser is non-malicious you can exclude those 5 rows with a single click on any row in the actor column that matches ExampleUser.

Then you can continue to explore the remaining potential issues in other detection categories.

Create your own benchmarks, detections, and dashboards

The Powerpipe Hub contains ready-made benchmarks with sets of detections that you can simply install and run. But Powerpipe also makes it simple to write your detections and benchmarks, and build dashboards to analyze your log data and share with others!