Powerpipe: Dashboards for DevOps

Introducing Powerpipe

Powerpipe is an open-source tool from Turbot that supports DevOps with:

Benchmarks - 5,000+ open-source controls from CIS, NIST, PCI, HIPAA, FedRamp and more. Run instantly on your machine or as part of your deployment pipeline.

Relationship Diagrams - The only dashboarding tool designed from the ground up to visualize DevOps data. Explore your cloud, understand relationships, drill down to the details.

Dashboards & Reports - High-level dashboards provide a quick overview. Use them to highlight misconfigurations and hotspots. Filter, pivot, and snapshot results.

Code, not clicks - Our dashboards are code: version-controlled, composable, shareable, easy to edit — designed for the way you work. Join our open-source community!

About Powerpipe dashboards and benchmarks

Powerpipe supports an ecosystem of dashboards and benchmarks. These open-source components use HCL to define and compose elements that include tables, charts, relationship graphs, benchmarks, and controls. And they use SQL to query for the data that animates them, typically by way of a parallel ecosystem of plugins that translate cloud APIs to database tables.

For example, the AWS Insights mod provides 135+ overview and detail dashboards for AWS services: EBS, ECS, EC2, IAM, and more.

Use the Powerpipe server to view dashboards

Likewise the AWS Compliance mod provides 1100+ controls for Audit Manager Control Tower, AWS Foundational Security Best Practices, CIS, and many more.

Powerpipe does not require Steampipe, but mods like AWS Insights rely on it. So to run the AWS Insights dashboards:

• Install Powerpipe
• Install Steampipe
• Install the AWS plugin for Steampipe

Then, in a fresh directory:

• powerpipe mod init (Powerpipe always runs in the context of a mod.)

• powerpipe mod install github.com/turbot/steampipe-mod-aws-insights (There are commands for mod management.)

• steampipe service start (Give Powerpipe access to Steampipe plugins.)

• powerpipe server (Many commands run client-only but dashboards use a server.)

• browse to http://localhost:9033

Dashboards are grouped by AWS service, here's the overview for EC2 instances.

And here's the detail view for an instance, featuring a relationship graph that shows what it mounts, runs as, and connects to.

Although these dashboards use Steampipe plugins, you can also connect Powerpipe to Postgres, SQLite, MySQL, or DuckDB. This mod, for example, visualizes biomedical data in a hosted instance of Postgres.

Use the Powerpipe server to run benchmarks

To run the AWS Compliance benchmarks, follow the same recipe but use https://hub.powerpipe.io/mods/turbot/aws_compliance. These dashboards are grouped by benchmark, here's the CIS v300 benchmark.

You can adjust how results are filtered and grouped. Here we filter to just section 1, and group by account_id.

Use the Powerpipe CLI to run benchmarks

You can alternatively run that same benchmark, from the same folder. List the benchmarks.

powerpipe benchmark list

Run a benchmark.

powerpipe benchmark run cis_v300

The documentation covers a range of options including --export to save results in asff, csv, and other file formats, and --snapshot to create a snapshot in Turbot Pipes.

Compose and create dashboards and benchmarks

The thousands of pre-built dashboards and benchmarks on the hub are all available for reuse and remixing. You can use mod management commands to create your own mod, install a local copy of one or more published mods, rearrange the resources they provide, and add your dashboards or benchmarks. Or you can create your own dashboards and benchmarks from scratch. In either case, you use HCL to define these elements, and SQL to fill them with data.

Here's a simple dashboard to chart Turbot blog posts by month. It uses the RSS plugin to aggregate posts and the chart resource to visualize the data.

dashboard "blog_posts" {

  title = "Blog posts"

  chart {
    type = "column"
    title = "posts by month"
    sql = <<EOQ
      select
        to_char(published, 'YYYY-MM') as published,
        count(*)
      from
        rss_item
      where
        feed_link = 'https://turbot.com/blog/feed.xml'
     group by published
     order by published
    EOQ
    }

}

To replicate this example: install Powerpipe and Steampipe as above, install the RSS plugin (steampipe plugin install rss), start the Steampipe service, save this HCL code in a file (e.g. dashboard.pp), run powerpipe server, and visit http://localhost:9093. Note that Powerpipe watches its source files and reacts to edits of HCL and/or SQL. For example, in an auto-saving editor, try changing the chart type from column to bar. Then change the aggregation from YYYY-MM to YYYY. You'll see the chart reflect both changes in realtime.

And here's a benchmark that defines a single control to ensure that each post has at least one category.

benchmark  "rss" {
  title    = "RSS compliance"
  children = [
    control.has_categories
  ]
}

control "has_categories" {
  title = "Ensure each item has one or more categories."
  sql = <<EOT
    select
      title as resource,
      case
        when jsonb_array_length(categories) > 0 then 'ok'
        else 'alarm'
      end as status,
      case
        when jsonb_array_length(categories) > 0 then 'Has ' || jsonb_array_length(categories) || ' categories'
        else 'Must have at least one category'
      end as reason,
      title,
      categories::text
    from
      rss_item
    where
      feed_link = 'https://turbot.com/blog/feed.xml'
    EOT
}

A benchmark lists one or more child controls. Each control runs a query that reports the required columns status, reason, and resource, along with optional dimensions — in this case, title and categories.

See it in action

Data visualizations and compliance controls as code

Powerpipe is a flexible open-source platform to visualize and monitor your cloud infrastructure, ensure security compliance, and even build your own custom dashboards and benchmarks. With pre-built modules connecting to AWS, Azure, GCP, and more, along with easy ways to create and share your own mods, Powerpipe is an essential GitOps-friendly tool for DevOps teams wrangling cloud complexity. To get started, download the tool, follow the tutorial, explore the mods, then let us know how it goes!