flow
A flow allows visualization of queries using types such as sankey
. Flows are node/edge visualizations. The data to be displayed is specified using a series of nodes and edges. The nodes define the vertices of the graph, and the edges define the connections between them.
Flow blocks can be declared as named resources at the top level of a mod, or they can be declared as anonymous blocks inside a dashboard
or container
, or be re-used inside a dashboard
or container
by using a flow
with base = <mod>.flow.<flow_resource_name>
.
Example Usage
dashboard "flow_sankey_ex_1" {input "vpc_input" {width = 4sql = <<-EOQselecttitle as label,vpc_id as valuefromaws_vpcEOQ}flow {title = "AWS VPC Subnets by AZ"node {sql = <<-EOQselectvpc_id as id,vpc_id as titlefromaws_vpcwherevpc_id = $1EOQargs = [self.input.vpc_input.value]}node {sql = <<-EOQselectdistinct on (availability_zone)availability_zone as id,availability_zone as titlefromaws_vpc_subnetwherevpc_id = $1EOQargs = [self.input.vpc_input.value]}node {sql = <<-EOQselectsubnet_id as id,subnet_id as titlefromaws_vpc_subnetwherevpc_id = $1EOQargs = [self.input.vpc_input.value]}edge {sql = <<-EOQselectdistinct on (availability_zone)vpc_id as from_id,availability_zone as to_idfromaws_vpc_subnetwherevpc_id = $1EOQargs = [self.input.vpc_input.value]}edge {sql = <<-EOQselectavailability_zone as from_id,subnet_id as to_idfromaws_vpc_subnetwherevpc_id = $1EOQargs = [self.input.vpc_input.value]}}}
Argument Reference
Argument | Type | Optional? | Description |
---|---|---|---|
args | Map | Optional | A map of arguments to pass to the query. |
base | flow Reference | Optional | A reference to a named flow resource that this flow should source its definition from. title and width can be overridden after sourcing via base . |
category | Block | Optional | category blocks that specify display options for nodes with that category. |
database | String | Optional | A database connection string for the database you wish to query. If not specified, the active database will be used. |
edge | Block | Optional | edge blocks that define the edges in the flow. |
node | Block | Optional | node blocks that define the nodes in the flow. |
param | Block | Optional | param blocks that defines the parameters that can be passed in to the query. param blocks may only be specified for hierarchies that specify the sql argument. |
query | Query Reference | Optional | A reference to a query resource that defines the query to run. A flow may either specify the query argument or the sql argument, but not both. |
sql | String | Optional | An SQL string to provide data for the flow . A flow may either specify the query argument or the sql argument, but not both. |
--search-path | String | Optional | Set a comma-separated list of connections to use as a custom search path for the query |
--search-path-prefix | String | Optional | Set a comma-separated list of connections to use as a prefix to the current search path for the query. |
title | String | Optional | A plain text title to display for this flow. |
type | String | Optional | The type of the flow. Can be sankey or table . |
width | Number | Optional | The width as a number of grid units that this item should consume from its parent. |
with | Block | Optional | with blocks that define prerequisite queries to run. with blocks may only be specified when the flow is defined as a top-level (mod level), named resource. |
Common Flow Properties
category
Property | Type | Default | Values | Description |
---|---|---|---|---|
color | string | The matching color from the default theme for the data series index. | A valid color value. This may be a named color, RGB or RGBA string, or a control status color. | The color to display for this category. |
Data Format
Flow data must be provided in a format where each row represents a node (vertex), an edge (connecting 2 vertices), or both.
Note that both column names and their relative position are important in flow queries; Powerpipe looks for columns by name in the result set, but Postgres union queries will append the rows based on the column's position, not the name of the column. All the union
queries must return the same columns, in the same order.
Significant columns are:
Name | Description |
---|---|
id | A unique identifier for the node. Nodes have an id , edges do not. |
title | A title to display for the node. |
category | A display category name. Both nodes and edges may specify a category to dictate how the item is displayed. By default, items of the same category are displayed with the same appearance (color), distinct from other categories. You can specify display options with a category block. |
depth | An integer to set the position of the node. The layout of the nodes is inferred from the query, however you can force placement with the depth column if you need to override the default behavior. |
from_id | The id of the source side of an edge. |
to_id | The id of the destination side of an edge. |
Generally speaking, there are 2 data formats commonly used for flows. If the data is hierarchical, it is often simpler to specify results where each row species a node (with an id
, and optionally title
, category
, and/or depth
) and an edge, by specifying a from_id
:
from_id | id | title | category |
---|---|---|---|
<null> | 1 | foo | root |
1 | 2 | bar | widget |
1 | 3 | baz | widget |
2 | 4 | foobar | fidget |
For flows that do not conform to a single-parent hierarchical structure, it's' usually easier to specify nodes and edges as separate rows. In this case, nodes will have an id
and optionally title
, category
, and/or depth
, but to_id
and from_id
will be null. Edges will populate to_id
and from_id
and optionally category
, and will have null id
, depth
, and title
:
from_id | to_id | id | title | category |
---|---|---|---|---|
<null> | <null> | 1 | foo | root |
<null> | <null> | 2 | bar | widget |
<null> | <null> | 3 | baz | widget |
<null> | <null> | 4 | foobar | fidget |
1 | 2 | <null> | <null> | widget |
1 | 3 | <null> | <null> | widget |
2 | 4 | <null> | <null> | fidget |
3 | 4 | <null> | <null> | fidget |
More Examples
Sankey with color by category
dashboard "flow_sankey_ex_category" {input "vpc_input" {width = 4sql = <<-EOQselecttitle as label,vpc_id as valuefromaws_vpcEOQ}flow {title = "AWS VPC Subnets by AZ"node {category = category.aws_vpcsql = <<-EOQselectvpc_id as id,vpc_id as titlefromaws_vpcwherevpc_id = $1EOQargs = [self.input.vpc_input.value]}node {category = category.aws_availability_zonesql = <<-EOQselectdistinct on (availability_zone)availability_zone as id,availability_zone as titlefromaws_vpc_subnetwherevpc_id = $1EOQargs = [self.input.vpc_input.value]}node {category = category.aws_vpc_subnetsql = <<-EOQselectsubnet_id as id,subnet_id as titlefromaws_vpc_subnetwherevpc_id = $1EOQargs = [self.input.vpc_input.value]}edge {sql = <<-EOQselectdistinct on (availability_zone)vpc_id as from_id,availability_zone as to_idfromaws_vpc_subnetwherevpc_id = $1EOQargs = [self.input.vpc_input.value]}edge {sql = <<-EOQselectavailability_zone as from_id,subnet_id as to_idfromaws_vpc_subnetwherevpc_id = $1EOQargs = [self.input.vpc_input.value]}}}category "aws_vpc" {color = "orange"}category "aws_availability_zone" {color = "tan"}category "aws_vpc_subnet" {color = "green"}
sankey with monolithic query
dashboard "flow_sankey_ex_1_mono" {flow {type = "sankey"title = "AWS VPC Subnets by AZ"sql = <<-EOQwith vpc as(select 'vpc-9d7ae1e7' as vpc_id)selectnull as from_id,vpc_id as id,vpc_id as title,0 as depth,'aws_vpc' as categoryfromaws_vpcwherevpc_id in (select vpc_id from vpc)union allselectdistinct on (availability_zone)vpc_id as from_id,availability_zone as id,availability_zone as title,1 as depth,'aws_availability_zone' as categoryfromaws_vpc_subnetwherevpc_id in (select vpc_id from vpc)union allselectavailability_zone as from_id,subnet_id as id,subnet_id as title,2 as depth,'aws_vpc_subnet' as categoryfromaws_vpc_subnetwherevpc_id in (select vpc_id from vpc)EOQ}}
Sankey with color by category (monolithic query)
dashboard "flow_sankey_ex_category_mono" {flow {type = "sankey"title = "AWS VPC Subnets by AZ"category "aws_vpc" {color = "orange"}category "aws_availability_zone" {color = "tan"}category "aws_vpc_subnet" {color = "green"}sql = <<-EOQwith vpc as(select 'vpc-9d7ae1e7' as vpc_id)selectnull as from_id,vpc_id as id,vpc_id as title,0 as depth,'aws_vpc' as categoryfromaws_vpcwherevpc_id in (select vpc_id from vpc)union allselectdistinct on (availability_zone)vpc_id as from_id,availability_zone as id,availability_zone as title,1 as depth,'aws_availability_zone' as categoryfromaws_vpc_subnetwherevpc_id in (select vpc_id from vpc)union allselectavailability_zone as from_id,subnet_id as id,subnet_id as title,2 as depth,'aws_vpc_subnet' as categoryfromaws_vpc_subnetwherevpc_id in (select vpc_id from vpc)EOQ}}
Sankey with node/edge data format, color by category, depth
dashboard "flow_sankey_ex_2" {flow {type = "sankey"title = "AWS IAM Managed Policies for User"category "direct" {color = "alert"}category "indirect" {color = "ok"}sql = <<-EOQwith user_list as(select 'arn:aws:iam::111111111111:user/jsmyth' as arn)-- User Nodesselectarn as id,name as title,0 as depth,'user' as category,null as from_id,null as to_idfromaws_iam_userwherearn in (select arn from user_list)-- Group Nodesunion selectg ->> 'Arn' as id,g ->> 'GroupName' as title,1 as depth,'group' as category,null as from_id,null as to_idfromaws_iam_user,jsonb_array_elements(groups) as gwherearn in (select arn from user_list)-- Policy Nodes (attached to groups)union selectp.arn as id,p.name as title,2 as depth,'policy' as category,null as from_id,null as to_idfromaws_iam_user as u,jsonb_array_elements(groups) as g,aws_iam_group as grp,jsonb_array_elements_text(grp.attached_policy_arns) as pol_arn,aws_iam_policy as pwhereg ->> 'Arn' = grp.arnand pol_arn = p.arnand u.arn in (select arn from user_list)-- Policy Nodes (attached to user)union selectp.arn as id,p.name as title,2 as depth,'policy' as category,null as from_id,null as to_idfromaws_iam_user as u,jsonb_array_elements_text(attached_policy_arns) as pol_arn,aws_iam_policy as pwherepol_arn = p.arnand u.arn in (select arn from user_list)-- User-> Group Edgeunion selectnull as id,null as title,null as depth,'indirect' as category,arn as from_id,g ->> 'Arn' as to_idfromaws_iam_user,jsonb_array_elements(groups) as gwherearn in (select arn from user_list)-- User -> Policy Edgeunion selectnull as id,null as title,null as depth,'direct' as category,arn as from_id,pol_arnfromaws_iam_user,jsonb_array_elements_text(attached_policy_arns) as pol_arnwherearn in (select arn from user_list)-- Group -> Policy Edgeunion selectnull as id,null as title,null as depth,'indirect' as category,grp.arn as from_id,pol_arnfromaws_iam_user as u,jsonb_array_elements(groups) as g,aws_iam_group as grp,jsonb_array_elements_text(grp.attached_policy_arns) as pol_arn,aws_iam_policy as pwhereg ->> 'Arn' = grp.arnand pol_arn = p.arnand u.arn in (select arn from user_list)EOQ}}