graph
A graph is a dashboard component that allows you to visualize the relationships between different resources and understand how they are connected.
Graphs can be declared as a block inside a dashboard or container. The data to be displayed in the graph is specified using a series of nodes and edges. The nodes define the vertices of the graph, and the edges define the connections between them.
Example Usage
dashboard "graph_ex1" {graph {title = "Schemas & Tables"node {category = category.catalogsql = <<-EOQselectdistinct on (catalog_name)concat('catalog:',catalog_name) as id,catalog_name as titlefrominformation_schema.schematawhereschema_name = 'net'EOQ}node {category = category.schemasql = <<-EOQselectconcat('schema:',schema_name) as id,schema_name as title,json_build_object('catalog', catalog_name,'schema', schema_name,'owner', schema_owner) as propertiesfrominformation_schema.schematawhereschema_name = 'net'EOQ}node {category = category.tablesql = <<-EOQselectconcat('table:',table_name) as id,table_name as title,json_build_object('catalog', table_catalog,'schema', table_schema,'type', table_type) as propertiesfrominformation_schema.tableswheretable_schema = 'net'EOQ}edge {sql = <<-EOQselectconcat('catalog:',catalog_name) as from_id,concat('schema:',schema_name) as to_idfrominformation_schema.schemataEOQ}edge {sql = <<-EOQselectconcat('schema:',table_schema) as from_id,concat('table:',table_name) as to_idfrominformation_schema.tablesEOQ}}}category "catalog" {title = "catalog"icon = "library_books"color = "orange"}category "schema" {title = "schema"icon = "schema"color = "blue"}category "table" {title = "table"icon = "table"color = "green"}
Argument Reference
Argument | Type | Optional? | Description |
---|---|---|---|
args | Map | Optional | A map of arguments to pass to the query. |
base | Graph Reference | Optional | A reference to a named graph resource that this graph should source its definition from. |
category | Block | Optional | category blocks that specify display options for nodes and edges with that category. |
database | String | Optional | A database connection string for the database you wish to query. If not specified, the active database will be used. |
direction | String | Optional | The direction of the graph layout. Valid options are left_right and top_down . The default is top_down . |
edge | Block | Optional | edge blocks that define the edges in the graph. |
node | Block | Optional | node blocks that define the nodes in the graph. |
param | Block | Optional | A param block that defines the parameters that can be passed in to the graph. You can only specify param blocks when the graph is defined as a top-level, named resource. |
query | Query Reference | Optional | A reference to a query resource that defines the query to run. A graph may either specify the query argument or the sql argument, but not both. |
--search-path | String | Optional | Set a comma-separated list of connections to use as a custom search path for the query |
--search-path-prefix | String | Optional | Set a comma-separated list of connections to use as a prefix to the current search path for the query. |
sql | String | Optional | A SQL string to provide data for the graph. A graph may either specify the query argument or the sql argument, but not both. |
title | String | Optional | The title to display above the graph. |
type | String | Optional | The type of graph to display. Currently, only graph is supported. The default is graph . |
width | Number | Optional | The width as a number of grid units that this item should consume from its parent. |
with | Block | Optional | with blocks that define prerequisite queries to run. with blocks may only be specified when the graph is defined as a top-level (mod level), named resource. |
Node/Edge Visualizations
Some Powerpipe dashboard elements can include node
and edge
blocks. These elements are sometimes referred to as node/edge visualizations and include graph
, flow
, and hierarchy
. These resources essentially implement the same interface:
- They support
node
andedge
blocks as children - They are also query-based resources, and support using the
sql
andquery
arguments instead ofnode
andedge
blocks - They must appear in a dashboard to be displayed, but they may be defined as top-level resources and referenced with
base
- They support
param
andwith
blocks, but only when in a top-level resource - They support
category
blocks - They have similar data formats
Monolithic Query v/s Node & Edge Blocks
Node/edge visualizations allow you to specify a monolithic query that returns a row for each node and edge, or you can specify node
and edge
blocks to define the nodes and edges separately.
In either case, the concept of nodes and edges is the same. Nodes and edges represent points and connections in a graph
, hierarchy
or flow
. A node is a vertex in the diagram, whereas an edge is a relationship between 2 nodes (usually represented by a line connecting them).
Key differences between nodes and edges are:
- A
node
MUST have anid
. Anedge
CANNOT have anid
. - An
edge
must have afrom_id
and ato_id
. Anode
CANNOT have ato_id
. Nodes USUALLY do not have afrom_id
either, but for simple single-parent hierarchies, it is often simpler to create a simple edge by specifyingfrom_id
on thenode
instead of creating separate node and edge blocks/rows.
Node & Edge Blocks
Typically, it is preferable to specify node
and edge
blocks than to use the monolithic query format:
- Using
node
andedge
results in a simpler, more readable, maintainable configuration. - Developing large union queries is difficult. Errors are hard to find and fix. The
node
andedge
model provides smaller, simpler queries that can be run and tested independently. - You can reuse nodes and edges in multiple node/edge visualizations.
- Powerpipe can run the node/edge queries in parallel and can provide better status information while the visualization is loading.
In the node/edge model, your graph
, flow
, or hierarchy
block will not specify the sql
or query
argument but instead will contain one or more node
blocks and edge
blocks. The node
and edge
blocks will specify the sql
or query
argument to retrieve the data for the node or edge.
The SQL column names are identical to the monolithic query format. Note that some fields (category
, title
) may be specified in HCL or in the query results. When both are specified, the SQL value takes precedence.
Example: Graph with node/edge blocks
dashboard "plugin_versions" {graph {title = "LDAP Plugin Versions"node {category = category.pluginsql = <<-EOQselectname as id,name as title,jsonb_build_object('name', name,'created', create_time,'updated', update_time) as propertiesfromsteampipe_registry_pluginwherename = 'turbot/ldap'EOQ}node {category = category.plugin_versionsql = <<-EOQselectdigest as id,left(split_part(digest,':',2),12) as title,jsonb_build_object('digest', digest,'created', create_time,'updated', update_time) as propertiesfromsteampipe_registry_plugin_versionwherename = 'turbot/ldap'EOQ}node {category = category.plugin_tagsql = <<-EOQselectconcat(digest,':',tag) as id,tag as titlefromsteampipe_registry_plugin_version,jsonb_array_elements(tags) as tagwherename = 'turbot/ldap'EOQ}edge {title = "version"sql = <<-EOQselectname as from_id,digest as to_idfromsteampipe_registry_plugin_versionwherename = 'turbot/ldap'EOQ}edge {title = "tag"sql = <<-EOQselectdigest as from_id,concat(digest,':',tag) as to_idfromsteampipe_registry_plugin_version,jsonb_array_elements(tags) as tagwherename = 'turbot/ldap'EOQ}}}category "plugin" {title = "plugin"icon = "extension"color = "darkred"}category "plugin_version" {title = "version"icon = "difference"color = "darkred"}category "plugin_tag" {title = "tag"icon = "sell"color = "black"}
Including the full node and edge block definitions within the graph can become unwieldy as the graph becomes more complex. It is often preferable to define the node and edge blocks as top-level resources and include them in the graph with the base
argument. This allows you to reuse the nodes and edges in multiple node/edge visualizations. You can even define parameters in the nodes and edges, and pass arguments from the graph.
dashboard "plugin_versions_example" {input "plugin_name" {query = query.plugin_inputwidth = 4}graph {title = "Plugin Versions"node {base = node.pluginargs = {plugin_name = self.input.plugin_name.value}}node {base = node.plugin_versionargs = {plugin_name = self.input.plugin_name.value}}node {base = node.plugin_tagargs = {plugin_name = self.input.plugin_name.value}}edge {base = edge.plugin_to_versionargs = {plugin_name = self.input.plugin_name.value}}edge {base = edge.version_to_tagargs = {plugin_name = self.input.plugin_name.value}}}}query "plugin_input" {sql = <<-EOQselectname as value,name as labelfromsteampipe_registry_pluginEOQ}node "plugin"{category = category.pluginsql = <<-EOQselectname as id,name as title,json_build_object('name', name,'created', create_time,'updated', update_time) as propertiesfromsteampipe_registry_pluginwherename = $1EOQparam "plugin_name" {}}node "plugin_version" {category = category.plugin_versionsql = <<-EOQselectdigest as id,left(split_part(digest,':',2),12) as title,json_build_object('digest', digest,'created', create_time,'updated', update_time) as propertiesfromsteampipe_registry_plugin_versionwherename = $1EOQparam "plugin_name" {}}node "plugin_tag" {category = category.plugin_tagsql = <<-EOQselectconcat(digest,':',tag) as id,tag as titlefromsteampipe_registry_plugin_version,jsonb_array_elements(tags) as tagwherename = $1EOQparam "plugin_name" {}}edge "plugin_to_version" {title = "version"sql = <<-EOQselectname as from_id,digest as to_idfromsteampipe_registry_plugin_versionwherename = $1EOQparam "plugin_name" {}}edge "version_to_tag" {title = "tag"sql = <<-EOQselectdigest as from_id,concat(digest,':',tag) as to_idfromsteampipe_registry_plugin_version,jsonb_array_elements(tags) as tagwherename = $1EOQparam "plugin_name" {}}category "plugin" {title = "plugin"icon = "extension"color = "darkred"}category "plugin_version" {title = "version"icon = "difference"color = "darkred"}category "plugin_tag" {title = "tag"icon = "sell"color = "black"}
Monolithic query
Node/edge visualizations are also query-based resources, and support using either the sql
or query
argument (but not both). When using a monolithic query, the query must return a row for each node and edge. Note that using a single query is an older format; it is usually simpler to use node
and edge
blocks instead.
Significant columns are:
Name | Applies To | Description |
---|---|---|
id | node | A unique identifier for the node. Nodes have an id , edges do not. id is required for nodes. |
title | node, edge | A title to display for the node. |
category | node, edge | A display category. This can be a category block or a reference to a named category . |
depth | node (flow only) | An integer to set the position of the node in a flow. The layout of the nodes is inferred from the query, however you can force placement with the depth column if you need to override the default behavior. |
from_id | node, edge | The id of the source side of an edge. from_id is required for edges, optional for nodes. |
properties | node, edge (graph only) | A jsonb key/value map of properties to display for the node/edge when the user hovers over it. The properties column is optional for nodes and edges. |
to_id | edge | The id of the destination side of an edge. to_id is required for edges. |
Typically, the monolithic query will be a large union
query. Note that both column names and their relative position are important! Powerpipe looks for columns by name in the result set, but Postgres union
queries will append the rows based on the column's position, not the name of the column. All the union
queries must return the same columns, in the same order.
Most commonly, you should specify nodes and edges as separate rows. In this case, nodes will have an id
and optionally title
, category
, and/or depth
, but to_id
and from_id
will be null. Edges will populate to_id
and from_id
and optionally category
, and will have null id
, depth
, and title
:
from_id | to_id | id | title | category |
---|---|---|---|---|
<null> | <null> | 1 | foo | root |
<null> | <null> | 2 | bar | widget |
<null> | <null> | 3 | baz | widget |
<null> | <null> | 4 | foobar | fidget |
1 | 2 | <null> | <null> | widget |
1 | 3 | <null> | <null> | widget |
2 | 4 | <null> | <null> | fidget |
3 | 4 | <null> | <null> | fidget |
If the data is strictly hierarchical (where each node can only have a single parent, as for a hierarchy
), it may be simpler to format the results such that each row species a node (with an id
, and optionally title
, category
, and/or depth
) and an edge, by specifying a from_id
:
from_id | id | title | category |
---|---|---|---|
<null> | 1 | foo | root |
1 | 2 | bar | widget |
1 | 3 | baz | widget |
2 | 4 | foobar | fidget |
Example: Graph with monolithic query
dashboard "plugin_versions_mono" {graph {title = "LDAP Plugin Versions"category "plugin" {title = "plugin"icon = "extension"color = "darkred"}category "plugin_version" {title = "version"icon = "difference"color = "darkred"}category "plugin_tag" {title = "tag"icon = "sell"color = "black"}sql = <<-EOQ-- plugin nodesselectname as id,null as from_id,null as to_id,name as title,'plugin' as category,jsonb_build_object('name', name,'created', create_time,'updated', update_time) as propertiesfromsteampipe_registry_pluginwherename = 'turbot/ldap'-- plugin version nodesunion allselectdigest as id,null as from_id,null as to_id,left(split_part(digest,':',2),12) as title,'plugin_version' as category,jsonb_build_object('digest', digest,'created', create_time,'updated', update_time) as propertiesfromsteampipe_registry_plugin_versionwherename = 'turbot/ldap'-- plugin tag nodesunion allselectconcat(digest,':',tag) as id,null as from_id,null as to_id,tag as title,'plugin_tag' as category,null as propertiesfromsteampipe_registry_plugin_version,jsonb_array_elements_text(tags) as tagwherename = 'turbot/ldap'-- plugin version edgesunion allselectnull as id,name as from_id,digest as to_id,'version' as title,null as category,null as propertiesfromsteampipe_registry_plugin_versionwherename = 'turbot/ldap'-- plugin tag edgesunion allselectnull as id,digest as from_id,concat(digest,':',tag) as to_id,'tag' as title,null as category,null as propertiesfromsteampipe_registry_plugin_version,jsonb_array_elements_text(tags) as tagwherename = 'turbot/ldap'EOQ}}
Categories
Node/Edge visualizations allow you to specify a category for each node and edge. Categories are used to define display properties such as color, title, and icon to provide a consistent look and feel across panels and dashboards.
Categories may be defined either at the mod level or in a graph
, flow
, or hierarchy
. When using node
and edge
blocks, it is typically preferable to define the categories as top-level, named mod resources. This allows you to reference them via the category
HCL argument in a node or edge. When specifying a category
column in SQL, such as when using the monolithic query approach, you will need to define the category in the graph, flow, or hierarchy.
With blocks
Node/Edge visualizations support with
blocks. Similar to a with
clause in a PostgreSQL CTE, the with
block allows you to specify additional queries or SQL statements to run first, and then pass the query results as arguments to sql
, query
, and node
& edge
blocks.
You can only specify with
blocks on top-level named resources in your mod. The results of the with
query can be referenced only within the resource in which it is defined (including any sub-blocks).