Skip to main content

Querying data in Infrahub

We can query data in 3 ways using the SDK:

  • querying a single node of a given kind, based on some filters, using the get method
  • querying multiple nodes of a given kind, based on some filters, using the filters method
  • querying all the nodes of a given kind, using the all method
  • querying with a GraphQL query, using the execute_graphql method

Query filters​

The get and filters query methods allow you to use filters. Filters specify on which attribute(s) or attributes of a relationship(s) the resulting node(s) should match. The available filters depend on the kind of Node you want to query and are dynamically generated from the schema.

Discovering available filters​

The easiest way to discover the available filters for a kind of node is by opening the GraphQLi Explorer in the GraphQL sandbox. Under each GraphQL query you will find the available filters.

Attribute filters​

For every attribute the following filters will be generated (replace attribute with the name of the attribute):

  • attribute__value: filter for a single attribute value
  • attribute__values: (list) filter for multiple attribute values
  • attribute__is_visible: (boolean) filter for the is_visible property of an attribute
  • attribute__is_protected: (boolean) filter for the is_protected property of an attribute
  • attribute__source__id: filter for the source property of an attribute
  • attribute__owner__id: filter for the owner property of an attribute

Relationship filters​

For every attribute of a relationship of the node, the same filters will be generated. The name of the filter will be prefixed with the relationship name followed by 2 underscores relationship__attribute__value.

Using filters​

Filters can be passed as arguments to the get or filters method.

tag = await client.get(kind="BuiltinTag", name__value="RED")

Using multiple filters​

When you pass multiple filters as argument to the get or filters method, they will combined in a logical AND operation. The resulting nodes of your query will match on all the filters in that case.

tags = await client.filters(kind="BuiltinTag", name__values=["RED", "BLUE"], name__is_protected=True)

Querying a single node​

You can query Infrahub for a single node of a particular kind, by using the get method and using 1 or more filters.

tag = await client.get(kind="BuiltinTag", name__value="RED")

Querying multiple nodes​

You can query Infrahub for multiple nodes of a particular kind, by using the filters method and using 1 or more filters.

tags = client.filters(kind="BuiltinTag", name__values=["RED", "BLUE"])

Querying all nodes​

You can query Infrahub for all nodes of a particular kind, by using the all method.

tags = await client.all(kind="BuiltinTag")

Querying with a GraphQL query​

In some scenarios it might be more convenient to query Infrahub using a GraphQL query, rather than using the builtin mechanisms in the SDK.

An example might be finding all the devices connected to a given circuit. This would require us to execute multiple queries using the nodes provided by the SDK. However with a GraphQL query this can be achieved using only a single query.

The downside of using this method, is that it will not construct Python objects for the resulting data. Instead the SDK will return a Python dictionary containing the deserialized JSON data returned by the GraphQL API.

query = """query {
  BuiltinTag(name__values: ["RED", "BLUE"]) {
    edges {
      node {
        name {
          value
        }
      }
    }
  }
}"""
data = await client.execute_graphql(query=query)
for tag in data["BuiltinTag"]["edges"]:
    print(tag["node"]["name"]["value"])

Attributes and relationships​

By default, the result of a query will include attributes, relationships of cardinality one and relationships of kind Attribute or Parent. Relationships that are included in a query will be automatically initialized which means the ID, type and display name of the peers will be included in the query. But the related node itself will not be included.

To explore this in a bit more details, we are going to assume the following schema has been loaded into Infrahub

---
version: "1.0"
nodes:
  - name: Device
    namespace: Test
    attributes:
      - name: name
        kind: Text
        unique: true
        optional: false
    relationships:
      - name: tags
        cardinality: many
        kind: Attribute
        peer: BuiltinTag
        optional: true
      - name: site
        cardinality: one
        kind: Attribute
        peer: TestSite
        optional: false
      - name: interfaces
        cardinality: many
        kind: Component
        peer: TestInterface
        optional: false
  - name: Site
    namespace: Test
    attributes:
      - name: name
        kind: Text
        unique: true
        optional: false
    relationships:
      - name: device
        cardinality: many
        kind: Attribute
        peer: TestDevice
        optional: true
  - name: Interface
    namespace: Test
    human_friendly_id: ["device__name__value", "name__value"]
    attributes:
      - name: name
        kind: Text
        unique: true
        optional: false
    relationships:
      - name: device
        cardinality: one
        kind: Parent
        peer: TestDevice
        optional: false

Attributes​

Attributes are included by default.

device = await client.get(kind="BuiltinTag", name__value="atl1-edge1")
print(device.name.value)

Relationships of cardinality one​

Relationships of cardinality one are included by default and will be initialized.

device = await client.get(kind="TestDevice", name__value="atl1-edge1")
print(device.site.initialized) # True
print(device.site.id, device.site.display_label, device.site.typename)
print(device.site.peer) # the related node is not included

Relationships of kind Attribute, Parent are included by default and will be initialized.

device = await client.get(kind="TestDevice", name__value="atl1-edge1")
print(device.tags.initialized) # True
print(device.tags.peers)

for tag in device.tags.peers:
    print(tag.id, tag.display_label, tag.typename)

Relationships of cardinality many are not included by default​

Relationships of cardinality many are not included by default.

device = await client.get(kind="TestDevice", name__value="atl1-edge1")
print(device.interfaces.initialized) # False
print(device.interfaces.peers) # empty list []

Including attributes and relationships​

You can include attributes and relationships that are not retrieved as part of a query by default. The included relationships will be initialized and the related nodes (peers) will be initialized.

device = await client.get(kind="TestDevice", name__value="atl1-edge1", include=["interfaces"])
print(device.interfaces.initialized) # True
print(device.interfaces.peers)

for interface in device.interfaces.peers:
    print(interface.id, interface.display_label, interface.typename)

Excluding attribute and relationships​

You can exclude attributes and relationships that are retrieved with a query by default. This can be useful if you need to optimize or speed up a particular query.

device = await client.get(kind="TestDevice", exclude=["site"])
print(device.site) # None

Fetching only the named attributes and relationships​

Pass only to get, filters or all to request the attributes and relationships you name and no others. Each node also gets its id, display_label and __typename. The query that the SDK sends names only those fields, so the response does not contain the other attributes and relationships of the kind.

devices = await client.all(kind="TestDevice", only=["name", "site"])
for device in devices:
    print(device.name.value, device.site.display_label)

The names in only are the names of the attributes and relationships in the schema of the kind, plus hfid and the hierarchy fields described below. With only=[], the query requests the id, display_label and __typename of each node and nothing else. only accepts id and display_label, but they add nothing, because every node gets them.

property=True and include_metadata=True work with only. property=True adds the properties of the named fields, and include_metadata=True adds the metadata of the node and of the named fields. only cannot be combined with include or exclude.

Reading a field that was not fetched​

A node returned by a query with only contains only the fields that the query fetched. Reading any other field raises FieldNotLoadedError, so a field that was not fetched is never read as an empty value. To know whether a node contains a field, check is_loaded on the attribute or the relationship. is_loaded never raises.

from infrahub_sdk.exceptions import FieldNotLoadedError

device = await client.get(kind="TestDevice", name__value="atl1-edge1", only=["site"])
print(device.site.id)
print(device.name.is_loaded) # False

try:
    print(device.name.value)
except FieldNotLoadedError as exc:
    print(exc)

The message names the kind, the field and the only list:

TestDevice.name was not fetched: the node was queried with only=['site']. Add 'name' to only, or check is_loaded before reading it.

These reads raise FieldNotLoadedError when the field was not fetched:

  • on an attribute: value;
  • on a relationship of cardinality one: id, hfid, hfid_str, display_label, typename, kind, peer, get() and fetch();
  • on a relationship of cardinality many: peers, peer_ids, peer_hfids, peer_hfids_str, is_from_profile, indexing and iteration.

Node methods that read these fields raise in the same cases, for example get_path_value and get_flat_value. On a file node, matches_local_checksum and upload_if_changed raise when the query did not fetch checksum, and so does download_file(skip_if_unchanged=True) when dest is an existing file.

The SDK raises FieldNotLoadedError only on nodes returned by a query with only, on the peers returned with them, and on peers fetched with fetch(only=...). On any other node, these reads behave as before, even when is_loaded is False, for example for an attribute that a query with exclude did not fetch.

Saving a node from a query with only​

You can change and save a node returned by a query with only. save() sends the node's id and the fields you changed, so the fields that the query did not fetch keep their values in Infrahub. Reading node.hfid and adding the node to the store do not raise either.

device = await client.get(kind="TestDevice", name__value="atl1-edge1", only=["name"])
device.name.value = "atl1-edge01"
await device.save()

Relationships named in only​

A relationship named in only is returned as a reference to its peers: their id, hfid, display_label and __typename. A relationship that you do not name is not requested, whatever its kind or cardinality.

With prefetch_relationships=True, the SDK also requests the default fields of the peers of the named relationships, builds peer nodes from them and adds the peer nodes to the store. The peer nodes follow the same rule as the node: reading a field that they do not contain raises FieldNotLoadedError.

device = await client.get(
    kind="TestDevice", name__value="atl1-edge1", only=["name", "site"], prefetch_relationships=True
)
print(device.site.peer.name.value)

Human-friendly IDs of nodes from a query with only​

Name hfid in only to fetch the human-friendly ID that Infrahub stores for each node. node.hfid, node.hfid_str, get_human_friendly_id() and get_human_friendly_id_as_string() return that value, and the SDK adds the node to the store under it. If you change a field that the human-friendly ID is built from, the SDK stops using the stored value and calculates the human-friendly ID from the node's fields, which gives None when one of its parts was not fetched. Naming hfid does not fetch the attributes and relationships that make up the human-friendly ID: reading them raises FieldNotLoadedError unless you name them too.

interfaces = await client.filters(kind="TestInterface", device__name__value="atl1-edge1", only=["hfid"])
for interface in interfaces:
    print(interface.hfid) # for example ["atl1-edge1", "Ethernet1"]

If only does not name hfid, the SDK calculates node.hfid from the fields that the query fetched. node.hfid is then None when one of those fields was not fetched.

For a generic kind, the query requests hfid when the generic or one of the kinds that implement it has a human-friendly ID. For a kind without a human-friendly ID, naming hfid changes nothing.

Hierarchy fields named in only​

A kind that is part of a hierarchy has four more fields: parent, children, ancestors and descendants. They are different from relationships of kind Parent, such as device on TestInterface, which only handles like any other relationship.

only accepts the four hierarchy fields only on a kind that is part of a hierarchy, and the query requests a hierarchy field only when you name it. A named hierarchy field is returned as a reference. With prefetch_relationships=True, the SDK also requests the default fields of the peers, without their own hierarchy fields, and builds peer nodes and adds them to the store for parent and children only. The peers of parent and children are of the peer kind that the schema of the kind declares for them, or of the hierarchy generic when the schema does not declare them. The peers of ancestors and descendants are of the hierarchy generic.

Generic kinds queried with only​

When you query a generic kind, only accepts the attributes and relationships of the generic and of any kind that implements it. If only some implementing kinds have a field, the query requests it for the nodes of those kinds. fragment=True adds no fields to a query with only.

Errors raised before the nodes are queried​

Two errors from infrahub_sdk.exceptions report a mistake in the arguments before the SDK requests any node data:

  • FieldSelectionConflictError: only is combined with include or exclude, even with an empty list. The SDK raises it before it sends any request.
  • FieldNotFoundError: a name in only is not a field of the kind or of any kind that implements it. The error names the field and the kind.

Fetching relationships manually​

The fetch method can be used to retrieve relationships, initialize them; and retrieving the related nodes manually. The fetch method can also be used on relationship that were previously initialized.

device = await client.get(kind="TestDevice", name__value="atl1-edge1")
print(device.interfaces.initialized) # False

await device.interfaces.fetch()
print(device.interfaces.initialized) # True
for interface in device.interfaces.peers:
    print(interface.id, interface.display_label, interface.typename, interface.peer.name.value)
device = await client.get(kind="TestDevice", name__value="atl1-edge1", include=["interfaces"])
print(device.interfaces.initialized) # True

await device.interfaces.fetch()
for interface in device.interfaces.peers:
    print(interface.id, interface.display_label, interface.typename, interface.peer.name.value)

Fetching only the named fields of the peers​

fetch accepts only on relationships of cardinality one and of cardinality many. Each peer gets its id, display_label, __typename and the named fields, and reading any other field of the peer raises FieldNotLoadedError.

device = await client.get(kind="TestDevice", name__value="atl1-edge1", only=["name", "site"])

await device.site.fetch(only=["name"])
print(device.site.peer.name.value)

await device.interfaces.fetch(only=["name"])
for interface in device.interfaces.peers:
    print(interface.peer.name.value)

On a relationship of cardinality one, fetch(only=[...]) sends one query for the peer, keeps the peer on the relationship and adds it to the store. If the peer has no human-friendly ID, because only did not name hfid and the fetched fields are not enough to calculate it, hfid and hfid_str on the relationship return the human-friendly ID that Infrahub returned with the reference.

On a relationship of cardinality many, fetch(only=[...]) works in this order:

  1. It checks the names against the peer kind of the relationship, before it sends any request.
  2. If the relationship was not fetched, it requests it. On a node returned by a query with only, it requests that relationship alone and does not change the node in the store. This also works for ancestors and descendants.
  3. It checks that every name is a field of the kind of every peer. If a name is missing on one of them, it raises FieldNotFoundError before it queries any peer.
  4. For each peer kind, it sends a query that counts the peers of that kind, then one query for each page of those peers, and adds the peers to the store.

On a node returned by a query with only, fetch() on a relationship of cardinality one that the query did not fetch raises FieldNotLoadedError. To fetch that peer, name the relationship in only.

Prefetch relationships​

Related nodes of a relationship can be retrieved, using the prefetch_relationships argument for the different query methods. But this requires the usage of the internal client store . More information can be found in the Using the client store guide.

Query a node(s) in the past​

To query the state of a Node in the past, you can pass the at argument to all the query methods. The at argument accepts a str, DateTime or Timestamp object as value. Values of type str will be parsed using the Pendulum library.

device = await client.get(kind="TestDevice", name__value="atl1-edge1", at="10:10:10")

Properties of attributes and relationships​

By default, the meta data or properties of attributes and relationships are not included. We can include these properties using the property argument of the SDK client's all, filters or get method.

device = await client.get(kind="TestDevice", name__value="atl1-edge1", property=True)
print(device.name.is_protected)
print(device.name.source.display_label)

Node metadata​

Node metadata provides information about when a node was created or last updated, and by whom. This includes timestamps and references to the accounts that made the changes.

Including node metadata in queries​

By default, node metadata is not included in query results. You can include it using the include_metadata argument of the SDK client's all, filters, or get method.

device = await client.get(kind="TestDevice", name__value="atl1-edge1", include_metadata=True)

Accessing node metadata​

Once metadata is included in the query, you can access it using the get_node_metadata() method. The metadata object contains the following fields:

  • created_at: Timestamp when the node was created
  • created_by: Reference to the account that created the node
  • updated_at: Timestamp when the node was last updated
  • updated_by: Reference to the account that last updated the node
device = await client.get(kind="TestDevice", name__value="atl1-edge1", include_metadata=True)

# Get the metadata object
metadata = device.get_node_metadata()

# Access creation metadata
print(metadata.created_at)  # e.g., "2024-01-15T10:30:00Z"
print(metadata.created_by.display_label)  # e.g., "admin"

# Access update metadata
print(metadata.updated_at)  # e.g., "2024-01-20T14:45:00Z"
print(metadata.updated_by.display_label)  # e.g., "admin"

The created_by and updated_by fields are NodeProperty objects that include:

  • id: The unique identifier of the account
  • display_label: A human-readable label for the account
  • typename: The GraphQL type name of the account

Relationship metadata​

When querying with include_metadata=True, you can also access metadata about relationship edges themselves. This tells you when a specific relationship (the connection between two nodes) was last modified and by whom.

Accessing relationship metadata​

Use the get_relationship_metadata() method on a related node to access the relationship edge metadata. This is different from node metadata - it describes when the relationship itself was created or modified, not the connected node.

device = await client.get(kind="TestDevice", name__value="atl1-edge1", include_metadata=True)

# For a cardinality-one relationship
rel_metadata = device.site.get_relationship_metadata()
if rel_metadata:
    print(rel_metadata.updated_at)  # e.g., "2024-01-17T08:00:00Z"
    print(rel_metadata.updated_by.display_label)  # e.g., "admin"

# For a cardinality-many relationship
for tag in device.tags.peers:
    rel_metadata = tag.get_relationship_metadata()
    if rel_metadata:
        print(f"Tag relationship updated at: {rel_metadata.updated_at}")
        print(f"Updated by: {rel_metadata.updated_by.display_label}")

The RelationshipMetadata object contains:

  • updated_at: Timestamp when the relationship was last updated
  • updated_by: Reference to the account that last updated the relationship (a NodeProperty object with id, display_label, and typename)
note

Relationship metadata tracks changes to the relationship edge itself (for example, when the relationship was created or when its properties were modified), not changes to the connected nodes. For node-level metadata, use get_node_metadata() on the node itself.

Ordering query results​

You can control the order in which query results are returned using the order argument. Results can be ordered by attributes, related-node attributes, or object metadata such as creation and update timestamps.

Ordering by fields​

Use the Order and OrderByEntry classes along with OrderDirection to order results by one or more fields. Each OrderByEntry takes a field and an optional direction (defaulting to OrderDirection.ASC). The entries are applied in order, so the first entry is the primary sort key.

A field can reference:

  • an attribute, e.g. name__value
  • a related-node attribute, e.g. owner__name__value
  • node metadata, e.g. node_metadata__created_at or node_metadata__updated_at
from infrahub_sdk.enums import OrderDirection
from infrahub_sdk.types import Order, OrderByEntry

# Get devices ordered by name (ascending is the default)
devices = await client.all(
    kind="TestDevice",
    order=Order(by=[OrderByEntry(field="name__value")])
)

# Get devices ordered by creation time (oldest first)
devices = await client.all(
    kind="TestDevice",
    order=Order(by=[OrderByEntry(field="node_metadata__created_at", direction=OrderDirection.ASC)])
)

# Get devices ordered by last update time (most recent first)
devices = await client.all(
    kind="TestDevice",
    order=Order(by=[OrderByEntry(field="node_metadata__updated_at", direction=OrderDirection.DESC)])
)

# Order by name ascending, then by creation time descending
devices = await client.all(
    kind="TestDevice",
    order=Order(by=[
        OrderByEntry(field="name__value", direction=OrderDirection.ASC),
        OrderByEntry(field="node_metadata__created_at", direction=OrderDirection.DESC),
    ])
)

The available order directions are:

  • OrderDirection.ASC: Ascending order (oldest/smallest first)
  • OrderDirection.DESC: Descending order (newest/largest first)
note

The node_metadata argument on Order is deprecated in favor of the by form shown above. It cannot be combined with by in the same Order.

Disabling default ordering​

For performance optimization, you can disable the default ordering behavior entirely:

from infrahub_sdk.types import Order

# Disable ordering to improve query performance
devices = await client.all(kind="TestDevice", order=Order(disable=True))

Combining metadata and ordering​

You can include metadata and order results in the same query:

from infrahub_sdk.enums import OrderDirection
from infrahub_sdk.types import NodeMetaOrder, Order

# Get the 10 most recently updated devices with their metadata
devices = await client.filters(
    kind="TestDevice",
    limit=10,
    include_metadata=True,
    order=Order(node_metadata=NodeMetaOrder(updated_at=OrderDirection.DESC))
)

for device in devices:
    metadata = device.get_node_metadata()
    print(f"{device.name.value} - Last updated: {metadata.updated_at}")

Query a node(s) in a different branch​

If you want to query a node(s) in a different branch than the default branch with which the SDK client was initiated, then you can use the branch argument of the query methods.

device = await client.get(kind="TestDevice", name__value="atl1-edge1", branch="refresh-site-atl1")

Query for a large amount of nodes​

When you have a query that matches a large amount of nodes, it could have an implication on the performance of the query. The SDK tries to optimize this process by using pagination, which happens transparently. However, even when using pagination, such queries could take a long time to complete.

Using the parallel argument, we can enable concurrent retrieval of pages of data for large queries, which can significantly improve the resolution time of the query.

The parallel argument is available for the SDK client's filters and all method.

device = await client.all(kind="TestDevice", parallel=True)