Measurements

Description

A Measurement is a source of sampled data that emits Records — timestamped JSON documents — on its own ROS topic. For example, a Record from the Memory Measurement:

{
    "flattened": false,
    "memory": {
        "used": 76.007431
    },
    "nested": true,
    "run_id": "358"
}

flattened/nested/run_id are added unconditionally by this node — see below. Fields like robot_name only appear once configured as a custom key (see custom_key_str_list above), and date/host/source_type/tag/ timestamp are added later, by Vector once the Bridge forwards the Record — see a captured example on any demo page for the full shape a Destination actually receives.

Node parameters

This node collects data and publishes it as Records. Each Measurement is a pluginlib plugin loaded into this node and publishes on its own topic_output; the Bridge (dc_bridge) subscribes to those topics and forwards the Records to the Destinations that list them in inputs (see Destinations). Conditions are pluginlib plugins loaded here too — optional predicates that gate whether a Measurement collects, e.g. only when the robot is not moving.

Parameter nameDescriptionType(s)Default
measurement_pluginsName of the measurement plugins to loadlist[str]N/A (mandatory)
condition_pluginsName of the condition plugins to loadlist[str][]
save_local_base_pathPath where files will be saved locally (e.g camera images). Expands $X to environment variables and =Y to custom string parametersstr"$HOME/ros2/data/%Y/%M/%D/%H"
all_base_pathPath where files will be saved at their destination (S3, RustFS...). Expands $X to environment variables and =Y to custom string parametersstr""
custom_key_str_listCustom strings to use in other parameters. They are also appended in the json sent to the destination, and to the File metadata Records of the same Measurementlist[str]N/A
custom_keys_str.force_overrideOverride values if the keys are already present in the measurement. Applies to all and can be overridden by custom_keys_str.<param_name>.force_overrideboolfalse
custom_keys_str.<param_name>.nameKey to add in the serialized datastrN/A (optional)
custom_keys_str.<param_name>.valueValue to set for the key as a fixed stringstrN/A (optional)
custom_keys_str.<param_name>.value_from_filePath to a file containing the value to setstrN/A (optional)
custom_keys_str.<param_name>.force_overrideOverride value if the key is already present in the measurementboolfalse
run_id.enabledIdentify which run the robot is. A new one is generated at every start of the node. Uses either a counter or UUIDbooltrue
run_id.counterEnable counter for the run_idbooltrue
run_id.counter_pathPath to store the last run. It is expanded with environment variables idstr"$HOME/run_id"
run_id.uuidGenerate a new run ID by using a random UUIDboolfalse

robot_name resolution

robot_name is a custom key like any other, but when it appears in custom_key_str_list its value resolves in a fixed order rather than always being a literal string, so a fleet does not need one hand-edited params file per robot:

  1. custom_keys_str.robot_name.value — a literal string, unchanged from before.
  2. custom_keys_str.robot_name.value_from_file — the contents of a file, e.g. one written by the provisioning process.
  3. The machine's hostname — the default when neither of the above is set.

A value_from_file that names a file that cannot be read, or any source that resolves to an empty string, fails node configuration with a clear error rather than shipping Records with a missing or blank robot_name.

Custom keys on Files

The keys listed in custom_key_str_list label a Measurement's Files as well as its Records: dc_uploader (ADR-0014, a separate process from the Bridge) writes them into the file_status and group_complete Records it emits for that Measurement's Files, so both sides of a Destination carry the same labelling. A Record names its custom keys in a custom_keys field for that purpose.

Two limits are worth knowing. A custom key whose name is one the Uploader computes itself (group_name, local_path, remote_path, storage_type, uploaded, size, …) is not written — the Uploader's own value is kept and dc_uploader logs the collision. The keys the rows already carry, robot_name and id (as robot_id), are likewise not repeated, and are not reported: those values are in the row either way. And the column still has to exist in the Destination: the PostgreSQL sink maps JSON keys onto existing columns 1:1, so a new custom key needs an ALTER TABLE on dc_files the same way it needs one on dc_records.

Plugin parameters

Each measurement is collected through a node and has these configuration parameters:

Parameter nameDescriptionType(s)Default
buffer_duration_secSeconds of history to buffer instead of publishing live; 0 disables buffering and preserves normal live publishingfloat0
condition_max_measurementsCollect a maximum of n measurements when conditions are activated (-1 = never, 0 = infinite)int0
cooldown_secSeconds to ignore further FlushEvents once post-roll ends, before buffering re-arms itself; 0 re-arms immediatelyfloat0
debugMore verbose outputboolfalse
enable_validatorWill validate the data against a JSON schemabooltrue
flush_topicTopic to receive the FlushEvent (see Triggers) that releases the buffered window, tagging each Record with the event's incident_idstr"/dc/flush"
gate_conditionName of a Condition that must become true once before any collection is published; then latches open permanently and is never consulted againstrN/A (optional)
group_keyValue of the key used when groupedstr"" (Optional)
if_all_conditionsCollect only if all conditions are activatedlist[str]N/A (optional)
if_any_conditionsCollect if any conditions is activatedlist[str]N/A (optional)
if_none_conditionsCollect only if all conditions are not activatedlist[str]N/A (optional)
include_measurement_nameInclude measurement name in the JSON databooltrue
include_measurement_pluginInclude measurement plugin name in the JSON databoolfalse
init_collectCollect when the node starts instead of waiting the first tickbooltrue
init_max_measurementsCollect a maximum of n measurements when starting the node (-1 = never, 0 = infinite)int0
json_schema_pathPath to the JSON schema, ignored if empty stringstrN/A (optional)
max_flush_rate_hzCeiling on how fast the buffered window is emitted once a flush releases it; 0 releases the whole window in one burstfloat0
pluginName of the plugin to loadstrN/A (mandatory)
polling_intervalInterval to which data is collected in millisecondsint (>=100)1000
post_roll_duration_secSeconds to keep publishing live after a flush, still tagged with the same incident_id; 0 means pre-roll onlyfloat0
remote_keysDestination names the Files this Measurement produces are uploaded to; each becomes a key under the Record's remote_pathslist[str]N/A (optional)
remote_prefixesPrefixes to apply to the remote paths of the Files this Measurement producesstrN/A (optional)
topic_outputTopic where result will be publishedstr"/dc/measurement/<measurement_name>"

buffer_duration_sec and friends: pre-event circular-buffer capture

When buffer_duration_sec is set above 0, this Measurement stops publishing live: each collected sample is instead pushed into an in-memory ring buffer covering the last buffer_duration_sec seconds. A FlushEvent on flush_topic (published by a dc_triggers broadcast node when its Trigger fires) then drives one incident-capture cycle:

  1. Buffering — the default, armed state: samples accumulate in the ring buffer and nothing is published. Only in this state does a FlushEvent start a cycle.
  2. Flushing — the buffered window is published oldest first, each Record tagged with the event's incident_id and stamped with when it was collected, not when it was released. With max_flush_rate_hz left at 0 the whole window goes out in one burst; set above 0, it is emitted at no more than that many Records per second, so a robot recovering from an incident does not also have to absorb the entire window at once. The window is consumed, so the next incident releases its own history rather than replaying this one. Samples collected while a rate-limited release is still draining are buffered, not published, and so become part of the next incident's pre-roll.
  3. PostRoll — for post_roll_duration_sec after the release finishes, samples are published live as they are collected, still tagged with the same incident_id, so the aftermath of the incident is captured too. It runs from the end of the release, not from the FlushEvent, so a rate-limited release does not eat into it. Left at its default 0, this phase is skipped entirely: pre-roll only.
  4. Cooldown — for cooldown_sec after post-roll ends, further FlushEvents are ignored, so a flapping Trigger cannot produce a flood of overlapping incidents. Samples are buffered again during this phase, so the next incident still gets a full pre-roll window.

Files follow their Records. A Measurement that produces Files (camera, map, …) normally leaves them under save_local_base_path for the Bridge to upload as soon as the Record naming them is published — but an armed Measurement publishes nothing, so every File it produces while buffering (also during cooldown, and while a rate-limited release is still draining) is instead moved into a scratch directory beside the save path, <save_local_base_path>/.dc_incident_scratch/<measurement_name>/, and the buffered Record is rewritten to reference the staged copy. That scratch directory rolls on the same buffer_duration_sec window as the Records themselves: a staged File is deleted from disk at the same moment its Record ages out of the ring buffer, so an armed Measurement's Files stay bounded instead of accumulating images no Record will ever carry to the Bridge. On release the staged copies are handed on with the Records referencing them — remote_paths is untouched, so each File uploads to exactly the Destination key it was collected under — and the scratch ring stops tracking them, leaving the Bridge's usual retention sweep and delete_when_sent to clean them up. The Bridge needs no configuration for any of this: a released Record is an ordinary Files Record that happens to be older than usual. Files collected during PostRoll are published live and never staged at all.

The Measurement then re-arms itself back to Buffering with no manual intervention — a second incident is captured exactly like the first. With both post_roll_duration_sec and cooldown_sec left at 0, a flush releases the pre-roll window and the Measurement is armed again immediately.

incident_id is a top-level field of the Record envelope, beside tags, run_id and name — not a key nested inside the measurement's own data — so a postgres Destination stores it in its own incident_id column and "everything from this one event" is a plain WHERE incident_id = '…'. See Destinations for the column the table needs. A Record collected outside an incident carries no incident_id at all, leaving the column NULL. A Group lifts a member's incident_id onto the merged Record the same way it does tags, so grouping does not bury it.

gate_condition vs. if_all/if_any/if_none_conditions

gate_condition is a one-shot arming latch: it names a single Condition plugin (any type under dc_measurements/plugins/conditions/) that suppresses every collection — including the init_collect Record normally published on activation — until that Condition becomes true for the first time. Once armed, the Condition is never consulted again for the lifetime of the node, even if it later becomes false again; re-arming does not happen. This is unlike if_all_conditions/ if_any_conditions/if_none_conditions, which are re-evaluated on every collection and can suppress publishing again once their Conditions change. If the named Condition doesn't exist among condition_plugins, collection is held back permanently and an error is logged.

How if_all/if_any/if_none_conditions combine

The three lists are evaluated on every collection and ANDed together: if_all_conditions objects unless every Condition it names is active, if_any_conditions unless at least one of its Conditions is active, and if_none_conditions unless every Condition it names is inactive. A list left empty never objects, so a Measurement naming no Condition at all always collects. A name that is not among condition_plugins reads as inactive and an error is logged: it blocks collection when listed in if_all_conditions or if_any_conditions, and is accepted by if_none_conditions.

Available plugins:

NameDescription
BatteryCharge percentage, voltage and current of a pack, plus charging sessions and completed cycles
CameraCamera images, images can be rotated and inspected to detect content in images. They are saved as files
Command velocityCommand velocity: navigation commands
CPUCPU statistics
Diagnostics/diagnostics DiagnosticStatus entries, converted to Records so hardware/driver health reaches Destinations
Distance traveledTotal distance traveled by the robot
Driving typeCurrent operating mode — autonomous, manual, teleop or unknown
DummyDummy event, for testing and debugging
Fast DDS statisticseProsima Fast DDS's own Statistics Module: latency, throughput, RTPS packets, physical-layer data. Fast-DDS-specific
FaultComponent diagnostic level transitions: one Record per raise, change or clear, a source for MTBF/MTTR
InterventionHuman takeovers: how often, how long autonomous beforehand, how long the takeover lasted
IP CameraIP camera videos as files
ManipulationOne MoveIt MoveGroup goal's lifecycle — start and end Records with outcome and timing
MapROS map files (yaml and pgm) and metadata used by the robot to localize and navigate
MemorySystem memory usage
Mission Nav2 (NavigateToPose)Nav2 adapter of the Mission Measurement for a single-pose NavigateToPose goal
Mission (nav2 FollowWaypoints)Outcome of a nav2 FollowWaypoints patrol/waypoint-following run, including per-waypoint failures
Mission Nav2 (NavigateThroughPoses)Nav2 adapter of the Mission Measurement for a NavigateThroughPoses goal
Mission (Open-RMF)Open-RMF adapter of the Mission Measurement, reading rmf-web's per-task TaskState
NetworkNetwork interfaces, availability
OSOperating System information
PermissionsPermissions of a file or directory
PositionRobot position
RandomA randomly generated value every polling interval — for exercising the pipeline without robot infrastructure
ROS2 control statusWhen a ros2_control controller or hardware component crosses into or out of the active state
Serial interfaceLine-delimited data off a configurable serial port, parsed into named Record fields
slam_toolbox qualityLocalization quality from slam_toolbox's /pose and loop-closure topics
SpeedRobot speed
StorageAvailable and used space in a directory
String stampedRepublish a string stamped message, can be used for external data
TCP HealthHealth status of a TCP Server
ThermalTemperatures (CPU, GPU, board, …) from the kernel's thermal sysfs interface
UptimeHow long the machine has been turned on