Mesh Storage
Redaction rules file (advanced)
Manual redaction.yaml schema for Alloy Edge v0.7 to v0.9.8
This page documents hand-written redaction.yaml rules files for Alloy Edge
v0.7 to v0.9.8. The rule editor in Edge settings can write
these rules for you; see
Configure.
From v0.9.0, a rules file must contain the top-level field version: 1. Without it, the file
fails to load. v0.7 and v0.8 releases ignore the field, so you can keep it in
every version.
# Required from v0.9.0.
version: 1
# Optional. Merge named functions from additional files.
includes:
- /etc/alloy/transforms-common.yaml
# Topic-level allow/deny. Runs before any other rule.
channels:
allow: ["*"]
deny: ["/user/*", "/audio/**"]
# Salt for hash(...). ${VAR} is read from the environment at load time.
hash_salt: "${ALLOY_HASH_SALT}"
# Per-channel mappings. First match wins.
transforms: [...]
# Per-metadata-record mappings. Same shape as transforms.
metadata: [...]
# The named-function library. Referenced by transforms / metadata via `function:`.
functions: { ... }
# Optional. Rule-file-level audit defaults. Settings in edge-sync.yaml's
# `audit:` block and CLI flags take precedence.
audit:
embed_in_mcap: trueTop-level
| Field | Type | Default | Description |
|---|---|---|---|
version | integer | — | Schema version. Required from v0.9.0, where any value other than 1 fails to load. |
enabled | bool | false | Accepted for compatibility, but Alloy Edge does not read it. A rules file applies whenever edge-sync.yaml references it: through a pipeline transform: step, or through redaction.enabled: true with rules_file. |
includes | list of paths | [] | Additional files merged into the same functions: namespace, in declaration order. Last file wins on name collision; the entry-point file's own functions: always wins overall. Resolved relative to the directory of the file that contains them. |
channels | object | {} | Topic-level allow/deny. See below. |
hash_salt | string | — | Salt for hash(...). ${VAR} is interpolated from the environment at load time. Missing env var → load-time error. |
functions | map | {} | Named transform bodies — the "library" referenced by transforms: and metadata:. |
transforms | list | [] | Per-channel mappings. First match wins. |
metadata | list | [] | Per-metadata-record mappings. Same shape as transforms but match: globs against record names. |
default_scope | list | [mcap, json, jsonl] | File types that metadata: rules apply to. A rule's own scope: overrides it. An empty list applies to all three. |
audit | object | — | Rule-file-level audit defaults. Currently one field: embed_in_mcap (bool). Absent → defer to edge-sync.yaml's audit: block / CLI flags. |
channels
Topic-level allow/deny. It runs before any other rule.
| Field | Type | Default | Description |
|---|---|---|---|
allow | list of globs | — | Fail-closed. null or [] → no topic passes. ["*"] → all topics pass (only deny filters). ["/odom", "/tf*"] → only matching topics pass. |
deny | list of globs | [] | Always wins on conflict — a deny match drops the topic regardless of allow. |
Globs use shell-style wildcards. * matches any characters, including /, so
deny: ["/user/*"] also drops /user/a/b.
transforms[] / metadata[]
Each entry has exactly one of function: or transform:. Both or neither → load-time error.
| Field | Type | Required | Description |
|---|---|---|---|
match | string or object | yes | Channel/record selector. See Match selector below. |
function | string | one of | Reference to a named function in functions: (or merged from includes:). |
transform | object | one of | Inline transform body — same shape as a functions: entry. |
scope | list | default_scope | metadata: rules only. File types the rule applies to: mcap, json or jsonl. Ignored on transforms: entries. |
Match selector
Two equivalent forms — pick the shorthand for the common case.
# Shorthand — channel only (or record-name only for `metadata:`)
match: "/diagnostics"
# Object form — full selector
match:
channel: "/diagnostics"
schema: "diagnostic_msgs/DiagnosticArray" # disambiguates same-topic-different-schema (rare but legal)
encoding: "cdr" # or "ros2msg", "protobuf", etc.For the metadata: block, the shorthand maps to record_name instead of channel:
match: "operator_*" # shorthand → record_name
match:
record_name: "operator_*" # object form
key: "operator_email" # optional inner-key glob| Field | Used by | Description |
|---|---|---|
channel | transforms | Topic glob (e.g. /diagnostics, /sensors/*). |
schema | transforms | Optional schema-name glob — disambiguates two channels with the same topic but different schemas. |
encoding | transforms | Optional encoding glob — cdr, ros2msg, protobuf. |
record_name | metadata | MCAP metadata-record name glob. |
key | metadata | Optional inner-key glob within a metadata record. |
Transform bodies
Each transform body has a type: field. These types are available in every
version from v0.7 to v0.9.8:
| Type | Fields | Effect |
|---|---|---|
drop | — | Drop the matched topic. |
decimate | n | Keep every Nth message. n must be at least 1. |
rate_cap | hz | Keep at most hz messages per second. hz must be greater than 0. |
remap | to | Rename the matched topic. The new name must not collide with another topic in the same file. |
put | schema, available_fields, template | Whitelist. See below. |
patch | schema, overrides | Denylist. See below. |
From v0.9.1, metadata: rules can also use drop_if. It drops the matched
value when a condition is true. Set operator to equals, not_equals,
contains, empty, or not_empty, and set value for the operators that
compare. transforms: rules reject drop_if.
type: put — whitelist
Spell out the entire output. Fields not mentioned in the template disappear. New upstream fields don't leak.
| Field | Type | Default | Description |
|---|---|---|---|
schema | string | — | ROS2 schema name the template targets (e.g. diagnostic_msgs/DiagnosticArray). Optional for metadata: rules. |
available_fields | list of strings | [] | Documentation-only — human-readable list of fields the template references. Not enforced. |
template | string (Jinja2) | — | Jinja2 template rendering the full output JSON. Omit template: for the identity put. The message passes through unchanged. Use it to record a deliberate "this channel is fine, leave it alone" decision in the config. |
type: patch — denylist
Pass everything through, override only the listed fields.
| Field | Type | Default | Description |
|---|---|---|---|
schema | string | — | ROS2 schema name. Optional for metadata: rules. |
overrides | map | {} | Field-path → Jinja2 expression. Path syntax is field, field.sub, array[N].field (one element), or array[].field (every element). |
functions
Named transform bodies. Same shape as a transform: entry, minus the match: key (which lives on the call site).
functions:
# Identity put: the message passes through unchanged.
identity_pass:
type: put
# Whitelist redactor for an operator-command message.
redact_operator_cmd:
type: put
schema: "your_msgs/OperatorCommand"
template: |-
{
"operator_id": {{ operator_id | hash(algo="sha256") | tojson }},
"command": {{ command | tojson }},
"timestamp": {{ timestamp | tojson }}
}
# Denylist counterpart — same schema, only override two fields.
redact_operator_cmd_lite:
type: patch
schema: "your_msgs/OperatorCommand"
overrides:
operator_id: '{{ original | hash(algo="sha256") }}'
notes: '""'Functions from includes: land in the same namespace; local definitions in the entry-point file always win on collision.
Template helpers
template: and overrides: expressions use Jinja2 syntax. On top of the
standard filters, Alloy Edge adds these helpers. Each one works as a filter
and as a function.
| Helper | Use | Example |
|---|---|---|
hash(algo=...) | Full hex digest of the salt, a | character, and the value. algo is md5 (the default) or sha256. Use this for de-identification. Requires hash_salt; a template that calls hash without a salt fails when it renders. From v0.9.0, an empty or whitespace-only salt also fails. | '{{ original | hash(algo="sha256") }}' |
sha256_short | First 8 hex chars of sha256(value) — unsalted. Content fingerprint, not de-identification (reversible with a wordlist). | '{{ s.name | sha256_short }}' |
regex_strip(pattern, replacement) | Replace regex matches in the value with a literal string. replacement is optional; without it, matches are removed. | '{{ original | regex_strip("[A-Z]+", "[X]") }}' |
regex_redact([patterns]) | Apply a list of regex patterns; replace every match with [REDACTED]. Convenience wrapper for chaining several regex_strip calls — patterns are user-supplied. | '{{ original | regex_redact([email_re, phone_re]) }}' |
redact_if(cond, value, replacement) | Return replacement when cond is true, otherwise return value. Write the condition as the first argument. In filter form, the piped value becomes cond. | '{{ redact_if(original == "lab-host", original, "[HOST]") }}' |
zero | Type-aware zero — "" for strings, 0 for numbers, etc. | '{{ original | zero }}' |
tojson | JSON-encode a value. | '{{ header | tojson }}' |
original is a context variable, not a filter. It's bound only inside patch.overrides (the pre-override field value) and metadata: rule expressions (the pre-redaction string). put.template does not see original — it sees the full decoded message and addresses fields by name ({{ header | tojson }}, {{ status[0].level }}).
hash_salt must be set if any rule calls hash(...). sha256_short does not consume the salt.
Resolution rules
- Channel filter runs first. Denied topics are dropped before any other rule runs.
- First match wins per channel / record name. A topic not listed in
transforms:passes through unchanged. includes:merge in declaration order (last include wins on function-name collision); localfunctions:in the entry-point file override every include.- Bad templates fail at load time. A Jinja syntax error in any template / override expression rejects the rules file before the agent starts.
Reload semantics
Alloy Edge reads the rules file, and every included file, when the sync process starts. Restart Alloy Edge after you edit them.
The merged config is hashed (sha256:<hex>, recorded as rules_hash on every audit entry) so you can prove later which version of the rules produced a given file.