Root Cause Analysis

What it is

Finding out why a robot failed: the moment it happened, whether it has happened before, and what caused it.

How Alloy does it

1. It reads your logs as they are. MCAP, ROS 1 bags, PX4, ArduPilot and MAVLink, decoded message by message. Several logs from the same run are read together.
2. It computes, it does not guess. The agent queries the recorded signals and runs its own analysis code on them: a frequency spectrum, a comparison across flights, a count across releases.
3. It checks the history and your other systems. Every run you have uploaded, plus the tickets, metrics and code you connect.
4. It shows its work. Each finding links to the moment and the chart behind it. You can open the code it ran, see what it ruled out and why, and see what it has not confirmed.

Ask in the Alloy app, in Slack, on a Jira ticket, or from your coding agent.

Example

Fictional example: the drones and numbers are invented.

A survey team flies 12 quadcopters on PX4. An operator reports that SV-14 "bounces" in altitude hold on some flights and not others. An engineer asks:

"SV-14 bounces in altitude hold on some flights. Find out why, and check whether other drones do it."

The agent answers:

The pattern: in hover, altitude oscillates ±0.6 m at 1.9 Hz. It happens on 3 of SV-14's 11 flights this week.

Ruled out, wind: the bounce appears on flights with wind under 2 m/s and is absent on a flight in 7 m/s.

What differs: all 3 bouncing flights carry the new gimbal mount; none of the 8 others do. On those 3 flights, vertical acceleration shows a strong vibration peak at 112 Hz, above the 80 Hz accelerometer filter. Vertical-velocity estimate noise is 3 times higher. [Spectrum chart] [Code it ran]

Not reported: SV-09 also flew with the new mount on Tuesday. It shows the same 112 Hz peak and a smaller bounce (±0.2 m). That flight was logged as normal.

Likely cause: gimbal-mount vibration at 112 Hz passes the accelerometer filter and corrupts the vertical-velocity estimate.

Not confirmed: the vibration source is not isolated. Suggested tests: a bench run-up with and without the mount, and one flight with a notch filter at 112 Hz.

The engineer runs the bench test, confirms the mount resonance, and asks the agent to check every flight on every drone for the 112 Hz peak from now on.

How to start

Ready to start. Talk to us. Alloy sets up your workspace with you and connects your data, and anyone on your team can start an investigation on the next incident.

Want proof first. Send the recordings from two or three incidents you have already solved, without the answer. Alloy investigates them cold, so you can compare its answer and its working with your team's.

Just looking. See the agent work on sample robots in Try Alloy.

What we need from you: access to your robot data, and Jira, Grafana and GitHub if causes often live there.

Outcomes for your team

Your senior engineers get their time back. Investigations stop landing on the few people who know the subsystem.

"Alloy skipped that entire step where I would've set up this dashboard … turned two days into 10 minutes."

Tom Cameron, Tech Lead, AI Engineering, on a fleet-wide investigation. How Andromeda uses Alloy

Your fleet grows without the escalation queue growing. Anyone can start an investigation. The expert joins with the evidence already gathered.

"We haven't necessarily needed to hire anyone yet."

Tom Brodie, Head of Autonomy. How SwarmFarm is scaling its robot fleet

You fix what matters most first. Every cause comes with a count by robot and by release, and a problem seen before is recognised.

"Our rate of duplicate tickets has gone down a fair bit, and the mean time to resolution for previously seen bugs has gone down."

Tom Cameron, Andromeda. How Andromeda uses Alloy

Related

• Scenarios & Regression Testing: watch for the same failure on every new run.
• Support Agent: the first pass before an incident reaches engineering.
• Fleet Monitoring: find the next incident before the field reports it.
• GitHub, Grafana, Jira