Getting good results

The agent gives better answers when it knows the robot, the time, what went wrong and what you already checked. Your engineers know these things best. A good request passes that knowledge on, and the agent does the log work. This page shows you how to ask, what to upload, and where the agent needs your help.

When the agent is a good fit

Rule of thumb. Give the agent a task when an engineer would otherwise pull logs from one or more runs, line up the signals, and look for the cause. Do it yourself when the task needs a physical look at the robot, or data that you did not record.

The agent does well when:

• the answer is in the recordings, or in a tool you connected (tickets, code, metrics);
• the question names a robot, a time window or a software version;
• you can check the answer by looking at a plot or a moment in the recording.

The agent does less well when:

• the robot did not record the signal that explains the event;
• the question is about a hardware fault that leaves no trace in the logs;
• the data comes from a test that deliberately causes faults, and the agent does not know the test plan.

How to write a good request

Put these five things in a request. You do not need all five every time.

1. The robot and the time. "AMR-12, yesterday between 14:00 and 15:00."
2. What you saw. "It stopped at dock 4 and needed two restarts."
3. What normal looks like. "It usually docks in under 40 s."
4. What you already know. "The battery was above 60%. We ruled out the charger."
5. What you want back. "The likely cause, the evidence with timestamps, and whether it happened on other robots this month."

Two more tips:

• Ask for a length. "Answer in five lines" or "a table, one row per robot". Answers can run long otherwise.
• If it stops short, ask for the next step. More runs, another signal or a comparison.

Good and bad requests

Investigate a field fault

Why the good request works

• It names the robot, the place and a time window, so the agent opens the right mission.
• It says what normal looks like, so the agent knows what counts as wrong.
• It rules out one cause, so the agent does not spend effort there.
• It asks for evidence and for other occurrences, so you can check the answer and size the problem.

Compare two releases

Why the good request works

• It names both versions and the time range.
• It names the metrics that matter for this release.
• It asks for a split, so one bad vehicle does not hide in an average.
• It asks for the runs behind each claim.

Define a scenario

Why the good request works

• It has a signal, a threshold, a duration and a context.
• You can confirm each match by looking at one plot.
• It describes one pattern. Make a second scenario for a second pattern.

Write a recurring report

Why the good request works

• It lists the sections, so every week's report has the same shape.
• It asks for links, so a reader can check each fault.
• It sets a length, so the report stays easy to read.

Ask a support question in Slack

Why the good request works

• It gives the robot, the site and the time.
• It asks whether the fault is known, so the agent checks past cases.
• It asks for an action that the operator can take now.

Follow up on an answer

Why the good request works

• It separates evidence from inference.
• It asks for the moment, so you can see it yourself.
• It names a signal that the agent may have skipped.

Before you upload

Use this checklist before you upload test or field data.

• One run per mission. In the browser, upload one run at a time: all the files in one upload become one mission. In a connected bucket, put each run in its own folder: each folder becomes one mission.
• Add a metadata.json. Any key-value pairs work, and each key becomes a mission filter. Useful keys are the robot ID, the software version, the site, the operator and the purpose of the test.
• Add field notes. Write what happened and what looked wrong in a .txt or .md file. The agent reads it as context.
• Keep topic names the same across robots and runs. Then one question or scenario works across the fleet.
• Record the signals that explain an event. Diagnostics, commands and state at full rate. Images at a lower rate are often enough.
• Include the test plan if the test causes faults on purpose. Otherwise the agent can read an induced fault as a real one.
• Use a supported format. The Agent Platform accepts MCAP (including ROS 2 recorded to MCAP), ROS 1 .bag, MAVLink .tlog, PX4 .ulg and ArduPilot .bin. Mesh Storage browser upload accepts .mcap, with .json or .yaml metadata.

Chat, reports, automations or scenarios

Start in chat. When you ask the same question a second time, turn it into a recurring report. When you need to count a pattern across the fleet, make it a scenario.

Correct it once

When the agent gets something wrong about your robots, correct it in the chat and say "remember that". Choose whether the note is for you or your whole organisation. The next person who asks gets the corrected answer.

Check the work

• Open the cited moment in Replay before you act on a finding.
• Ask the agent what it checked and ruled out.
• For a report, read its version history to see what changed.
• Correct the agent in chat when it is wrong. Save team facts to memory, for example "Missions from 20 to 30 September used a temporary LiDAR mount."

See How you can check the agent's work.

Known limits

These limits are specific to how you work with the agent. The boundaries that apply to all of Alloy are in Where Alloy stops.

• Test context. In a test that causes faults on purpose, the agent needs your test plan or labels to tell planned behaviour from a real failure.
• The agent can reuse earlier context without a fresh check. In a long chat, ask it to check the data again before it concludes.
• The agent can generalise from too few runs. Ask how many runs support a claim.
• Answers about software come from what the agent can read. If the agent describes your code or your UI, check it against the source.
• Missions still processing. You can ask about a mission's raw data while it processes, but some heavy topics are available only when processing ends. Recurring reports run after processing finishes.
• Credits. When your organisation runs out of credits, the in-app chat stops taking new messages. Ask Alloy how your plan handles recurring reports and scenario scans.
• Jira ticket changes beyond the agent's own replies need a person to agree. Ticket create, edit and transition: Alloy switches this on with you.

Related

• What changes for your team
• Alloy for AI agents
• Security and trust
• When to use Alloy