Running the agent
Start a run, watch it as it happens, and know what the agent can do and where it stops.
A run sends the agent to your live site as one persona, trying one goal. It starts on the homepage and works the way a visitor would: it reads the page, decides what to do, does it, and looks again.
Starting a run
Choose a Persona and what it's Trying to do. Type to filter either list, or use the + beside it to add a new one.
The site has to be verified first. You can also start a run from an empty cell of the Scores grid, which opens Run once with that pair already chosen. To run a pair every day or week, schedule it.
One run at a time
Only one run goes on a site at a time, so you can't start another on a goal's site while one is going there. Across PersonaMark at most two runs go at once, so yours may wait a moment for its turn.
While it waits, Run once says A run is waiting to start. If it hasn't started after two minutes, the page says the worker that runs agents may be off. You can cancel it.
Watching it live
Once it starts, Run once says The agent is working on this site, with a link to Watch it as it happens. That opens the run's report, which counts the steps taken so far, adds each step as the agent takes it, and shows the score once the run finishes and is scored.
What the agent can do
| Action | What it does |
|---|---|
| Go to an address | Opens an address on your site. |
| Look again | Reads the page again, for example after something loads. |
| Click | Clicks an element. |
| Type | Types into a field, replacing what's there. |
| Type the login | Types the test account's username or password, without seeing it. Only offered when the goal has a login. |
| Choose an option | Picks an option in a dropdown or list by its exact label. |
| Press a key | Enter, Escape, Tab, or the up or down arrow, on whatever has focus. |
| Wait | Waits up to 5 seconds for something to appear. |
| Scroll | Scrolls one screen up or down. |
| Go back | Returns to the previous page. |
| Finish | Ends the run, saying the goal is done or that this person would give up here. |
Actions are at least a second apart. A Phone persona uses a phone-sized touch screen, and a Desktop persona a laptop-sized one.
The agent finds elements the way a screen reader does, by their role and their name: a button called "Send invoice", a link called "Pricing". A control with no clear name is hard for it to use, and that tends to show up as friction in the report. It's usually hard for people using assistive technology too. Collapsed sections, the kind you open by clicking their heading, are listed for the agent so it can open them.
Where it can go
A run covers your site's whole domain, including subdomains such as an app, since a goal like signing up can lead there. It never leaves your domain.
Windows it opens
If a click opens a new window or tab on your domain, the agent carries on there, and the page that opened it closes. A window on any other site is closed straight away, and the step says so. The scorer counts that as friction, since a visitor would have been taken somewhere else.
Cookie banners
The agent isn't given a way around cookie or consent banners. Dealing with one is part of the visit, so it deals with it the way the persona would. A banner inside a frame the agent can't reach counts as friction too, since it can get in the way of the task.
How a run ends
| Outcome | What happened |
|---|---|
| Completed | The agent said the goal is done. |
| Gave up | The agent said this person would give up. |
| Hit the step limit | It took 40 steps without finishing. |
| Hit the time limit | It ran for 8 minutes without finishing. |
| Stopped for safety | A guardrail or the cost cap ended it. No score. |
| Cancelled | Someone cancelled it. No score. |
| Didn't finish | The run stopped before it could save an outcome. |
The cost cap is $2.00 of model calls in one run, counted from what the model provider reports for each call. It's there to stop a run that's gone wrong, not as a budget. Passing it ends the run before the agent acts on its last answer. A run's cost shows on its report, and the site's total is in model spend.
A run never retries
If a run fails partway, it isn't tried again automatically. A retry would repeat real actions on your real site, such as submitting a form twice. Run once says The last run failed with the reason, and you can start another when you're ready.
Cancelling a run
Choose Cancel run, either on Run once or on the run's report while it's going. The agent stops before its next step. A cancelled run keeps the steps it took, but gets no score, and finishing never overwrites a cancel. If it had already finished, you'll see "That run has already finished."