Skip to Content
PersonaMark docs
Personas and goals

Running the agent

Start a run, watch it as it happens, and know what the agent can do and where it stops.

A run sends the agent to your live site as one persona, trying one goal. It starts on the homepage and works the way a visitor would: it reads the page, decides what to do, does it, and looks again.

Starting a run

On the project's Personas page, choose Run once.

Choose a Persona and what it's Trying to do. Type to filter either list, or use the + beside it to add a new one.

Choose Run agent.

The site has to be verified first. You can also start a run from an empty cell of the Scores grid, which opens Run once with that pair already chosen. To run a pair every day or week, schedule it.

One run at a time

Only one run goes on a site at a time, so you can't start another on a goal's site while one is going there. Across PersonaMark at most two runs go at once, so yours may wait a moment for its turn.

While it waits, Run once says A run is waiting to start. If it hasn't started after two minutes, the page says the worker that runs agents may be off. You can cancel it.

Watching it live

Once it starts, Run once says The agent is working on this site, with a link to Watch it as it happens. That opens the run's report, which counts the steps taken so far, adds each step as the agent takes it, and shows the score once the run finishes and is scored.

What the agent can do

ActionWhat it does
Go to an addressOpens an address on your site.
Look againReads the page again, for example after something loads.
ClickClicks an element.
TypeTypes into a field, replacing what's there.
Type the loginTypes the test account's username or password, without seeing it. Only offered when the goal has a login.
Choose an optionPicks an option in a dropdown or list by its exact label.
Press a keyEnter, Escape, Tab, or the up or down arrow, on whatever has focus.
WaitWaits up to 5 seconds for something to appear.
ScrollScrolls one screen up or down.
Go backReturns to the previous page.
FinishEnds the run, saying the goal is done or that this person would give up here.

Actions are at least a second apart. A Phone persona uses a phone-sized touch screen, and a Desktop persona a laptop-sized one.

The agent finds elements the way a screen reader does, by their role and their name: a button called "Send invoice", a link called "Pricing". A control with no clear name is hard for it to use, and that tends to show up as friction in the report. It's usually hard for people using assistive technology too. Collapsed sections, the kind you open by clicking their heading, are listed for the agent so it can open them.

Where it can go

A run covers your site's whole domain, including subdomains such as an app, since a goal like signing up can lead there. It never leaves your domain.

Windows it opens

If a click opens a new window or tab on your domain, the agent carries on there, and the page that opened it closes. A window on any other site is closed straight away, and the step says so. The scorer counts that as friction, since a visitor would have been taken somewhere else.

The agent isn't given a way around cookie or consent banners. Dealing with one is part of the visit, so it deals with it the way the persona would. A banner inside a frame the agent can't reach counts as friction too, since it can get in the way of the task.

How a run ends

OutcomeWhat happened
CompletedThe agent said the goal is done.
Gave upThe agent said this person would give up.
Hit the step limitIt took 40 steps without finishing.
Hit the time limitIt ran for 8 minutes without finishing.
Stopped for safetyA guardrail or the cost cap ended it. No score.
CancelledSomeone cancelled it. No score.
Didn't finishThe run stopped before it could save an outcome.

The cost cap is $2.00 of model calls in one run, counted from what the model provider reports for each call. It's there to stop a run that's gone wrong, not as a budget. Passing it ends the run before the agent acts on its last answer. A run's cost shows on its report, and the site's total is in model spend.

A run never retries

If a run fails partway, it isn't tried again automatically. A retry would repeat real actions on your real site, such as submitting a form twice. Run once says The last run failed with the reason, and you can start another when you're ready.

Cancelling a run

Choose Cancel run, either on Run once or on the run's report while it's going. The agent stops before its next step. A cancelled run keeps the steps it took, but gets no score, and finishing never overwrites a cancel. If it had already finished, you'll see "That run has already finished."

On this page