browser-use: who decides what

Read the one-page summary first · The gallery · Make a map of your own system · The repository

browser-use's Agent at its defaults (the step loop, what the model sees, the actions it can take, the browser session's watchdogs, limits and the final judgment), in browser-use/browser-use, read at commit b49a68e4. Separate entry points (the beta agent, the sandbox runner, the MCP server and the command line) are outside this map, and so are Browser Use's cloud browsers and other features that are off until a setting turns them on. Claims about how the code behaves refer to that commit. Their code is MIT-licensed; this page quotes short identifiers and cites files and lines at that commit.

What will it click or submit on its own, and what stops it?

Nothing asks first. The model's chosen actions run as soon as they are parsed, so no person approves a form submit, a purchase or a page script D17. The ordinary built-in actions are offered on every page, evaluate among them, which runs JavaScript the model writes B10.

Some page events are answered by code before the model sees them: it clicks OK on alerts and confirm dialogs the moment they open D7, grants clipboard and notification permissions to every site D8, and saves offered PDFs and attachments without a click D9.

A run normally ends when the model calls done G1. The automatic stops are counts, not judgments: 500 steps F1, or 5 failed steps in a row followed by one last forced answer F5. On Mac and Linux a person at the terminal can pause it with Ctrl+C; on Windows the first press quits D18. An integrator who wants an approval step adds it in their own code, through the step callback D17.

How to read this page

A seat is a judgment the agent makes over and over: what the model sees of the page, which action runs next, may this page be opened, is the task done. Each card records the question, how it is answered, and who or what holds it, with the code that implements or exposes that decision. Seat ids (A1, B12 …) are links.

Each answer describes what browser-use does by default. Exceptions named are the main ones, not a complete list.

The chip on each card has two words. The first is about the question: a RULE has one right answer given its inputs, so a run that answers differently is a bug; a RULING is one that competent people could answer differently. Enforcing a limit someone set is a RULE; choosing what to keep, what to trust or when the work is done is a RULING, even when code holds it.

The second word is who holds it: llm = a language-model call; model = another learned model; code = code in their repository; human = a person; external = code outside the repository that the agent hands the decision to.

Paths are relative to the repository root. A bare :120-140 or (323) means lines in the file named just before it.

Counts

90 seats across 9 stages: A 10, B 12, C 7, D 18, E 17, F 15, G 6, H 1, I 4.

Kind of question · holder	Seats
RULING · code	48
RULE · code	27
RULING · llm	11
RULING · external	3
RULING · human	1

What this edition does not have

There is no list of gaps. No coverage figure: this edition does not say how many of browser-use's model calls and decision points the seats leave out. Each card shows the question, the answer and its holder; for most seats we also recorded what checks the judgment and how it has failed, but those notes are not shown here.

What is left out of this edition

5 of the 90 cards are left out of this edition. The counts above include them.

A. Intake and routing (10 seats)

A1 When an agent is built without a model, which model answers its steps? RULING · code

Answered how: With no model passed and no DEFAULT_LLM environment variable, code builds Browser Use's own hosted model bu-2-0, which needs a BROWSER_USE_API_KEY or construction fails; DEFAULT_LLM can name a different model instead.
Held by: browser_use/agent/service.py:226-236 (llm is None -> CONFIG.DEFAULT_LLM env name via get_llm_by_name, else ChatBrowserUse()); ChatBrowserUse default model bu-2-0 at https://llm.api.browser-use.com (browser_use/llm/browser_use/chat.py:47,97).

A2 Does the agent ask the model for the short memory-plus-action reply (no thinking, no evaluation, no plan), and is planning switched off? RULE · code

Answered how: Code forces flash mode for any model from the browser-use provider, the default included, so the model writes only a memory note and actions and planning is off; with another provider, the developer's flash_mode (default off) decides.
Held by: browser_use/agent/service.py:238-244: llm.provider == 'browser-use' forces flash_mode = True, and flash_mode forces enable_planning = False; the reply schema follows at :786-791 (AgentOutput.type_with_custom_actions_flash_mode, browser_use/agent/views.py:458-473, required ['memory','action']).

A3 Which of the eight standing instruction files does this run's model receive? RULE · code

Answered how: Code picks the file once at construction: the default hosted model, always in flash mode, gets system_prompt_flash.md. Other files go to Claude models in flash mode, to browser-use/ open-weights ids, or when flash is off; override_system_message replaces the file.
Held by: browser_use/agent/prompts.py:60-92 SystemPrompt._load_prompt_template, keyed on is_browser_use_model ('browser-use/' in model, browser_use/agent/service.py:503), is_anthropic (isinstance(llm, ChatAnthropic), browser_use/agent/service.py:498-500), Opus/Haiku 4.5 by name (browser_use/agent/prompts.py:17-25), flash_mode, use_thinking; built once in __init__ (browser_use/agent/service.py:507-518) and sent with cache=True (browser_use/agent/prompts.py:58); not called when override_system_message is set (browser_use/agent/prompts.py:49-50).

A4 When the agent extracts page content, judges its own run or compacts its history, which model does it ask? RULE · code

Answered how: Code sends page extraction, the end-of-run judge and history compaction to the same model object as the main loop unless the developer passes another; a later switch to fallback_llm moves only the main loop.
Held by: browser_use/agent/service.py:254-257 (page_extraction_llm and judge_llm default to the main llm object); compaction settings.compaction_llm or self.settings.page_extraction_llm or self.llm (browser_use/agent/service.py:1169), where settings.page_extraction_llm is always set by __init__ (browser_use/agent/service.py:408), so compaction uses the ORIGINAL main model unless compaction_llm is given (browser_use/agent/views.py:46, default None).

A5 After a rate limit or provider error, does the agent switch to a backup model, and for which errors? RULING · code

Answered how: With no fallback_llm (the default), a provider error fails the step. With one set, code moves the rest of the run to it on the first rate limit, truncated reply, or 401/402/429/500/502/503/504 error; others, such as a 400, do not switch.
Held by: browser_use/agent/service.py:1973-1979 (catch in get_model_output) and _try_switch_to_fallback_llm :1981-2023: switch once, on ModelRateLimitError, ModelOutputTruncatedError, or status in {401, 402, 429, 500, 502, 503, 504} (:2001-2004), only if fallback_llm is set (:2010-2012); sticky for the rest of the run (:2017-2018).

A6 Does the agent navigate somewhere before asking the model anything, and which URL does it pick from the task text? RULING · code

Answered how: By default (directly_open_url=True) code opens the one web address it finds in the task text before the model's first turn, and tells the model; with none, several, a file-like name, or a negation such as "never go to" just before it, nothing opens.
Held by: browser_use/agent/service.py:463-467 (when directly_open_url is True, the default at :200, with no follow-up and no initial_actions) -> _extract_start_url browser_use/agent/service.py:2307-2454: strip e-mail addresses (:2313), match two URL patterns (:2316-2319), drop placeholders (:2410), drop any candidate containing a listed file extension anywhere (.{ext} substring test, list :2323-2392, test :2418-2423) or .htm without a scheme (:2424-2425), drop URLs negated within 20 characters before them (:2431-2436), add https:// (:2439-2440), and navigate only when exactly ONE unique URL remains (:2444-2452). Executed with no model call by _execute_initial_actions (:3309-3317) under step_timeout (:2599-2613); integrator initial_actions run the same way instead.

A7 When a structured result type is declared, what is added to the task the model reads? RULE · code

Answered how: With no output schema (the default), the task is unchanged; with one set on the agent or its tools, code appends Expected output format: and the full JSON schema to the task text the model reads every step.
Held by: browser_use/agent/service.py:351-365 (agent output_model_schema wins over the Tools one, with a warning) and _enhance_task_with_schema :612-628 (appends Expected output format: <Name> and the full JSON schema to the task text, :623-624); the done action is swapped for a structured one (:364-365, G5).

A8 When the developer passes passwords or keys for the agent to type, does the run proceed if the browser may visit any site, or if a secret's site is not on the allowed list? RULING · code

Answered how: Code accepts the secrets in every case and only logs a warning: when allowed_domains is unset (the default) or when a per-site secret's domain is not covered by it. The run is never stopped.
Held by: browser_use/agent/service.py:534-581: no allowed_domains -> a warning ("your sensitive_data may be exposed", :539-544); per-domain secrets not covered by allowed_domains -> a warning (:577-581); the run proceeds in both cases (no raise, no flag).

A9 When the developer adds a new task to the same agent, how does the model see the old and the new one? RULE · code

Answered how: When the developer calls add_new_task, code wraps the first task as <initial_user_request> and appends the new one as <follow_up_user_request>; the agent keeps its history, browser settings and step count, and nothing re-opens a page (a finished run may already have closed the browser).
Held by: browser_use/agent/service.py:998-1012 add_new_task (sets follow_up_task, clears stop/pause, new event bus) and browser_use/agent/message_manager/service.py:191-197 (wraps as <initial_user_request> + <follow_up_user_request> and adds a history item); initial actions and URL auto-open are skipped for follow-ups (browser_use/agent/service.py:463, :3311).

A10 From the model's name, does the agent turn screenshots off, turn coordinate clicking on, or shrink screenshots? RULING · code

Answered how: Code matches the model's name: the default bu-2-0 matches none, so it gets no coordinate clicking and full-size screenshots. Names containing e.g. claude-sonnet-4 turn coordinate clicking on, Sonnet screenshots are shrunk, and DeepSeek or Grok-3 names lose vision.
Held by: browser_use/agent/service.py:327-334 coordinate clicking on when the lower-cased model name contains claude-sonnet-4, claude-opus-4, claude-fable-5, gemini-3-pro or browser-use/; screenshots resized to 1400x850 when the name after any provider/ prefix starts claude-sonnet and no llm_screenshot_size was given (browser_use/agent/service.py:246-252); vision forced off for names containing deepseek (browser_use/agent/service.py:476-479) or grok-3 / grok-code (browser_use/agent/service.py:481-486).

B. Context: what the model sees (12 seats)

B1 Which sections make up the message the model reads every step, and in what order? RULE · code

Answered how: Code builds one fresh message each step: the task, the run's history notes, files and plan, the page as a numbered element list, any one-step read results, then step number and date last; a screenshot follows when vision applies (B7).
Held by: browser_use/agent/prompts.py:404-506 AgentMessagePrompt.get_user_message: <user_request> (:361-362) -> <agent_history> -> <agent_state> (file system, todo.md, plan, sensitive-data placeholders, available file paths, :337-359) -> <browser_state> (:224-335) -> <read_state> if any -> <page_specific_actions> if any -> unavailable-skills text if any -> <step_info> (step N of max, today's date, :364-373) last for caching; then sample images, screenshot(s), read-file images, only when a screenshot or read-file image is present (:444-504). Context messages are appended after it (browser_use/agent/message_manager/service.py:570-574).

B2 From the live page, which elements appear in the model's view, and which get a number it can click or type into? RULING · code

Answered how: Code shows the model visible text and controls within about 1,000 px above and below the current view, and numbers only the elements its heuristics judge interactive (click listeners, form controls and the like); scripts, styles and SVG innards are dropped.
Held by: browser_use/dom/serializer/serializer.py:455-560 _create_simplified_tree (drops style, script, head, meta, link, title, browser_use/dom/serializer/serializer.py:18, and SVG children, :485-486; keeps visible, scrollable or shadow-hosting nodes; forces "visible" for aria-*/pseudo attributes and hidden file inputs, :519-531), visibility from browser_use/dom/service.py:252-343 (CSS display/visibility/opacity, then inside the viewport plus viewport_threshold=1000 px vertically, browser_use/dom/service.py:64, never set from the profile, browser_use/browser/watchdogs/dom_watchdog.py:558-565), interactivity from browser_use/dom/serializer/clickable_elements.py:6-246 ClickableElementDetector.is_interactive (JS click listeners, form controls, roles, search-class hints, iframes larger than 100x100 px), numbering in _assign_interactive_indices_and_mark_new_nodes (browser_use/dom/serializer/serializer.py:660-770; scrollable containers get a number only if they have no interactive child, unless they look like a dropdown, :716-746; new-since-last-step elements get *, :759-766).

B3 Does the model lose sight of text or controls painted over by another element, or nested inside a larger button or link? RULING · code

Answered how: By default code drops text painted under another element (an open modal or dropdown) and unnumbers covered controls; a child 99% inside a link or button loses its own number but keeps its text, unless it is a form field or labelled control.
Held by: paint order: browser_use/dom/serializer/serializer.py:135-136 PaintOrderRemover.calculate_paint_order (browser_use/dom/serializer/paint_order.py:165-219: a node fully inside the union of opaque rects painted above it is marked ignored_by_paint_order), on by BrowserProfile.paint_order_filtering=True (browser_use/browser/profile.py:704, "Slightly experimental"); covered text is skipped (browser_use/dom/serializer/serializer.py:1163-1175) and covered elements get no number (browser_use/dom/serializer/serializer.py:666). Containment: _apply_bounding_box_filtering (browser_use/dom/serializer/serializer.py:147-153, :772-881; on by default, browser_use/dom/service.py:1140-1141 passes no enable_bbox_filtering) marks a child 99% inside a propagating parent (a, button, button/combobox roles, :46-58) unless it is a form control, itself propagating, has onclick, an aria-label or an interactive role (:854-878); a marked element gets no line and no number, but its children, text included, are still rendered (:994-1001).

B4 Does an attribute in the page's own markup remove an element from what the model sees? RULE · code

Answered how: Code drops any element, with everything inside it, whose data-browser-use-exclude attribute (or the per-session variant the demo panel sets) is true; any website can set the plain one, and the element is then missing from the element list the model reads (a screenshot can still show it).
Held by: browser_use/dom/serializer/serializer.py:489-501: any element whose data-browser-use-exclude-<session> OR legacy data-browser-use-exclude attribute is "true" is dropped with its whole subtree.

B5 Is content inside iframes (including frames from other websites) and inside web components shown to the model? RULING · code

Answered how: By default code reads frames from other websites into the model's view alongside the page (up to 100 frame documents; cross-origin frames five levels deep, visible and at least 10x10 px); web-component (shadow) content is always included.
Held by: browser_use/browser/profile.py:675-678 cross_origin_iframes=True (DomService's own default is False, browser_use/dom/service.py:60, but the watchdog passes the profile value, browser_use/browser/watchdogs/dom_watchdog.py:561), max_iframes=100 (browser_use/browser/profile.py:679-681, snapshot documents cut at browser_use/dom/service.py:669-674), max_iframe_depth=5 (browser_use/browser/profile.py:683-686, browser_use/dom/service.py:981-984), cross-origin frames only when the frame element is visible and at least 10x10 px (browser_use/dom/service.py:37-42, :989-999); frame documents are walked (browser_use/dom/serializer/serializer.py:503-510); shadow roots always included, closed ones labelled |SHADOW(closed)| (browser_use/dom/serializer/serializer.py:467-477, :1018-1026); hidden-below-the-fold hint inside frames (:1184-1199).

B6 When the page's element list is long, where is it cut, and what is the model told? RULING · code

Answered how: Code gives the model the first 40,000 characters of the element list (the max_clickable_elements_length default), cut mid-element, and says so; the rest is not in that list, though scrolling, extraction or the screenshot can show it. Each attribute value is cut to 100 characters.
Held by: browser_use/agent/prompts.py:252-258: the serialized element text is cut at max_clickable_elements_length characters (default 40000, browser_use/agent/service.py:211), mid-element, and the header says "(truncated to 40000 characters)" (browser_use/agent/prompts.py:332); each attribute value capped at 100 characters (browser_use/dom/serializer/serializer.py:1398) and interactive images' alt/title/label/src at 100 (browser_use/dom/serializer/serializer.py:957,961); first 4 <select> options at 30 characters (browser_use/dom/serializer/serializer.py:407-412); scroll position as "pages above/below" (browser_use/agent/prompts.py:264-275).

B7 Is a picture of the page attached to this step's message? RULE · code

Answered how: With use_vision=True (the default) code attaches a fresh screenshot to every step on a web page; with 'auto' only after an action asks, with False never. Blank new-tab pages, non-web pages, DeepSeek or Grok-3 models and failed captures get none.
Held by: browser_use/agent/message_manager/service.py:461-478: use_vision=True (default, browser_use/agent/service.py:170) always; 'auto' only after an action result asks (include_screenshot metadata, browser_use/agent/message_manager/service.py:454-459; the screenshot action exists only in 'auto', browser_use/agent/service.py:319-325); False never. Overrides: new-tab pages never (browser_use/agent/prompts.py:406-408); non-http(s) pages capture none (browser_use/browser/watchdogs/dom_watchdog.py:266, :319); placeholder screenshots dropped (browser_use/agent/prompts.py:442); DeepSeek and some Grok models forced off (A10); resized when llm_screenshot_size is set (browser_use/agent/prompts.py:375-401); vision_detail_level='auto' (browser_use/agent/service.py:197). A screenshot is captured every step even with vision off (browser_use/agent/service.py:1097-1100).

B8 Of the steps already taken, which ones does the model see, and in what form? RULING · code

Answered how: By default (max_history_items=None) code shows the model a short record of every earlier step; compaction, also on by default, swaps older steps for a summary once 25 steps pass and the record exceeds 40,000 characters. Long errors keep only their ends.
Held by: browser_use/agent/message_manager/service.py:152-189 agent_history_description: by default (max_history_items=None, browser_use/agent/service.py:183) EVERY step's note and result; with a limit, the first item, an "N previous steps omitted" line and the last N-1 (:176-187); a compacted summary is prefixed as <compacted_memory> with "Treat as unverified context" (:155-163). Each step's errors are cut to first and last 100 characters when over 200 (:342-347); results capped at 60,000 characters (:350-367).

B9 When an action returns text, images or an error, does the model see it once, keep seeing it, or not see it? RULE · code

Answered how: Code shows content flagged show-once (such as long page extracts) only on the next step, keeps each action's short memory line in the history for the run, and keeps errors there too, cut to their first and last 100 characters.
Held by: browser_use/agent/message_manager/service.py:304-367 _update_agent_history_description: results flagged include_extracted_content_only_once go to <read_state> for the NEXT step only (:323-328, cleared :316-317); long_term_memory, else extracted_content, goes into the permanent history line (:335-340); images ride along once (:331-333); a step with no parsed output becomes "Agent failed to output in the right format." (:377-380).

B10 Which tools is the model offered this step, and how are they described to it? RULE · code

Answered how: Each step code offers every built-in action on every page, including evaluate (runs any page JavaScript; screenshot is dropped at the default use_vision=True), except any the integrator excluded, plus custom actions whose site filter matches; on the last step or after repeated failures only done is offered.
Held by: browser_use/agent/service.py:4049-4068 _update_action_models_for_page rebuilds the reply schema every step from Registry.create_action_model(page_url=...) (browser_use/tools/registry/service.py:519-605: actions with no domain filter plus those whose domains match the URL; a restricted action is hidden on an empty URL, browser_use/tools/registry/views.py:110-118); domain-limited actions are also listed in <page_specific_actions> (browser_use/agent/service.py:1117, browser_use/tools/registry/views.py:128-154); actions named in exclude_actions are never registered (browser_use/tools/registry/service.py:307-309); screenshot removed unless use_vision='auto' (browser_use/agent/service.py:319-325); coordinate click per A10; on the last step, or after max_failures with final_response_after_failure=True (the default), the schema is done only (browser_use/agent/service.py:1569-1591, F3, F6).

B12 When building the page view fails, does the model learn that, or does it see an empty page? RULE · code

Answered how: If reading the page times out, code tells the model so and that no element numbers are safe; if reading the page throws instead, the model is shown an empty element list and told to consider waiting, with nothing saying the read failed (a screenshot may still show the page).
Held by: two paths with different answers. Timeout: browser_use/browser/session.py:1649-1694 sets state_error ("no element indices are safe to use"), clears the cached selector maps (:1659-1661), and the prompt shows it as <browser_state_error> (browser_use/agent/prompts.py:301-303). Exception: browser_use/browser/watchdogs/dom_watchdog.py:394-400 (DOM build raised: empty DOM, browser_errors=[], the rest of the state built normally, :488-506) and :515-547 (any other failure: empty DOM, title 'Error', browser_errors=[str(e)]) set NO state_error; nothing reads browser_errors (Grep browser_errors over browser_use/: 5 hits, all writes or the field at browser_use/browser/views.py:106), so the model sees Page appears empty - consider waiting and empty page (browser_use/agent/prompts.py:230-231, :281-282).

B13 Does the agent tell the model that the page looks empty, still loading, scrollable, a PDF, or that a dialog was closed for it? RULING · code

Answered how: Code adds short status hints above the element list: "appears empty" under 10 elements, "may still be loading" when requests are pending and text is sparse, pages above and below, a PDF warning judged by the address, and dialogs it auto-closed.
Held by: browser_use/agent/prompts.py:224-335: fewer than 10 elements -> "Page appears empty - consider waiting" (:230-231); pending requests and more than 20 elements with under 5 text characters per element -> "page may still be loading" (:233-242); pages above/below and "scroll down" past 0.2 pages (:264-275); PDF viewer by URL suffix or /pdf/ (browser_use/browser/watchdogs/dom_watchdog.py:471) -> "DO NOT use the extract action" (prompts.py:306-313); auto-closed JavaScript dialogs listed (:321-326); current tab marked only when exactly one tab matches URL and title (:287-299). Non-http(s) pages skip the DOM build and screenshot (dom_watchdog.py:266, :311-356).

C. Choosing the next action (7 seats)

C1 Given the task, the page and its history, what should the agent do next (which actions, with which arguments)? RULING · llm

Answered how: Every step, the hosted Browser Use model (default bu-2-0) reads the task, the page state and the history and returns up to five actions as JSON; code runs them in order. Another model is used only if the integrator passes one.
Held by: browser_use/agent/service.py:1953 (get_model_output): the agent's llm, by default ChatBrowserUse() model bu-2-0 on the hosted gateway (browser_use/agent/service.py:226-236 unless config DEFAULT_LLM names another; browser_use/llm/browser_use/chat.py:47), called once per step via _get_model_output_with_retry (browser_use/agent/service.py:1668-1702) under asyncio.wait_for(..., llm_timeout) (browser_use/agent/service.py:1185-1187).

C2 What structured action list does the model's reply mean? RULE · code

Answered how: The Browser Use gateway returns the reply as structured data, and code validates it against the actions available on the current page; a reply that does not fit fails the step.
Held by: browser_use/llm/browser_use/chat.py:228-248 (default path): the gateway returns completion as a dict; action dicts are validated one by one into the Union ActionModel, then the whole reply via output_format.model_validate(completion_data). Other providers: provider-side JSON-schema mode plus output_format.model_validate_json(...) (e.g. browser_use/llm/openai/chat.py:225-233, :253-258, :301). Schema prepared by SchemaOptimizer.create_optimized_json_schema (browser_use/llm/schema.py:12, adds additionalProperties: false :128-152).

C3 When Claude returns a malformed tool call, what output did it mean to give? RULING · code

Answered how: Not on the default path: only when the integrator uses ChatAnthropic, code tries several repairs of a malformed tool call (decoding stringified fields, parsing a call written into text) before failing the step.
Held by: browser_use/llm/anthropic/chat.py:470-509: validate tool input; on failure decode double-serialized string fields (_repair_serialized_fields :287-299); then recover a whole call written into the thinking string as <parameter> markup or JSON (_completion_from_serialized_tool_input :306-350); with auto tool choice, parse JSON out of plain text (_completion_from_text_response :352-372, :503-506).

C4 When the model returns no action at all, what happens? RULING · code

Answered how: Code asks the model once more; if the reply is still empty it tries to insert a failed done, but by the code and upstream #5360 that fallback raises instead, so the step fails and counts as a failure.
Held by: browser_use/agent/service.py:1675-1700 (_get_model_output_with_retry): re-asks once with "You forgot to return an action..." (:1682-1687); if still empty, intends to insert done(success=False, text='No next action returned by LLM!') (:1690-1700).

C5 If the model asks for more actions than the per-step limit, which ones run? RULE · code

Answered how: Only the first five actions of a reply run (the max_actions_per_step default); the rest are dropped and the model is not told which.
Held by: browser_use/agent/service.py:1960-1962: parsed.action = parsed.action[: max_actions_per_step].

C6 Which real URL does a shortened URL in the model's reply stand for? RULE · code

Answered how: Before each model call, code shortens long URL query strings in user and assistant messages; afterwards it puts the full URL back into every string of the model's reply. A URL the model alters slightly is not restored.
Held by: shortening before the call browser_use/agent/service.py:1791-1865 (query+fragment over _url_shortening_limit=25 chars cut and suffixed with a 7-char md5, :1818-1832); restoration in every string of the parsed reply :1956-1958, :1867-1937.

C7 Does the model's plan update change the plan the agent tracks? RULE · code

Answered how: Not at defaults: the default Browser Use model forces flash mode, which turns planning off. With planning on, code replaces the plan when the model sends a new one and moves the current item when the model names one.
Held by: browser_use/agent/service.py:1418-1451 (_update_plan_from_model_output): plan_update replaces the plan; current_plan_item is clamped into range and earlier items are marked done.

D. Gatekeeping: may this action run? (18 seats)

D2 After a tab lands on a page or a new tab opens, does that page pass the domain policy, and if not, is it sent away? RULE · code

Answered how: With no domain lists set (the default), a page with a host name is never sent away; one with no host name, such as a file: page, can be. With lists set, a page reached by redirect or by the page itself is caught only at the next browser-state read, after it has loaded and run, and its tab is sent to about:blank.
Held by: browser_use/browser/watchdogs/security_watchdog.py:50-71 (on_NavigationCompleteEvent: navigate the tab to about:blank) and :73-92 (on_TabCreatedEvent: close the tab), both via _is_url_allowed :176. The current-URL re-check fires from browser_use/browser/watchdogs/downloads_watchdog.py:219-249, which on each browser-state request dispatches (without awaiting, :241) a NavigationCompleteEvent carrying the current page URL.

D5 Which secret values are hidden from the model, and on which channels? RULE · code

Answered how: With no sensitive_data (the default) there is nothing to hide. When secrets are given, the model sees placeholder names, and real values are scrubbed from the per-step state message; extra context messages and screenshots, sent every step by default, are not scrubbed.
Held by: placeholder list shown to the model browser_use/agent/message_manager/service.py:391-418 (only names for the current page's domain, :399-406); value redaction on the per-step state message :556-566 via _filter_sensitive_data :576-600 (browser_use/utils.py redact_sensitive_string); context messages are NOT filtered :570-574; history saved to file filters only input actions browser_use/agent/views.py:545-556.

D7 When a page opens a JavaScript dialog, who decides OK or Cancel? RULING · code

Answered how: Code clicks OK on every alert, confirm and leave-page dialog the moment it opens, and Cancel on every text prompt, before the model sees it; the model is told afterwards through the next browser state.
Held by: browser_use/browser/watchdogs/popups_watchdog.py:62-123; the choice is should_accept = dialog_type in ('alert', 'confirm', 'beforeunload') at :79 (prompt → Cancel). The dialog text is queued for the next browser state :68-72.

D8 Which browser permissions (clipboard, notifications, camera...) are granted to the sites the agent visits? RULING · code

Answered how: When the browser connects, code grants clipboard read/write and notifications to every site (the default permissions list); any other permission a page asks for is left to Chrome.
Held by: browser_use/browser/watchdogs/permissions_watchdog.py:23-43 (Browser.grantPermissions with no origin, i.e. every origin, :35-39); default list browser_use/browser/profile.py:363-368 (clipboardReadWrite, notifications).

D9 When a page offers a file (PDF, attachment, download link), is it saved to disk? RULING · code

Answered how: Yes. Downloads are always allowed, into a fresh temp folder, and with auto_download_pdfs on (the default) PDFs and attachment responses are also fetched and saved without a click. Setting accept_downloads=False changes nothing.
Held by: browser_use/browser/watchdogs/downloads_watchdog.py:500-513 (Browser.setDownloadBehavior behavior: 'allow', into downloads_path, by default a fresh temp folder browser_use/browser/profile.py:475-491); when auto_download_pdfs (default True, browser_use/browser/profile.py:712; checked browser_use/browser/watchdogs/downloads_watchdog.py:286-288,547-549): network auto-download :557-712 with the rule _should_auto_download_network_response :91-104 (PDF → yes; attachment → yes unless a generic text name :77-88), and PDF-viewer auto-download :277-301. Downloads are then added to the agent's uploadable/readable files browser_use/agent/service.py:658-693.

D10 Is the file name a page supplies safe to write, and does the path stay inside the downloads folder? RULE · code

Answered how: When the library itself saves a file, code cuts the page-supplied name down to a bare name and refuses the write if its real path would leave the downloads folder. Files Chrome downloads on its own are named by Chrome (not traced).
Held by: browser_use/browser/watchdogs/downloads_watchdog.py:1482-1493 (_sanitize_download_filename, called at :317, :770, :956, :1051) and :1495-1500 (_is_path_contained, applied at :838-840, :1063, :1411).

D11 When the model asks to upload a file, is that file one the agent may hand to a website? RULE · code

Answered how: Code allows only files the integrator listed, files downloaded in this run, or files the agent itself wrote, and does not check which site receives the upload; on a remote browser any path passes.
Held by: browser_use/tools/service.py:864-910: allowed if in available_file_paths (:869), or a file downloaded this session (:871-872), or a FileSystem-managed file resolved by basename with a realpath containment check (:879-899, GHSA-j9hj-92j8-jv9h cited at :890); on a remote browser any path passes (:905-907).

D12 When the model reads or writes a file, which names and places are allowed? RULE · code

Answered how: The agent may write only plainly named files with a supported extension, in its own folder in a new temporary directory that is emptied at start (a bad name is auto-corrected or refused), and may read an outside file only when its exact path is in available_file_paths, which includes this run's downloads.
Held by: browser_use/filesystem/file_system.py:562-576 (_is_valid_filename: basename, character class, extension allow-list), :606-625 (_resolve_filename: basename first); outside files only when listed in available_file_paths browser_use/tools/service.py:1798-1802.

D13 When a site puts up a CAPTCHA, who solves it, and how long does the agent wait? RULING · external

Answered how: On a local browser (the default) nobody solves a CAPTCHA and the agent does not wait. On Browser Use cloud browsers their service solves it and the agent pauses at the start of its next step, up to 120 seconds.
Held by: browser_use/browser/watchdogs/captcha_watchdog.py:80-155 (registers for the cloud browser's BrowserUse.captchaSolverStarted/Finished CDP events; the solver itself is outside the repo) and browser_use/browser/watchdogs/captcha_watchdog.py:170-207 (the wait); the agent waits at the start of each step browser_use/agent/service.py:1047-1062; watchdog attached when captcha_solver (default True, browser_use/browser/profile.py:652-655) browser_use/browser/session.py:1848-1852.

D14 When a site asks for cookie consent, who answers it? RULING · external

Answered how: By default code downloads a third-party extension ("I still don't care about cookies"), caches it, and loads it through the browser's launch arguments to dismiss cookie banners, with two sites configured as exceptions; if that setup fails the browser starts without it; with enable_default_extensions=False the banner is just part of the page and the model handles it.
Held by: the third-party extension "I still don't care about cookies" (browser_use/browser/profile.py:1036-1040), downloaded and loaded when enable_default_extensions (default True, :648-651, env override :19-25) via :939,987-1001; exempt sites cookie_whitelist_domains (default nature.com, qatarairways.com, :660-663) patched in at :1106-1109,1118-1139.

D15 Which of the browser's built-in protections and prompts does the agent's browser run with? RULING · code

Answered how: When browser-use launches Chrome, it keeps normal web security and certificate checks but turns off popup blocking, the form-resubmit prompt and client-side phishing detection; disable_security=True also removes web security and certificate checks.
Held by: browser_use/browser/profile.py:162-230 (CHROME_DEFAULT_ARGS: --disable-popup-blocking :181, --disable-prompt-on-repost :182, --disable-client-side-phishing-detection :170, --safebrowsing-disable-auto-update :220, --disable-external-intent-requests :221); disable_security (default False :626) adds CHROME_DISABLE_SECURITY_ARGS (:141-149: web security off, certificate errors ignored) at :927; assembled in get_args :906-985.

D16 Is the browser started with Chromium's process sandbox on? RULING · code

Answered how: Chrome's sandbox is on unless the library guesses it is inside a container (for example, PID 1's command line contains py, uv or app); then it is off, and chromium_sandbox=True cannot turn it back on.
Held by: browser_use/browser/profile.py:447-449 (chromium_sandbox default not CONFIG.IN_DOCKER) and :925 (--no-sandbox etc. when CONFIG.IN_DOCKER or not self.chromium_sandbox); Docker guess browser_use/config.py:20-44 (/.dockerenv, cgroup, PID 1 command line containing py, uv or app :31-33, fewer than 10 processes :39-40); env IN_DOCKER=true can force the guess on, never off (browser_use/config.py:175-176).

D17 Before the agent submits, buys, sends, deletes or runs page JavaScript, does anyone approve it? RULING · code

Answered how: Nobody is asked: the model's chosen actions run as soon as they are parsed. An integrator can add a gate in code through register_new_step_callback, which sees the proposed actions before they run.
Held by: nobody is asked by default: the step goes from model output straight to execution, browser_use/agent/service.py:1201-1217 (_get_next_action → _execute_actions → multi_act), and multi_act runs each action after only a stop/pause check (browser_use/agent/service.py:2797-2815). No ask-a-person action is registered in Tools (Grep (?i)ask_human|ask_user|approval|approve|confirm_action over browser_use/ → 3 files, none on the default path: integrations/openai/browser_use.py, integrations/anthropic/browser_use.py, skills/browser-use/SKILL.md).

D18 While the agent runs, may a person at the terminal halt it, and how does it resume? RULING · human

Answered how: On Mac and Linux a person at the terminal can press Ctrl+C to pause the run and press it again to quit; on Windows the first press quits. A pause is checked between steps and between queued actions. Pressing Enter does not resume, despite the printed hint: only code calling agent.resume() does.
Held by: the person running the script, by Ctrl+C: browser_use/utils.py:324-360 (first Ctrl+C cancels the step and calls pause), :362-377 (SIGTERM exits), registered by browser_use/agent/service.py:2547-2555 when enable_signal_handler (default True, :213); Agent.pause prints "Press [Enter] to resume or [Ctrl+C] again to quit" browser_use/agent/service.py:3942-3946; the loop waits on _external_pause_event :2624-2627.

D19 Is this element one the click action may press, or must it be handled another way? RULE · code

Answered how: Code refuses clicks on file-upload inputs and <select> dropdowns and tells the model which action to use instead; an element whose onclick mentions print is saved as a PDF instead of opening the print dialog. Everything else may be clicked.
Held by: browser_use/browser/watchdogs/default_action_watchdog.py:712-725 (<select> and file inputs refused, the model told which action to use), also :352-355 (file input, index click) and :417-428 (file input and <select>, coordinate click; skipped when force=True :400-404); elements whose onclick contains "print" (:225-237) are saved as a PDF instead :357-369,430-439.

E. Execution and observation (17 seats)

E2 Did this action succeed or fail, as reported back to the loop and the model? RULING · code

Answered how: Each action handler decides what counts as an error. Several failures, such as a missing element for click or input and every file-system failure, come back as ordinary text, so the loop treats them as successes.
Held by: browser_use/tools/service.py:714-718 (click: element-not-found returned as extracted_content, not error) and each other handler's choice between ActionResult(error=...) and ActionResult(extracted_content=...) in browser_use/tools/service.py. Element-not-found is NOT an error for input (:785-789), dropdown_options (:1686-1690), select_dropdown (:1714-1718); IS an error for scroll (:1389-1393) and upload (:924-926). find_text not found is not an error (:1515-1522); a multi-page scroll whose every attempt failed still reports success (:1441-1465); close tab always succeeds (:1058-1065); every file-system failure comes back as plain text (browser_use/filesystem/file_system.py:883, :904-907, :921, :952) wrapped as extracted_content (browser_use/tools/service.py:1785, :1793).

E4 After each action, should the rest of the queued actions still run? RULING · code

Answered how: Code runs a reply's actions in order and stops the queue at an error, a done, a navigation, search, back, tab switch or evaluate, or any URL or focused-tab change. A page change without a URL change does not stop it.
Held by: browser_use/agent/service.py:2755-2873 (multi_act): done after the first action stops the queue (:2785-2790); stop after done, error or last action (:2834-2835); stop after an action flagged terminates_sequence (navigate, search, go_back, switch, evaluate: browser_use/tools/service.py:461, :505, :583, :1007, :1835; check browser_use/agent/service.py:2840-2845); stop when the URL or focused tab changed (:2847-2853); wait_between_actions 0.1 s between actions (:2793-2795, browser_use/browser/profile.py:694).

E5 How is a click carried out, and what is clicked instead when the target is special? RULING · code

Answered how: Code clicks the centre of the element the model named by its number, using a JavaScript click when something covers it. It refuses file inputs and turns a click on a <select> or a print button into a different action.
Held by: index → node from the cached selector map of the last observation (browser_use/browser/session.py:2501-2516, via the alias :2544-2546); _click_by_index browser_use/tools/service.py:704-765 (index 0 refused by assert :709-711; a <select> is turned into a dropdown_options read :740-748); on_ClickElementEvent browser_use/browser/watchdogs/default_action_watchdog.py:338-388 (file input refused :352-355; a "print" button is replaced by a PDF save :357-369); _click_element_node_impl :703-1060 (select refused :718, occlusion check :877, JS click fallback :880-902, checkbox JS fallback :975). Coordinate clicks are enabled only for model names matching claude-sonnet-4, claude-opus-4, claude-fable-5, gemini-3-pro or browser-use/ (browser_use/agent/service.py:328-334), so NOT for the default bu-2-0; when on they dispatch force=True (browser_use/tools/service.py:678-680), which skips the file-input, select and print checks (default_action_watchdog.py:399-404).

E7 When a click opens a new tab, does the agent move to it? RULING · code

Answered how: Yes: 50 ms after a click, code switches to the first new tab it finds and tells the model so. A tab that opens later than that is not caught here.
Held by: browser_use/tools/service.py:629-654 (_detect_new_tab_opened): after 50 ms, any new tab is switched to automatically (first new one).

E8 Has the page finished loading after a navigation? RULING · code

Answered how: Code waits up to 3 seconds (same site) or 8 seconds (other site) for a load or network-idle signal, then carries on either way; the model is told it navigated, not that the wait ran out.
Held by: browser_use/browser/session.py:1034-1155 (_navigate_and_wait): Page.navigate capped at nav_timeout, which the caller sets from NavigateToUrlEvent.event_timeout (browser_use/browser/session.py:990-996), default 30 s (TIMEOUT_NavigateToUrlEvent, browser_use/browser/events.py:122; the 20 s at browser_use/browser/session.py:1070-1071 applies only when none is passed); then wait for load or networkIdle lifecycle events (default wait_until='load', browser_use/browser/events.py:114) for 3 s on the same domain, 8 s otherwise (browser_use/browser/session.py:1056-1064, :1110-1145); same-document navigations return at once (:1098-1101).

E9 Did the page come back empty, and should it be reloaded? RULING · code

Answered how: After navigating in the current tab, code checks whether the page looks empty, waits 3 seconds, then reloads once; it reports an error only if the page still has no document root.
Held by: browser_use/tools/service.py:514-544 (inside navigate): if an http(s) page has no DOM root or an empty LLM view, wait 3 s and recheck; still empty → navigate to the same URL again, wait 5 s; error only if the DOM root is still None (:539-544).

E10 Into which file input does an upload go? RULING · code

Answered how: Code uploads into the file input next to the element the model named or, if there is none, into the file input nearest the current scroll position; the model is told it uploaded to the index it named.
Held by: browser_use/tools/service.py:928-985: a file input "near" the element the model named (browser_session.find_file_input_near_element, browser_use/browser/session.py:2717); else the file input in the selector map closest to the current scroll position (browser_use/tools/service.py:941-980). Which files may be uploaded is D11 (:869-920).

E11 What does the current page say in answer to the model's extraction query? RULING · llm

Answered how: When the agent calls extract, the same Browser Use model reads the page as text (up to 100,000 characters per call) and answers the query in its own words; nothing checks the answer against the page.
Held by: browser_use/agent/service.py:254-255 (page_extraction_llm defaults to the agent's llm), called at browser_use/tools/service.py:1275-1278 (free text) or :1196-1202 (structured, when the model passes output_schema or the integrator sets extraction_schema), each under a 120 s cap.

E12 Which part of the page does the extraction model get to read? RULING · code

Answered how: Code converts the page to markdown, strips scripts, styles and JSON-like blobs, and hands the extraction model a first chunk of about 100,000 characters with a note on how to ask for the next.
Held by: browser_use/dom/markdown_extractor.py:113-191 (markdownify with script/style stripped; regex removal of JSON-like blobs, including any inline code span that starts with { :163-165; long lines that parse as JSON dropped :177-184); chunk_markdown_by_structure :410 with max_chunk_chars=100000 (browser_use/tools/service.py:1079, :1125); image URLs auto-included when the query contains words like "image" or "photo" (:1090-1092).

E13 How much of an action's output stays in the model's memory, and for how long? RULING · code

Answered how: Each action decides: large extract or evaluate output (evaluate is first cut at 20,000 characters), file reads and dropdown lists are shown once, then kept only as a short note or a saved file the agent can read back.
Held by: browser_use/tools/service.py:1222-1229 (extract, structured path; free-text path :1286-1293): output over 10,000 chars is saved to extracted_content_N.md and shown once; evaluate output is cut at 20,000 chars and over 10,000 shown once (:1913-1927); read_file keeps about 1,000 chars in memory and shows the full text once (:1807-1831); dropdown options shown once (:1701-1705). The flags long_term_memory / include_extracted_content_only_once are defined at browser_use/agent/views.py:327-332; how the message manager uses them is B9.

E14 Should the model's JavaScript be rewritten before it runs? RULING · code

Answered how: Before running the model's JavaScript, code rewrites some quoting and escaping patterns it treats as mistakes, across the whole code, so a legitimately escaped quote inside a string is changed too.
Held by: browser_use/tools/service.py:1943-2006 (_validate_and_fix_javascript): unescape \" everywhere (:1949), collapse double-escaped regex escapes (:1953-1954), turn double-quoted arguments of document.evaluate, querySelector(All), .closest, .matches into template literals (:1957-1991); applied at :1844, run with Runtime.evaluate (:1847-1850).

E16 How is a dropdown option chosen, and what if the page undoes it? RULING · code

Answered how: Code picks the option whose text or value matches the model's text, ignoring case; if the page reverts it, it tries clicking the option. A failed selection usually comes back as text, not as an error.
Held by: browser_use/tools/service.py:1711-1750 → on_SelectDropdownOptionEvent browser_use/browser/watchdogs/default_action_watchdog.py:3300 (native <select>: case-insensitive exact match on trimmed option text or on value :3332-3341; if the page framework reverts the selection, a click fallback :3608-3690); options read by on_GetDropdownOptionsEvent :2824, ARIA combobox :3094.

E18 Should a failed model request be retried, and how many times? RULING · code

Answered how: The Browser Use client makes up to five attempts on server errors, rate limits and network failures, with growing pauses, all inside the agent's 75-second limit for each model call.
Held by: default path: browser_use/llm/browser_use/chat.py:176-225: up to max_retries=5 attempts (:51) on 429/500/502/503/504 (:28) and on httpx timeouts/connect errors, exponential backoff from 1 s capped at 60 s with 10% jitter (:186-193). Other providers hand the retry to the vendor SDK (max_retries passed to the client, e.g. browser_use/llm/openai/chat.py:59, :94; browser_use/llm/anthropic/chat.py:60, :78).

E19 After a step raises an error, does the run go on, wait for the browser, or stop? RULING · code

Answered how: The error becomes the next step's input for the model and counts as a failure; the run continues. A closed browser ends the run, and a connection being rebuilt is waited for first.
Held by: browser_use/agent/service.py:1259-1315 (_handle_step_error): an interruption is not a failure (:1263-1267); a connection-like error (string match, :1317-1331) while a reconnect is in progress waits up to RECONNECT_WAIT_TIMEOUT, and if the connection came back records "Connection lost and recovered" as the result without counting a failure and the run moves on (:1270-1286); a closed browser stops the run (:1289-1293, :1333-1355); anything else, including a connection-like error that did not recover and is not a closed browser, becomes the next step's error message and counts (:1296-1314).

E20 When a step runs out of time part-way, what does the model learn about what already happened? RULING · code

Answered how: If a step runs past 180 seconds, it is cut off and the model is told only that the step timed out, not which of its actions had already run.
Held by: browser_use/agent/service.py:2481-2497 (_execute_step): on the 180 s step timeout, last_result is replaced with one error "Step N timed out after 180 seconds". The actions that already ran are kept only in multi_act's local results (:2764), assigned to state.last_result only after it returns (:1217-1218); step() cleared last_result before the model call (:1072) and _finalize writes no history item when last_result is empty (:1360-1361).

F. Limits and memory (15 seats)

F1 When has the run used up its step budget? RULE · code

Answered how: Code stops a run after 500 steps by default (max_steps of run()), records "Failed to complete task in maximum steps" and returns with no answer; a follow-up task on the same Agent keeps counting from the first run's steps.
Held by: browser_use/agent/service.py:2620 (while self.state.n_steps <= max_steps), default max_steps=500 at browser_use/agent/service.py:2523 (and run_sync :4088); the loop's else branch at browser_use/agent/service.py:2654-2672 appends a history item with error='Failed to complete task in maximum steps'.

F2 When should the agent be warned to wrap up? RULING · code

Answered how: From 75% of the step budget (step 375 of the default 500) until the last step, code adds a "BUDGET WARNING" to each prompt telling the model to consolidate its results and call done.
Held by: browser_use/agent/service.py:1543-1567 (_inject_budget_warning, fires at steps_used / max_steps >= 0.75 and not on the last step), called from _prepare_context at :1154.

F3 On the last allowed step, what may the agent do? RULE · code

Answered how: On the last step (step 500 by default) code offers the model only the done action and tells it to set success to false if the task is not fully finished.
Held by: browser_use/agent/service.py:1569-1578 (_force_done_after_last_step: adds a message and swaps self.AgentOutput to self.DoneAgentOutput, built with only the done action at :794-800 and :4062-4068). is_last_step is step_number >= max_steps - 1 (browser_use/agent/views.py:283-285).

F4 Does this step count as a failure? RULING · code

Answered how: Code counts a step as a failure when its only result is an error (one action, or several where the first errored), the model call fails, or the step times out. A step whose later action errors resets the count to zero, by design.
Held by: browser_use/agent/service.py:1234-1243 (_post_process: increments consecutive_failures only when the step produced exactly ONE result and it carries error; otherwise RESETS the counter to 0); also :1298-1300 (any exception in a step), :2492 (step timeout), :2613 (initial actions timeout).

F5 When does the agent give up after repeated failures? RULING · code

Answered how: By default, after 5 failed steps in a row code gives the model one forced-done step (F6); if that fails too, the run ends with "Stopped due to 5 consecutive failures" and returns the history.
Held by: browser_use/agent/service.py:2629-2635 (stop when consecutive_failures >= max_failures + int(final_response_after_failure), i.e. 5 + 1 = 6 by default); defaults max_failures=5 (:173), final_response_after_failure=True (:203).

F6 After repeated failures, who writes the run's final answer? RULING · llm

Answered how: After 5 failures in a row (the default), the main model gets one step with only done and is told to report what it found and set success false unless finished; nothing checks that it does.
Held by: browser_use/agent/service.py:1580-1591 (_force_done_after_failure: at consecutive_failures >= max_failures with final_response_after_failure True, adds the message and swaps to DoneAgentOutput), answered by the main llm on that step.

F7 How long may one model call take? RULING · code

Answered how: With the default model (bu-2-0) code gives each model call 75 seconds (90 for Claude, o3, DeepSeek and Gemini 3 Pro names, 30 for Groq); a call that runs over counts as a failed step.
Held by: browser_use/agent/service.py:262-278 (_get_model_timeout: 90 s for gemini 3-pro, o3, claude, sonnet, deepseek; 75 s other gemini; 30 s groq; 75 s otherwise), enforced at :1185-1199 (asyncio.wait_for(..., timeout=self.settings.llm_timeout)).

F8 How long may a whole step take? RULING · code

Answered how: Code gives a whole step (browser state, model call and actions) 180 seconds by default; a step that runs over is cancelled, recorded as timed out and counted as a failure.
Held by: browser_use/agent/service.py:2481-2497 (asyncio.wait_for(self.step(...), timeout=self.settings.step_timeout), default 180 s at :199); initial actions get the same bound at :2599-2613.

F9 Is the agent stuck in a loop, and what is it told? RULING · code

Answered how: On by default: code counts repeated similar actions (nudges at 5, 8 and 12 repeats in the last 20; wait, done and go_back are not counted) and unchanged pages, and adds a "you might be looping" note to the next prompt; it never blocks or stops the agent.
Held by: browser_use/agent/views.py:157-248 (ActionLoopDetector: rolling window of 20 action hashes, nudges at 5/8/12 repeats, page-stagnation nudge once the page fingerprint is unchanged for 5 consecutive steps); wired at browser_use/agent/service.py:1497-1541 and :1157-1158, :1232.

F10 Should the agent be told to replan? RULING · code

Answered how: Off at the default model: ChatBrowserUse forces flash mode, which turns planning off. With planning on and a plan written, code adds "REPLAN SUGGESTED" after 3 counted failures in a row (F4's counter).
Held by: browser_use/agent/service.py:1465-1479 (when a plan exists and consecutive_failures >= planning_replan_on_stall, default 3 at :205).

F11 Should the agent be told to make a plan or finish? RULING · code

Answered how: Off at the default model (flash mode turns planning off). With planning on, from step 5 code tells an agent with no plan, on every step, to write one or call done.
Held by: browser_use/agent/service.py:1481-1495 (no plan and n_steps >= planning_exploration_limit, default 5 at :206).

F12 Which older history is summarised away, and what does the summary keep? RULING · llm

Answered how: On by default: once 25 steps have passed and the history text exceeds 40,000 characters, the main model summarises it, and code keeps that summary plus the first and last six steps; a failed summary is retried every step.
Held by: browser_use/agent/service.py:1169 (settings.compaction_llm or self.settings.page_extraction_llm or self.llm, i.e. the main llm by default); call at browser_use/agent/message_manager/service.py:216-302. Gates: every 25 steps (compact_every_n_steps, browser_use/agent/views.py:39) AND history text >= 40,000 chars (:55, browser_use/agent/message_manager/service.py:241-243). After a summary it keeps the first history item plus the last 6 (keep_last_items, browser_use/agent/views.py:43; browser_use/agent/message_manager/service.py:292-298).

F13 What is the agent's plan, and which item is it on? RULING · llm

Answered how: Off at the default model (flash mode removes the plan fields). With planning on, the model may write a todo list and say which item it is on; code stores it and shows it back each step.
Held by: browser_use/agent/views.py:395-396 (the main llm's plan_update / current_plan_item fields; dropped from the flash-mode schema at browser_use/agent/views.py:469-470), applied by browser_use/agent/service.py:1418-1451 (index clamped to range at browser_use/agent/service.py:1439).

F14 What should the agent remember from this step? RULING · llm

Answered how: Each step the model may write a short memory note (the default prompt asks for up to five sentences that also judge the last step and plan the next); code carries the notes it writes forward as the run's history text.
Held by: browser_use/agent/views.py:393 (the main llm's memory field), stored per step at browser_use/agent/message_manager/service.py:382-389 and rendered back by HistoryItem.to_string (browser_use/agent/message_manager/views.py:33-55).

F16 When a history cap is set, which past steps does the model stop seeing? RULING · code

Answered how: Off by default (max_history_items=None): every step stays in view until compaction (F12). When set (it must be above 5), code shows the first step, a "steps omitted" marker and the most recent steps up to the cap.
Held by: browser_use/agent/message_manager/service.py:165-189 (keeps the first item and the last max_history_items - 1, inserts "[... N previous steps omitted...]"); max_history_items=None by default (browser_use/agent/service.py:183), must be > 5 (browser_use/agent/message_manager/service.py:141).

G. Judging the result (6 seats)

G1 Is the task finished? RULING · llm

Answered how: The model decides the task is finished by calling done; code ends the run on that call without checking whether the answer is right (the judge, G3, only grades it afterwards).
Held by: browser_use/tools/service.py:2049-2090 (the done action the main llm calls; structured variant browser_use/tools/service.py:2012-2045); the loop reads it through AgentHistoryList.is_done (browser_use/agent/views.py:722-727, last result of the last step) at browser_use/agent/service.py:2502. multi_act refuses done anywhere but first in a step (browser_use/agent/service.py:2786-2790) and stops after it (:2834).

G2 Did the task succeed (the agent's own report)? RULING · llm

Answered how: The run's success flag is whatever the model puts in done; if the model leaves it out code records success, and with an output schema the field is hidden from the model, so a structured run reads as successful unless the model sets it anyway.
Held by: browser_use/tools/views.py:100 (DoneAction.success, filled by the model's done call; StructuredOutputAction.success at browser_use/tools/views.py:117); read back by AgentHistoryList.is_successful (browser_use/agent/views.py:729-735). This is the value integrators read and the value telemetry sends (browser_use/agent/service.py:2261); the judge does NOT override it (:1632-1634).

G3 Does an independent judge think the task succeeded? RULING · llm

Answered how: On by default: after a done, a second call through the agent's own model client grades the run against the task and the last 10 screenshots; code attaches and logs the verdict but keeps the agent's own success flag. No done, no judging.
Held by: browser_use/agent/service.py:1594-1627 (_judge_trace, calling judge_llm, which defaults to the main llm at browser_use/agent/service.py:256-257; use_judge=True by default at :186), with messages from construct_judge_messages (browser_use/agent/judge.py:44-225); runs after a done, before the done callback (browser_use/agent/service.py:2502-2507, :2291-2296).

G4 Did the last action achieve what it was meant to? RULING · llm

Answered how: Absent at the default model: flash mode removes the field (its prompt asks for the same judgement inside memory, F14). With a model not in flash mode, the model writes a one-line verdict each step; code shows it back and acts on nothing in it.
Held by: browser_use/agent/views.py:392 (the main llm's evaluation_previous_goal field, required in the schema at browser_use/agent/views.py:405, removed from the flash-mode schema at browser_use/agent/views.py:467), stored at browser_use/agent/message_manager/service.py:384 and shown back as history text (browser_use/agent/message_manager/views.py:45-47).

G5 Does the final output match the schema the integrator asked for? RULE · code

Answered how: Absent by default. When the integrator passes output_model_schema, pydantic checks the final done against it; a misfit is a failed step the model sees, and a run can still end at its step or failure limit with no fitting answer.
Held by: browser_use/tools/service.py:2012-2016 (pydantic validation of the done parameters against StructuredOutputAction[output_model] at action dispatch), then AgentHistoryList.structured_output re-parses the final JSON (browser_use/agent/views.py:914-926). Present only with output_model_schema or a Tools output model (browser_use/agent/service.py:351-365).

G6 Did a replayed run succeed? RULING · llm

Answered how: Only when an integrator calls rerun_history or load_and_rerun: a model (by default the agent's own) judges a final screenshot and step counts; if that call fails, code calls the replay successful when no step recorded an error.
Held by: browser_use/agent/service.py:2965-2967 (summary_llm, defaulting to the main llm) in _generate_rerun_summary (browser_use/agent/service.py:2932-3021), called at the end of rerun_history (browser_use/agent/service.py:3299-3302). A separate entry point (I-stage), not reached by Agent.run.

H. Delegation (1 seat)

H1 When an integrator adds MCP tools, what does an outside tool do and return? RULING · external

Answered how: Absent by default. When integrator code registers an MCP server's tools, each one (or each in tool_filter) becomes an action the model can call, and the outside server decides what it does and returns.
Held by: browser_use/mcp/client.py:309-342 (mcp_action_wrapper hands the call to the MCP server via self.session.call_tool; no-parameter twin browser_use/mcp/client.py:360-390); registered only when integrator code calls MCPClient.register_to_tools (browser_use/mcp/client.py:212-246; Grep for register_to_tools|MCPClient\( over browser_use/ finds only the module's own docstring and definition).

I. Learning and feedback (4 seats)

I1 What browser state (cookies, logins) carries into the next run? RULING · code

Answered how: By default no saved cookies or logins are loaded: with no storage_state named, code loads and saves none. The default browser profile folder is a temporary one made once when the library loads, so later runs in the same Python process can find what earlier ones left there. With a storage_state named, code loads it at start and merges changed cookies back about once a minute.
Held by: browser_use/browser/watchdogs/storage_state_watchdog.py:49-57 (StorageStateWatchdog loads storage_state on connect, browser_use/browser/watchdogs/storage_state_watchdog.py:224-322), saves on cookie change every 30 s (:41, :119-133) and on demand, MERGING with the existing file (new cookie wins per name/domain/path, :185-191, :324-345), atomic write with .bak (:194-204); an in-memory dict is loaded but never saved (:168-172). With a non-temp Chrome user_data_dir, the profile is COPIED to a temp dir first (browser_use/browser/profile.py:849-904), so the real profile is read, not written.

I2 Which values typed in a recorded run are reusable variables? RULING · code

Answered how: Only when an integrator calls detect_variables or load_and_rerun(variables=...): code guesses which typed values are variables to swap on replay, from field type and id/name/label keywords, else from email, phone, date, name or number patterns.
Held by: browser_use/agent/variable_detector.py:9-46 (detect_variables_in_history): element attributes first, then value patterns for email, phone, YYYY-MM-DD date, capitalised name and short number (browser_use/agent/variable_detector.py:98-120, :213-256); reached through Agent.detect_variables (browser_use/agent/service.py:4097-4101) and load_and_rerun(variables=...) → _substitute_variables_in_history (:3931-3932, :4103-4163). Only text in text and query fields (browser_use/agent/variable_detector.py:63).

I3 On replay, which element on today's page is the one the recording clicked? RULING · code

Answered how: Only when an integrator replays a run: code matches each recorded element to today's page by a fixed cascade (exact hash, stable hash, XPath, accessible name, attribute) and acts on the first element matched at the first level that finds one.
Held by: browser_use/agent/service.py:3554-3712 (_update_action_indices): five-level cascade EXACT hash → STABLE hash → XPATH → accessible name → attribute (name, id, aria-label), first match wins at each level, same-frame elements first (browser_use/agent/service.py:3576-3583).

I4 On replay, is this step a needless repeat that should be skipped? RULING · code

Answered how: Only when an integrator replays a run: code skips a step when the previous step worked and both first actions are the same action type on the same element, whatever their parameters, and records the skip as no error.
Held by: browser_use/agent/service.py:3739-3808 (_is_redundant_retry_step), applied at browser_use/agent/service.py:3226-3235: skip when the previous step succeeded and both steps' FIRST action has the same action type on the same element (by hash, stable hash or xpath).

How this map was made

Four readers each took a part of the agent and wrote seat cards from the source, and four more rewrote each answer in a short form that describes the default. The map's author then re-read every seat against the cited lines and corrected what the source did not support. Reviews by a model from another family (read-only, at the same commit) checked the answers against the code, and each of their findings was checked against the code before anything changed. A checker confirms that each seat's recorded holder line exists at the pinned commit and carries its anchor text; the few holder lines edited for this page were checked by the cross-family page review instead. Nothing on this page was produced by running their code.