# spotlit: a 6.7 MB two-stage element ranker for browser agents

How a 2 MB text embedder became a two-stage model that ranks a web page's elements for AI browser agents, in the browser.

Sardor Madaminov · October 22, 2026 · https://sardorm.io/research/spotlit

---

Most browser agents send the whole page to their language model at every step. A page has hundreds to thousands of elements, and the next action usually needs one of them. On one recorded step on Viator, the page has 1,373 elements, about 75,131 tokens of text, and the goal needs exactly one: the Coffee & Tea link.

spotlit ranks a page's elements against the agent's goal and hands the model the 50 that matter. It is 6.7 MB of open weights, runs in the browser on WebAssembly with no server and no GPU, and takes about a quarter of a second per page. This is how it got there, including what did not work.

> Figure: One page step. The Viator step: every element is scored by stage 1, the best 400 are re-read by stage 2, the agent's model reads the top 50 and picks one action. 75,131 tokens of page become 507 tokens of element lines.

## Where it started

It began in late July, when I came across [Moka](https://github.com/millionco/moka), a tiny Go-playing network that runs in the browser. Seeing how small a useful model can get made me look for other things a model that size could do, so I tried my own. My first was a text embedder with no tokenizer: byte 3, 4 and 5-grams hashed into a 6,144 x 320 table, 1.99 MB at INT8, fitted to imitate `bge-small-en-v1.5` [[4]](#references). The goal was to add it to keyword search (BM25) and beat BM25 alone by at least +0.03 nDCG@10.

| Corpus | BM25 | BM25 + student | BM25 + teacher |
| :--- | ---: | ---: | ---: |
| Rust book, 458 queries | 0.589 | **0.613** | 0.687 |
| Private project docs, dev, 597 queries | 0.265 | 0.262 | 0.321 |

On technical prose it added +0.024; on dense design documents it made search worse. I tried gradient-trained students, more and different training text, two or three hash functions per n-gram, a table twice the size, a 100K-word static vocabulary and a small transformer. None closed the gap, and a single change of hash function moved the score by 0.015, so most of those differences were noise. (The conclusion I wrote down then: without pretraining, a model this small learns too little language from half a million labelled texts to beat BM25 on varied documentation.) That lesson came back later.

## A new job for the same table

So I gave the 2 MB table a different job: rank a page's elements against an agent's goal, trained on Mind2Web [[1]](#references), where people completed tasks on real websites, one target element per step: the element they clicked, typed into or chose from.

Every element becomes one short line: its tag, its labels (`aria-label`, `title`, `alt`, `placeholder` and a few others), up to 24 words of its own text, then `|` and up to 12 words of its parent's text. A link reads `a Coffee & Tea | Coffee & Tea`. The goal is the task plus up to three previous actions.

**Stage 1** cuts the goal and every line into overlapping 3, 4 and 5-byte pieces, hashes each piece into one of 6,144 rows [[3]](#references), averages the rows into a 320-dimensional vector, and ranks by cosine. There is no vocabulary, so typos, case and punctuation change little; it is the subword idea of fastText [[5]](#references) taken down to bytes. Type below to see the pieces two texts share.

> Figure: Byte pieces. The pieces stage 1 hashes, after its normalisation (lower-case, punctuation to spaces, one space at each end). Highlighted pieces appear in both texts. Through the trained table, these two texts have cosine 0.69.

I split Mind2Web's training websites by a salted hash: 6,669 steps to train on, and 1,106 dev steps from websites the model never sees. Training ran one page per update, with a softmax over the target element and every other candidate on the page, at temperature 0.05: Adam, learning rate 5e-4 with cosine decay, 3 epochs, 18,804 updates, 20 minutes on an M4 Pro at 0.34 GB.

Dev recall@50, the share of steps whose target element lands in the top 50, went from 38.2% for keyword search to 55.3%. Then it flattened: on its logging sample it levelled off near 53% from update 11,000 while training loss kept falling. It was memorising websites, not learning what elements mean.

## A second stage that reads

The fix was a second stage with pretraining: **BERT-tiny** (`google/bert_uncased_L-2_H-128_A-2`, 4.4M parameters) [[2]](#references) as a cross-encoder that reads `[CLS] goal [SEP] element [SEP]` for each of stage 1's top 400 and scores the pair. Reading the two together is what separates "Cancel" from "Subscribe" for the goal "Cancel my subscription", which share letters.

It trains on stage 1's own mistakes: each batch holds 8 steps of 32 pairs, the target element and 31 negatives drawn from stage 1's top 400, with a softmax per step. AdamW at 1e-4 with warmup and decay, 3 epochs, 2,350 batches, five and a half minutes on the laptop's GPU at 1.1 GB.

Dev recall@50 went from 55.3% to 72.5%, and on the Viator step the Coffee & Tea link climbs from #363 to #10.

> Figure: Stage 2 on the Viator step. The target element's rank on a log axis. Stage 1 keeps it, at #363, inside the 400 it passes on; stage 2 lifts it into the top 50 the agent reads.

## Choices, and what decided them

| Question | What I measured | Chosen |
| :--- | :--- | :--- |
| BERT-tiny or BERT-mini? | Mini is 6 points better on dev (78.7% vs 72.5% recall@50) but re-scoring 400 pairs takes 1.16 s in Chrome against 0.19 s | BERT-tiny: the target was one second per page step |
| INT8 or full precision? | INT8 export moved dev recall@50 by +0.6 points for BERT-tiny and shrank it to 4.49 MB | INT8 |
| WebGPU or WebAssembly? | INT8 graphs on ONNX Runtime Web's WebGPU backend gave wrong scores, agreeing with Python's top 50 on only 9-13 of 40 steps | WebAssembly |
| Fuse stage 1 with BM25? | Reciprocal rank fusion scored 54.3% recall@50 against 55.3% for stage 1 alone | No fusion |
| How deep should stage 2 read? | Stage 1's top 300 / 400 holds the target element on 84.7% / 87.6% of dev steps, a ceiling for any re-ranker | 400 |

> Figure: One page step in Chrome. Median over 38 dev steps, Apple M4 Pro, eight WebAssembly threads in a cross-origin isolated page. FP32 on WebGPU is faster but a 17.6 MB download, and INT8 on WebGPU returns wrong scores.

## Results on dev

Every decision above was made on the dev split. The table and chart compare the systems I tried; "released" is the INT8 files that ship.

| System | Recall@1 | Recall@10 | Recall@50 | Recall@100 | MRR | Tokens, top 50 |
| :--- | ---: | ---: | ---: | ---: | ---: | ---: |
| BM25 | 3.6% | 13.0% | 38.2% | 50.8% | 0.073 | 21x fewer |
| bge-small, untrained | 3.3% | 12.5% | 31.4% | | 0.066 | 21x fewer |
| Stage 1 | 7.8% | 31.1% | 55.3% | 66.9% | 0.155 | 34x fewer |
| Stage 1 + BM25 fusion | 5.1% | 24.1% | 54.3% | 68.7% | 0.114 | 26x fewer |
| Stage 1 + BERT-mini | 16.5% | 54.2% | 78.7% | 84.2% | 0.285 | 44x fewer |
| **Released, stage 1 + BERT-tiny, INT8** | **10.9%** | **49.3%** | **73.1%** | **80.1%** | **0.229** | **46x fewer** |

> Figure: Recall at k, dev split. The share of dev steps whose target element is in the top k, on a log axis. Hover the chart, or select it and use the arrow keys.

## Opening the test splits once

Then I froze the files, recorded their checksums, and opened Mind2Web's three test splits once.

| Test split | Steps | BM25 @50 | Stage 1 @50 | **spotlit @50** | @10 | @100 | MRR |
| :--- | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| Cross-task | 2,094 | 44.2% | 69.2% | **80.8%** | 57.2% | 85.8% | 0.274 |
| Cross-website | 1,373 | 43.8% | 56.7% | **75.8%** | 48.7% | 82.3% | 0.219 |
| Cross-domain | 5,911 | 44.5% | 57.7% | **75.9%** | 51.5% | 83.6% | 0.251 |

The columns are recall at 50, 10 and 100. On every split the top 50 is 32-43x fewer tokens than the full page.

Dev agrees with cross-website and cross-domain within three points; cross-task is higher because its websites appear in training. MindAct's ranker from the Mind2Web paper [[1]](#references) reports 88.9 / 85.3 / 85.7% on the same splits with 184M parameters, so spotlit is 8-10 points behind at about a 29th of the size, with element descriptions that differ, so the gap is approximate.

## In the browser

Stage 1 and the WordPiece tokenizer are plain JavaScript, stage 2 is an INT8 ONNX file on [ONNX Runtime Web](https://onnxruntime.ai/docs/tutorials/web/). I checked every runtime against the Python reference before trusting a number from it:

| Check | Result |
| :--- | :--- |
| Byte n-gram encoder, JavaScript and C++ | 31 of 31 golden cases identical |
| WordPiece tokenizer, JavaScript, against the reference fast tokenizer | 667 of 667 pairs give identical token ids, including accents, CJK, emoji and truncation |
| Whole pipeline in Chrome | top-50 membership agrees with Python on 40 of 40 dev steps |

A page step takes 236 ms on eight threads: 35-45 ms for stage 1, about 10 ms for tokenization, 185 ms for stage 2. Without cross-origin isolation, as on most hosted pages, it runs on one thread and takes about a second. (The tokenizer was the hardest part to match: the Rust tokenizer's pair truncation is not the textbook longest-first rule, so the JavaScript version was fitted to 1,849 length pairs.)

## Inside real agents

Offline recall is not task success, so I gave Claude Code a small browser server with two modes: one returns every visible element, the other only spotlit's top 50 for the goal the agent passes.

| Seven multi-page tasks, Claude Sonnet, one run each | Every element | spotlit top 50 |
| :--- | ---: | ---: |
| Tasks finished | 7 of 7 | 7 of 7 |
| Input tokens | 2,700,384 | **427,787** |
| Cost | $3.55 | **$0.48** |
| Time | 149 s | 173 s |
| Tool calls | 48 | 55 |

I also replayed 30 random dev steps, asking the model to pick the element from each list:

| Agent | List shown | Right element | Target element in the list | Input tokens |
| :--- | :--- | ---: | ---: | ---: |
| Claude Code, Sonnet | every element | 15 of 30 | 30 of 30 | 964,166 |
| Claude Code, Sonnet | spotlit top 50 | 13 of 30 | 26 of 30 | 76,740 |
| Codex | every element | 12 of 23 | 23 of 23 | 560,558 |
| Codex | spotlit top 50 | 9 of 23 | 20 of 23 | 210,525 |

Thirty steps is too few to separate a difference of two from noise, but the pattern is consistent: a little accuracy for a large saving in tokens. It was not faster: with spotlit the agent made more calls and took longer.

## What I learned

- **Pretraining carried over; table size did not.** Stage 1 memorised the websites it saw, and every bigger or cleverer table I tried before it stalled. The gains on new websites came from stage 2's pretrained weights.
- **Train the second stage on the first stage's mistakes.** Negatives drawn from stage 1's own top 400 taught stage 2 exactly the confusions it would face.
- **Measure in the runtime you ship.** The WebGPU bug only showed up in Chrome, and only for INT8; Python said everything was fine.
- **Watch for leaks in the data.** Mind2Web lists the target element first among the candidates, so every tie breaks by element id, never by position.
- **Open the test set once.** All the choices above were made on dev, and the test numbers were the last thing I looked at.

## Where it falls short

| Limit | Detail |
| :--- | :--- |
| A shortlist, not a picker | The target element is first on about 11% of dev steps; the agent's model still decides |
| Misses about one step in four on new websites | Cross-website recall@50 is 75.8%; an agent needs a fallback such as a wider list or the full page |
| Stage 1 caps stage 2 | Its top 400 holds the target element on 87.6% of dev steps |
| Look-alikes | For "Sign in to my account", stage 2 ranks "Sign up free" first and "Log in" fifth |
| English sites from 2023, text only | No screenshot, position or visibility; iframes and shadow roots are not collected |

Try it on a real page in the [demo](https://spotlit-v1.pages.dev/#demo), which also has the weights to download, and see each stage interactively in the [full write-up](https://spotlit-v1.pages.dev/blog/spotlit/).

## References

1. Deng, Gu, Zheng, Chen, Stevens, Wang, Sun and Su. [Mind2Web: Towards a Generalist Agent for the Web](https://arxiv.org/abs/2306.06070). NeurIPS 2023, Datasets and Benchmarks Track. The data, and the MindAct ranker spotlit is compared with.
2. Turc, Chang, Lee and Toutanova. [Well-Read Students Learn Better: On the Importance of Pre-training Compact Models](https://arxiv.org/abs/1908.08962). 2019. The BERT-tiny checkpoint stage 2 starts from.
3. Weinberger, Dasgupta, Langford, Smola and Attenberg. [Feature Hashing for Large Scale Multitask Learning](https://arxiv.org/abs/0902.2206). ICML 2009. Signed hashing into a fixed number of rows, as stage 1 does.
4. Xiao, Liu, Zhang, Muennighoff, Lian and Nie. [C-Pack: Packed Resources For General Chinese Embeddings](https://arxiv.org/abs/2309.07597). 2023. The BGE models, including the `bge-small-en-v1.5` teacher.
5. Bojanowski, Grave, Joulin and Mikolov. [Enriching Word Vectors with Subword Information](https://arxiv.org/abs/1607.04606). TACL 2017. Words as bags of character n-grams.