The Masterlens benchmark
How well does a campaign memory tool remember what happened at the table? We gave Masterlens and Archivist AI the same two sessions and the same 34 questions, then graded every answer by hand.
- Masterlens 90%
- Archivist AI 60%
- Masterlens 0
- Archivist AI 2
The sessions
Two back-to-back four-hour sessions from one of the most-watched actual-play campaigns online. A large cast, a game master voicing dozens of characters, politics, flashbacks and secrets: the kind of story that is hard to keep straight.
About 66,000 words of table talk, taken from the show’s published captions, one line per speaker turn. Neither tool got anything else: no character list, no prep notes, no summaries.
The questions
34 questions, written from the transcripts before this round of testing. They are the questions a game master asks between sessions. The examples below are made up, so as not to give the test away.
Lookups 13
A name, an object or a place that came up at the table.
“What did the innkeeper give the party before they left town?”
Several parts 3
Plans, histories and relationships that build up over many scenes.
“How was the jailbreak supposed to work, and what went wrong?”
Claim or fact 7
A character says something. A good answer says who said it and what the session actually shows.
“Did the captain really sell out the rebels?”
Across sessions 4
Answers that only come together with both sessions.
“Who had the map at the end of the first session, and where did it go next?”
Trick questions 7
About things that never happen in these sessions. The only right answer is that the notes don’t say.
“What happens when the party reaches the floating city?”
The answer key
Each question lists the key facts a complete answer has to contain, 70 in all, with the transcript lines they come from. It also lists what an answer must not say, such as a rumour stated as fact or a detail that belongs to another character.
How we asked
- Both sessions went into each tool through its normal import.
- Every question was asked in a new chat, so no answer could lean on an earlier one.
- Answers were saved exactly as each tool returned them.
How we graded
Every answer was checked by hand against the key and given one grade.
- Correct
- Every key fact the question calls for.
- Partly correct
- Some of the key facts, and nothing false.
- Missed
- None of the key facts, including saying the notes don’t cover it when they do.
- Correctly unknown
- A trick question answered with “your notes don’t say”, and nothing invented.
- Wrong
- States something the sessions contradict, or turns what a character claimed into fact.
- Made up
- Invents an answer to a trick question.
Key facts found counts every key fact present across all answers, out of 70. An incomplete answer counts only the key facts it contains, so a partly correct answer earns part of the score, not all of it. When a tool said the notes don’t cover something the sessions do show, that question counts as missed: it earns no key facts, but it isn’t a wrong answer.
Across all 34 questions, Masterlens gave no wrong answers: every answer either matched the sessions or said the notes don’t cover it. That is not the same as every answer being complete; 21 of the 34 were fully correct.
Results
| Measure | Masterlens | Archivist AI |
|---|---|---|
| Key facts found | 63 of 70 (90%) | 42 of 70 (60%) |
| Fully correct answers | 21 of 34 | 11 of 34 |
| Answered at least in part | 27 of 27 | 22 of 27 |
| Wrong answers | 0 | 2 |
| Made-up answers to trick questions | 0 of 7 | 0 of 7 |
Key facts by kind of question
| Kind | Masterlens | Archivist AI |
|---|---|---|
| Lookups | 26 of 31 | 22 of 31 |
| Several parts | 12 of 12 | 7 of 12 |
| Claim or fact | 18 of 20 | 6 of 20 |
| Across sessions | 7 of 7 | 7 of 7 |
What we saw
- Both tools were honest about what the sessions never covered. Neither made up an answer to a trick question.
- Archivist AI’s two wrong answers both turned something a character claimed into settled fact. Masterlens didn’t treat either claim as fact.
- The biggest gap was on claims. When characters disagree, Masterlens more often said who claimed what and what the session showed.
- Archivist AI found a few details Masterlens missed, such as how many of the raiders died and that the magpie was attacking the driver.
- Masterlens still misses details that come up only once in conversation. That is where we are working next.
Limits
- One campaign and one run per tool. AI answers vary from run to run: when we repeated earlier versions of this test, Masterlens moved by a few key facts between identical runs.
- We wrote the questions and graded the answers ourselves, using the same key and the same rules for both tools.
- Both tools change often. This test used the versions available on 17 and 18 November 2026.
- Your campaign is not this campaign. Results for your table will differ.