The Masterlens benchmark

How well does a campaign memory tool remember what happened at the table? We gave Masterlens and Archivist AI the same two sessions and the same 34 questions, then graded every answer by hand.

Key facts found in the answers
  • Masterlens 90%
  • Archivist AI 60%
Wrong answers
  • Masterlens 0
  • Archivist AI 2

The sessions

Two back-to-back four-hour sessions from one of the most-watched actual-play campaigns online. A large cast, a game master voicing dozens of characters, politics, flashbacks and secrets: the kind of story that is hard to keep straight.

About 66,000 words of table talk, taken from the show’s published captions, one line per speaker turn. Neither tool got anything else: no character list, no prep notes, no summaries.

The questions

34 questions, written from the transcripts before this round of testing. They are the questions a game master asks between sessions. The examples below are made up, so as not to give the test away.

  • Lookups 13

    A name, an object or a place that came up at the table.

    “What did the innkeeper give the party before they left town?”

  • Several parts 3

    Plans, histories and relationships that build up over many scenes.

    “How was the jailbreak supposed to work, and what went wrong?”

  • Claim or fact 7

    A character says something. A good answer says who said it and what the session actually shows.

    “Did the captain really sell out the rebels?”

  • Across sessions 4

    Answers that only come together with both sessions.

    “Who had the map at the end of the first session, and where did it go next?”

  • Trick questions 7

    About things that never happen in these sessions. The only right answer is that the notes don’t say.

    “What happens when the party reaches the floating city?”

The answer key

Each question lists the key facts a complete answer has to contain, 70 in all, with the transcript lines they come from. It also lists what an answer must not say, such as a rumour stated as fact or a detail that belongs to another character.

How we asked

  1. Both sessions went into each tool through its normal import.
  2. Every question was asked in a new chat, so no answer could lean on an earlier one.
  3. Answers were saved exactly as each tool returned them.

How we graded

Every answer was checked by hand against the key and given one grade.

Correct
Every key fact the question calls for.
Partly correct
Some of the key facts, and nothing false.
Missed
None of the key facts, including saying the notes don’t cover it when they do.
Correctly unknown
A trick question answered with “your notes don’t say”, and nothing invented.
Wrong
States something the sessions contradict, or turns what a character claimed into fact.
Made up
Invents an answer to a trick question.

Key facts found counts every key fact present across all answers, out of 70. An incomplete answer counts only the key facts it contains, so a partly correct answer earns part of the score, not all of it. When a tool said the notes don’t cover something the sessions do show, that question counts as missed: it earns no key facts, but it isn’t a wrong answer.

Across all 34 questions, Masterlens gave no wrong answers: every answer either matched the sessions or said the notes don’t cover it. That is not the same as every answer being complete; 21 of the 34 were fully correct.

Results

MeasureMasterlensArchivist AI
Key facts found63 of 70 (90%)42 of 70 (60%)
Fully correct answers21 of 3411 of 34
Answered at least in part27 of 2722 of 27
Wrong answers02
Made-up answers to trick questions0 of 70 of 7

Key facts by kind of question

KindMasterlensArchivist AI
Lookups26 of 3122 of 31
Several parts12 of 127 of 12
Claim or fact18 of 206 of 20
Across sessions7 of 77 of 7

What we saw

  • Both tools were honest about what the sessions never covered. Neither made up an answer to a trick question.
  • Archivist AI’s two wrong answers both turned something a character claimed into settled fact. Masterlens didn’t treat either claim as fact.
  • The biggest gap was on claims. When characters disagree, Masterlens more often said who claimed what and what the session showed.
  • Archivist AI found a few details Masterlens missed, such as how many of the raiders died and that the magpie was attacking the driver.
  • Masterlens still misses details that come up only once in conversation. That is where we are working next.

Limits

  • One campaign and one run per tool. AI answers vary from run to run: when we repeated earlier versions of this test, Masterlens moved by a few key facts between identical runs.
  • We wrote the questions and graded the answers ourselves, using the same key and the same rules for both tools.
  • Both tools change often. This test used the versions available on 17 and 18 November 2026.
  • Your campaign is not this campaign. Results for your table will differ.