Close a replay gap in a TOTP verify path
Measured at 33% over 6 counted trials, which is inside the band gate 5 admits. The breakdown below says which model that figure came from.
- 33%
- measured pass rate
- 2 of 6 counted trials met the reward threshold.
- 2
- models measured
- Each cell is one agent against one model, run for the same number of attempts.
- 0
- errored trials
- Trials that produced no reward. Counted separately and never in the rate above.
- 6
- trials dispatched
- Every attempt this round started, including the ones that errored before a reward.
What each model did
One row per agent-and-model cell of the round that produced the figure above. The bar is passes over counted trials.
The table scrolls sideways.
| Agent | Model | Passed | Rate | Errored |
|---|---|---|---|---|
| Terminus 2 | GPT-6 Astraopenai | 2 of 3 | 67% | none |
| Terminus 2 | Claude Fable 5.1anthropic | 0 of 3 | 0% | none |
A cell with no passes is not a broken task: it is the point of one. Gate 5 admits a task only when the full matrix lands between 10 and 60 percent, so a set that every model cleared would have been rejected for being too easy before it reached this page.
The task
A login flow verifies TOTP codes statelessly, so the same six digits submitted twice inside one step both pass. Make a code work once across every account, while the store of accepted codes stays bounded on a box that authenticates far more accounts than it ever sees twice.
- Category
- Security
- Declared tier
- hard. The tier is the author's declaration and it decides which band passes: a hard-tier task may legitimately sit under 10 percent, where a standard one may not.
- Languages
- python
- Catalogue reference
- pyotp-replay-guard-skew-window. The name a licensed copy carries, so a run of it is traceable back to this row.
- What a licence includes
- The graded task, its environment and verifier images pinned by digest, and the trial record behind the figures on this page.
- What stays with us
- The instruction, the test suite and the reference solution. Publishing those would retire the task, because a measurement you can read the answer to is not a measurement.