Code & Agentic RL · SWE, terminal, OS-use, science, security, legacy migration

Agentic & Long-Horizon Benchmark Tasks

Long-horizon agent tasks with environments, programmatic graders and reference runs, built by Perit.

ENV · 11 FAMILIES · ZIP · GRADED
AvailabilitySample available
11task families
Gradedprogrammatic
Refruns included
Perit-built
01

Preview

SWERepository-level bug fix and feature tasks
TerminalMulti-step shell workflows
OS-useDesktop and file-system operation
ScienceAnalysis pipelines with checked outputs
SecurityPatch and triage tasks
Legacy migrationPort and verify old code
+5 familiesListed in the PDF brief
# task: legacy-migration/007

Goal     Port the billing module from Python 2 to 3.11
         without changing invoice totals.
Env      docker image, 1 repo, 2 services
Grader   pytest suite (214 cases) + diff of 1,000 invoices
Budget   40 tool calls, 30 min wall clock
Ref run  passed 214/214, 31 tool calls

Scoring  pass = all tests green AND 0 invoice deltas
Preview · task cards + README
02

At a glance

Volume11 task families
FormatsZIP (env + graders + reference runs) · PDF brief
Languagesmulti (programming)
AnnotationGraded (programmatic graders)
Licence statusPerit-built
AvailabilitySample available
Date addedSep 25, 2026
Update cadenceNew families quarterly
03

Provenance ledger

Rights / consentCleared
PII handlingScrubbed
QA statusPerit QA passed
LicencePerit-built

No supplier, broker, client or personal names appear on this page or in its samples. Previews are Perit-owned copies with container metadata stripped.

04

Schema and anonymised sample record

{
"repo_id": "PD-009-r007",
"primary_language": "Multiple",
"lines_of_code": 58214,
"commits": 4113,
"first_commit": "2016-04-02",
"last_commit": "2026-08-19",
"test_files": 212,
"ci_configured": true
}

The record above is synthetic: the field list and types are the real delivery schema, the values are not from the set. Full field documentation, including every optional column, ships with the pilot batch so you can write the loader before the data lands.

05

How delivery works. Four steps, and the first one is a request.

01 · Day 0RequestTell us the set, the use and the volume.
02 · 48 hoursSpec and quoteWritten spec, rights summary and price.
03 · 1-2 weeksPilot batchA QA'd slice in your data room to test against.
04 · OngoingDelivery to your bucketFull set or scheduled drops, same schema.
Access

Tell us the run, and we will tell you what clears for it.

Written spec, rights summary and price inside 48 hours, and a straight answer on anything still in review, including when the answer is that it will not clear.