Every entry states its question, environment, limits, and reproduction path. A benchmark specification without an execution is not a score; scores across different tasks, versions, and models are not comparable.
2published evaluation reports
5local experiments summarized
3topology protocols pending
RESULTS AND EXPERIMENTS
What the runs measured.
The figures below come from reports and artifacts in the source repository. Task contexts and datasets differ; do not compare scores across them.
01 / REPORTED EVALUATION
LoCoMo · conversational memory
Evaluation of GenOS V3 and Qwen 2.5 Coder 7B on the LoCoMo corpus, with 1,986 questions across five categories.
18.39%Overall F1 · 1,986 questions
23.68%F1 · factual questions (cats. 1–4)
5.84%Overall exact match
QUESTION
What absolute recall score did this GenOS configuration achieve on LoCoMo? No same-model raw-context ablation was reported.
By category: event 30.71%, factual 17.02%, causal 16.92%, temporal 13.23%, adversarial 0.09%. The adversarial score penalizes explicit refusals when the gold answer is “undefined”; the report proposes a separate abstention metric.
REPRODUCTION
See the LoCoMo report for corpus version and harness commit.
Fixes are validated with FAIL_TO_PASS and PASS_TO_PASS tests. The 26-task denominator limits the scope: this rate is not a result on all 300 SWE-bench Lite tasks.
REPRODUCTION
FAIL_TO_PASS + PASS_TO_PASS validation per the linked report.
Workspace ablation scores 11/12 versus 11/12 in this campaign. The same report also documents a Lipson-like run of a simulated trajectory (reward 7.7369) and MBH-RNN (100,000 iterations, ten simulated trials); neither is a replication or a measure of generalization.
REPRODUCTION
Campaign report documents seeds and ablation arms.
In this run, ReAct passes 9/12 and MCTS 6/12 (175.9 average expansions). This is a small local synthetic suite, not an independent standard benchmark; a scenario correction was also recorded with the result.
The result depends on a decision harness without an LLM. Effects from some layers remain to be confirmed in runtime missions; the report explicitly distinguishes behavioral change from a status that is merely displayed.
Benefit profiles are defined in the model: the experiment informs selection under these assumptions, not an empirical discovery about the mechanisms. The report also flags an uninterpretable contrast in the predictable environment.
REPRODUCTION
Model assumptions and benefit profiles in the linked research doc.
Ablation on three problems: the harness compares the structural diversity of cognitive recipes composed at dispatch.
3Recipes · diversity arm E
0.51Pairwise distance · E
1.0Requirement coverage · E
QUESTION
Does diversity-arm composition produce structurally varied recipes?
ENVIRONMENT
Ablation harness · 3 problems · no LLM in the loop
LIMITS
Arm D (relevance only) produces one repeated recipe and a distance of 0; E produces three recipes with 1.0 coverage. This result concerns structure, not agent response quality: runtime missions remain to be measured.
12 complex missions used the full local backend and SQLite WAL. 9 cycles blocked and 3 completed; 0 outcomes received verified promotion. This is a functional qualification run, not a comparative benchmark. Its event persistence result does not imply mission success.
Choose a domain, task and policy to inspect its recorded plan, outcome and expansions from September 30, 2026. This view does not rerun the benchmark.
BLOCKSWORLD · bw-sussman
GenOS A*
GOAL REACHED
YesHarness goal met
3Expansions
2Plan actions
2BFS optimum (if computed)
RECORDED PLAN
01unstack C
02stack C on B
Deterministic comparison on synthetic tasks with a 240-expansion cap. These values come from the source runner; this selector does not launch a new run.
REPRODUCTION COMMANDnode backend/tests/search/test_planning_gap.jsThe result is specific to this suite, budget and artifact version. Replaying the command requires the GenOS source repository and its environment.
These evaluations reflect the documentation reviewed on 2026-10-18 at GenOS 5091f70. Each result applies only to the tasks, versions, models, and conditions stated in its source. A test result alone does not prove business correctness or generalization.