EVALARC / DEVELOPMENT AUDIT

Test the grader.
Then trust the signal.

Behavioral controls for task-specific agent evaluations. Correct and faulty submissions are evaluated against the same externally enforced contract.

8/8
declared faults detected
1.000
reference correctness
docker
execution backend

Does the grader detect plausible defects?

Negative controlTargetCandidate scoreResultEvidence
ack-without-workbasic0.000Detectedjson-types-and-overwrite, read-write-delete, seeded-state-machine
memory-onlypersistence0.783Detectedrestart-preserves-acknowledged-write, restart-preserves-cas, restart-preserves-transaction
partial-batchtransactions0.900Detectedrollback-batch
cas-always-winscompare_swap0.850Detectedcas-match-and-mismatch, cas-type-sensitivity
boolean-equals-onecompare_swap0.925Detectedcas-type-sensitivity
delete-noopbasic0.650Detectedread-write-delete, seeded-state-machine
accept-nonstring-keysvalidation0.925Detectedreject-and-continue
commit-on-exitcrash_recovery0.900Detectedsigkill-after-acknowledgement, sigkill-after-atomic-batch, sigkill-after-delete

Positive control

basic1.003/3
validation1.002/2
transactions1.002/2
compare_swap1.002/2
persistence1.003/3
crash_recovery1.003/3

Reproduction record

Task: durable-kv v0.1.0
Seeds: [17]
Grader SHA-256: 52caa3136434569f96097860bc9bf85eb7ada5f7f9890c0540db81afb441871d
Cases SHA-256: 842a16a1e65700cde92ee5f5aaaa10d494a9fc9d71bd2d99fb68147b19aad63a
Image ID: sha256:78387bc3881b8273120a12ebe6c1ab22b018ccc2c9adf565ae1ac9b536e184ea
Created: 2026-09-14T01:29:51.125110+00:00

Complete JSON evidence for all controls