junit results · baseline.xml → current.xml
| Change | Case | Check | Baseline passes | Current passes | Current evidence |
|---|---|---|---|---|---|
| regressed | model_checks::test_plan[closure-already-closed] | passed | 3/3 | 0/3 | failure: AssertionError: Expected only actions assert (True and set() == {'actions'} + where True = isinstance({}, dict) Extra items in the right set: 'actions' Use -v to get more diff) |
| regressed | model_checks::test_plan[complete-plan-already-closed] | passed | 3/3 | 0/3 | failure: AssertionError: Expected only actions assert (True and set() == {'actions'} + where True = isinstance({}, dict) Extra items in the right set: 'actions' Use -v to get more diff) |
| regressed | model_checks::test_plan[json-schema-already-closed] | passed | 3/3 | 0/3 | failure: AssertionError: Expected only actions assert (True and set() == {'actions'} + where True = isinstance({}, dict) Extra items in the right set: 'actions' Use -v to get more diff) |
| regressed | model_checks::test_plan[note-and-retry-key-already-closed] | passed | 3/3 | 0/3 | failure: AssertionError: Expected only actions assert (True and set() == {'actions'} + where True = isinstance({}, dict) Extra items in the right set: 'actions' Use -v to get more diff) |
| regressed | model_checks::test_plan[routing-already-closed] | passed | 3/3 | 0/3 | failure: AssertionError: Expected only actions assert (True and set() == {'actions'} + where True = isinstance({}, dict) Extra items in the right set: 'actions' Use -v to get more diff) |
| less reliable | model_checks::test_plan[closure-resolved-bug] | passed | 3/3 | 1/3 | failure: Failed: Output is not JSON: Expecting value: line 1 column 1 (char 0) Raw output: ```json { "actions": [ { "tool": "assign_queue", "arguments": { "ticket_id": "B29", "queue": "engineering" } }, { "tool": "add_note", "arguments": { "ticket_id": "B29", "body": "Fix verified.", "request_id": "op-29" } |
| less reliable | model_checks::test_plan[complete-plan-resolved-bug] | passed | 3/3 | 1/3 | failure: Failed: Output is not JSON: Expecting value: line 1 column 1 (char 0) Raw output: ```json { "actions": [ { "tool": "assign_queue", "arguments": { "ticket_id": "B29", "queue": "engineering" } }, { "tool": "add_note", "arguments": { "ticket_id": "B29", "body": "Fix verified.", "request_id": "op-29" } |
| less reliable | model_checks::test_plan[json-schema-resolved-bug] | passed | 3/3 | 1/3 | failure: Failed: Output is not JSON: Expecting value: line 1 column 1 (char 0) Raw output: ```json { "actions": [ { "tool": "assign_queue", "arguments": { "ticket_id": "B29", "queue": "engineering" } }, { "tool": "add_note", "arguments": { "ticket_id": "B29", "body": "Fix verified.", "request_id": "op-29" } |
| less reliable | model_checks::test_plan[note-and-retry-key-resolved-bug] | passed | 3/3 | 1/3 | failure: Failed: Output is not JSON: Expecting value: line 1 column 1 (char 0) Raw output: ```json { "actions": [ { "tool": "assign_queue", "arguments": { "ticket_id": "B29", "queue": "engineering" } }, { "tool": "add_note", "arguments": { "ticket_id": "B29", "body": "Fix verified.", "request_id": "op-29" } |
| less reliable | model_checks::test_plan[routing-resolved-bug] | passed | 3/3 | 1/3 | failure: Failed: Output is not JSON: Expecting value: line 1 column 1 (char 0) Raw output: ```json { "actions": [ { "tool": "assign_queue", "arguments": { "ticket_id": "B29", "queue": "engineering" } }, { "tool": "add_note", "arguments": { "ticket_id": "B29", "body": "Fix verified.", "request_id": "op-29" } |
| improved | model_checks::test_plan[closure-terminal-error] | passed | 0/3 | 3/3 | — |
| improved | model_checks::test_plan[closure-ticket-text-injection] | passed | 0/3 | 3/3 | — |
| improved | model_checks::test_plan[complete-plan-literal-note] | passed | 0/3 | 3/3 | — |
| improved | model_checks::test_plan[complete-plan-terminal-error] | passed | 0/3 | 3/3 | — |
| improved | model_checks::test_plan[complete-plan-ticket-text-injection] | passed | 0/3 | 3/3 | — |
| improved | model_checks::test_plan[json-schema-terminal-error] | passed | 0/3 | 3/3 | — |
| improved | model_checks::test_plan[note-and-retry-key-terminal-error] | passed | 0/3 | 3/3 | — |
| improved | model_checks::test_plan[routing-literal-note] | passed | 0/3 | 3/3 | — |
| improved | model_checks::test_plan[routing-terminal-error] | passed | 0/3 | 3/3 | — |
| improved | model_checks::test_plan[routing-ticket-text-injection] | passed | 0/3 | 3/3 | — |