Comparisons
Compare your current and changed assistant on the same tests.
On this page
Use the same tests for both versions
Choose an exact published suite version and the cases to compare. Both assistants receive the same saved requests, restaurant list and practical needs. Each produces new answers; old discovery answers are not reused.
Profile A / Profile B mean the diners. Baseline / Candidate mean your current assistant and the changed version you want to try.
Run your current and changed assistant
You can use one model with two different prompts. Configure each version in your local assistant code, set its matching release label and run the runner for its queued work. Changing the label alone does not change the prompt or model.
export TASTECASE_ASSISTANT_VERSION="concierge-prompt-v1" node packages/runner/dist/cli.js run-once
The label must match the version chosen in TasteCase. A wrong or missing label cannot claim comparison work.
Read what changed
When both versions finish, freeze the report to save it. You may also choose to freeze a partial report, but missing runs stay missing.
Read rule outcomes separately from changed recommendations. A different answer is not automatically worse. Use Changes only to focus on restaurants that moved, appeared or disappeared.
Run limits
A comparison supports up to 10 cases and 120 planned runs, with a deadline within one hour. Additional limits on reclaimed work are listed in Current limitations.