mirror of
https://github.com/discourse/discourse.git
synced 2026-08-06 13:08:40 +08:00
c7ba71bfb3The new --dataset PATH flag lets you drive evals from a CSV instead of YAML files. Pair it with --feature module:feature_name; each row is turned into its own eval case using that feature’s runner. CSVs must include content (input) and expected_output (expected result) columns; rows with either missing will fail fast. Eval ids are auto-derived from the dataset filename plus row index, so you can reference or inspect them in logs. Example: `./run --dataset evals/datasets/spam.csv --feature spam:inspect_posts --models gpt-4o-mini` runs every row through the spam inspector and validates outputs against the expected values. |
||
|---|---|---|
| .. | ||
| ai_helper.rb | ||
| base.rb | ||
| discoveries.rb | ||
| hyde.rb | ||
| inference.rb | ||
| spam.rb | ||
| summarization.rb | ||
| translation.rb | ||
