Evaluation
What is ML evaluation?
ML evaluation scores model or pipeline outputs against labels or a judge, so you can compare versions.
Updated
How it works
- Fix the inputs at a table version.
- Score the output columns.
- Compare scores across versions.
What it is not
It is not the database that stores those outputs. An eval framework is not a multimodal table.
ML evaluation: this, and the thing it is confused with
| This | Not this | |
|---|---|---|
| Scores | Outputs | The storage engine |
| Needs | Labels or a judge | Only a metric name |
| Lives beside | The table of outputs | Instead of the table |
Where Pixeltable fits
Pixeltable stores the outputs you score. It is not an eval product. A score can still be a column or an external judge reading the table.
Questions
How does ML evaluation work?
Fix the inputs at a table version. Score the output columns. Compare scores across versions.
What is ML evaluation often confused with?
It is not the database that stores those outputs. An eval framework is not a multimodal table.
Next step
Install the Pixeltable skill, or sign up for Cloud and run the same file on a hosted database.