Hugging Face Announces Evaluation On The Hub
HuggingFace has announced Evaluation on the Hub, a quick and easy to use tool to evaluate any model on any dataset available on the Hugging Face Hub.
Created on June 28|Last edited on June 28
Comment
Hugging Face's newest addition to its expansive lineup of tools is called Evaluation on the Hub. This feature, powered by AutoTrain, allows anyone to easily evaluate any model on any dataset available on the Hugging Face Hub.
There's no code required, so it's accessible to everyone, especially those who just want to quickly see how a model performs on some dataset without needing to go through an annoying code setup.
Datasets feature a leaderboard of models evaluated on them, showing which models perform the best on the data. Evaluating models with the new tool will add them to a dataset's leaderboard, so if you have a particular model in mind that's not on the dataset's leaderboard yet, you can simply evaluate it with the tool to get a good idea of its standing compared to other models.
How do I use Evaluation on the Hub?
To use Evaluate on the Hub, you'll first enter the dataset that you want to evaluate. Some datasets have pre-set configurations, but for the most part, you'll have to take a dive into the advanced configuration tab. Within the advanced configuration tab, you'll find a number of options to define exactly how the evaluation will be carried out, such as the task, split, and columns.
Once the dataset configuration is complete, you can select the model you would like to evaluate on the dataset. From there, just enter your username to be notified once it's complete, and check back later to see the results.
I've found the UI to be a little frustrating and buggy at times, so your mileage may vary.
Find out more
Add a comment
Tags: ML News
Iterate on AI agents and models faster. Try Weights & Biases today.