Introducing semantic based coloring for W&B Models
We've made it easier to understand key relationships among configuration parameters and metrics in your model runs. Here's what you need to know.
Created on July 21|Last edited on July 21
Comment
When evaluating AI and machine learning models, enterprises frequently rely on metrics such as accuracy, loss, or perplexity. These clear, quantitative measures reveal not only how well a model understands data and minimizes errors but if a model meets the standards required for production deployment.
If we only cared about hitting performance targets, we could automate model promotion once critical metrics reach a set threshold. But for AI and ML model builders, improving outcomes also comes from understanding and refining models by visually inspecting the data. W&B Workspace empowers your teams to dig deeper into the results and learn from each iteration.
In this piece, we'll be highlighting the benefits of semantic coloring, which, along with a number of other Workspace features, empowers easy exploration of the connections among key metrics and configuration parameters. Users can easily explore and customize plots and tables and entire dashboard views to quickly identify what works, and what does not.
Semantic coloring
Validation accuracy chart without semantic coloring (left) vs. validation accuracy chart with semantic coloring (right)
Semantic coloring lets you highlight and better understand relationships among configuration parameters and metrics by generating a color palette based on a single parameter or metric and applying it to Workspace line charts, clarifying your analysis and letting teams quickly spot patterns and correlations that would otherwise be hidden in the data.
For example, if you’re plotting validation accuracy over training steps and coloring each run by MFU, you can instantly notice whether models with higher hardware utilization also reach peak accuracy faster, which can help diagnose if infrastructure changes are translating into meaningful gains in real model performance. Or, when tracking accuracy or loss over epochs and coloring the data by learning rate, you can quickly see which learning rates lead to faster convergence or more stable training. Understanding how these configuration parameters and metrics interact provides a deeper grasp of model behavior, improving your ability to diagnose issues and fine-tune models for optimal results.
Semantic coloring helps you:
- Spot where configuration parameters and metrics align or conflict,
- Detect outliers and bias,
- Efficiently compare experiments, and
- Make more informed model selection and tuning decisions.
Color palettes can be used to visualize ranges for any numerical configuration parameter or metric. Opportunities to control semantic coloring include:
- Choosing the Y value for bucket creation. Options include Max, Min, and Latest
- Selecting the number of buckets to create from the range to define different levels of granularity in each color scheme
- Inverting the existing color assignments or choosing custom colors for each bucket
These settings allow you to re-color charts for improved pattern detection and aesthetic appeal, making complex configuration parameter and metric relationships intuitively visible and turning result data into actionable visual insights for more nuanced and robust AI model evaluation.
Parameter importance
Recognizing which hyperparameters have the greatest impact on model performance is essential. Developing a top-performing model means thoughtfully tuning these parameters, not just hoping to stumble upon good results with random guesses. While exploring random combinations may be a starting point, genuine improvement relies on identifying the most effective configuration for your model.
The W&B Workspace parameter importance chart helps you quickly identify which parameters most impact your ML and AI model training outcomes. Just select the metric you want to analyze, and the chart will display two columns for each parameter.
Correlation
Correlation is the linear correlation between the hyperparameter and the chosen metric. So a high correlation means that when the hyperparameter has a higher value, the metric also has higher values and vice versa. Correlation is a great metric to look at but it can’t capture second order interactions between inputs and it can get messy to compare inputs with wildly different ranges.
Importance
Using hyperparameters as inputs and the performance metric as the target, W&B trains a random forest and reports each hyperparameter’s importance, helping you identify which hyperparameters truly affect model results, streamlining your tuning process.
Parallel coordinates plots
W&B Workspace line and bar charts show aggregated results and trends over time. Parallel coordinate plots reveal key parameter and metric combinations tied to a chosen “metric of interest.” While you can review every run to find model checkpoints with the highest accuracy, parallel coordinate plots help you quickly spot patterns between hyperparameters and metrics.
The parallel coordinate chart consists of multiple vertical axes representing either categoricals or value ranges for a specific input parameter. The user chooses a key performance metric, like accuracy or loss, to display on the final axis. Each curved line in the chart represents a single experiment run.
Patterns become more easily identifiable when applying a filter on the goal metric, validation accuracy in the image above, coloring only the runs that fall within a defined range. For example, the image above highlights only runs that resulted in validation accuracy between .93 and .94.
Parallel coordinate plots extend beyond ML model training and are also valuable for training or fine-tuning AI models. Rather than focusing on accuracy or loss, you can also analyze hyperparameter relationships to metrics like BLEU and ROUGE-2 scores, or benchmarks such as HellaSwag and MMLU to fine-tune AI models effectively.
Conclusion
Choosing the right model for production deployment depends on thoroughly understanding performance metrics and evaluation outcomes. With intuitive visualization and analysis tools, you can turn experiment data into actionable insights. W&B Workspace’s tables and charts provide important connections among configuration parameters and metrics, supporting confident model selection.
Semantic coloring joins the list of W&B Models features available to help ML and AI model builders develop a better understanding which hyperparameter and metric relationships create the best performing models. By visualizing correlations among configuration parameters and metrics in a meaningful and intuitive way, Weights & Biases equips its users with the tools they need to build better models faster.
Add a comment
Iterate on AI agents and models faster. Try Weights & Biases today.