Artifacts Lineage Clustering in Weights & Biases
We just made a change to W&B Artifacts. Here's what you need to know
Created on March 12|Last edited on March 13
Comment
Sophisticated enterprise machine learning teams need full visibility into their model and dataset pipelines.
Effective tracking and overall workflow management of production models starts with maintaining a strong audit trail and building lineage - the traceability and reproducibility of any model, with full details on how and why it was developed.
W&B automatically tracks and versions all the artifacts used and logged in any given run. W&B Artifacts then builds graphical views of these logged artifacts, allowing users to explore the full lineage of an artifact either within the W&B App UI or programmatically. Many of our enterprise users rely on Artifacts lineage to see which version of a dataset was used to train the current model in production or see all models that were trained with a specific version of data.
They also need to maintain reliably detailed audit trails for data lineage, gain observability into their pipelines, improve model exploration and build collaborative workflows across their development and production team. Artifacts Lineage tackles that with a graph-based view that shows a general overview of your model pipeline, while allowing for deep dives into specific areas.
In our efforts to continue making this valuable and integral for ML practitioners, we’re releasing a brand new UI for the Artifacts Lineage Direct Acyclic Graph (DAG). This updated UI introduces clustering so that users can incrementally load new branches, leading to a huge decrease in loading time for larger lineage graphs and improving the overall usability and functionality of the graph view.
See Artifacts Lineage Clustering In Action
With this new update, when a level of the graph (i.e. a vertical column of nodes) has 5 or more runs or artifacts, a cluster will now be created, with a search bar within the cluster to find specific versions. To continue investigating the lineage of a node in a cluster, a node can be pulled out from a cluster.
Clicking on a node—whether it’s an artifact or a run node—will open up a preview drawer with an overview of the node instead of opening up a new browser tab as it did previously. This change makes it easier for users to view run or artifact details without losing context. Users can easily see all the important information about that particular node, including a description of it, the time and date it was logged and executed, the user who logged the artifact or executed the run, and the size and number of files it contains.
This updated UI will greatly improve the performance time for loading entire lineage graphs with all parent, children and sibling nodes, enabling much faster queries and delightful experiences for users when exploring the lineages of their Artifacts. The new UI also makes navigating through certain artifact versions and tracing their entire histories much easier, allowing for better evaluations and providing peace of mind for audit requests.
Check out the updated Weights & Biases Artifacts UI today and let us know what you think! Feel free to explore in this example project to see what the new UI looks and feels like in action.
Add a comment
Iterate on AI agents and models faster. Try Weights & Biases today.