Organizing registries in W&B Models
Learn how best to organize your team's models in our Registry in this detailed walkthrough
Created on October 15|Last edited on October 15
Comment
Registries in Weights & Biases (W&B) are organization-wide hubs for managing curated collections of artifacts such as models and datasets. They serve as living systems where teams can track versions, assign aliases (e.g. production, staging, candidate), and collaborate around shared assets.
Rather than thinking of it as a static catalog, your Registry can serve as the central meeting place for your organization’s key ML assets, enabling discovery, collaboration, and governance across teams and workflows.
Why registries matter
Registries underpin the modern ML lifecycle, creating a shared foundation that connects research and operations. They deliver value in several key ways:
- Collaboration: Teams can share datasets and models across projects and business units, reducing duplication and siloing.
- Discoverability: Assets are easy to search, filter, and browse, so it’s always clear which version to use.
- Governance and auditability: Every artifact version is immutable and fully tracked. Lineage graphs show which data, code, and runs produced each model.
- Source of truth: Each collection becomes the definitive record for a family of assets, removing ambiguity over “what’s live.”
- Workflow integration: Registries plug into experiment tracking, CI/CD, and deployment systems so the same assets tested during research are those deployed to production.
In short: registries aren’t just storage; they’re the backbone that supports the responsible scaling of your ML systems.
Key concepts and definitions
As we continue, you'll want to familiarize yourself with the following four terms:
- Registry: The org-level hub for artifact collections.
- Collection: A family of related artifacts (e.g. all versions of a churn model).
- Artifact version: A specific, immutable snapshot of a model, dataset, or other asset.
- Alias: A mutable label pointing to a version (e.g. production → v1.2.0).
Setting up a registry
Getting started with registries is straightforward:
1. Create a registry
- Registries are created at the organization level.
- Choose a scope that reflects a meaningful boundary — e.g. compliance needs, a business unit, or a problem domain.
2. Add a collection
- Within a registry, create collections to represent families of artifacts (for example, all versions of your customer churn model).
- Collections scale easily, so don’t hesitate to create one per dataset or model family.
3. Log artifacts to the collection
- Use the W&B SDK to log datasets, models, or other assets into the appropriate collection.
- Each log creates a new artifact version, automatically tracked and immutable.
4. Assign aliases
- Add aliases such as production, staging, or candidate to indicate which version is currently active in different environments.
- Aliases are mutable, so you can seamlessly promote new versions without breaking downstream workflows.
5. Integrate with workflows
- Connect registries to experiment tracking, evaluation pipelines, and CI/CD so the same assets move cleanly from research to production.
Best practice: Favor collections over registries
One of the biggest mistakes organizations make is creating too many registries. In most cases, it’s better to maintain fewer registries with more collections.
Registries should represent hard boundaries — ownership, compliance, environment. Collections, on the other hand, are lightweight and flexible. They’re the right tool for organizing datasets and models at scale.
A helpful analogy: a registry is like a library, while collections are shelves. You don’t want 50 tiny libraries scattered around; you want a few well-structured ones with plenty of shelves.
Common patterns for organizing registries
Different organizations gravitate toward different setups depending on their size, compliance requirements, and structure. Three patterns are most common:
Domain registries
A domain registry organizes assets around a problem space. For example, natural language processing (NLP), computer vision, or recommender systems. Each registry then contains multiple collections, one per family of related datasets or models.
This approach emphasizes discoverability by expertise: if you’re an NLP researcher, you know exactly where to look for translation models and datasets. If you’re on the vision team, everything you need is in the Vision registry.
Domain registries make sense when:
- You have multiple or tandem ML problem areas. Large organizations often run parallel efforts across NLP, Vision, Speech, or Tabular ML.
- Discoverability is a priority. Grouping assets by domain makes it easy for practitioners to find and build upon relevant existing work.
- Cross-team collaboration matters. Teams within the same problem area can reuse datasets, models, and evaluations without duplication.
Domain registries are less useful in smaller organizations (where all work falls into a single domain) or in compliance-heavy industries (where environment registries or organizationally scoped registries may be a better fit).
How it works in practice
- Teams log artifacts into collections: When the NLP group trains a new translation model, it’s logged into the translation-model collection inside the NLP registry.
- Versions are tracked automatically: Each iteration of the model is immutable, with lineage showing which datasets and code produced it.
- Aliases designate “live” assets: The best-performing translation model gets an alias production → v5.2.0, so downstream systems always know which version is active.
Advantages
- Discoverability drives reuse: A researcher working on summarization can quickly see if a translation dataset could be repurposed, rather than sourcing or labeling data from scratch.
- Clear separation by expertise: NLP specialists don’t have to wade through computer vision datasets.
- Cross-team leverage: Vision researchers can build on existing datasets (e.g. faces or satellite imagery) without duplicating work.
- Scalability: As new problem domains emerge—say, reinforcement learning or multimodal—a new domain registry can be added without disrupting the existing structure.
Tradeoffs
- Overlap between domains: Some assets (e.g. multimodal datasets) don’t neatly fit into a single domain. These require clear ownership rules.
- Potential for silos: While discoverability within a domain improves, discoverability across domains may suffer if registries aren’t well-tagged or searchable.
- Governance overhead: Each registry needs a curator to enforce naming conventions and ensure consistency.
Domain registries are a natural fit for large, research-heavy organizations like Moonshots R Us, where multiple ML problem areas coexist. They optimize for discoverability, collaboration, and scalability by grouping assets according to domain expertise.
Organization unit registries
Instead of organizing by technical domain, this pattern organizes registries by organizational ownership, assigning registries to business units or teams. Each unit owns its registry and manages the collections within it. To put this into practice, you’d delegate ownership of assets to specific teams or divisions. Each unit gets its own registry, where it curates collections of models and datasets relevant to its scope.
This approach emphasizes autonomy and accountability. Teams can manage their own registries end-to-end, while still contributing to a broader organizational ecosystem.
This structure makes sense when:
- You’re a federated organization: Large enterprises (banks, telecoms, global corporations) often operate with semi-independent units.
- Autonomy is important: Each team or division may need the freedom to manage its own assets without central bottlenecks.
- Ownership boundaries are clear: Assets map directly to business functions (e.g. credit risk models, marketing datasets).
Organization unit registries may be less effective in smaller organizations (where units overlap heavily) or highly research-driven contexts (where domain registries often work better).
Let's walk through a setup with a fictional company we'll call Moonshots R Us. Imagine it has grown from a pure R&D shop into a diversified enterprise with multiple product lines. They decide to structure their registries around business units:
Credit Risk Registry
- default-prediction-model (collection)
- loan-performance-dataset (collection)
Marketing Registry
- churn-model (collection)
- campaign-dataset (collection)
- segmentation-model (collection)
Operations Registry
- supply-forecasting-model (collection)
- inventory-dataset (collection)
How it works in practice
- Each unit manages its own registry: The Credit Risk team owns the Credit Risk registry. They decide how to curate versions, enforce access, and promote assets.
- Collections map to business outcomes: The Marketing registry includes a churn-model collection with multiple versions, plus a campaign-dataset collection tied to specific promotions.
- Aliases capture status: The Marketing team tags the current production churn model with production → v3.4.1, while keeping older versions available for audit.
- Autonomy with alignment: Each unit works independently, but naming conventions and metadata tags (team=marketing, env=prod) ensure the broader organization can still search and discover assets across registries.
Advantages
- Clear ownership: Every registry has an accountable team that manages curation, access, and promotion.
- Operational autonomy: Units can move at their own pace without waiting on central approval.
- Alignment with business goals: Collections naturally map to the models and datasets that drive outcomes for that business function.
Tradeoffs
- Risk of duplication: Different units may create overlapping datasets or models if coordination is weak.
- Cross-unit discoverability: It’s harder for one team to find and reuse another team’s work unless metadata is standardized.
- Governance complexity: Without strong conventions, business unit registries can drift into inconsistency.
Business unit registries are a strong fit for federated enterprises like Moonshots R Us. They optimize for ownership and autonomy by aligning registries with organizational units, ensuring each team can manage assets independently while contributing to the larger system.
This pattern is particularly useful when clear accountability is more important than technical grouping, and when different teams need the freedom to manage their own pipelines, datasets, and models.
Environment registries
An environment registry separates assets by stage of the ML lifecycle — for example, Research, Staging, and Production. Collections often mirror across environments, but only vetted versions move forward into later stages.
When to use environment registries:
- Strict compliance or audit needs: Regulated industries (finance, healthcare, government) often require clear lifecycle separation.
- Operational clarity: Everyone knows which assets are experimental versus production-ready.
- Governance: Promotion into production registries can be tied to review or approval workflows.
Here's an example:
Research Registry
- churn-model (collection)
- transactions-dataset (collection)
Staging Registry
- churn-model (collection) – candidate versions under evaluation
Production Registry
- churn-model (collection) – current alias: production → v1.2.0
This pattern works especially well for enterprises that need strict auditability and want lifecycle stages to be obvious. It’s generally not the best model for most customers, but when it��s the right fit, it’s really the right fit.
Anonymized, real customer examples
These are anonymized and lightly simplified examples of real registry setups in production today.
Customer 1: Defense contractor
Customer 2: Pharmaceutical
Customer 3: Robotics lab
Recommendations
- Define trust boundaries carefully: Only create a registry when isolation is required (compliance, domain, ownership).
- Scale with collections, not registries: They’re flexible, lightweight, and easier to manage.
- Leverage aliases: Use production, staging, candidate to signal status without duplicating registries.
- Assign ownership: Each registry should have a clear curator or owning team.
- Standardize conventions: Apply consistent naming and metadata tags (e.g. domain=nlp, team=fraud, env=prod) to keep collections organized.
To recap
Registries serve as hubs for discovery, collaboration, and governance across the ML lifecycle. They bring structure to research and operations while enabling teams to scale responsibly. They can serve as a dual discovery and enforcement mechanism, becoming the source of truth for your organization’s ML function overall.
The key principle: fewer registries, more collections. Use registries for hard boundaries; let collections, versions, and aliases handle day-to-day organization. We will continue to augment registries and collections and deliver functional and usability enhancements. Let us know how you use your registry!
Add a comment