Introducing Queue Config Templating for W&B Launch
Our new Launch feature makes managing complex machine learning workflows a whole lot easier. Learn how to get started today.
Created on November 14|Last edited on November 16
Comment
Introduction
W&B Launch was built to help ML engineers take advantage of specialized hardware and powerful compute resources to run more experiments. But running more experiments at dramatically increased scale requires ML and MLOps teams to rapidly alter both model configuration (hyperparameters and training data) and hardware requests as seamlessly as possible.
With the introduction of Queue Config Templating for W&B Launch, we want to make it even easier for ML engineers to alter their compute configurations, and for MLOps engineers to administer and manage secure guardrails to ensure responsible compute consumption.
Admins can now add template variables to their queue configurations for key fields like memory, GPU, and duration. ML engineers can quickly alter these fields whenever they launch a new job. Admins can also set defaults, mins, and maxes for each of these fields in an optimal way that allows their ML team to effectively and efficiently use the pool of resources together.
Here’s a walkthrough example of how to set up queue allowlisting for your ML engineers:
First, head to your user settings page and toggle on the Launch allow list feature flag.
MLOps: Queue Configuration
Next, we need to work with MLOps to configure our queue (in this example, a Kubernetes queue). Let’s edit the queue config to add templated fields for memory, GPUs, and maximum run time.
Note that we need to append units (in this case (Mi) when requesting memory. We’ve also added metadata fields for W&B project and entity to observe that usage in our Kubernetes console.
When we click Parse configuration, we’ll see tiles for each of the three templated fields appear below. This is where we’ll edit the display name, min/max/default values, and label for the field.
These fields can also include enumerations of strings, like specific base images you’ve created, specific GPU types on your cluster (e.g. A100 vs. H100), or instance types in a cloud environment (e.g. p2.large vs. p3.2xlarge).
Once we save the config, our ML engineers will see a new experience when launching jobs to this queue.
ML Engineers: Requesting Resources
Now when we select this queue to launch a job to, we'll see each of the fields we just curated:
We can now easily change our configuration requests for each of these fields depending on the context of the job and scenario and with limit ranges clearly displayed, and then submit the job as usual. We can still see the full configuration if we want but note that non-admins cannot edit the raw queue configuration. This allows admins to expose powerful compute clusters fairly and equitably to all members of the team without having to worry about configurations changing.
Queue Config Templating for W&B Launch is currently available behind a User Settings feature flag for all W&B users in Cloud, Dedicated Cloud, and Customer Managed environments. Toggle it on to try it out for yourself, and please reach out if you have any questions about Queue Config Templating or refer to Launch docs for more information.
Add a comment
Iterate on AI agents and models faster. Try Weights & Biases today.