Access Serverless GPUs with W&B Launch and Optumi
Learn more about Optumi's recent integration with W&B Launch
Created on May 1|Last edited on May 8
Comment
As I’m sure many of you know, Weights & Biases recently introduced their new Launch module with the stated goal of providing "easy access to computing–with none of the complexity." Launch makes it easy to "package up code and launch a job into any target environment," building a bridge between ML practitioners and computing resources. This is an exciting step forward for ML productivity!
Most often, compute resources are virtual machines or Kubernetes clusters in one cloud provider, a setup that makes sense for many companies and workloads. However, things are changing very quickly in the ML space. It feels like a groundbreaking generative model gets released every few days 🤯. It’s not uncommon to get excited by a model on Hugging Face, only to find it doesn’t run well (or at all) on the GPU you have access to.
In a dynamic environment, you need dynamic infrastructure. This is why Optumi is integrating our serverless compute platform with W&B Launch. Our goal is to extend the vision for W&B Launch by building a bridge between ML practitioners and affordable, on-demand GPUs in competitive cloud providers.
Here’s an overview of how it will work, through the lens of an ML practitioner:
- Create an Optumi account and subscribe (there's no monthly charge and you'll pay for resources as you go).
- Create a new launch queue of type Optumi. For this example, let’s name it “OptumiQueue."
Now that we have a queue configured, we need to choose a machine to run the launch agent. On that machine install both W&B and Optumi libraries, login and run the W&B agent for the Optumi queue. That code is here:
pip install wandb[launch]wandb loginoptumi loginwandb launch-agent -q OptumiQueue
Next, launch a job from the W&B GUI into the Optumi queue. You’ll be able to describe the machine you want to run on in the side panel (see screenshot below) . For this example, we’ll specify a machine from cloud provider "Lambda Labs" with an "A10" GPU. When you click "Launch now," Optumi will allocate a new machine, run the job and deallocate the machine afterwards.
That's it. You'll see the job result in the W&B GUI.
It’s that easy!
Here are three example runs on different GPUs using Optumi. We ran on NVIDIA V100, T4 and A10 GPUs. You can use WandB telemetry to assess the runs, as well as other Optumi-specific information like training cost. Check the run set section below these plots for the GPU type.
Run set
3
If you’re interested in learning more about the W&B Launch and Optumi integration, reach out to the team at Optumi. You can find them at contact@optumi.com or by filling out this form.
Add a comment
Iterate on AI agents and models faster. Try Weights & Biases today.