Our Partnership with AWS: A Year in Review
With AWS re:Invent 2022 in full swing, we wanted to take a moment and look back at our growing partnership
Created on November 30|Last edited on November 30
Comment
Introduction
Amazon Web Services (AWS) and Weights & Biases is a strong combination for organizations of all sizes to build and leverage machine learning. 2022 was a strong year for the W&B and AWS partnership, with progress on multiple fronts including product development, technical content, and partnership differentiation. Here are the highlights:
Partnership Differentiation
In 2022, W&B focused on differentiating our product in the AWS ecosystem. We started with the AWS Foundational Technical Review, ensuring the W&B platform meets AWS’ well architected principles across operational excellence, performance, reliability, cost optimization, and security. W&B then achieved the AWS Partner Network Machine Learning (ML) competency for technology partners. To achieve the ML competency, W&B completed a technical review of the W&B platform with an AWS ML expert, validating W&B’s expertise integrating with ML workloads on AWS. W&B also demonstrated customer success on ML use cases, validating the impact W&B has on customer’s ML workloads. Finally, as of early 2022, W&B is listed on the AWS Marketplace. Customers purchasing W&B via the Marketplace can leverage their existing billing mechanism with AWS and customers who have an annual spend commitment with AWS can allocate a proportion or all of their W&B spend towards their AWS annual commitment.
Today, we are proud to announce W&B is a launch partner for the Amazon SageMaker Service Ready program. The designation validates W&B’s integration with Amazon SageMaker, AWS’ industry leading MLOps platform. If you’re interested in learning how to use W&B with Amazon SageMaker, check out some of our technical content listed below.
“Weights and Biases has made tremendous progress developing on AWS technologies and differentiating their technology in the AWS ecosystem,” said Harish Surendranath, Head of ISV Partnerships and Marketplace for AI, HPC, and IoT. “Weights and Biases are providing Customers developing their ML applications on AWS a great developer experience, especially for deep learning and AI use cases. We are excited to work with Weights & Biases to provide ML practitioners with advanced MLOps capabilities on AWS.”
Product Development
The foundation of the partnership is W&B’s technical development on AWS. In 2022, the product development focused on two areas: creating flexible W&B server deployment options on AWS and integrating the W&B platform with AWS services such as Amazon SageMaker and Amazon Elastic Kubernetes Service (EKS).
A new feature within Weights & Biases, W&B Launch, will support Amazon EKS and Amazon SageMaker training as first class citizens. W&B Launch enables users to easily queue and launch jobs (no pun intended), and easily tweak hyperparameters and input data to retrain models. W&B Launch is currently in private preview, if you’re interested in testing out the capability, please reach out to the W&B team.
Hosting and managing the W&B server on AWS was also a key focus in 2022. For customers who would like the convenience of a solution fully hosted by W&B, and the data security of a single tenant, we released the W&B Dedicated Cloud for AWS. For customers who choose to manage the software themselves, W&B has created a Terraform for deployment on AWS to accelerate the deployment in their AWS VPC. If you have questions about either of these deployment methods, please contact us.
Finally, W&B is compatible with training workloads on SageMaker Studio, can track experiments with SageMaker’s training infrastructure, and is fully compatible with training workloads on Amazon EKS. A complete walkthrough of leveraging W&B with SageMaker Studio can be found on our joint blog post.
Finally, W&B is compatible with training workloads on SageMaker Studio, can track experiments with SageMaker’s training infrastructure, and is fully compatible with training workloads on Amazon EKS. A complete walkthrough of leveraging W&B with SageMaker Studio can be found on our joint blog post.
Technical Content
Finally, the W&B and AWS teams collaborated on technical content to help bring the innovations into the hands of ML developers. We teamed up to create technical examples, blogs, and finally a live workshop. Each of these pieces of collateral walks through a use case such as sentiment analysis or image segmentation, and how to leverage the various parts of the W&B platform such as experiment tracking, tables, artifacts, and sweeps with a jupyter environment on Amazon SageMaker. Links to the collateral are below, with more to come in 2023.
Deploy Sentiment Analyzer Using SageMaker and W&B
Exploring wandb's SageMaker integration to train, tune and deploy a sentiment analysis API
Direct Marketing with XGBoost and Amazon SageMaker
Analysis of experiments from SageMaker Immersion Day direct marketing notebook.
Text Classification With AWS SageMaker, HuggingFace, and W&B
This article provides a tutorial on running exploratory data analysis and natural language processing experiments using AWS SageMaker, HuggingFace, and W&B.
Using AWS Sagemaker and Weights & Biases Together on Digit Recognition with MNIST
We've received a few questions about how W&B works with Amazon Sagemaker. Here's a quick tutorial on a simple dataset to get your started.
Conclusion
As more organizations continue to build innovative applications leveraging machine learning, deep learning, and AI on AWS, Weights & Biases will continue to build critical MLOps functionality for both the individual ML researchers and the teams that support them. Weights & Biases will continue to leverage innovative capabilities created by AWS to improve ML researchers productivity, provide insights to help ensure full AWS resource utilization, and provide a single system of record for all ML experiments in an organization. There’s much more to come in 2023!
Add a comment
Iterate on AI agents and models faster. Try Weights & Biases today.