Square Brings Conversational AI to Businesses of All Sizes With W&B

"Now when we train, we don’t talk to S3. We don’t need to know details about where it’s stored. We just talk to the Artifacts registry and we can track the lineage for everything."
Ethan Rosenthal
Staff AI Engineer

A Customer-Centric Company

As a business owner, how you talk the talk says a lot about how you walk the walk.
Building customer relationships are critical to ensuring long-term success. And good customer relationships are based on customer experience—what it’s like for people to do business with you. A single interaction can be the difference between a rave review or a catastrophic complaint.
But to deliver consistently effective customer communication takes time, effort, and at times, patience. This is where Square comes in.
Known for its reliability, ease of use, and reasonable cost, Square’s mission is to help businesses participate and thrive in the economy. And the ML team at Square is working on tools to take that one step further to help the everyday entrepreneur make each customer interaction a success.

Square Messages and Square Assistant

Whether it’s owning a hair salon, restaurant, or boutique, managing a business means having thoughtful conversations with your customers. Often this communication comes in the form of convenient texting and can be for all kinds of reasons—from checking in with vendors and giving employees direction to juggling client appointments. These one-to-one messages are what businesses rely on to run their operations every day.
To help track all those conversations, Square introduced Square Messages, a messaging hub designed to keep all customer communication in one place. Anything from texts, receipts, invoices, feedback, and more can be sent and received through Square Messages. But that’s not all—busy entrepreneurs can also use Square Assistant, an automated messaging tool that responds to customers in real-time. Business owners can think of the tool as their personal AI assistant.
Take, for example, managing appointments. If a customer wants to confirm, reschedule or cancel an appointment, Square Assistant can instantly reply to the customer to provide a solution for each of those scenarios. What’s great about leveraging Assistant is that it ensures all customer interactions are direct, quick, and professional.

Machine Learning at Scale

What goes on behind the scenes to make both tools successful? A robust automatic retraining framework, production-ready metrics, and complete lineage of the lifecycle for all models. At Square, W&B had a part to play in each of those key items.
Conversations are happening every second between customers and businesses that use Square. With all the new data generated from those messages, Square sets up an automated retraining of their model—a GPT-style large language model—that trains the model with the conversational data that occurred in the previous month. All the training data and models are stored in W&B to keep the entire team’s progress in an organized, standardized, and central location.
From there, Square also tags the models in W&B based on the stage of the model. And Square has a unique way of categorizing its models to determine which gets deployed in production.

A battle of the models

When Square automatically retrains its model monthly with fresh data, it first gets deployed as a Shadow Model. This is where the model is quietly used in the background: it generates text, but no one sees it. Next, the Shadow Model gets promoted to become a Challenger Model, and as the name implies, it challenges the existing model—the Champion Model. The Challenger and Champion go through an automated A/B test, and if the Challenger performs better, it gets promoted as the new Champion model. This process leverages not only W&B but scalable infrastructure provided by Amazon Web Services, including both data storage and access to GPUs for model training.
Request routing in shadow mode:
Request routing during the A/B test:
Square’s “jobs” framework for how a model moves from shadow → challenger → champion:

Production-ready metrics

Throughout all this, W&B acts as the source of truth to help version models and transition them through those three stages: Shadow, Challenger, and Champion. Since W&B allows Square to monitor their training in real-time, Square can then quickly and iteratively debug their models, ensuring performance metrics are met for deployment. “We use Weights & Biases as the brain of our operations. It ensures the highest quality models make it to production,” said Gabor Angeli, Senior Manager at Square.

End-to-End Lineage

Now, with all the different models leveraged at Square, maintaining model lineage becomes increasingly important. This is where W&B Artifacts come into play. With Artifacts, Square knows exactly which datasets are used to create their models—without second guessing or asking others.
“Now when we train, we don’t talk to S3. We don’t need to know details about where it’s stored. We just talk to the Artifacts registry and we can track the lineage for everything,” said Ethan Rosenthal, AI Engineering Manager at Square.
Past that, Artifacts have also been helpful in giving Square confidence when building new models and testing them against existing Champion models. Now, Square can rest assured that when comparing model performance, the models are trained with the same data and then evaluated with the same test data too. In comparison to its previous solution, Square had only basic experiment tracking features and no capability similar to Artifacts that could assist with tracking every step of their ML pipeline.
Another aspect of W&B that is particularly beneficial for Square is W&B Reports. Reports allow feedback to be easily passed on from one team member to another. At a glance, the dynamic document shows all the runs and their relevant parameters, metrics, logs, and code—giving the full context of one’s work. “Creating a report has been much more helpful to see what everyone’s working on, as opposed to getting a Google sheet with metrics that are probably outdated by the time I read it,” explains Ethan.

The Businesses of Tomorrow

With more workers seemingly leaving their 9-5 grind to become their own bosses, having a solution like Square becomes all the more important. Since the start, Square has always focused on helping businesses grow, thrive, and succeed. And that mission isn’t going anywhere.
As Square continues to build tools for entrepreneurs to work smarter and save time, having a comprehensive MLOps platform is key. Leveraging W&B means Square can focus on developing the best models for production, share learnings to build on top of each other’s work, and gain a complete understanding of data and model lineage. For a mission-driven organization like Square, having an enterprise-grade, end-to-end MLOps platform is non-negotiable—and to no surprise, W&B is exactly that.