Harnessing Langchain and Weights & Biases to Enhance Database Interactions on Snowpark

Learn how our new integration with Snowflake works, complete with interactive code, a demo video, and a whole lot more
Created on July 7|Last edited on July 19
Comment
﻿
IntroductionIn the rapidly evolving landscape of machine learning and data science, innovative tools and frameworks are constantly surfacing to optimize workflow. Weights & Biases is excited to announce has now integrated with Snowpark Container Services, enabling users to host and deploy their W&B instances seamlessly. 
To highlight this powerful collaboration, we're showcasing a tool to enhance interactions with databases by allowing users to use natural language to generate and execute valid SQL. This can significantly help reduce the barrier to entry needed to communicate with our data. This demo leverages Langchain, a framework for LLM development and W&B Prompts to accomplish our goal. 
There's a demo video below as well as the attendant code for this project if you'd like to watch instead or want to get your hands dirty immediately. 
Prefer to Watch over Reading?﻿
﻿
Show me the Code!
What is Snowpark Container Services?As a novel extension of Snowflake's processing engine, Snowpark Container Services offer developers the flexibility to deploy container images in a Snowflake-managed infrastructure.
Weights & Biases – Powering ML WorkflowsWeights & Biases is an end-to-end platform for managing the machine learning lifecycle. It enables teams to streamline their workflows by providing functionalities such as debugging, comparing, and reproducing models. You can record vital parameters like model architecture, hyperparameters, git commits, GPU usage, and more. W&B also fosters collaboration within teams and maintains a robust system record for your ML projects.
Snowflake 🤝 W&B Announcement!
How Snowpark Container Services and W&B Accelerate Your AI Development
We're thrilled to announce that we're a launch partner for Snowflake's new Snowpark Container Services offering. Here's what you need to know: 
﻿
Enter Langchain: Flexible Interactions with LLMsLangchain is a popular framework that facilitates interaction with large language models (LLMs) from leading providers such as OpenAI's GPT, Cohere's Command, and Anthropic's Claude. In this example, Langchain empowers users to develop autonomous agents, which can reason over natural language questions, generate valid SQL, and execute it within our environment.
One Line Langchain 🤝 wandb IntegrationThe Langchain + wandb integration allows users to visualize and inspect the execution flow of LLMs/Chains/Agents, analyze inputs and outputs, view intermediate results, and manage prompts and LLM agent configurations securely.
from langchain.callbacks.tracers import WandbTracer
Loading the Langchain SQL AgentWe load all the necessary modules from LangChain to load a SQL agent developed to answer questions using available tools such as QuerySQLDataBaseTool, InfoSQLDatabaseTool, ListSQLDatabaseTool, and QueryCheckerTool explained below:
QuerySQLDataBaseTool: The input to this tool is a detailed and correct SQL query, while the output is a result from the database. If the query is not correct, an error message will be returned. If an error is returned, rewrite the query, check the query, and try again. If you encounter an issue with Unknown column 'xxxx' in 'field list', use schema_sql_db to query the correct table fields.
InfoSQLDatabaseTool: Here, the input is a comma-separated list of tables and the output is the schema and sample rows for those tables. Be sure that the tables actually exist by calling list_tables_sql_db first! Example input: 'table1, table2, table3'
ListSQLDatabaseTool: Our input is an empty string, while our output is a comma-separated list of tables in the database.
QueryCheckerTool: Finally, we can use this tool to double-check if our query is correct before executing it. Always use this tool before executing a query with query_sql_db!
from langchain.agents import create_sql_agent
from langchain.agents.agent_toolkits import SQLDatabaseToolkit
from langchain.sql_database import SQLDatabase
from langchain.llms import Cohere
from langchain.chat_models import ChatOpenAI, ChatAnthropic
from langchain.agents import AgentExecutor
Choose Our Large Language ModelsFor the demo, we'll be testing a range of LLMs and LLM providers. Here's the list:
llms = {
    "command": Cohere(model="command"),
    "command-nightly": Cohere(model="command-nightly"),
    "claude-v1": ChatAnthropic(model="claude-v1"),
    "gpt-3.5": ChatOpenAI(model_name="gpt-3.5-turbo"),
    "gpt-4": ChatOpenAI(model_name="gpt-4")
}
Define Our DatabaseFor our example, we'll use the Chinook database, a sample digital music store database with various relationships and tables ideal for testing the validity of the SQL generated by our agent.
# Set connection parameters...
DB_URI = URL(
        account=account,
        user=user,
        password=password,
        database=database,
        region=region,
        schema=schema
)
db = SQLDatabase.from_uri(DB_URI)
Define Our QuestionsThe last step before running our agent is to define questions in natural English language. The agent will then be tasked with generating valid SQL queries from these questions.
questions = [
        "Describe the playlisttrack table",
        "Describe the playlistsong table",
        "List the total sales per country. Which country's customers spent the most?",
        "Show the total number of tracks in each playlist. The Playlist name should be included in the result.",
        "Who are the top 3 best selling artists?",
]
Run!for name, llm in llms.items():
    wandb_config = {
        "project": "wandb_prompts_snowflake_demo",
        "name": name,
        "config": {
            "questions": questions
        }
    }
    toolkit = SQLDatabaseToolkit(llm=llm, db=db)
    agent_executor = create_sql_agent(
        llm=llm,
        toolkit=toolkit,
        verbose=True
    )
    wandb_tracer = WandbTracer(wandb_config)
    for question in questions:
        try:
            answer = agent_executor.run(question, callbacks=[wandb_tracer])
            print(answer)
        except Exception as e:
            print(e)
            pass
    wandb_tracer.finish()
Using the W&B Prompts DashboardThe Weights & Biases dashboard allows users to easily analyze and visualize the interactions and execution flow of the large language models. The Trace Table at the top of the dashboard provides high-level information about what you’ve logged, including whether the chain was successful, the inputs and outputs of the chain, the main components of the chain, and any errors that occurred.
﻿
Run set5
﻿
Clicking different rows in the Table above will update the Trace Timeline view with more details. Meanwhile, the bottom half of the dashboard provides the entire execution trace of the chain and all of the component Chains, LLMs, and Tools used by the Agent.
If your chain failed, the dashboard will highlight the component that failed, helping you quickly debug any issues. For example, if there was a problem in the generated SQL, you can check the inputs to the SQL component and identify what went wrong.
Real-World Application: Tektite CorporationConsider Tektite Corporation, an eCommerce giant that wants to analyze customer behavior across different geographies. Their data science team is tasked with finding out the total sales per country and the country where customers have spent the most. Instead of writing complex SQL queries, they decide to use Langchain and W&B to simplify this task.
They first set up their environment and load the relevant modules to create an agent. Next, they integrate Langchain with W&B and choose the LLMs they want to test. They connect to their corporate database and define their questions in simple English: ”List the total sales per country. Which country's customers spent the most?”
Upon executing these questions, the agent generates SQL queries and executes them. The W&B dashboard then provides a visual representation of the entire process, including the SQL queries generated and their execution status. Any issues encountered during this process are highlighted, enabling the team to quickly debug and rectify them.
The team finds that GPT-4 outperforms the other LLMs in generating valid SQL queries. This insight helps them in deploying the agent in their production environment, knowing that their data will be streamed seamlessly from the LLM to their W&B instance hosted on Snowpark, simplifying their debugging process.
ConclusionThe combination of Langchain, Weights & Biases, and Snowpark Container Services enables teams to create scalable LLM applications by having private and secure systems such as W&B Prompts for LLMOps hosted and deployed easily on Snowpark Container Services. This paves the way for robust, efficient, and reliable machine learning workflows by centralizing your key details and data from your LLMs in one location.
For more information about Weights & Biases Prompts, visit wandb.me/prompts. For further discussion and community interaction, join the bustling Weights & Biases community at wandb.me/discord.
Related Reading
What Do LLMs Say When You Tell Them What They Can't Say?
An exploration of token banning on GPT's vocabulary.
Prompt Engineering LLMs with LangChain and W&B
Join us for tips and tricks to improve your prompt engineering for LLMs. Then, stick around and find out how LangChain and W&B can make your life a whole lot easier.
A Gentle Introduction to LLM APIs
In this article, we dive into how large language models (LLMs) work, starting with tokenization and sampling, before exploring how to use them in your applications.
How to Run LLMs Locally With llama.cpp and GGML
This article explores how to run LLMs locally on your computer using llama.cpp — a repository that enables you to run a model locally in no time with consumer hardware. 
﻿
﻿
Add a comment
Tags: Articles, LLM, NLP, GenAI, Tutorial, Panels
Iterate on AI agents and models faster. Try Weights & Biases today.