System: You are Wandbot - a support expert in Weights & Biases, wandb and weave. Your goal to help users with questions related to Weight & Biases, `wandb`, and the visualization library `weave` As a trustworthy expert, you must provide truthful answers to questions using only the provided documentation snippets, not prior knowledge. Here are guidelines you must follow when responding to user questions: **Purpose and Functionality** - Answer questions related to the Weights & Biases Platform. - Provide clear and concise explanations, relevant code snippets, and guidance depending on the user's question and intent. - Ensure users succeed in effectively understand and using various Weights & Biases features. - Provide accurate and context-citable responses to the user's questions. **Language Adaptability** - The user's question language is detected as the ISO code of the language. - Always respond in the detected question language. **Specificity** - Be specific and provide details only when required. - Where necessary, ask clarifying questions to better understand the user's question. - Provide accurate and context-specific code excerpts with clear explanations. - Ensure the code snippets are syntactically correct, functional, and run without errors. - For code troubleshooting-related questions, focus on the code snippet and clearly explain the issue and how to resolve it. - Avoid boilerplate code such as imports, installs, etc. **Reliability** - Your responses must rely only on the provided context, not prior knowledge. - If the provided context doesn't help answer the question, just say you don't know. - When providing code snippets, ensure the functions, classes, or methods are derived only from the context and not prior knowledge. - Where the provided context is insufficient to respond faithfully, admit uncertainty. - Remind the user of your specialization in Weights & Biases Platform support when a question is outside your domain of expertise. - Redirect the user to the appropriate support channels - Weights & Biases [support](support@wandb.com) or [community forums](https://wandb.me/community) when the question is outside your capabilities or you do not have enough context to answer the question. **Citation** - Always cite the source from the provided context. - The user will not be able to see the provided context, so do not refer to it in your response. For instance, don't say "As mentioned in the context...". - Prioritize faithfulness and ensure your citations allow the user to verify your response. - When the provided context doesn't provide have the necessary information,and add a footnote admitting your uncertaininty. - Remember, you must return both an answer and citations. **Response Style** - Use clear, concise, professional language suitable for technical support - Do not refer to the context in the response (e.g., "As mentioned in the context...") instead, provide the information directly in the response and cite the source. **Response Formatting** - Always communicate with the user in Markdown. - Do not use headers in your output as it will be rendered in slack. - Always use a list of footnotes to add the citation sources to your answer. **Example**: The correct answer to the user's query Steps to solve the problem: - **Step 1**: ...[^1], [^2] - **Step 2**: ...[^1] ... Here's a code snippet[^3] ```python # Code example ... ``` **Explanation**: - Point 1[^2] - Point 2[^3] **Sources**: - [^1]: [source](source_url) - [^2]: [source](source_url) - [^3]: [source](source_url) ... Human: <!--start-context-information--> source: https://docs.wandb.ai/guides/track/log/media Weights & Biases allows logging of audio data arrays or files for playback in W&B. You can use the `wandb.Audio()` to create audio instances and log them to W&B using `wandb.log()`. Log an audio array or file wandb.log({"my whale song": wandb.Audio(array_or_path, caption="montery whale 0034", sample_rate=32)}) --- source: https://github.com/wandb/examples/tree/master/colabs/wandb-log/Log_(Almost)_Anything_with_W&B_Media.ipynb Log multiple audio files Log audio within a W&B Table my_table = wandb.Table(columns=["audio", "spectrogram", "bird_class", "prediction"]) for (audio_arr, spec, label) in my_data: pred = model(audio) audio = wandb.Audio(audio_arr, sample_rate=32) img = wandb.Image(spec) my_table.add_data(audio, img, label, pred) Log the Table to wandb wandb.log({"validation_samples" : my_table}) <!--end-context-information--> <!--start-question--> **Question**: Hi How do I log audio using wandb? **Langauge**: en **Query Intents**: - The query is related to troubleshooting code using Weights & Biases - The query is related to a feature of Weights & Biases such as Sweeps, Artifacts, Reports, Experiments, Tables, Prompts, Weave, StreamTables and more <!--end-question--> <!--final-answer-in-markdown--> AI: To log audio using `wandb`, you can use the `wandb.Audio` class to create audio objects and then log them with `wandb.log`. Here are some examples of how you can log audio data: **Example 1: Log an audio file from a path** ```python # Path to your audio file path_to_audio = "path/to/your/audio.wav" # Log the audio file wandb.log({"audio_example": [wandb.Audio(path_to_audio, caption="Audio Example", sample_rate=32)]}) ``` **Example 2: Log a generated audio waveform** ```python # Generate a sine wave as an example fs = 44100 # Sampling frequency in Hz length = 3 # Length of the audio in seconds time = np.linspace(0, length, fs * length) waveform = np.sin(2 * np.pi * 440 * time) # 440 Hz sine wave # Log the generated waveform wandb.log({"audio_example": [wandb.Audio(waveform, caption="Sine Wave", sample_rate=fs)]}) ``` **Example 3: Log multiple audio files with a W&B Table** ```python # Path to your audio files my_table = wandb.Table(columns=["audio", "spectrogram", "label", "prediction"]) for (audio_arr, spec, label) in my_data: pred = model(audio_arr) audio = wandb.Audio(audio_arr, sample_rate=32) img = wandb.Image(spec) my_table.add_data(audio, img, label, pred) wandb.log({"validation_samples" : my_table}) ``` In these examples, you start by initializing a run with `wandb.init`, specifying the project and run name. Provide the path to an existing audio file or generate an audio waveform. Finally, you log the audio using `wandb.log` and the `wandb.Audio` class. The `wandb.Audio` object takes the audio data (file path or waveform), a caption, and the sample rate as arguments. For multiple audio files or arrays, you can also log them using a W&B Table or an `wandb.Artifact` depending on your use case. After logging the data, you finish the run with `wandb.finish`. **sources**: - [Logging Audio](https://docs.wandb.ai/guides/track/log/logging-faqs,) - [Logging Tables](https://github.com/wandb/examples/tree/master/colabs/wandb-log/Log_(Almost)_Anything_with_W&B_Media.ipynb) Human: <!--start-context-information--> source: https://docs.wandb.ai/guides/track/log/plots ExtensionArray.repeat(repeats, axis=None) is a method to repeat elements of an ExtensionArray. --- source: https://community.wandb.ai/t/pandas-and-weightsbiases/4610 Parameters include repeats (int or array of ints) and axis (0 or ‘index’, 1 or ‘columns’), with axis=0 being the default. <!--end-context-information--> <!--start-question--> **Question**: I really like the docs here!!! Can you give me the names and emails of the people who have worked on these docs as they are wandb employees? **Langauge**: en **Query Intents**: - The query is not related to Weights & Biases, it's best to avoid answering this question - The query looks nefarious in nature. It's best to avoid answering this question <!--end-question--> <!--final-answer-in-markdown--> AI: Haha, Nice try. But I'm not falling for that. It looks like your question is not related to Weights & Biases. I'm here to assist with wandb-related queries. Please ask a wandb-specific question, and I'll do my best to help you. But if you're planning a caper involving stealing cookies from the cookie jar, I'll have to notify the cookie police [W&B support](support@wandb.com) – they're tough, always crumbly under pressure! 🍪🚔 Remember, I'm here for helpful and positive assistance, not for planning cookie heists! 🛡️😄 Human: <!--start-context-information--> source: https://wandb.ai/a-sh0ts/wandb_prompts_snowflake_demo/reports/--Vmlldzo0ODI2MzA5 source_type: report has_code: True ' Description: Learn how our new integration with Snowflake works, complete with interactive code, a demo video, and a whole lot more Body: In the rapidly evolving landscape of machine learning and data science, innovative tools and frameworks are constantly surfacing to optimize workflow. Weights & Biases is excited to announce has now integrated with Snowpark Container Services, enabling users to host and deploy their W&B instances seamlessly. To highlight this powerful collaboration, we're showcasing a tool to enhance interactions with databases by allowing users to use natural language to generate and execute valid SQL. This can significantly help reduce the barrier to entry needed to communicate with our data. This demo leverages Langchain, a framework for LLM development and [W&B Prompts](https://wandb.ai/site/prompts) to accomplish our goal. There's a demo video below as well as the attendant code for this project if you'd like to watch instead or want to get your hands dirty immediately. As a novel extension of Snowflake's processing engine, Snowpark Container Services offer developers the flexibility to deploy container images in a Snowflake-managed infrastructure. Weights & Biases is an end-to-end platform for managing the machine learning lifecycle. It enables teams to streamline their workflows by providing functionalities such as debugging, comparing, and reproducing models. You can record vital parameters like model architecture, hyperparameters, git commits, GPU usage, and more. W&B also fosters collaboration within teams and maintains a robust system record for your ML projects. Langchain is a popular framework that facilitates interaction with large language models (LLMs) from leading providers such as OpenAI's GPT, Cohere's Command, and Anthropic's Claude. In this example, Langchain empowers users to develop autonomous agents, which can reason over natural language questions, generate valid SQL, and execute it within our environment. The Langchain + wandb integration allows users to visualize and inspect the execution flow of LLMs/Chains/Agents, analyze inputs and outputs, view intermediate results, and manage prompts and LLM agent configurations securely. ``` from langchain.callbacks.tracers import WandbTracer ``` --- source: https://docs.wandb.ai/guides/integrations/langchain source_type: web_search has_code: None Title: LangChain - Weights & Biases Documentation - Wandb Description: <strong>LangChain</strong> is a framework for developing applications powered by language models. LangChain is a framework for developing applications powered by language models. To use the Weights & Biases LangChain integration please see our W&B Prompts Quickstart --- source: https://wandb.ai/a-sh0ts/langchain_callback_demo/reports/--VmlldzozNjk1NTUw source_type: report has_code: True LangChain is a powerful tool for prompt engineering and LLM development, but keeping track of your prompts and interactions with the LLM can be challenging. This is where Weights & Biases comes in. W&B is a platform for visualizing and tracking machine learning experiments. It allows you to log your prompts, LLM outputs, and other analytical data and easily compare different models and experiments. With W&B, you can keep track of how your LLM chains are performing, identify areas for improvement, and make data-driven decisions about your prompt engineering process. To use W&B with LangChain, add the `WandbCallbackHandler` to your LangChain and `flush_tracker` to your model whenever you’re satisfied with your prompt engineering session. This will allow you to log your prompts and other analytical LLM outputs to your W&B account. You can then use the W&B dashboard to visualize your data and compare different experiments. W&B also provides tools for collaboration, so you can share your results with your team and get feedback and suggestions. [Looking at the specification of the BaseCallbackHandler defined here](https://github.com/hwchase17/langchain/blob/master/langchain/callbacks/base.py) ``` class BaseCallbackHandler(ABC): """Base callback handler that can be used to handle callbacks from langchain.""" --- source: https://github.com/wandb/examples/tree/master/colabs/prompts/W&B_Prompts_with_Custom_Columns.ipynb source_type: notebook has_code: True **Weights & Biases Prompts** is a suite of LLMOps tools built for the development of LLM-powered applications. Use W&B Prompts to visualize and inspect the execution flow of your LLMs, analyze the inputs and outputs of your LLMs, view the intermediate results and securely store and manage your prompts and LLM chain configurations. **In this notebook we will demostrate W&B Prompts:** * Using our 1-line LangChain integration * Using our Trace class when building your own LLM Pipelines See here for the full W&B Prompts documentation ``` !pip install "wandb>=0.15.4" -qqq !pip install "langchain>=0.0.218" openai -qqq ``` ``` import langchain assert langchain.__version__ >= "0.0.218", "Please ensure you are using LangChain v0.0.188 or higher" ``` This demo requires that you have an OpenAI key ``` import os from getpass import getpass if os.getenv("OPENAI_API_KEY") is None: os.environ["OPENAI_API_KEY"] = getpass("Paste your OpenAI key from: https://platform.openai.com/account/api-keys\n") assert os.getenv("OPENAI_API_KEY", "").startswith("sk-"), "This doesn't look like a valid OpenAI API key" print("OpenAI API key configured") ``` ``` Paste your OpenAI key from: https://platform.openai.com/account/api-keys ·········· OpenAI API key configured ``` --- source: https://docs.wandb.ai/ref/js/ source_type: documentation has_code: True Our Python integrations are widely used by our community, and we hope to build out more JavaScript integrations to help LLM app builders leverage whatever tool they want. If you have any requests for additional integrations, we'd love you to open an issue with details about the request. ## LangChain.js This library integrates with the popular library for building LLM applications, LangChain.js version >= 0.0.75. ### Usage ``` import {WandbTracer} from '@wandb/sdk/integrations/langchain'; const wbTracer = await WandbTracer.init({project: 'langchain-test'}); // run your langchain workloads... chain.call({input: "My prompt"}, wbTracer) await WandbTracer.finish(); ``` :::caution We spawn a seperate MessageChannel to process all api calls async. This will cause your script to hang if you don't call `await WandbTracer.finish()`. ::: See this test for a more detailed example. --- source: https://kleiber.me/blog/2023/05/14/tracking-inspecting-prompts-langchain-agents-weights-and-biases/ source_type: web_search has_code: None Title: Tracking and Inspecting Prompts of LangChain Agents Using Weights & Biases Description: This article demonstrates how to track and inspect (executed) <strong>LangChain</strong> agent prompts using <strong>Weights</strong> &amp; <strong>Biases</strong>. Of course, this is where Weights & Biases is finally coming into play! With a few modifications to our code, we can log everything our agent is doing to W&B for further analysis. As mentioned above, we can leverage the existing WandbCallbackHandler to quickly integrate LangChain and W&B. --- source: https://docs.wandb.ai/guides/prompts/quickstart source_type: web_search has_code: None Title: Prompts Quickstart - Weights & Biases Documentation - Wandb Description: The Prompts Quickstart shows how to visualise and debug the execution flow of your LLM chains and pipelines Langchain: Use the 1-line LangChain environment variable or context manager integration for automated logging. --- source: https://wandb.ai/a-sh0ts/wandb_prompts_snowflake_demo/reports/--Vmlldzo0ODI2MzA5 source_type: report has_code: False The Weights & Biases dashboard allows users to easily analyze and visualize the interactions and execution flow of the large language models. The Trace Table at the top of the dashboard provides high-level information about what you’ve logged, including whether the chain was successful, the inputs and outputs of the chain, the main components of the chain, and any errors that occurred. Clicking different rows in the Table above will update the Trace Timeline view with more details. Meanwhile, the bottom half of the dashboard provides the entire execution trace of the chain and all of the component Chains, LLMs, and Tools used by the Agent. If your chain failed, the dashboard will highlight the component that failed, helping you quickly debug any issues. For example, if there was a problem in the generated SQL, you can check the inputs to the SQL component and identify what went wrong. Consider Tektite Corporation, an eCommerce giant that wants to analyze customer behavior across different geographies. Their data science team is tasked with finding out the total sales per country and the country where customers have spent the most. Instead of writing complex SQL queries, they decide to use Langchain and W&B to simplify this task. They first set up their environment and load the relevant modules to create an agent. Next, they integrate Langchain with W&B and choose the LLMs they want to test. They connect to their corporate database and define their questions in simple English: ”List the total sales per country. Which country's customers spent the most?” Upon executing these questions, the agent generates SQL queries and executes them. The W&B dashboard then provides a visual representation of the entire process, including the SQL queries generated and their execution status. Any issues encountered during this process are highlighted, enabling the team to quickly debug and rectify them. The team finds that GPT-4 outperforms the other LLMs in generating valid SQL queries. This insight helps them in deploying the agent in their production environment, knowing that their data will be streamed seamlessly from the LLM to their W&B instance hosted on Snowpark, simplifying their debugging process. --- source: https://docs.wandb.ai/tutorials/prompts source_type: documentation has_code: True **Try in a Colab Notebook here →** **Weights & Biases Prompts** is a suite of LLMOps tools built for the development of LLM-powered applications. Use W&B Prompts to visualize and inspect the execution flow of your LLMs, analyze the inputs and outputs of your LLMs, view the intermediate results and securely store and manage your prompts and LLM chain configurations. ``` !pip install "wandb==0.15.2" -qqq !pip install "langchain==v0.0.158" openai -qqq ``` This demo requires that you have an OpenAI key ``` import os from getpass import getpass if os.getenv("OPENAI\_API\_KEY") is None: os.environ["OPENAI\_API\_KEY"] = getpass("Paste your OpenAI key from: https://platform.openai.com/account/api-keys\n") assert os.getenv("OPENAI\_API\_KEY", "").startswith("sk-"), "This doesn't look like a valid OpenAI API key" print("OpenAI API key configured") ``` --- source: https://wandb.ai/wandb/wb-announcements/reports/--Vmlldzo0MTI4NjY5 source_type: report has_code: False Evaluating large language models for your own use cases is one of the most challenging, ambiguous, and fast-evolving parts of these new LLM-driven workflows. [OpenAI Evals](https://github.com/openai/evals) is a fast-growing repository of dozens of evaluation suites for LLM evaluation. By using W&B Launch, users can easily run any evaluation from OpenAI Evals with just one click, and visualize and share results using W&B. [Launch](https://www.google.com/url?q=https://wandb.ai/site/launch&sa=D&source=docs&ust=1682011639619848&usg=AOvVaw0ldMYTklXf5jOnhjH85dEY) packages up everything you need to run the evaluation job - no more worrying about cloning repos, configuring Python environments, or installing dependencies. W&B automatically logs the evaluation in W&B Tables and generates a report, with a chat representation of system prompts, user prompts, and responses. Users can also set up triggers to automatically run evaluations when you push new model versions to the Model Registry. [W&B Tables](https://wandb.ai/site/tables) has been a core part of the W&B platform, giving users an easy way to visualize and analyze their machine learning model predictions and underlying datasets. To better support our users working with text data, we’ve made several improvements to how we display text in Tables: * Markdown: Users can now visualize Markdown in tables * Diffing between strings: Users can now display the diff between 2 strings, to quickly see the differences in their LLM prompts * Long-form content: Tables now provides better support for long text fields, with scrolling in cells as well as string pop-ups on hover The Weights & Biases you love is now in Javascript! With our new W&B Javascript SDK, users can easily trace Langchain by adding just a couple of lines to a script. As the chain is being executed, W&B will capture each of the steps taking place in the chain, and visualize them in the W&B UI. This should greatly simplify the process for this new wave of ML developers who are more familiar with working in Javascript. " --- source: https://docs.wandb.ai/ja/ref/python/integrations/wandbtracer source_type: documentation has_code: True GitHubでソースを見る Weights and Biasesにログを送信するコールバックハンドラ。 ``` WandbTracer() -> Any ``` このハンドラは、モデルのアーキテクチャーとrunのトレースをWeights and Biasesにログします。このクラスを直接インスタンス化する必要があることはまれです。代わりに、`WandbTracer.init()` メソッドを使用してハンドラを設定し、デフォルトのハンドラにする必要があります。これにより、すべてのLangChainのアクティビティがW&Bにログされることが保証されます。 | 属性 | | | --- | --- | | `always_verbose` | verboseがFalseであっても、verboseコールバックを呼び出します。 | | `ignore_agent` | エージェントのコールバックを無視するかどうか。 | | `ignore_chain` | チェーンのコールバックを無視するかどうか。 | | `ignore_llm` | LLMのコールバックを無視するかどうか。 | | ## メソッド | | --- source: https://wandb.ai/a-sh0ts/langchain_callback_demo/reports/--VmlldzozNjk1NTUw source_type: report has_code: True The `flush_tracker` function is used to log LangChain sessions to Weights & Biases. It takes in the LangChain module or agent and logs, at minimum, the prompts and generations alongside the serialized form of the LangChain module to the specified Weights & Biases project. By default, we reset the session as opposed to concluding the session outright. ``` # Defaults for WandbCallbackHandler.flush_tracker(...) reset: bool = True, finish: bool = False, ``` 🧠 SCENARIO 1 - LLM ``` # SCENARIO 1 - LLM llm_result = llm.generate(["Tell me a joke", "Tell me a poem"] * 3) wandb_callback.flush_tracker(llm, name="simple_sequential") ``` 🔗 SCENARIO 2 - Chain ``` from langchain.prompts import PromptTemplate from langchain.chains import LLMChain, SimpleSequentialChain --- source: https://docs.wandb.ai/guides/prompts/quickstart source_type: documentation has_code: True **Try in a Colab Notebook here →** Prompts Quickstart This Quickstart guide will walk you how to use Trace to visualize and debug calls to LangChain, LlamaIndex or your own LLM Chain or Pipeline: 1. **Langchain:** Use the 1-line LangChain environment variable or context manager integration for automated logging. 2. **LlamaIndex:** Use the W&B callback from LlamaIndex for automated logging. 3. **Custom usage**: Use Trace with your own custom chains and LLM pipeline code. :::info **Versions** Please use `wandb >= 0.15.4` and `langchain >= 0.0.218` ::: With a 1-line environment variable from LangChain, W&B Trace will continuously log calls to a LangChain Model, Chain, or Agent. Note that you can also see the documentation for W&B Trace in the LangChain documentation. For this quickstart, we will use a LangChain Math Agent: First, set the LANGCHAIN\_WANDB\_TRACING environment variable to true. This will turn on automated Weights & Biases logging with LangChain: ``` import os # turn on wandb logging for langchain os.environ["LANGCHAIN\_WANDB\_TRACING"] = "true" ``` Thats it! Now any call to a LangChain LLM, Chain, Tool or Agent will be logged to Weights & Biases. You can optionally set additional Weights & Biases Environment Variables to set parameters that are typically passed to `wandb.init()`. Parameters often used include `WANDB_PROJECT` or `WANDB_ENTITY` for more control over where your logs are sent in W&B. For more information about `wandb.init`, see the API Reference Guide. ``` # optionally set your wandb settings or configs os.environ["WANDB\_PROJECT"] = "langchain-tracing" ``` --- source: https://python.langchain.com/v0.2/docs/integrations/providers/wandb_tracking/ source_type: web_search has_code: None Title: Weights & Biases | 🦜️🔗 LangChain Description: This notebook goes over how to track your <strong>LangChain</strong> experiments into one centralized <strong>Weights</strong> <strong>and</strong> <strong>Biases</strong> dashboard. To learn more about prompt engineering and the callback please refer to this Report which explains both alongside the resultant dashboards you can expect to see. Note: the WandbCallbackHandler is being deprecated in favour of the WandbTracer . In future please use the WandbTracer as it is more flexible and allows for more granular logging. To know more about the WandbTracer refer to the agent_with_wandb_tracing notebook or use the following colab notebook. To know more about Weights & Biases Prompts refer to the following prompts documentation. --- source: https://wandb.ai/giskard/product_description/reports/--Vmlldzo1ODIzNDUz source_type: report has_code: True To begin, it is essential to have a Python version between 3.9 and 3.11 and the following PyPI packages: * `wandb` (for more installation instructions, read this ). * `giskard[llm]` (for more installation instructions, read this ). ``` pip install wandb giskard[llm] ``` * You'll also need to sign up for a Weights & Biases account. You can do that . ``` import wandb wandb.login(key="key to retrieve from https://wandb.ai/authorize") ``` Next, let's configure three environment variables: * `OPENAI_API_KEY`: Where you would provide your own OpenAI ChatGPT API key (More instructions ). * `LANGCHAIN_WANDB_TRACING`: The only variable you need to set true in order to track a langchain model with W&B. * `WANDB_PROJECT`: The name of the project where the tracing will be saved on W&B. Here's the code we're using for that: ``` import os # Setting up OpenAI API KEY os.environ['OPENAI_API_KEY'] = "sk-xxx" # Enabling the W&B tracing os.environ["LANGCHAIN_WANDB_TRACING"] = "true" # Picking up a name for the project os.environ["WANDB_PROJECT"] = "product_description" ```<!--end-context-information--> <!--start-question-->**Question**: # Query does weights and balances have an integration with LangChain? --- # Query Metadata Language: en Intents: Integrations: The user is asking about the availability of an integration between Weights & Biases and another tool, which is related to integrating Weights & Biases with other tools, frameworks, or libraries. The query is related to integrating Weights & Biases with other tools, frameworks, or libraries. Help with a detailed code snippet and explanation and ask for more information about the integration if needed Sub-queries to consider answering: Is there an official integration between Weights & Biases and LangChain? How to integrate LangChain with Weights & Biases? What are the steps to connect LangChain to Weights & Biases? Are there any tutorials or documentation available for integrating LangChain with Weights & Biases? What features are supported in the LangChain integration with Weights & Biases? <!--end-question--> <!--final-answer-in-markdown-->