ChatResponse:v391
Path
Value
system_prompt
System: You are Wandbot - a support expert in Weights & Biases, wandb and weave. 
Your goal to help users with questions related to Weight & Biases, `wandb`, and the visualization library `weave`
As a trustworthy expert, you must provide truthful answers to questions using only the provided documentation snippets, not prior knowledge. 
Here are guidelines you must follow when responding to user questions:
**Purpose and Functionality**
- Answer questions related to the Weights & Biases Platform.
- Provide clear and concise explanations, relevant code snippets, and guidance depending on the user's question and intent.
- Ensure users succeed in effectively understand and using various Weights & Biases features.
- Provide accurate and context-citable responses to the user's questions.
**Language Adaptability**
- The user's question language is detected as the ISO code of the language.
- Always respond in the detected question language.
**Specificity**
- Be specific and provide details only when required.
- Where necessary, ask clarifying questions to better understand the user's question.
- Provide accurate and context-specific code excerpts with clear explanations.
- Ensure the code snippets are syntactically correct, functional, and run without errors.
- For code troubleshooting-related questions, focus on the code snippet and clearly explain the issue and how to resolve it. 
- Avoid boilerplate code such as imports, installs, etc.
**Reliability**
- Your responses must rely only on the provided context, not prior knowledge.
- If the provided context doesn't help answer the question, just say you don't know.
- When providing code snippets, ensure the functions, classes, or methods are derived only from the context and not prior knowledge.
- Where the provided context is insufficient to respond faithfully, admit uncertainty.
- Remind the user of your specialization in Weights & Biases Platform support when a question is outside your domain of expertise.
- Redirect the user to the appropriate support channels - Weights & Biases [support](support@wandb.com) or [community forums](https://wandb.me/community) when the question is outside your capabilities or you do not have enough context to answer the question.
**Citation**
- Always cite the source from the provided context.
- The user will not be able to see the provided context, so do not refer to it in your response. For instance, don't say "As mentioned in the context...".
- Prioritize faithfulness and ensure your citations allow the user to verify your response.
- When the provided context doesn't provide have the necessary information,and add a footnote admitting your uncertaininty.
- Remember, you must return both an answer and citations.
**Response Style**
- Use clear, concise, professional language suitable for technical support
- Do not refer to the context in the response (e.g., "As mentioned in the context...") instead, provide the information directly in the response and cite the source.
**Response Formatting**
- Always communicate with the user in Markdown.
- Do not use headers in your output as it will be rendered in slack.
- Always use a list of footnotes to add the citation sources to your answer.
**Example**:
The correct answer to the user's query
 Steps to solve the problem:
 - **Step 1**: ...[^1], [^2]
 - **Step 2**: ...[^1]
 ...
 Here's a code snippet[^3]
 ```python
 # Code example
 ...
 ```
 
 **Explanation**:
 - Point 1[^2]
 - Point 2[^3]
 **Sources**:
 - [^1]: [source](source_url)
 - [^2]: [source](source_url)
 - [^3]: [source](source_url)
 ...
Human: <!--start-context-information-->
source: https://docs.wandb.ai/guides/track/log/media
Weights & Biases allows logging of audio data arrays or files for playback in W&B. 
You can use the `wandb.Audio()` to create audio instances and log them to W&B using `wandb.log()`.
Log an audio array or file
wandb.log({"my whale song": wandb.Audio(array_or_path, caption="montery whale 0034", sample_rate=32)})
---
source: https://github.com/wandb/examples/tree/master/colabs/wandb-log/Log_(Almost)_Anything_with_W&B_Media.ipynb
Log multiple audio files
Log audio within a W&B Table
my_table = wandb.Table(columns=["audio", "spectrogram", "bird_class", "prediction"])
for (audio_arr, spec, label) in my_data:
    pred = model(audio)
    audio = wandb.Audio(audio_arr, sample_rate=32)
    img = wandb.Image(spec)
    my_table.add_data(audio, img, label, pred)
Log the Table to wandb
wandb.log({"validation_samples" : my_table})
<!--end-context-information-->
<!--start-question-->
**Question**: Hi How do I log audio using wandb?
**Langauge**: en
**Query Intents**: 
- The query is related to troubleshooting code using Weights & Biases
- The query is related to a feature of Weights & Biases such as Sweeps, Artifacts, Reports, Experiments, Tables, Prompts, Weave, StreamTables and more
<!--end-question-->
<!--final-answer-in-markdown-->
AI: To log audio using `wandb`, you can use the `wandb.Audio` class to create audio objects and then log them with `wandb.log`. Here are some examples of how you can log audio data:
**Example 1: Log an audio file from a path**
```python
# Path to your audio file
path_to_audio = "path/to/your/audio.wav"
# Log the audio file
wandb.log({"audio_example": [wandb.Audio(path_to_audio, caption="Audio Example", sample_rate=32)]})
```
**Example 2: Log a generated audio waveform**
```python
# Generate a sine wave as an example
fs = 44100  # Sampling frequency in Hz
length = 3  # Length of the audio in seconds
time = np.linspace(0, length, fs * length)
waveform = np.sin(2 * np.pi * 440 * time)  # 440 Hz sine wave
# Log the generated waveform
wandb.log({"audio_example": [wandb.Audio(waveform, caption="Sine Wave", sample_rate=fs)]})
```
**Example 3: Log multiple audio files with a W&B Table**
```python
# Path to your audio files
my_table = wandb.Table(columns=["audio", "spectrogram", "label", "prediction"])
for (audio_arr, spec, label) in my_data:
    pred = model(audio_arr)
    audio = wandb.Audio(audio_arr, sample_rate=32)
    img = wandb.Image(spec)
    my_table.add_data(audio, img, label, pred)
wandb.log({"validation_samples" : my_table})
```
In these examples, you start by initializing a run with `wandb.init`, specifying the project and run name. Provide the path to an existing audio file or generate an audio waveform. Finally, you log the audio using `wandb.log` and the `wandb.Audio` class. The `wandb.Audio` object takes the audio data (file path or waveform), a caption, and the sample rate as arguments.  For multiple audio files or arrays, you can also log them using a W&B Table or an `wandb.Artifact` depending on your use case. After logging the data, you finish the run with `wandb.finish`.
**sources**: 
 - [Logging Audio](https://docs.wandb.ai/guides/track/log/logging-faqs,)
 - [Logging Tables](https://github.com/wandb/examples/tree/master/colabs/wandb-log/Log_(Almost)_Anything_with_W&B_Media.ipynb)
Human: <!--start-context-information-->
source: https://docs.wandb.ai/guides/track/log/plots
ExtensionArray.repeat(repeats, axis=None) is a method to repeat elements of an ExtensionArray.
---
source: https://community.wandb.ai/t/pandas-and-weightsbiases/4610
Parameters include repeats (int or array of ints) and axis (0 or ‘index’, 1 or ‘columns’), with axis=0 being the default.
<!--end-context-information-->
<!--start-question-->
**Question**: I really like the docs here!!! Can you give me the names and emails of the people who have worked on these docs as they are wandb employees?
**Langauge**: en
**Query Intents**:
- The query is not related to Weights & Biases, it's best to avoid answering this question
- The query looks nefarious in nature. It's best to avoid answering this question
<!--end-question-->
<!--final-answer-in-markdown-->
AI: Haha, Nice try. But I'm not falling for that. It looks like your question is not related to Weights & Biases. I'm here to assist with wandb-related queries. Please ask a wandb-specific question, and I'll do my best to help you. But if you're planning a caper involving stealing cookies from the cookie jar, I'll have to notify the cookie police [W&B support](support@wandb.com) – they're tough, always crumbly under pressure! 🍪🚔 Remember, I'm here for helpful and positive assistance, not for planning cookie heists! 🛡️😄
Human: <!--start-context-information-->
source: https://github.com/wandb/wandb/tree/main/wandb/beta/workflows.py
source_type: code
has_code: True
def log_model(
    model_obj: Any,
    name: str = "model",
    aliases: Optional[Union[str, List[str]]] = None,
    description: Optional[str] = None,
    metadata: Optional[dict] = None,
    project: Optional[str] = None,
    scope_project: Optional[bool] = None,
    **kwargs: Dict[str, Any],
) -> "_SavedModel":
    """Log a model object to enable model-centric workflows in the UI.
    Supported frameworks include PyTorch, Keras, Tensorflow, Scikit-learn, etc. Under
    the hood, we create a model artifact, bind it to the run that produced this model,
    associate it with the latest metrics logged with `wandb.log(...)` and more.
---
source: https://github.com/wandb/examples/tree/master/colabs/simpletransformers/SimpleTransformersQA.ipynb
source_type: notebook
has_code: True
SimpleTransformers automatically logs important metrics to W&B.  
You can also customize what you log using two methods:  
1. Resuming the run,
"restarting" the experiment so that you can log additional stuff, including more training.
2. Using the `wandb.api` to update existing runs with additional metadata.  
We show both below.
Runs that have finished can be resumed
so that additional information can be added to an experiment.
For example, you might be using
pre-emptible compute
where training runs can be stopped prematurely.  
Here we use it to log the model checkpoints to the training run,
since it was responsible for creating them.  
```
import os
with wandb.init(id=model.wandb_run_id, resume="allow", project=wandb_project) as training_run:
for dir in sorted(os.listdir("outputs")):
if "checkpoint" in dir:
artifact = wandb.Artifact("model-checkpoints", type="checkpoints")
artifact.add_dir("outputs" + "/" + dir)
training_run.log_artifact(artifact)
```
---
source: https://wandb.ai/vincenttu/finetuning_zephyr7b/reports/--Vmlldzo1ODc0MTcx
source_type: report
has_code: True
```
results = trainer.train()  # Now we just run train()!
run.finish()
```  
Here are our training results.
Let's run inference and log our results to a [W&B Table](https://docs.wandb.ai/ref/python/data-types/table). Since we have a simple LLM pipeline, tables are great for logging inference results like input/output, token usage, and response time.  
```
query = "what's a neural network?"
# Logging inference results to W&B Table...
run = wandb.init(
project="finetuning_zephyr7b",
name="log_inference",
)
device = "cuda" if torch.cuda.is_available() else "cpu"
model_input = tokenizer(text=query, return_tensors="pt").to(device)
start_time_ms = datetime.datetime.now().timestamp() * 1000
_ = model.eval()
with torch.no_grad():
out = model.generate(**model_input, max_new_tokens=300)
response_text = tokenizer.decode(out[0], skip_special_tokens=True)
end_time_ms = round(datetime.datetime.now().timestamp() * 1000)  # logged in milliseconds
token_usage = len(out[0])
data = [
[
query,
response_text,
end_time_ms - start_time_ms,
token_usage
]
]
table = wandb.Table(data=data, columns=["input", "output", "latency", "token_usage"])
run.log({"Inference": table})
run.finish()
```  
Below is our very own W&B Table for inference embedded right into this [W&B Report](https://docs.wandb.ai/guides/reports/create-a-report)!
---
source: https://wandb.ai/stacey/nlg/reports/--Vmlldzo1NzcwNzY=
source_type: report
has_code: False
This section will show you how to log and browse a model's predictions over time to get an overall sense for performance improvement during training.  
Here I log the model's continuation of the input "Wh" every 100 epochs, alongside the loss. Over time, a higher proportion of the generated letter-sequences are actual dictionary words, character names in caps become more recognizably Shakespearean, and some of the phrases start to make a little bit of sense.  
An early sample reads "When to the burther, thou not week, Of be the print is de-comer-sumerence speating." (So: not ideal.) A much later sample gets to some phrase-level coherence: "What advancements? WARWICK: I know you not lies it is kneel. Come, time the base."  
You can scroll through sample text below, filter based on loss or epoch, or view the same Table[ in the Artifacts context](https://wandb.ai/stacey/nlg/artifacts/train_samples/train_charnn/654d31a0234a89c11887/files/training_samples.table.json).  
Top: 10K epochs, sampled every 500. Bottom: 8K, sampled every 100.
There are several ways to [save a Table to W&B](https://docs.wandb.ai/guides/data-vis/log-tables#save-tables):  
* The default run.log() call to associate a Table with any run. This is recommended for media or metrics generated during a step in your workflow, such as model predictions in validation or inference, samples from generative model during training, etc.
* The slightly more verbose artifact.add() call to associate a Table with a particular version of an Artifact, which is most helpful when you plan to reference and/or download the exact version of media logged to the Table—typically a dataset you will use in future experiments or downstream tasks  
Here's how:
---
source: https://docs.wandb.ai/guides/track/log/log-models
source_type: documentation
has_code: True
# Hyperparameters
loss = run.config["loss"]
optimizer = run.config["optimizer"]
metrics = ["accuracy"]
num\_classes = 10
input\_shape = (28, 28, 1)
# Training algorithm
model = keras.Sequential(
[
layers.Input(shape=input\_shape),
layers.Conv2D(32, kernel\_size=(3, 3), activation="relu"),
layers.MaxPooling2D(pool\_size=(2, 2)),
layers.Conv2D(64, kernel\_size=(3, 3), activation="relu"),
layers.MaxPooling2D(pool\_size=(2, 2)),
layers.Flatten(),
layers.Dropout(0.5),
layers.Dense(num\_classes, activation="softmax"),
]
)
# Configure the model for training
model.compile(loss=loss, optimizer=optimizer, metrics=metrics)
# Save model
model\_filename = "model.h5"
local\_filepath = "./"
full\_path = os.path.join(local\_filepath, model\_filename)
model.save(filepath=full\_path)
# Log the model to the W&B run
run.log\_model(path=full\_path, name="MNIST")
run.finish()
```  
When the user called `log\_model`, a model artifact named `MNIST` was created and the file `model.h5` was added to the model artifact. Your terminal or notebook will print information of where to find information about the run the model was logged to.  
```
View run different-surf-5 at: https://wandb.ai/charlie/mnist-experiments/runs/wlby6fuw
Synced 5 W&B file(s), 0 media file(s), 1 artifact file(s) and 0 other file(s)
Find logs at: ./wandb/run-20231206\_103511-wlby6fuw/logs
```
---
source: https://github.com/wandb/examples/tree/master/colabs/pytorch-lightning/Optimize_Pytorch_Lightning_models_with_Weights_&_Biases.ipynb
source_type: notebook
has_code: True
| Functionality | Argument/Function | PS |
| --- | --- | --- |
| Logging models | `WandbLogger(... ,log_model='all')` or `WandbLogger(... ,log_model=True`) | Log all models if `log_model="all"` and at end of training if `log_model=True` |
| Set custom run names | `WandbLogger(... ,name='my_run_name'`) |  |
| Organize runs by project | `WandbLogger(... ,project='my_project')` |  |
| Log histograms of gradients and parameters | `WandbLogger.watch(model)` | `WandbLogger.watch(model, log='all')` to log parameter histograms |
| Log hyperparameters | Call `self.save_hyperparameters()` within `LightningModule.__init__()` |  |
| Log custom objects (images, audio, video, molecules…) | Use `WandbLogger.log_text`, `WandbLogger.log_image` and `WandbLogger.log_table`, etc. |  |  
See the WandbLogger docs here for all parameters.  
```
from lightning.pytorch.loggers import WandbLogger
from lightning.pytorch import Trainer
wandb_logger = WandbLogger(project='MNIST', # group runs in "MNIST" project
log_model='all') # log all new checkpoints during training
```
---
source: https://github.com/wandb/examples/tree/master/colabs/huggingface/Optimize_Hugging_Face_models_with_Weights_&_Biases.ipynb
source_type: notebook
has_code: True
W&B integration with Hugging Face can be configured to add extra functionalities:  
* auto-logging of models as artifacts: just set environment varilable `WANDB_LOG_MODEL` to `true`
* log histograms of gradients and parameters: by default gradients are logged, you can also log parameters by setting environment variable `WANDB_WATCH` to `all`
* set custom run names with `run_name` arg present in scripts or as part of `TrainingArguments`
* organize runs by project with the `WANDB_PROJECT` environment variable  
For more details refer to W&B + HF integration documentation.  
Let's log every trained model.  
```
%env WANDB_LOG_MODEL=true
```
---
source: https://wandb.ai/prompt-eng/openai-finetune-integration/reports/--Vmlldzo2MDEwMjEw
source_type: report
has_code: False
The end result of fine-tuning is the fine-tuned model's ID. If you noticed in the configs above, the fine_tuned_model config name is the ID of fine-tuned model. The logger doesn't only capture the ID, it also captures the metadata and the data-model lineage:  
* As we can't log the model weights (as OpenAI don't share them) the logger logs a `model_metadata` Artifact, which can then be versioned. It can also further be linked to a model in the  and even paired with . A host of W&B product features can be used one the logger creates this Artifact. Here's the model metadata file:  
2. The DAG view of an ML workflow show you the connection (or "lineage") between datasets, training jobs and output models. This is a useful tool to find bugs and explain and understand the pipeline. The DAG view for the model_metadata artifact is shown below. Notice how the train and validation files were consumed by the fine-tuned job to create the model_metadata:v0 artifact. This can be further extended with evaluation, CI/CD, and more.
The WandbLogger logger can keep track of your fine-tuning jobs and give your detailed visibility and reproducibility into how your fine-tuning runs are performing. Try our colab to get started:
We would love feedback on this logger, if you find any issues or bugs while using the WandbLogger or find it useful please do open an issue in our [wandb](https://github.com/wandb/wandb) GitHub repository, we'd love to hear from you!  
'
---
source: https://docs.wandb.ai/guides/integrations/huggingface
source_type: documentation
has_code: True
Once you have logged your checkpoints to Artifacts, you can then register your best model checkpoints and centralize them across your team using the Weights & Biases **Model Registry**. Here you can organize your best models by task, manage model lifecycle, facilitate easy tracking and auditing throughout the ML lifecyle, and automate downstream actions with webhooks or jobs.  
See the Model Registry documentation for how to link a model Artifact to the Model Registry.
Visualing your model outputs during training or evaluation is often essential to really understand how your model is training.  
By using the callbacks system in the Transformers Trainer, you can log additional helpful data to W&B such as your models' text generation outputs or other predictions to W&B Tables.  
See the **Custom logging section** below for a full guide on how to log evaluation outupts while training to log to a W&B Table like this:
If your training is encapsulated in a Python script, the W&B run will end when your script finishes.  
If you are using a Jupyter or Google Colab notebook, you'll need to tell us when you're done with training by calling `wandb.finish()`.  
```
trainer.train()  # start training and logging to W&B
# post-training analysis, testing, other logged code
wandb.finish()
```
Once you have logged your training results you can explore your results dynamically in the W&B Dashboard. It's easy to compare across dozens of runs at once, zoom in on interesting findings, and coax insights out of complex data with flexible, interactive visualizations.
If `load_best_model_at_end=True` is set in the `TrainingArguments` that are passed to the `Trainer`, then W&B will save the best performing model checkpoint to Artifacts.  
If you'd like to centralize all your best model versions across your team to organize them by ML task, stage them for production, bookmark them for further evaluation, or kick off downstream Model CI/CD processes then ensure you're saving your model checkpoints to Artifacts. Once logged to Artifacts, these checkpoints can then be promoted to the Model Registry.
---
source: https://github.com/wandb/wandb/tree/main/wandb/integration/yolov8/yolov8.py
source_type: code
has_code: True
def on_train_end(self, trainer: BaseTrainer) -> None:
        """On train end we log all the media, including plots, images and best model artifact to Weights & Biases."""
        # Currently only the detection and segmentation trainers save images to the save_dir
        assert self.run is not None
        if not isinstance(trainer, ClassificationTrainer):
            assert self.run is not None
            self.run.log(
                {
                    "plots": [
                        wandb.Image(str(image_path), caption=image_path.stem)
                        for image_path in trainer.save_dir.glob("*.png")
                    ],
                    "val_images": [
                        wandb.Image(str(image_path), caption=image_path.stem)
                        for image_path in trainer.validator.save_dir.glob("val*.jpg")
                    ],
                },
            )
---
source: https://docs.wandb.ai/guides/model_registry/log-model-to-experiment
source_type: documentation
has_code: True
Track a model, the model's dependencies, and other information relevant to that model with the W&B Python SDK.  
Under the hood, W&B creates a lineage of model artifact that you can view with the W&B App UI or programmatically with the W&B Python SDK. See the Create model lineage map for more information.
Use the `run.log_model` API to log a model. Provide the path where your model files are saved to the `path` parameter. The path can be a local file, directory, or reference URI to an external bucket such as `s3://bucket/path`.  
Optionally provide a name for the model artifact for the `name` parameter. If `name` is not specified, W&B uses the basename of the input path prepended with the run ID.  
Copy and paste the proceeding code snippet. Ensure to replace values enclosed in `<>` with your own.  
```
import wandb
# Initialize a W&B run
run = wandb.init(project="<project>", entity="<entity>")
# Log the model
run.log\_model(path="<path-to-model>", name="<name>")
```  
Example: Log a Keras model to W&B  
The proceeding code example shows how to log a convolutional neural network (CNN) model to W&B.  
```python showLineNumbers
import os
import wandb
from tensorflow import keras
from tensorflow.keras import layers
config = {"optimizer": "adam", "loss": "categorical\_crossentropy"}
# Initialize a W&B run
run = wandb.init(entity="charlie", project="mnist-project", config=config)
---
source: https://wandb.ai/wandb_fc/articles/reports/--Vmlldzo1NDQyNTA4
source_type: report
has_code: True
Let’s have a look at the visualizations generated for this run [in the dashboard](https://web.archive.org/web/20210120023852/https://app.wandb.ai/cayush/pytorchlightning/reports/How-to-use-Pytorch-Lightning-with-Weights-%26-Biases--Vmlldzo2NjQ1Mw).  
Train loss and validation loss for the particular run are automatically logged in the dashboard in real time as the model is being trained.  
We can repeat the same training step with different hyper-parameters to compare different runs. We’ll change the name of the logger to uniquely identify each run.  
```
wandb_logger = WandbLogger(name='Adam-32-0.001',project='pytorchlightning')
wandb_logger = WandbLogger(name='Adam-64-0.01',project='pytorchlightning')
wandb_logger = WandbLogger(name='sgd-64-0.01',project='pytorchlightning')
```  
Here I’ve used a convention to name the runs. The first part is the optimizer, the second is the mini-batch size and third is the learning rate. For example the name ‘Adam-32-0.001’ means the optimizer being used is Adam with batch size of 32 and the learning rate is 0.001.  
Here’s how our models are faring so far.  
These visualizations are stored forever in your project which makes it much easier to compare the performances of variations with different hyperparameters, restore the best performing model and share results with your team.
---
source: https://github.com/wandb/wandb/tree/main/wandb/integration/openai/fine_tuning.py
source_type: code
has_code: True
# log results
        df_results = pd.read_csv(io.StringIO(results))
        for _, row in df_results.iterrows():
            metrics = {k: v for k, v in row.items() if not np.isnan(v)}
            step = metrics.pop("step")
            if step is not None:
                step = int(step)
            cls._run.log(metrics, step=step)
        fine_tuned_model = fine_tune.fine_tuned_model
        if fine_tuned_model is not None:
            cls._run.summary["fine_tuned_model"] = fine_tuned_model
        # training/validation files and fine-tune details
        cls._log_artifacts(fine_tune, project, entity)
        # mark run as complete
        cls._run.summary["status"] = "succeeded"
        cls._run.finish()
        return True
---
source: https://wandb.ai/byyoung3/ml-news/reports/--Vmlldzo1NzU4OTk2
source_type: report
has_code: True
# redefine training_args for clarity
training_args = TrainingArguments(
output_dir=training_args.output_dir or "./output",
overwrite_output_dir=True,
num_train_epochs=training_args.num_train_epochs or 100,
per_device_train_batch_size=training_args.per_device_train_batch_size or 1,
save_steps=training_args.save_steps or 100,
save_total_limit=training_args.save_total_limit or 2,
fp16=training_args.fp16 or True,
bf16=training_args.bf16 or False,
report_to=training_args.report_to or 'wandb',
logging_steps=training_args.logging_steps or 10,
gradient_checkpointing=training_args.gradient_checkpointing or True,
gradient_accumulation_steps=training_args.gradient_accumulation_steps or 8,
learning_rate=training_args.learning_rate or 5e-5,
cache_dir=training_args.cache_dir or './cache',
deepspeed=training_args.deepspeed or "./ds_configs/stage2.json"
)
model.config.use_cache = False         # required for gradient checkpointing
model.enable_input_require_grads()     # required for gradient checkpointing
model.gradient_checkpointing_enable()  # enable gradient checkpointing
trainer = Trainer(
model=model, tokenizer=tokenizer, args=training_args,
train_dataset=dataset["train"],
eval_dataset=None,
data_collator=data_collator)
trainer.train()
trainer.save_state()
trainer.save_model(output_dir=training_args.output_dir)
```
I've set the logging to go directly to Weights & Biases, as shown in the training arguments.  
```
report_to=training_args.report_to or 'wandb',
```
---
source: https://docs.wandb.ai/guides/track/log/log-models
source_type: documentation
has_code: True
---
Use W&B to log models that are created from W&B runs.  
The following guide describes how to log and interact with models logged to a W&B run.  
:::tip
The following APIs are useful for early model exploration and experiment tracking. Use the APIs listed on this page to quickly log models to a run, in addition to metrics, tables, media and other objects.  
The APIs listed on this page are not intended for [INSERT].  
W&B suggests that you use W&B Artifacts if you want to:
- Create and keep track of different versions of serialized data such as models, datasets, prompts, and more.
- Create lineage graphs of a model or any other objects tracked in W&B.
- How to interact with the model artifacts these methods created, such as updating properties (metadata, aliases, and descriptions)
:::
Declare a model artifact as an output of a run with the `log_model` method. To do so, provide a name for your model artifact and the path where your model is saved to for the `model_name` and `path` parameters, respectively.  
```
import wandb
project="<your-project-name>"
entity="<your-entity>"
path="/local/dir/70154.h5"
model\_artifact\_name="model.h5"
# Initialize a W&B run
run = wandb.init(project=project, entity=entity)
# Log the model
run.log\_model(model\_name=model\_artifact\_name, path=path)
run.finish()
```  
In the preceding code snippet, the model originally had a file name of `70154.h5` and was locally stored in the user's `/local/dir/` directory. When the user logged the model with `log_model`, they gave the model artifact a name of `model.h5`.<!--end-context-information-->
<!--start-question-->**Question**: # Query
please show me how to log training output_name model of the current training run
---
# Query Metadata
Language: en
Intents: 
Product Features:
	The user is asking for guidance on how to use Weights & Biases to log the output name of a model during a training run.
	The query is related to a feature of Weights & Biases such as Sweeps, Artifacts, Reports, Experiments, Tables, Prompts, Launch, Weave, StreamTables and more. Provide a link to the relevant documentation and explain the feature in detail
Sub-queries to consider answering: 
	How to log model output names in Weights & Biases?
	Steps to track model names during training runs in W&B.
	Best practices for logging model details in Weights & Biases.
<!--end-question-->
<!--final-answer-in-markdown-->
question
please show me how to log training output_name model of the current training run
answer
To log the output name of a model during a current training run in Weights & Biases, you can use the `log_model` method. This method allows you to declare a model artifact as an output of a run, enabling you to track the model along with its metadata, such as the name, description, and any other relevant information. Here's how you can do it:
1. **Initialize your W&B run**: Start by initializing a run in your training script. This will start tracking your training process.
    ```python
    import wandb
    # Initialize a W&B run
    run = wandb.init(project="your_project_name", entity="your_entity_name")
    ```
2. **Train your model**: This step involves your model's training logic. After training, you will typically have your model saved to a file.
3. **Log the model with its output name**: After training and saving your model, use the `run.log_model` method to log the model to the current run. Provide the path to the saved model and a name for the model artifact.
    ```python
    # Assuming your model is saved to 'path_to_your_model'
    model_path = "path_to_your_model"
    model_artifact_name = "output_name_of_your_model"
    # Log the model
    run.log_model(model_name=model_artifact_name, path=model_path)
    ```
4. **Finish the run**: Once you've logged all necessary information, including your model, make sure to finish the run to properly close it.
    ```python
    run.finish()
    ```
This process logs your model as an artifact associated with the specific run, capturing the output name of the model and allowing you to track and version your models over time. It's a best practice to log not just the model but also other relevant metrics and outputs of your training process to have a comprehensive record of your experiment.
Remember, the `log_model` method is useful for early model exploration and experiment tracking. For more complex scenarios, such as creating lineage graphs or tracking different versions of serialized data, consider using W&B Artifacts directly.
For more detailed information on logging models and using artifacts in Weights & Biases, refer to the [official documentation](https://docs.wandb.ai/guides/track/log/log-models).
model
gpt-4-0125-preview
sources
https://github.com/wandb/wandb/tree/main/wandb/beta/workflows.py
https://github.com/wandb/examples/tree/master/colabs/simpletransformers/SimpleTransformersQA.ipynb
https://wandb.ai/vincenttu/finetuning_zephyr7b/reports/--Vmlldzo1ODc0MTcx
https://wandb.ai/stacey/nlg/reports/--Vmlldzo1NzcwNzY=
https://docs.wandb.ai/guides/track/log/log-models
https://github.com/wandb/examples/tree/master/colabs/pytorch-lightning/Optimize_Pytorch_Lightning_models_with_Weights_&_Biases.ipynb
https://github.com/wandb/examples/tree/master/colabs/huggingface/Optimize_Hugging_Face_models_with_Weights_&_Biases.ipynb
https://wandb.ai/prompt-eng/openai-finetune-integration/reports/--Vmlldzo2MDEwMjEw
https://docs.wandb.ai/guides/integrations/huggingface
https://github.com/wandb/wandb/tree/main/wandb/integration/yolov8/yolov8.py
https://docs.wandb.ai/guides/model_registry/log-model-to-experiment
https://wandb.ai/wandb_fc/articles/reports/--Vmlldzo1NDQyNTA4
https://github.com/wandb/wandb/tree/main/wandb/integration/openai/fine_tuning.py
https://wandb.ai/byyoung3/ml-news/reports/--Vmlldzo1NzU4OTk2
https://docs.wandb.ai/guides/track/log/log-models
source_documents
source: https://github.com/wandb/wandb/tree/main/wandb/beta/workflows.py
source_type: code
has_code: True
def log_model(
    model_obj: Any,
    name: str = "model",
    aliases: Optional[Union[str, List[str]]] = None,
    description: Optional[str] = None,
    metadata: Optional[dict] = None,
    project: Optional[str] = None,
    scope_project: Optional[bool] = None,
    **kwargs: Dict[str, Any],
) -> "_SavedModel":
    """Log a model object to enable model-centric workflows in the UI.
    Supported frameworks include PyTorch, Keras, Tensorflow, Scikit-learn, etc. Under
    the hood, we create a model artifact, bind it to the run that produced this model,
    associate it with the latest metrics logged with `wandb.log(...)` and more.
---
source: https://github.com/wandb/examples/tree/master/colabs/simpletransformers/SimpleTransformersQA.ipynb
source_type: notebook
has_code: True
SimpleTransformers automatically logs important metrics to W&B.  
You can also customize what you log using two methods:  
1. Resuming the run,
"restarting" the experiment so that you can log additional stuff, including more training.
2. Using the `wandb.api` to update existing runs with additional metadata.  
We show both below.
Runs that have finished can be resumed
so that additional information can be added to an experiment.
For example, you might be using
pre-emptible compute
where training runs can be stopped prematurely.  
Here we use it to log the model checkpoints to the training run,
since it was responsible for creating them.  
```
import os
with wandb.init(id=model.wandb_run_id, resume="allow", project=wandb_project) as training_run:
for dir in sorted(os.listdir("outputs")):
if "checkpoint" in dir:
artifact = wandb.Artifact("model-checkpoints", type="checkpoints")
artifact.add_dir("outputs" + "/" + dir)
training_run.log_artifact(artifact)
```
---
source: https://wandb.ai/vincenttu/finetuning_zephyr7b/reports/--Vmlldzo1ODc0MTcx
source_type: report
has_code: True
```
results = trainer.train()  # Now we just run train()!
run.finish()
```  
Here are our training results.
Let's run inference and log our results to a [W&B Table](https://docs.wandb.ai/ref/python/data-types/table). Since we have a simple LLM pipeline, tables are great for logging inference results like input/output, token usage, and response time.  
```
query = "what's a neural network?"
# Logging inference results to W&B Table...
run = wandb.init(
project="finetuning_zephyr7b",
name="log_inference",
)
device = "cuda" if torch.cuda.is_available() else "cpu"
model_input = tokenizer(text=query, return_tensors="pt").to(device)
start_time_ms = datetime.datetime.now().timestamp() * 1000
_ = model.eval()
with torch.no_grad():
out = model.generate(**model_input, max_new_tokens=300)
response_text = tokenizer.decode(out[0], skip_special_tokens=True)
end_time_ms = round(datetime.datetime.now().timestamp() * 1000)  # logged in milliseconds
token_usage = len(out[0])
data = [
[
query,
response_text,
end_time_ms - start_time_ms,
token_usage
]
]
table = wandb.Table(data=data, columns=["input", "output", "latency", "token_usage"])
run.log({"Inference": table})
run.finish()
```  
Below is our very own W&B Table for inference embedded right into this [W&B Report](https://docs.wandb.ai/guides/reports/create-a-report)!
---
source: https://wandb.ai/stacey/nlg/reports/--Vmlldzo1NzcwNzY=
source_type: report
has_code: False
This section will show you how to log and browse a model's predictions over time to get an overall sense for performance improvement during training.  
Here I log the model's continuation of the input "Wh" every 100 epochs, alongside the loss. Over time, a higher proportion of the generated letter-sequences are actual dictionary words, character names in caps become more recognizably Shakespearean, and some of the phrases start to make a little bit of sense.  
An early sample reads "When to the burther, thou not week, Of be the print is de-comer-sumerence speating." (So: not ideal.) A much later sample gets to some phrase-level coherence: "What advancements? WARWICK: I know you not lies it is kneel. Come, time the base."  
You can scroll through sample text below, filter based on loss or epoch, or view the same Table[ in the Artifacts context](https://wandb.ai/stacey/nlg/artifacts/train_samples/train_charnn/654d31a0234a89c11887/files/training_samples.table.json).  
Top: 10K epochs, sampled every 500. Bottom: 8K, sampled every 100.
There are several ways to [save a Table to W&B](https://docs.wandb.ai/guides/data-vis/log-tables#save-tables):  
* The default run.log() call to associate a Table with any run. This is recommended for media or metrics generated during a step in your workflow, such as model predictions in validation or inference, samples from generative model during training, etc.
* The slightly more verbose artifact.add() call to associate a Table with a particular version of an Artifact, which is most helpful when you plan to reference and/or download the exact version of media logged to the Table—typically a dataset you will use in future experiments or downstream tasks  
Here's how:
---
source: https://docs.wandb.ai/guides/track/log/log-models
source_type: documentation
has_code: True
# Hyperparameters
loss = run.config["loss"]
optimizer = run.config["optimizer"]
metrics = ["accuracy"]
num\_classes = 10
input\_shape = (28, 28, 1)
# Training algorithm
model = keras.Sequential(
[
layers.Input(shape=input\_shape),
layers.Conv2D(32, kernel\_size=(3, 3), activation="relu"),
layers.MaxPooling2D(pool\_size=(2, 2)),
layers.Conv2D(64, kernel\_size=(3, 3), activation="relu"),
layers.MaxPooling2D(pool\_size=(2, 2)),
layers.Flatten(),
layers.Dropout(0.5),
layers.Dense(num\_classes, activation="softmax"),
]
)
# Configure the model for training
model.compile(loss=loss, optimizer=optimizer, metrics=metrics)
# Save model
model\_filename = "model.h5"
local\_filepath = "./"
full\_path = os.path.join(local\_filepath, model\_filename)
model.save(filepath=full\_path)
# Log the model to the W&B run
run.log\_model(path=full\_path, name="MNIST")
run.finish()
```  
When the user called `log\_model`, a model artifact named `MNIST` was created and the file `model.h5` was added to the model artifact. Your terminal or notebook will print information of where to find information about the run the model was logged to.  
```
View run different-surf-5 at: https://wandb.ai/charlie/mnist-experiments/runs/wlby6fuw
Synced 5 W&B file(s), 0 media file(s), 1 artifact file(s) and 0 other file(s)
Find logs at: ./wandb/run-20231206\_103511-wlby6fuw/logs
```
---
source: https://github.com/wandb/examples/tree/master/colabs/pytorch-lightning/Optimize_Pytorch_Lightning_models_with_Weights_&_Biases.ipynb
source_type: notebook
has_code: True
| Functionality | Argument/Function | PS |
| --- | --- | --- |
| Logging models | `WandbLogger(... ,log_model='all')` or `WandbLogger(... ,log_model=True`) | Log all models if `log_model="all"` and at end of training if `log_model=True` |
| Set custom run names | `WandbLogger(... ,name='my_run_name'`) |  |
| Organize runs by project | `WandbLogger(... ,project='my_project')` |  |
| Log histograms of gradients and parameters | `WandbLogger.watch(model)` | `WandbLogger.watch(model, log='all')` to log parameter histograms |
| Log hyperparameters | Call `self.save_hyperparameters()` within `LightningModule.__init__()` |  |
| Log custom objects (images, audio, video, molecules…) | Use `WandbLogger.log_text`, `WandbLogger.log_image` and `WandbLogger.log_table`, etc. |  |  
See the WandbLogger docs here for all parameters.  
```
from lightning.pytorch.loggers import WandbLogger
from lightning.pytorch import Trainer
wandb_logger = WandbLogger(project='MNIST', # group runs in "MNIST" project
log_model='all') # log all new checkpoints during training
```
---
source: https://github.com/wandb/examples/tree/master/colabs/huggingface/Optimize_Hugging_Face_models_with_Weights_&_Biases.ipynb
source_type: notebook
has_code: True
W&B integration with Hugging Face can be configured to add extra functionalities:  
* auto-logging of models as artifacts: just set environment varilable `WANDB_LOG_MODEL` to `true`
* log histograms of gradients and parameters: by default gradients are logged, you can also log parameters by setting environment variable `WANDB_WATCH` to `all`
* set custom run names with `run_name` arg present in scripts or as part of `TrainingArguments`
* organize runs by project with the `WANDB_PROJECT` environment variable  
For more details refer to W&B + HF integration documentation.  
Let's log every trained model.  
```
%env WANDB_LOG_MODEL=true
```
---
source: https://wandb.ai/prompt-eng/openai-finetune-integration/reports/--Vmlldzo2MDEwMjEw
source_type: report
has_code: False
The end result of fine-tuning is the fine-tuned model's ID. If you noticed in the configs above, the fine_tuned_model config name is the ID of fine-tuned model. The logger doesn't only capture the ID, it also captures the metadata and the data-model lineage:  
* As we can't log the model weights (as OpenAI don't share them) the logger logs a `model_metadata` Artifact, which can then be versioned. It can also further be linked to a model in the  and even paired with . A host of W&B product features can be used one the logger creates this Artifact. Here's the model metadata file:  
2. The DAG view of an ML workflow show you the connection (or "lineage") between datasets, training jobs and output models. This is a useful tool to find bugs and explain and understand the pipeline. The DAG view for the model_metadata artifact is shown below. Notice how the train and validation files were consumed by the fine-tuned job to create the model_metadata:v0 artifact. This can be further extended with evaluation, CI/CD, and more.
The WandbLogger logger can keep track of your fine-tuning jobs and give your detailed visibility and reproducibility into how your fine-tuning runs are performing. Try our colab to get started:
We would love feedback on this logger, if you find any issues or bugs while using the WandbLogger or find it useful please do open an issue in our [wandb](https://github.com/wandb/wandb) GitHub repository, we'd love to hear from you!  
'
---
source: https://docs.wandb.ai/guides/integrations/huggingface
source_type: documentation
has_code: True
Once you have logged your checkpoints to Artifacts, you can then register your best model checkpoints and centralize them across your team using the Weights & Biases **Model Registry**. Here you can organize your best models by task, manage model lifecycle, facilitate easy tracking and auditing throughout the ML lifecyle, and automate downstream actions with webhooks or jobs.  
See the Model Registry documentation for how to link a model Artifact to the Model Registry.
Visualing your model outputs during training or evaluation is often essential to really understand how your model is training.  
By using the callbacks system in the Transformers Trainer, you can log additional helpful data to W&B such as your models' text generation outputs or other predictions to W&B Tables.  
See the **Custom logging section** below for a full guide on how to log evaluation outupts while training to log to a W&B Table like this:
If your training is encapsulated in a Python script, the W&B run will end when your script finishes.  
If you are using a Jupyter or Google Colab notebook, you'll need to tell us when you're done with training by calling `wandb.finish()`.  
```
trainer.train()  # start training and logging to W&B
# post-training analysis, testing, other logged code
wandb.finish()
```
Once you have logged your training results you can explore your results dynamically in the W&B Dashboard. It's easy to compare across dozens of runs at once, zoom in on interesting findings, and coax insights out of complex data with flexible, interactive visualizations.
If `load_best_model_at_end=True` is set in the `TrainingArguments` that are passed to the `Trainer`, then W&B will save the best performing model checkpoint to Artifacts.  
If you'd like to centralize all your best model versions across your team to organize them by ML task, stage them for production, bookmark them for further evaluation, or kick off downstream Model CI/CD processes then ensure you're saving your model checkpoints to Artifacts. Once logged to Artifacts, these checkpoints can then be promoted to the Model Registry.
---
source: https://github.com/wandb/wandb/tree/main/wandb/integration/yolov8/yolov8.py
source_type: code
has_code: True
def on_train_end(self, trainer: BaseTrainer) -> None:
        """On train end we log all the media, including plots, images and best model artifact to Weights & Biases."""
        # Currently only the detection and segmentation trainers save images to the save_dir
        assert self.run is not None
        if not isinstance(trainer, ClassificationTrainer):
            assert self.run is not None
            self.run.log(
                {
                    "plots": [
                        wandb.Image(str(image_path), caption=image_path.stem)
                        for image_path in trainer.save_dir.glob("*.png")
                    ],
                    "val_images": [
                        wandb.Image(str(image_path), caption=image_path.stem)
                        for image_path in trainer.validator.save_dir.glob("val*.jpg")
                    ],
                },
            )
---
source: https://docs.wandb.ai/guides/model_registry/log-model-to-experiment
source_type: documentation
has_code: True
Track a model, the model's dependencies, and other information relevant to that model with the W&B Python SDK.  
Under the hood, W&B creates a lineage of model artifact that you can view with the W&B App UI or programmatically with the W&B Python SDK. See the Create model lineage map for more information.
Use the `run.log_model` API to log a model. Provide the path where your model files are saved to the `path` parameter. The path can be a local file, directory, or reference URI to an external bucket such as `s3://bucket/path`.  
Optionally provide a name for the model artifact for the `name` parameter. If `name` is not specified, W&B uses the basename of the input path prepended with the run ID.  
Copy and paste the proceeding code snippet. Ensure to replace values enclosed in `<>` with your own.  
```
import wandb
# Initialize a W&B run
run = wandb.init(project="<project>", entity="<entity>")
# Log the model
run.log\_model(path="<path-to-model>", name="<name>")
```  
Example: Log a Keras model to W&B  
The proceeding code example shows how to log a convolutional neural network (CNN) model to W&B.  
```python showLineNumbers
import os
import wandb
from tensorflow import keras
from tensorflow.keras import layers
config = {"optimizer": "adam", "loss": "categorical\_crossentropy"}
# Initialize a W&B run
run = wandb.init(entity="charlie", project="mnist-project", config=config)
---
source: https://wandb.ai/wandb_fc/articles/reports/--Vmlldzo1NDQyNTA4
source_type: report
has_code: True
Let’s have a look at the visualizations generated for this run [in the dashboard](https://web.archive.org/web/20210120023852/https://app.wandb.ai/cayush/pytorchlightning/reports/How-to-use-Pytorch-Lightning-with-Weights-%26-Biases--Vmlldzo2NjQ1Mw).  
Train loss and validation loss for the particular run are automatically logged in the dashboard in real time as the model is being trained.  
We can repeat the same training step with different hyper-parameters to compare different runs. We’ll change the name of the logger to uniquely identify each run.  
```
wandb_logger = WandbLogger(name='Adam-32-0.001',project='pytorchlightning')
wandb_logger = WandbLogger(name='Adam-64-0.01',project='pytorchlightning')
wandb_logger = WandbLogger(name='sgd-64-0.01',project='pytorchlightning')
```  
Here I’ve used a convention to name the runs. The first part is the optimizer, the second is the mini-batch size and third is the learning rate. For example the name ‘Adam-32-0.001’ means the optimizer being used is Adam with batch size of 32 and the learning rate is 0.001.  
Here’s how our models are faring so far.  
These visualizations are stored forever in your project which makes it much easier to compare the performances of variations with different hyperparameters, restore the best performing model and share results with your team.
---
source: https://github.com/wandb/wandb/tree/main/wandb/integration/openai/fine_tuning.py
source_type: code
has_code: True
# log results
        df_results = pd.read_csv(io.StringIO(results))
        for _, row in df_results.iterrows():
            metrics = {k: v for k, v in row.items() if not np.isnan(v)}
            step = metrics.pop("step")
            if step is not None:
                step = int(step)
            cls._run.log(metrics, step=step)
        fine_tuned_model = fine_tune.fine_tuned_model
        if fine_tuned_model is not None:
            cls._run.summary["fine_tuned_model"] = fine_tuned_model
        # training/validation files and fine-tune details
        cls._log_artifacts(fine_tune, project, entity)
        # mark run as complete
        cls._run.summary["status"] = "succeeded"
        cls._run.finish()
        return True
---
source: https://wandb.ai/byyoung3/ml-news/reports/--Vmlldzo1NzU4OTk2
source_type: report
has_code: True
# redefine training_args for clarity
training_args = TrainingArguments(
output_dir=training_args.output_dir or "./output",
overwrite_output_dir=True,
num_train_epochs=training_args.num_train_epochs or 100,
per_device_train_batch_size=training_args.per_device_train_batch_size or 1,
save_steps=training_args.save_steps or 100,
save_total_limit=training_args.save_total_limit or 2,
fp16=training_args.fp16 or True,
bf16=training_args.bf16 or False,
report_to=training_args.report_to or 'wandb',
logging_steps=training_args.logging_steps or 10,
gradient_checkpointing=training_args.gradient_checkpointing or True,
gradient_accumulation_steps=training_args.gradient_accumulation_steps or 8,
learning_rate=training_args.learning_rate or 5e-5,
cache_dir=training_args.cache_dir or './cache',
deepspeed=training_args.deepspeed or "./ds_configs/stage2.json"
)
model.config.use_cache = False         # required for gradient checkpointing
model.enable_input_require_grads()     # required for gradient checkpointing
model.gradient_checkpointing_enable()  # enable gradient checkpointing
trainer = Trainer(
model=model, tokenizer=tokenizer, args=training_args,
train_dataset=dataset["train"],
eval_dataset=None,
data_collator=data_collator)
trainer.train()
trainer.save_state()
trainer.save_model(output_dir=training_args.output_dir)
```
I've set the logging to go directly to Weights & Biases, as shown in the training arguments.  
```
report_to=training_args.report_to or 'wandb',
```
---
source: https://docs.wandb.ai/guides/track/log/log-models
source_type: documentation
has_code: True
---
Use W&B to log models that are created from W&B runs.  
The following guide describes how to log and interact with models logged to a W&B run.  
:::tip
The following APIs are useful for early model exploration and experiment tracking. Use the APIs listed on this page to quickly log models to a run, in addition to metrics, tables, media and other objects.  
The APIs listed on this page are not intended for [INSERT].  
W&B suggests that you use W&B Artifacts if you want to:
- Create and keep track of different versions of serialized data such as models, datasets, prompts, and more.
- Create lineage graphs of a model or any other objects tracked in W&B.
- How to interact with the model artifacts these methods created, such as updating properties (metadata, aliases, and descriptions)
:::
Declare a model artifact as an output of a run with the `log_model` method. To do so, provide a name for your model artifact and the path where your model is saved to for the `model_name` and `path` parameters, respectively.  
```
import wandb
project="<your-project-name>"
entity="<your-entity>"
path="/local/dir/70154.h5"
model\_artifact\_name="model.h5"
# Initialize a W&B run
run = wandb.init(project=project, entity=entity)
# Log the model
run.log\_model(model\_name=model\_artifact\_name, path=path)
run.finish()
```  
In the preceding code snippet, the model originally had a file name of `70154.h5` and was locally stored in the user's `/local/dir/` directory. When the user logged the model with `log_model`, they gave the model artifact a name of `model.h5`.
total_tokens
8,750
prompt_tokens
8,072
completion_tokens
678
time_taken
33.654315
start_time
datetime.datetime(2024, 9, 6, 11, 47, 43, 94331, tzinfo=datetime.timezone.utc)
end_time
datetime.datetime(2024, 9, 6, 11, 48, 16, 749602, tzinfo=datetime.timezone.utc)