Soley Therapeutics PoC Guide

One stop shop for everything you need to test out during the PoC.
Created on August 22|Last edited on August 22
Comment
﻿
Weights and Biases (W&B) 💫Workshop SessionsUse Cases / Test CasesEnvironmentGetting Started (SDK Installation and Login)Test CasesTest Case 1: End to End Artifacts TrackingTest Case 2: Track Model Training MetricsTest Case 3: Automate the end to end processTest Case 4: Track Hyperparameter tuning jobsExperiment Tracking 🍽 Artifact Tracking and VersioningW&B RegistryInteractive TablesHyperparameter SweepsReportsFAQs
﻿
Weights and Biases (W&B) 💫Weights and Biases is a ML Ops platform built to facilitate collaboration and reproducibility across the machine learning development lifecycle. Machine learning projects can quickly become a mess without some best practices in place to aid developers and scientists as they iterate on models and move them to production. 
W&B is lightweight enough to work with whatever framework or platform teams are currently using, but enables teams to quickly start logging their important results to a central system of record. On top of this system of record, W&B has built visualization, automation, and documentation capabilities for better debugging, model tuning, and project management. 
Workshop Sessions
















DateSessionRecording LinkTopics Discussed
Aug 22, 2024Discuss Use-Caseshttps://us-39259.app.gong.io/e/c-share/?tkn=mm9kmm0c975s60anz5bkswubOverview on W&B PyTorch Integration, Artifacts, Experiment Tracking, Registry, Automations, Reports
﻿
Use Cases / Test Cases
EnvironmentWeights & Biases Trial account is hosted here and everyone should have access. Let us know if you haven't received an invite.
Getting Started (SDK Installation and Login)To start using W&B, you first need to install the Python package (if it's not already there)
pip install wandb
Once it's installed, authenticate your user account by logging in through the CLI or SDK. You should have receive an email to sign up to the platform, after which you can obtain your API token (The API token is in your "Settings" section under your profile)
wandb login --host <YOUR W&B HOST URL> <YOUR API TOKEN>
OR through Python:
WANDB_BASE_URL = ﻿﻿﻿'https://wandb.ai'﻿
wandb.login(host=os.getenv("WANDB_BASE_URL"), key=os.getenv("WANDB_API_KEY"))
Once you are logged in, you are ready to track your workflows!
Test Cases





























S NoCapability & Success CriteriaW&B Product Area
1End to End Artifact TrackingW&B Artifacts
2Track Model Training MetricsW&B Experiment Tracking
3Automate the process with triggering downstream tasks automaticallyW&B Automations, W&B Launch, W&B Reports
4Track Hyperparameter tuning jobsW&B Sweeps
﻿
﻿
Test Case 1: End to End Artifacts Tracking
Test Case 2: Track Model Training Metrics
Test Case 3: Automate the end to end process
Test Case 4: Track Hyperparameter tuning jobs
Experiment Tracking 🍽 
Artifact Tracking and Versioning
W&B Registry
Interactive Tables
Hyperparameter SweepsOne of the more tedious aspects of training deep learning models is tuning hyper-parameters. When we log runs in W&B, we can make W&B aware of hyper-parameters. A central sweep controller can then delegate new hyper-parameter combinations based on a set of distributions we specify across the hyper-parameter space through a .yaml file. If we do a Bayes search, W&B can even seed the search with previous runs we've already logged. Below is an example where we have a simple training function which exposes several hyper-parameters to W&B via wandb.config. W&B sweep then initializes a hyper-parameter search using the dictionary of distributions for the hyper-parameter space. 
def train(config=None):
    # Initialize a new wandb run
    with wandb.init(config=config):
        # If called by wandb.agent, as below,
        # this config will be set by Sweep Controller
        config = wandb.config
﻿
        loader = build_dataset(config.batch_size)
        network = build_network(config.fc_layer_size, config.dropout)
        optimizer = build_optimizer(network, config.optimizer, config.learning_rate)
﻿
        for epoch in range(config.epochs):
            avg_loss = train_epoch(network, loader, optimizer)
            wandb.log({"loss": avg_loss, "epoch": epoch})
﻿
sweep_config = {
    'method': 'random',
    # 'method': 'grid',
    # 'method': 'bayes',
    }
﻿
parameters_dict = {
    'optimizer': {
        'values': ['adam', 'sgd']
    },
    'fc_layer_size': {
        'values': [128, 256, 512, 1024]
    },
    'dropout': {
        'values': [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7]
    },
    # Static hyperparameter, notice singular key
    'epochs': {
        'value': 10
    },
    'learning_rate': {
        # Flat distribution between 0 and 0.1
        'distribution': 'uniform',
        'min': 0,
        'max': 0.25
    },
    'batch_size': {
        'distribution': 'q_log_uniform',
        'q': 1,
        'min': math.log(32),
        'max': math.log(256),
    }
}
﻿
### Initializes the central sweep server
sweep_config['parameters'] = parameters_dict
sweep_id = wandb.sweep(sweep_config, project="sweeps-demo-pytorch")
﻿
### Run this in multiple machines/cores to distribute the hyperparameter search
wandb.agent(sweep_id, train, count=10)
﻿
﻿
W&B will automatically separate the runs associated with a sweep and create some charts automatically that allow us to do more meta-analysis on which combinations are working well. 
﻿
Sweep: 0ewut602 150
﻿
W&B provides a mechanism for automating hyper-parameter search through W&B Sweeps. Sweeps allows you to configure a large set of experiments across a pre-specified hyper-parameter space. To implement a sweep you just need to: 
Add wandb.init() to your training script, ensuring that all hyper-parameters are passed to your training logic via wandb.config. 
﻿Write a yaml file with your hyper-parameter search specified i.e. method of search, hyper-parameter distributions and values to search over. 
Run the sweep controller, which runs in W&B through wandb.sweep or through the UI. The controller will delegate new hyperparameter values to wandb.config of the various agents running. 
﻿Run agents in however many machines you want to run the experiments with wandb.agent
The agents will execute the training script replacing the wandb.config with queued hyper-parameter values the controller is keeping track of. 
If you prefer to use other hyper-parameter optimization frameworks, W&B has integrations with RayTune, Optuna, among others.
Reports
FAQs1. I didn't name my run. Where is the run name coming from?
Ans: If you do not explicitly name your run, a random run name will be assigned to the run to help identify the run in the UI. For instance, random run names will look like "pleasant-flower-4" or "misunderstood-glade-2".
2. How can I configure the name of the run in my training code?
Ans: At the top of your training script when you call wandb.init, pass in an experiment name, like this: 
wandb.init(name="my_awesome_run")
3. If wandb crashes, will it possibly crash my training run?
Ans: It is extremely important to us that we never interfere with your training runs. We run wandb in a separate process to make sure that if wandb somehow crashes, your training will continue to run. If the internet goes out, wandb will continue to retry sending data to wandb.ai.
4. Why is a run marked crashed in W&B when it’s training fine locally?
This is likely a connection problem — if your server loses internet access and data stops syncing to W&B, we mark the run as crashed after a short period of retrying.
5. How do I stop wandb from writing to my terminal or my jupyter notebook output?
Ans: Set the environment variable WANDB_SILENT to true.
In Python
os.environ["WANDB_SILENT"] = "true"
Within Jupyter Notebook
%env WANDB_SILENT=true
With Command Line
WANDB_SILENT=true
﻿
﻿
Date	Session	Recording Link	Topics Discussed
Aug 22, 2024	Discuss Use-Cases	https://us-39259.app.gong.io/e/c-share/?tkn=mm9kmm0c975s60anz5bkswub	Overview on W&B PyTorch Integration, Artifacts, Experiment Tracking, Registry, Automations, Reports
S No	Capability & Success Criteria	W&B Product Area
1	End to End Artifact Tracking	W&B Artifacts
2	Track Model Training Metrics	W&B Experiment Tracking
3	Automate the process with triggering downstream tasks automatically	W&B Automations, W&B Launch, W&B Reports
4	Track Hyperparameter tuning jobs	W&B Sweeps
Add a comment