Hyperparameter Optimization Report: ppo_sentiments_train_2022-11-10_17-53-33

This is a report that shows the results of a hyperparameter optimization experiment.
Created on November 10|Last edited on November 10
Comment
﻿
The following plots show the results of the hyperparameter optimization experiment. Use this as a starting point for your analysis. Go in the edit mode to customize the report. Share it with your team to collaborate on the analysis.
AnalysisParallel coordinates chart (top) summarize the relationship between large numbers of hyperparameters and model metrics at a glance. 
The scatter plot (right) compares the different trials and gives you a insight on how the trials progressed. 
The parameter importance plot(left) lists the hyperparameters that were the best predictors of, and highly correlated to desirable values of your metrics.
﻿
Run set1
﻿
MetricsThe following line plots show the metrics for each trial. Use this to investigate the performance of the model for each trial at the metrics level.
﻿
Run set1
﻿
Best ConfigThe code block shown below is the best config found by the hyperparameter optimization experiment according to Ray Tune.
{
    "model": {
        "model_path": "lvwerra/gpt2-imdb",
        "tokenizer_path": "gpt2",
        "model_type": "AcceleratePPOModel",
        "num_layers_unfrozen": 2
    },
    "train": {
        "seq_length": 36,
        "epochs": 2,
        "total_steps": 100,
        "batch_size": 128,
        "lr_ramp_steps": 100,
        "lr_decay_steps": 79000,
        "weight_decay": 1e-06,
        "learning_rate_init": 0.0001412,
        "learning_rate_target": 0.0001412,
        "opt_betas": [
            0.9,
            0.95
        ],
        "checkpoint_interval": 10000,
        "eval_interval": 4,
        "pipeline": "PPOPipeline",
        "orchestrator": "PPOOrchestrator",
        "project_name": "trlx-hyperopt-bohb"
    },
    "method": {
        "name": "ppoconfig",
        "num_rollouts": 96,
        "chunk_size": 128,
        "ppo_epochs": 3,
        "init_kl_coef": 0.1,
        "target": 6,
        "horizon": 10000,
        "gamma": 1,
        "lam": 0.9505779632324011,
        "cliprange": 0.2,
        "cliprange_value": 0.2,
        "vf_coef": 2.3,
        "gen_kwargs": {
            "max_length": 48,
            "min_length": 48,
            "top_k": 0.0,
            "top_p": 1.0,
            "do_sample": true
        }
    }
}
﻿
Add a comment