BoN @9680c9f/Merge branch 'main' into bon-x/2023-09-01 main @bf0fe38/fix(examples/hh): old gpt-j checkpoint loading (#552)/2023-09-01

BoN @a022d3f/Merge branch 'main' into BoN/2023-09-01 main @a7eb066/chore(requirements.txt): update everything to the latest (#547)/2023-08-31

update-requirements @bfb53f6/chore(requirements.txt): update everything to the latest/2023-08-21 main @900040a/mpt support (#546)/2023-08-21

BoN @36f06af/Remove prints/2023-08-07 main @2e667e6/fix(modeling): deepspeed checkpoint loading (#482)/2023-08-08

value-branch @d48c1a4/Fix: Include num_value_layers_frozen arg in Seq2Seq model init/2023-07-26 main @e36fe9d/fix(modeling_ppo): load reference head under zero3 (#489)/2023-07-24

value-branch @d48c1a4/Fix: Include num_value_layers_frozen arg in Seq2Seq model init/2023-07-26 main @e36fe9d/fix(modeling_ppo): load reference head under zero3 (#489)/2023-07-24

value-branch @d48c1a4/Fix: Include num_value_layers_frozen arg in Seq2Seq model init/2023-07-26 main @e36fe9d/fix(modeling_ppo): load reference head under zero3 (#489)/2023-07-24

value-branch @a875c4a/Fixing style/2023-07-18 main @e36fe9d/fix(modeling_ppo): load reference head under zero3 (#489)/2023-07-24

value-branch @a875c4a/Fixing style/2023-07-18 main @e36fe9d/fix(modeling_ppo): load reference head under zero3 (#489)/2023-07-24

value-branch @a875c4a/Fixing style/2023-07-18 main @e36fe9d/fix(modeling_ppo): load reference head under zero3 (#489)/2023-07-24

value-branch @a875c4a/Fixing style/2023-07-18 main @e36fe9d/fix(modeling_ppo): load reference head under zero3 (#489)/2023-07-24

fix-logging @7e4374e/Fix logging/2023-07-05 main @5d0f04d/Update README.md (#537)/2023-07-21

fix-logging @7e4374e/Fix logging/2023-07-05 main @5d0f04d/Update README.md (#537)/2023-07-21

fix-logging @7e4374e/Fix logging/2023-07-05 main @5d0f04d/Update README.md (#537)/2023-07-21

BoN @1fda0ce/fix(accelerate_base_trainer): order of keyword arguments/2023-07-22 main @5d0f04d/Update README.md (#537)/2023-07-21

main @ea7c2b0/revert(accelerate_ilql_trainer): remove `shuffle` option/2023-07-22 main @5d0f04d/Update README.md (#537)/2023-07-21

BoN @1fda0ce/fix(accelerate_base_trainer): order of keyword arguments/2023-07-22 main @5d0f04d/Update README.md (#537)/2023-07-21

BoN @3d7e0d5/Fix chunked generation/2023-07-21 main @5d0f04d/Update README.md (#537)/2023-07-21

main @10369a1/fix(accelerate_trainer): fill a missing `shuffle` argument/2023-07-22 main @5d0f04d/Update README.md (#537)/2023-07-21

main @8a943d9/Reset train dataloder at each iteration/2023-07-19 main @288d4cb/Fix: rename model_tok to tokenizer is reward_fn arg (#534)/2023-07-20

dense-reward-carper @6bc1c54/fix(ppo_randomwalks): `reward_fn` signature to accommodate tokenizer/2023-07-17 main @5c5abca/feat(readme): add instructions to avoid OOMs with hyperparameters (#470)/2023-07-13

main @fe33681/Fix ordering of ppo epoch iteration/2023-07-13 main @1446523/Update README.md/2023-07-10

dense-reward-carper @442423f/Fix missing dtype in trainer rewards tensor (#520)/2023-07-10 main @b9c16f5/fix peft config cause tensorboard type error (#515)/2023-07-05

dev_additional_special_tokens @d2c4b3b/fix(accelerate_ppo_trainer): resize token embeddings without hydra/2023-06-26 main @475a284/Release: v0.7.0 (#511)/2023-06-23

dev_additional_special_tokens @12560ac/Merge branch 'main' into dev_additional_special_tokens/2023-06-26 main @475a284/Release: v0.7.0 (#511)/2023-06-23

quantization @2b2c546/ 8-bit inference (#512)/2023-06-24 main @475a284/Release: v0.7.0 (#511)/2023-06-23

bigcode @a7e3b9c/feat(modeling_ppo): add GPTBigCodeModelBranch/2023-06-19 main @475a284/Release: v0.7.0 (#511)/2023-06-23

dev_additional_special_tokens @134bbf9/fix: move hydra heads resize_token_embeddings/2023-06-14 main @475a284/Release: v0.7.0 (#511)/2023-06-23

fix-default-gen_kwargs @087e73d/fix(base_trainer): force pad_token regardless of architecture/2023-06-23 main @093e89a/Change logging_dir to project_dir (#506)/2023-06-17

fix-logratio-bug @a451c94/Shift mask for seq2seq models + remove debug print statements/2023-06-22 main @093e89a/Change logging_dir to project_dir (#506)/2023-06-17

fix-logratio-bug @a451c94/Shift mask for seq2seq models + remove debug print statements/2023-06-22 main @093e89a/Change logging_dir to project_dir (#506)/2023-06-17

peft_migration @de52db0/ILQL generate when temperature is 0./2023-06-08 main @093e89a/Change logging_dir to project_dir (#506)/2023-06-17

peft_migration @de52db0/ILQL generate when temperature is 0./2023-06-08 main @093e89a/Change logging_dir to project_dir (#506)/2023-06-17

fix-ppo-zero3 @717d81b/style: satisfy black/2023-06-05 main @093e89a/Change logging_dir to project_dir (#506)/2023-06-17

fix-drop-last-in-dataloader @3cd05ed/style: satisfy flake/2023-05-10 main @9fdd0d7/feat(requirements.txt): upgrade dependencies (#465)/2023-05-01

update-requirements @3291d2e/fix(requirements.txt): relax dependencies/2023-05-01 main @a47611d/docs(base_trainer): fill in missing `prepare_learning` method (#449)/2023-04-30

update-requirements @8e64ad8/chore(requirements.txt): upgrade/2023-04-30 main @a47611d/docs(base_trainer): fill in missing `prepare_learning` method (#449)/2023-04-30

main @8540322/fix(ppo_modeling): condition passing `device` in `OPTModelBranch`/2023-04-25 main @07c962e/Create Example training scripts to run in Stability cluster (#419)/2023-04-24

fix-kl-computation @c37aa8b/feat(ppo_trainer): log token-wise KL/2023-04-20 main @9bc0836/fix(offline_pipeline): ILQL negative indexing under truncation (#435)/2023-04-18

fix-kl-computation @0920350/style: satisfy black/2023-04-19 main @9bc0836/fix(offline_pipeline): ILQL negative indexing under truncation (#435)/2023-04-18

fix-save-pretrained-zero3 @d94aac1/style/2023-04-10 main @a66a7da/[feat] Support tying metadata to each prompt (#421)/2023-04-10

add-prompts-metadata @8f11cca/style: satisfy flake/2023-04-06 main @565c316/Minibatch impl (#364)/2023-04-06

add-prompts-metadata @8f11cca/style: satisfy flake/2023-04-06 main @565c316/Minibatch impl (#364)/2023-04-06

add_eos_token_when_padded @f219302/Ensure trailing EOS token is added also when the generated output contains trailing padding as is often the case when doing batch generation/2023-04-05 main @d7225c2/set torch dependency to version 2.0.0 for CUDA in installation instructions (#409)/2023-04-05

ray-train-integration @ec638c3/style: satisfy black/2023-03-31 main @c77e304/Nemo ILQL fixes (#404)/2023-03-30

fix-preserve-eos-token @de3b04f/fix(ppo_trainer): preserve `<eos>` and append it after trimming/2023-03-28 main @114998b/[feat] Add benchmark tools (#357)/2023-03-28

add-benchmark-tools @fb8cdb7/chore(README): add a link to reference runs/2023-03-26 main @086a905/[feat] Add LLaMa Model support for PPO (#375)/2023-03-27

add-hypothesis @0390597/Merge branch 'main' of github.com:CarperAI/trlx into add-hypothesis/2023-03-22 main @b0c4ea9/feat: add requirements.txt from the most recent `pip freeze` (#381)/2023-03-21

add-hypothesis @0390597/Merge branch 'main' of github.com:CarperAI/trlx into add-hypothesis/2023-03-22 main @b0c4ea9/feat: add requirements.txt from the most recent `pip freeze` (#381)/2023-03-21

add-hypothesis @0390597/Merge branch 'main' of github.com:CarperAI/trlx into add-hypothesis/2023-03-22 main @b0c4ea9/feat: add requirements.txt from the most recent `pip freeze` (#381)/2023-03-21

add-benchmark-tools @711fb4d/fix(benchmark): ignore git apply patch failed error/2023-03-22 main @b0c4ea9/feat: add requirements.txt from the most recent `pip freeze` (#381)/2023-03-21

add-benchmark-tools @711fb4d/fix(benchmark): ignore git apply patch failed error/2023-03-22 main @b0c4ea9/feat: add requirements.txt from the most recent `pip freeze` (#381)/2023-03-21

add-benchmark-tools @711fb4d/fix(benchmark): ignore git apply patch failed error/2023-03-22 main @b0c4ea9/feat: add requirements.txt from the most recent `pip freeze` (#381)/2023-03-21

add-benchmark-tools @711fb4d/fix(benchmark): ignore git apply patch failed error/2023-03-22 main @b0c4ea9/feat: add requirements.txt from the most recent `pip freeze` (#381)/2023-03-21

half-exp-fix @4b095df/Merge branch 'main' into half-exp-fix/2023-03-20 main @79bfc6b/fix(docs): Update incorrect `PPORLElement` logprob tensor shape hint (#377)/2023-03-17

half-exp-fix @1a243ae/fixes half exp not implemented error/2023-03-13 main @79bfc6b/fix(docs): Update incorrect `PPORLElement` logprob tensor shape hint (#377)/2023-03-17

fix-total-steps-distributed @c883b0f/fix(ilql_trainer): `total_steps` calculation if running distributed/2023-03-16 main @4623105/[fix] Drop `<eos>` from ILQL sample's phrases (#362)/2023-03-14

fix-total-steps-distributed @c883b0f/fix(ilql_trainer): `total_steps` calculation if running distributed/2023-03-16 main @4623105/[fix] Drop `<eos>` from ILQL sample's phrases (#362)/2023-03-14

ray-train-integration @c7ac679/fix(setup.cfg): condition ray's pinned wheel/2023-03-15 main @4623105/[fix] Drop `<eos>` from ILQL sample's phrases (#362)/2023-03-14

ray-train-integration @a7e7bb4/merge: revert to upstream changes/2023-03-14 main @4623105/[fix] Drop `<eos>` from ILQL sample's phrases (#362)/2023-03-14

infinite_prompt_pipeline @a50cc07/Adding `infinite_dataloader` since previous solution doesn't seem to work as expected/2023-03-07 main @ded2e5e/fix(ppo_trainer): update `AdaptiveKLController` with correct KL (#361)/2023-03-13

half-exp-fix @1a243ae/fixes half exp not implemented error/2023-03-13 main @ded2e5e/fix(ppo_trainer): update `AdaptiveKLController` with correct KL (#361)/2023-03-13

fix-ilql-sample-endings @20eec76/fix(ilql_randomwalks): bump `seq_length` to not trim any samples/2023-03-10 main @ded2e5e/fix(ppo_trainer): update `AdaptiveKLController` with correct KL (#361)/2023-03-13

fix-kl-controller @ee8cd4a/fix(ppo_trainer): update `AdaptiveKLController` with correct KL/2023-03-08 main @adbf8fc/Add intermediate checkpointing to `accelerate` trainers (#349)/2023-03-08

fix-kl-controller @ee8cd4a/fix(ppo_trainer): update `AdaptiveKLController` with correct KL/2023-03-08 main @adbf8fc/Add intermediate checkpointing to `accelerate` trainers (#349)/2023-03-08

fix-ilql-sample-endings @20eec76/fix(ilql_randomwalks): bump `seq_length` to not trim any samples/2023-03-10 main @adbf8fc/Add intermediate checkpointing to `accelerate` trainers (#349)/2023-03-08

fix-ilql-sample-endings @20eec76/fix(ilql_randomwalks): bump `seq_length` to not trim any samples/2023-03-10 main @adbf8fc/Add intermediate checkpointing to `accelerate` trainers (#349)/2023-03-08

fix-kl-controller @ee8cd4a/fix(ppo_trainer): update `AdaptiveKLController` with correct KL/2023-03-08 main @adbf8fc/Add intermediate checkpointing to `accelerate` trainers (#349)/2023-03-08

fix-kl-controller @ee8cd4a/fix(ppo_trainer): update `AdaptiveKLController` with correct KL/2023-03-08 main @adbf8fc/Add intermediate checkpointing to `accelerate` trainers (#349)/2023-03-08

fix-kl-controller @ee8cd4a/fix(ppo_trainer): update `AdaptiveKLController` with correct KL/2023-03-08 main @adbf8fc/Add intermediate checkpointing to `accelerate` trainers (#349)/2023-03-08

fix-kl-controller@ee8cd4a/fix(ppo_trainer): update `AdaptiveKLController` with correct KL/2023-03-08 main@adbf8fc/Add intermediate checkpointing to `accelerate` trainers (#349)/2023-03-08

ee8cd4a/fix(ppo_trainer): update `AdaptiveKLController` with correct KL/2023-03-08 adbf8fc/Add intermediate checkpointing to `accelerate` trainers (#349)/2023-03-08

ee8cd4a/fix(ppo_trainer): update `AdaptiveKLController` with correct KL/2023-03-08 adbf8fc/Add intermediate checkpointing to `accelerate` trainers (#349)/2023-03-08

ee8cd4a/fix(ppo_trainer): update `AdaptiveKLController` with correct KL/2023-03-08 adbf8fc/Add intermediate checkpointing to `accelerate` trainers (#349)/2023-03-08

ee8cd4a/fix(ppo_trainer): update `AdaptiveKLController` with correct KL/2023-03-08 v. adbf8fc/Add intermediate checkpointing to `accelerate` trainers (#349)/2023-03-08

ee8cd4a/fix(ppo_trainer): update `AdaptiveKLController` with correct KL/2023-03-08 v. adbf8fc/Add intermediate checkpointing to `accelerate` trainers (#349)/2023-03-08

ee8cd4a/fix(ppo_trainer): update `AdaptiveKLController` with correct KL/2023-03-08 v. adbf8fc/Add intermediate checkpointing to `accelerate` trainers (#349)/2023-03-08

ee8cd4a/fix(ppo_trainer): update `AdaptiveKLController` with correct KL/2023-03-08 v. adbf8fc/Add intermediate checkpointing to `accelerate` trainers (#349)/2023-03-08

ee8cd4a/fix(ppo_trainer): update `AdaptiveKLController` with correct KL/2023-03-08 v. adbf8fc/Add intermediate checkpointing to `accelerate` trainers (#349)/2023-03-08