Skip to main content

Instruction finetuning

1. Instruction dataset gathered at https://huggingface.co/collections/mesolitica/malaysian-synthetic-dataset-656c2673fe7fe0b1e9e25fe2 2. Notebook to prepare dataset at https://github.com/mesolitica/malaysian-dataset/blob/master/llm-instruction/combine-malay-no-alignment-multitasks-v3.ipynb 3. Prepared data pushed to https://huggingface.co/datasets/malaysia-ai/mosaic-chat-instructions-v3
Created on December 25|Last edited on December 31

02k4k6k8k10k12k14kStep0.60.811.21.41.6
Group MaLLaM 5B
TinyLlama 1.1B
1
MaLLaM 1.1B
1
MaLLaM 5B
1
Mistral 7B
10