lm_eval_Llama_3B_tmmluplus Table

Ptst1110-ee10's workspace

Runs

task_configs.tmmluplus_accounting.doc_to_text

task_configs.tmmluplus_administrative_law.doc_to_text

Finished

ptst1110-ee10

7mo ago

20m 24s

[]

100000

cuda:1

1234

pretrained=lianghsun/Llama-3.2-Taiwan-3B-Instruct

torch.bfloat16

3212749824

main

9a5dedcac9431af0e490b59d432fefe155a82e51

1234

accounting

ZoneTwelve/tmmluplus

["A","B","C","D"]

answer

{{question.strip()}} A. {{choices[0]}} B. {{choices[1]}} C. {{choices[2]}} D. {{choices[3]}} Answer:

first_n

train

[{"metric":"acc","aggregation":"mean","higher_is_better":true},{"metric":"acc_norm","aggregation":"mean","higher_is_better":true}]

multiple_choice

def process_docs(dataset: datasets.Dataset) -> datasets.Dataset: def _helper(doc): # modifies the contents of a single # document in our dataset. answer_list = ["A", "B", "C", "D"] out_doc = { "questions": doc["question"], "choices": [doc["A"], doc["B"], doc["C"], doc["D"]], "goal": answer_list.index(doc["answer"]), } return out_doc return dataset.map(_helper) # returns back a datasets.Dataset object

false

tmmluplus_other_tasks

tmmluplus_accounting