lm-eval-Qwen_7B_tmmluplus Table

Ptst1110-ee10's workspace

Runs

task_configs.tmmluplus_accounting.doc_to_text

task_configs.tmmluplus_administrative_law.doc_to_text

Finished

ptst1110-ee10

7mo ago

24m 50s

[]

100000

cuda:1

1234

pretrained=benchang1110/Qwen2.5-Taiwan-7B-Instruct

torch.bfloat16

7615616512

main

80a408cdd0f1062e395ec19b2692519770172e2f

1234

accounting

ZoneTwelve/tmmluplus

["A","B","C","D"]

answer

{{question.strip()}} A. {{choices[0]}} B. {{choices[1]}} C. {{choices[2]}} D. {{choices[3]}} Answer:

first_n

train

[{"aggregation":"mean","higher_is_better":true,"metric":"acc"},{"metric":"acc_norm","aggregation":"mean","higher_is_better":true}]

multiple_choice

def process_docs(dataset: datasets.Dataset) -> datasets.Dataset: def _helper(doc): # modifies the contents of a single # document in our dataset. answer_list = ["A", "B", "C", "D"] out_doc = { "questions": doc["question"], "choices": [doc["A"], doc["B"], doc["C"], doc["D"]], "goal": answer_list.index(doc["answer"]), } return out_doc return dataset.map(_helper) # returns back a datasets.Dataset object

false

tmmluplus_other_tasks

tmmluplus_accounting