BaselineHF:v35
Path
Value
model_name
Qwen/Qwen2.5-0.5B-Instruct-GGUF
device
cpu
llm_model
<llama_cpp.llama.Llama object at 0x7a4344df3af0>
tokenizer
null
use_torch_compile
true
torch_dtype
torch.bfloat16
set_threads_and_interop
false
thread_count
6
max_new_tokens
1
use_llamacpp
true
predict