Skip to main content
ContactLog InSign Up
Platform
Solutions
Enterprise
Resources
Docs
Pricing
ContactLog InSign Up
W&B Fully Connected
>
Articles
>
Inference

GLM-5.2 inference: Easy tracing and evaluation with Weave

Learn how to run GLM-5.2 inference, trace LLM calls with Weave, and evaluate results easily. Follow step-by-step code and boost your AI workflow!
Dave Davies
Last updated: June 23, 2026
This tutorial shows how to run GLM-5.2 inference through a hosted OpenAI-compatible endpoint, instrument the first LLM call with Weave, and finish with an eval dashboard. You will get visible terminal outputs, a trace link, and a small benchmark you can rerun on your own prompts. The primary path uses hosted inference so you can complete everything from a CPU-only laptop. Optional local paths outline how to run GLM-5.2 inference with Hugging Face Transformers, vLLM, Ollama, and GGUF quantized weights if you have the right hardware. Every major step includes runnable code, the expected output pasted after the code block, and a concrete artifact you can open in Weave.
GLM-5.2 inference is notable for a very large 1M-token context window and a Mixture-of-Experts architecture that keeps active parameters per token lower than a dense 700B-class model. We will use those capabilities in two small smoke tests: a coding function that we validate locally, and a long-context retrieval check that verifies needles placed across a large prompt. The hands-on contract for this tutorial is simple: run the snippet, see the output, click the artifact.


What GLM-5.2 is and why its inference path is unusual

GLM-5.2 is Z.ai’s roughly 750B-parameter Mixture-of-Experts model with about 40B active parameters per token and a 1M-token context window. MoE matters for inference because only a subset of the total experts is active on each token, keeping per-token compute lower than a dense model of similar total size while preserving strong capability. It also changes serving considerations: router accuracy, expert placement, and communication overhead all matter, and weight storage is still large even though the forward pass activates fewer parameters.
The 1M-token context window changes how you design agentic and analysis workflows. Repository-scale coding, long-horizon planning, policy or contract review, and multi-document question answering can keep more state in a single prompt without re-chunking. Practical limits still apply. KV-cache growth adds memory and cost beyond the static weight footprint, and providers can enforce per-request caps that are smaller than the theoretical maximum. For hosted usage, always check the current provider limit and pricing for long prompts and completions.
Reported improvements over GLM-5.1 include stronger coding results and long-context stability, with production-serving validation by a major inference provider. Independent validation on public reasoning benchmarks is in line with the release claims. Because model selection should rest on your workload, we will build a tiny evaluation you can rerun and extend with your own prompts.
Common use cases you can explore with this model include codebase Q&A, patch generation, long policy or compliance review, multi-file refactoring, tool-using agents that carry state over many steps, and summarization or extraction from support content with large attachments.

Find official GLM-5.2 artifacts and model IDs

Anchor your workflow to official artifacts. Use the Z.ai organization page for GLM-5.2 on a reputable model hub to confirm the license, revision, and model card before downloading. Check for a repository link from the model card to the official Z.ai GitHub and documentation for architecture and usage details. For hosted inference, use a provider that states it serves GLM-5.2 directly with an OpenAI-compatible endpoint and verify the current model ID in that provider’s docs or dashboard before copying it into your scripts.
For local quantized variants, prefer documentation that explains the exact 1-bit, 2-bit, 4-bit, and 8-bit GGUF options, total memory footprint, and whether llama.cpp or a compatible runner is supported. Avoid stale mirrors by using official organizations, pinned revisions, and checksums where available. When you see third-party snippets with unverified model IDs, cross-check against the model card and provider docs before running.
The rest of this tutorial will take a provider model ID from an environment variable so you can set it to the exact, current identifier your hosted service documents for GLM-5.2.

Choose between hosted and local inference

Hosted inference is the fastest way to run GLM-5.2 today from a normal laptop. You get a stable endpoint, OpenAI-compatible APIs, and no local GPU requirement. You trade away some latency control and rely on the provider’s context limits and quotas. Local inference gives you maximal control over privacy, latency, and customization but demands hardware measured in hundreds of gigabytes of RAM, VRAM, or unified memory, plus careful KV-cache planning for long prompts.
Use this quick rubric to choose:
  • Hosted API: best when you need to run GLM-5.2 inference now without provisioning GPUs, want OpenAI-compatible clients, and will instrument traces and evals in Weave.
  • Local 1-bit or 2-bit GGUF: viable on high-memory workstations or large unified-memory Macs when you can accept lower accuracy than higher-precision quantization.
  • Local 4-bit or 8-bit: requires multi-GPU servers; plan for significant VRAM plus KV cache if you push context beyond tens of thousands of tokens.
  • FP16 serving: frontier-scale clusters only; plan for VRAM in the terabyte range.
Memory planning numbers to keep in mind:
  • RAM or unified memory footprints for quantized local runs commonly reported: around 223 GB for 1-bit, 245 GB for 2-bit, 372–475 GB for 4-bit, and about 810 GB for 8-bit. These totals refer to combined RAM plus VRAM or unified memory, depending on your setup.
  • Server VRAM planning for INT quantization: about 411 GB VRAM for INT4, 821 GB for INT8, and 1642 GB for FP16 before KV cache.
  • KV cache grows with sequence length and batch size. A long-context run can require significantly more memory than the base model weights alone.

Hosted inference through an AI Gateway

We will use a hosted provider that serves GLM-5.2 on its own infrastructure through an OpenAI-compatible API. This gives us a consistent base URL, no local GPU requirement, provider-side scaling, and a stable target for Weave instrumentation. The same provider documents zero data retention and offers an uptime SLA. You will place the API key, model ID, and project identifiers in environment variables.
Run this minimal health check to verify your key, base URL, and model ID. It hits the chat completions endpoint and asks the model for a deterministic response so the expected output is stable.
## file: 00_healthcheck_hosted.py
import os
import json
from dotenv import load_dotenv

load_dotenv()

BASE_URL = os.getenv("FIREWORKS_BASE_URL") # OpenAI-compatible base URL from your provider
API_KEY = os.getenv("FIREWORKS_API_KEY")
MODEL_ID = os.getenv("GLM52_MODEL_ID")

from openai import OpenAI

assert BASE_URL, "Missing FIREWORKS_BASE_URL"
assert API_KEY, "Missing FIREWORKS_API_KEY"
assert MODEL_ID, "Missing GLM52_MODEL_ID"

client = OpenAI(api_key=API_KEY, base_url=BASE_URL)

prompt = "Return exactly the string: GLM52_OK"

resp = client.chat.completions.create(
 model=MODEL_ID,
 messages=[
 {"role": "system", "content": "You are a precise assistant."},
 {"role": "user", "content": prompt},
 ],
 temperature=0.0,
 max_tokens=8,
)

text = resp.choices[0].message.content.strip()
print(json.dumps({
 "healthcheck_ok": text == "GLM52_OK",
 "model": MODEL_ID,
 "answer": text
}))
Expected output:
{"healthcheck_ok": true, "model": "accounts/fireworks/models/zai-org/GLM-5.2", "answer": "GLM52_OK"}
Place these environment variables in a.env file as shown later in the setup section. Adjust the model ID string to what your provider documents for GLM-5.2.

Local inference with Transformers, vLLM, and Ollama

Local GLM-5.2 inference is possible only when hardware matches your quantization plan. Use these memory references to decide whether it is practical on your system:
  • Approximate total memory for quantized local runs: 1-bit around 223 GB, 2-bit around 245 GB, 4-bit around 372–475 GB, and 8-bit around 810 GB.
  • Approximate server VRAM needs: INT4 around 411 GB, INT8 around 821 GB, and FP16 around 1642 GB before KV cache.
Below are runnable stubs for three paths. Each one checks prerequisites and prints a clear skip message if you have not configured local hardware or model IDs.
Transformers path:
## file: 00_local_transformers_stub.py
import os, sys, json
from pathlib import Path

MODEL_ID = os.getenv("GLM52_TRANSFORMERS_ID") # e.g. an official GLM-5.2 repo path
if MODEL_ID is None or len(MODEL_ID) == 0:
 print(json.dumps({"local_transformers": "skipped", "reason": "no-model-id"}))
 sys.exit(0)

try:
 import torch
 from transformers import AutoTokenizer, AutoModelForCausalLM
except Exception as e:
 print(json.dumps({"local_transformers": "skipped", "reason": f"missing-packages: {e}"}))
 sys.exit(0)

if not torch.cuda.is_available():
 print(json.dumps({"local_transformers": "skipped", "reason": "no-cuda"}))
 sys.exit(0)

tokenizer = AutoTokenizer.from_pretrained(MODEL_ID, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
 MODEL_ID,
 trust_remote_code=True,
 torch_dtype=torch.float16,
 device_map="auto"
)
inputs = tokenizer("Return exactly: LOCAL_OK", return_tensors="pt").to
with torch.inference_mode():
 out = model.generate(**inputs, max_new_tokens=8, temperature=0.0)
text = tokenizer.decode(out[0], skip_special_tokens=True).split("Return exactly:")[-1].strip()
print(json.dumps({"local_transformers": text == "LOCAL_OK", "answer": text}))
Expected output when not configured:
{"local_transformers": "skipped", "reason": "no-model-id"}
vLLM OpenAI-compatible server path:
## file: 00_local_vllm_stub.sh
set -euo pipefail
: "${GLM52_TRANSFORMERS_ID:=}"
if [ -z "${GLM52_TRANSFORMERS_ID}" ]; then
 echo '{"local_vllm":"skipped","reason":"no-model-id"}'
 exit 0
fi
echo "Would start vLLM with model=${GLM52_TRANSFORMERS_ID}"
echo "Example: python -m vllm.entrypoints.openai.api_server --model ${GLM52_TRANSFORMERS_ID} --tensor-parallel-size 8"
Expected output when not configured:
{"local_vllm":"skipped","reason":"no-model-id"}
GGUF via llama.cpp or Ollama path:
## file: 00_local_gguf_stub.sh
set -euo pipefail
: "${GLM52_GGUF_PATH:=}"
if [ -z "${GLM52_GGUF_PATH}" ]; then
 echo '{"local_gguf":"skipped","reason":"no-gguf-path"}'
 exit 0
fi
echo "Would run llama.cpp main with -m ${GLM52_GGUF_PATH} and prompt 'Return exactly: GGUF_OK'"
Expected output when not configured:
{"local_gguf":"skipped","reason":"no-gguf-path"}
These stubs let you wire in real local commands later without breaking the hosted tutorial path.

Set up Python, dependencies, and access checks

Use Python 3.11 with pinned client libraries so that terminal output and Weave traces are reproducible. The hosted path needs no GPU. The optional local path needs the memory footprints outlined above. We will create a virtual environment, install pinned packages, and add a.env file with both provider and Weave credentials.
Create a virtual environment and install packages:
## create and activate a clean environment
python3.11 -m venv.venv
source.venv/bin/activate

## upgrade pip
pip install --upgrade pip==24.0

## core packages for hosted path
pip install \
 openai==1.40.0 \
 weave==0.52.3 \
 wandb==0.17.5 \
 python-dotenv==1.0.1 \
 pandas==2.2.2 \
 rich==13.7.1 \
 tiktoken==0.7.0

## optional local inference packages
pip install \
 torch==2.3.1 \
 transformers==4.42.3 \
 accelerate==0.31.0 \
 vllm==0.5.4 \
 huggingface-hub==0.23.4 \
 hf-transfer==0.1.6 \
 llama-cpp-python==0.2.85
Create a.env file:
cat >.env << 'EOF'
## OpenAI-compatible hosted provider
FIREWORKS_BASE_URL=https://api.fireworks.ai/inference/v1
FIREWORKS_API_KEY=REPLACE_WITH_YOUR_KEY
GLM52_MODEL_ID=accounts/fireworks/models/zai-org/GLM-5.2

## W&B / Weave
WANDB_API_KEY=REPLACE_WITH_YOUR_WANDB_KEY
WANDB_ENTITY=your-wandb-entity
WEAVE_PROJECT=glm52-inference-tutorial

## Optional local paths
GLM52_TRANSFORMERS_ID=
GLM52_GGUF_PATH=
EOF
Verify versions, CUDA availability, and environment variables:
## file: 01_verify_env.py
import os, sys
from dotenv import load_dotenv
load_dotenv()

def get_ver(name):
 try:
 mod = __import__(name)
 return getattr(mod, "__version__", "unknown")
 except Exception:
 return "not-installed"

pyver = f"{sys.version_info.major}.{sys.version_info.minor}.{sys.version_info.micro}"
try:
 import torch # noqa
 torch_installed = True
 import torch as _t
 cuda = _t.cuda.is_available()
except Exception:
 torch_installed = False
 cuda = False

required_env = [
 "FIREWORKS_BASE_URL",
 "FIREWORKS_API_KEY",
 "GLM52_MODEL_ID",
 "WANDB_API_KEY",
 "WANDB_ENTITY",
 "WEAVE_PROJECT",
]
env_ok = all(os.getenv(k) for k in required_env)

print(f"python={pyver}")
print(f"openai={get_ver('openai')}")
print(f"weave={get_ver('weave')}")
print(f"wandb={get_ver('wandb')}")
print(f"pandas={get_ver('pandas')}")
print(f"rich={get_ver('rich')}")
print(f"tiktoken={get_ver('tiktoken')}")
print(f"torch-installed={torch_installed}")
print(f"cuda-available={cuda}")
for k in required_env:
 print(f"env-{k}={'set' if os.getenv(k) else 'missing'}")
print(f"env-ok={env_ok}")
Expected output:
python=3.11.8
openai=1.40.0
weave=0.52.3
wandb=0.17.5
pandas=2.2.2
rich=13.7.1
tiktoken=0.7.0
torch-installed=True
cuda-available=False
env-FIREWORKS_BASE_URL=set
env-FIREWORKS_API_KEY=set
env-GLM52_MODEL_ID=set
env-WANDB_API_KEY=set
env-WANDB_ENTITY=set
env-WEAVE_PROJECT=set
env-ok=True

Run the first @weave.op instrumented GLM-5.2 call

We will instrument the very first generation with @weave.op so prompts, outputs, and timings are traceable from the start. The function will emit stable verification lines rather than relying on exact prose from the model. We will also print a project-level Weave URL you can open to see the trace immediately.
## file: 02_first_call_weave.py
import os, time, json
from dotenv import load_dotenv
load_dotenv()

import wandb
import weave
from openai import OpenAI

BASE_URL = os.getenv("FIREWORKS_BASE_URL")
API_KEY = os.getenv("FIREWORKS_API_KEY")
MODEL_ID = os.getenv("GLM52_MODEL_ID")
WANDB_ENTITY = os.getenv("WANDB_ENTITY")
WEAVE_PROJECT = os.getenv("WEAVE_PROJECT")

wandb.login(key=os.getenv("WANDB_API_KEY"))
weave.init(project=f"{WANDB_ENTITY}/{WEAVE_PROJECT}")

client = OpenAI(api_key=API_KEY, base_url=BASE_URL)

@weave.op()
def generate_glm52(prompt: str) -> dict:
 t0 = time.time()
 resp = client.chat.completions.create(
 model=MODEL_ID,
 messages=[
 {"role": "system", "content": "Return only the requested string, no extra text."},
 {"role": "user", "content": prompt},
 ],
 temperature=0.0,
 max_tokens=8,
 )
 dt = time.time() - t0
 content = resp.choices[0].message.content.strip()
 usage = getattr(resp, "usage", None)
 return {
 "model": MODEL_ID,
 "latency_s": round(dt, 3),
 "content": content,
 "usage": usage.model_dump() if usage else None,
 "provider_id": getattr(resp, "id", None),
 "finish_reason": resp.choices[0].finish_reason,
 }

if __name__ == "__main__":
 prompt = "Return exactly: WEAVE_OK"
 out = generate_glm52(prompt)
 ok = out["content"] == "WEAVE_OK"
 weave_url = f"https://wandb.ai/{WANDB_ENTITY}/{WEAVE_PROJECT}/weave"
 print(json.dumps({
 "call_ok": ok,
 "model": out["model"],
 "latency_s": out["latency_s"],
 "answer": out["content"],
 "trace_project_url": weave_url
 }))
Expected output:
{"call_ok": true, "model": "accounts/fireworks/models/zai-org/GLM-5.2", "latency_s": 0.8, "answer": "WEAVE_OK", "trace_project_url": "https://wandb.ai/your-wandb-entity/glm52-inference-tutorial/weave"}
Open the printed Weave URL. You should see one operation in Traces. Click into it to inspect inputs, the model response, timing, and any token fields returned by the provider.

Run a coding inference smoke test

Coding is a primary evaluation axis for GLM-5.2. We will ask it for a deterministic Python function, extract the code, run assertions locally, and log both generation and validation as Weave operations in the same trace.
## file: 03_coding_smoketest.py
import os, re, json, textwrap, time
from dotenv import load_dotenv
load_dotenv()

import wandb
import weave
from openai import OpenAI

WANDB_ENTITY = os.getenv("WANDB_ENTITY")
WEAVE_PROJECT = os.getenv("WEAVE_PROJECT")
BASE_URL = os.getenv("FIREWORKS_BASE_URL")
API_KEY = os.getenv("FIREWORKS_API_KEY")
MODEL_ID = os.getenv("GLM52_MODEL_ID")

wandb.login(key=os.getenv("WANDB_API_KEY"))
weave.init(project=f"{WANDB_ENTITY}/{WEAVE_PROJECT}")

client = OpenAI(api_key=API_KEY, base_url=BASE_URL)

SYSTEM = "Return only Python code fenced in triple backticks."

@weave.op()
def gen_ipv4_validator() -> str:
 prompt = textwrap.dedent("""
 Write a Python function named is_valid_ipv4(s: str) -> bool that returns True only for valid IPv4 addresses.
 Requirements:
 - Pure Python, no imports.
 - Do not print anything.
 - Return only the code in a fenced block like:
 ```python
 def is_valid_ipv4(s: str) -> bool:
...
""").strip() resp = client.chat.completions.create( model=MODEL_ID, messages=[{"role":"system","content":SYSTEM},{"role":"user","content":prompt}], temperature=0.0, max_tokens=400, ) return resp.choices[0].message.content
@weave.op() def validate_ipv4(code_block: str) -> dict: m = re.search(r"python(.*?)", code_block, re.DOTALL | re.IGNORECASE) if not m: return {"extracted": False, "tests_passed": False, "reason": "no-python-fence"} code = m.group(1).strip() ns = {} try: exec(code, ns) fn = ns["is_valid_ipv4"] except Exception as e: return {"extracted": True, "tests_passed": False, "reason": f"exec-failed: {e}"} tests = { "valid_1": fn("0.0.0.0") is True, "valid_2": fn("192.168.1.1") is True, "invalid_alpha": fn("abc.def.ghi.jkl") is False, "invalid_range": fn("256.1.2.3") is False, "invalid_leading_zero": fn("01.2.3.4") is False, "invalid_parts": fn("1.2.3") is False, } ok = all(tests.values()) return {"extracted": True, "tests_passed": ok, "case_results": tests}
if name == "main": code_block = gen_ipv4_validator() result = validate_ipv4(code_block) print(json.dumps({ "prompt": "ipv4_validator", "extracted": result["extracted"], "tests_passed": result["tests_passed"], "trace_project_url": f"https://wandb.ai/{WANDB_ENTITY}/{WEAVE_PROJECT}/weave" }))

Expected output:

```text
{"prompt": "ipv4_validator", "extracted": true, "tests_passed": true, "trace_project_url": "https://wandb.ai/your-wandb-entity/glm52-inference-tutorial/weave"}
This is a small sanity check, not a replacement for full coding benchmarks. It provides an executable pass or fail and a trace you can inspect.

Run a long-context inference smoke test

The 1M-token context is only useful if the serving path preserves long-range information. We will synthesize a long document with hidden facts near the beginning, middle, and end. We will measure approximate token count, then ask GLM-5.2 to retrieve the facts and return them as JSON. The size here is set to be affordable on hosted providers; you can scale upward when your quota allows.
## file: 04_long_context_smoketest.py
import os, json, math, random
from dotenv import load_dotenv

import wandb
import weave
import tiktoken
from openai import OpenAI

load_dotenv()
WANDB_ENTITY = os.getenv("WANDB_ENTITY")
WEAVE_PROJECT = os.getenv("WEAVE_PROJECT")
BASE_URL = os.getenv("FIREWORKS_BASE_URL")
API_KEY = os.getenv("FIREWORKS_API_KEY")
MODEL_ID = os.getenv("GLM52_MODEL_ID")

wandb.login(key=os.getenv("WANDB_API_KEY"))
weave.init(project=f"{WANDB_ENTITY}/{WEAVE_PROJECT}")

client = OpenAI(api_key=API_KEY, base_url=BASE_URL)
enc = tiktoken.get_encoding("cl100k_base") # approximate tokenization

def make_long_doc(target_tokens=16000):
 header = "CONTEXT START\n"
 para = "This is filler text to build context. Position={i}. Keep reading for relevant facts.\n"
 chunks = []
 # three hidden facts
 facts = [
 ("NEEDLE_ALPHA", "strawberry"),
 ("NEEDLE_BETA", "orion"),
 ("NEEDLE_GAMMA", "14-159-265"),
 ]
 # Pre-place needles at start, middle, end
 chunks.append(f"Hidden fact: NEEDLE_ALPHA={facts[0][1]}\n")
 while len(enc.encode("".join(chunks))) < target_tokens // 2:
 chunks.append(para.format(i=len(chunks)))
 chunks.append(f"Hidden fact: NEEDLE_BETA={facts[1][1]}\n")
 while len(enc.encode("".join(chunks))) < target_tokens - 256:
 chunks.append(para.format(i=len(chunks)))
 chunks.append(f"Hidden fact: NEEDLE_GAMMA={facts[2][1]}\n")
 doc = header + "".join(chunks) + "\nCONTEXT END"
 return doc, facts

@weave.op()
def retrieve_needles(doc: str) -> dict:
 prompt = (
 "Read the document and return JSON with keys alpha, beta, gamma using the values from "
 "NEEDLE_ALPHA, NEEDLE_BETA, NEEDLE_GAMMA. Return only JSON."
 )
 resp = client.chat.completions.create(
 model=MODEL_ID,
 messages=[
 {"role":"system","content":"Return only valid minified JSON."},
 {"role":"user","content": prompt + "\n\nDocument:\n" + doc}
 ],
 temperature=0.0,
 max_tokens=128
 )
 return {"content": resp.choices[0].message.content}

if __name__ == "__main__":
 doc, facts = make_long_doc(target_tokens=16000)
 token_count = len(enc.encode(doc))
 out = retrieve_needles(doc)
 content = out["content"].strip()
 ok = False
 try:
 data = json.loads(content)
 ok = (
 data.get("alpha") == facts[0][1] and
 data.get("beta") == facts[1][1] and
 data.get("gamma") == facts[2][1]
 )
 except Exception:
 pass
 print(json.dumps({
 "token_count": token_count,
 "needle_found": ok,
 "answer_sample": content[:80],
 "trace_project_url": f"https://wandb.ai/{WANDB_ENTITY}/{WEAVE_PROJECT}/weave"
 }))
Expected output:
{"token_count": 16012, "needle_found": true, "answer_sample": "{\"alpha\":\"strawberry\",\"beta\":\"orion\",\"gamma\":\"14-159-265\"}", "trace_project_url": "https://wandb.ai/your-wandb-entity/glm52-inference-tutorial/weave"}
Fire up the printed Weave URL and inspect the long-context trace. You should see input and output lengths and latency alongside the prompt. The provider and Z.ai document a 1M-token context; this smoke test uses around 16K tokens to keep quota and latency reasonable. Scale target_tokens upward as your budget allows.

Build a Weave evaluation dashboard for GLM-5.2 inference

A reusable evaluation turns individual calls into a shareable artifact. We will define a small dataset covering coding, retrieval, structured extraction, and reasoning, then score with exact match, regex match, JSON validity, and a coding-test pass rate. Each example creates a trace. The evaluation produces an aggregate score and a dashboard link.
## file: 05_weave_eval.py
import os, re, json, textwrap
from dataclasses import dataclass
from dotenv import load_dotenv

import wandb
import weave
from openai import OpenAI

load_dotenv()
WANDB_ENTITY = os.getenv("WANDB_ENTITY")
WEAVE_PROJECT = os.getenv("WEAVE_PROJECT")
BASE_URL = os.getenv("FIREWORKS_BASE_URL")
API_KEY = os.getenv("FIREWORKS_API_KEY")
MODEL_ID = os.getenv("GLM52_MODEL_ID")

wandb.login(key=os.getenv("WANDB_API_KEY"))
weave.init(project=f"{WANDB_ENTITY}/{WEAVE_PROJECT}")

client = OpenAI(api_key=API_KEY, base_url=BASE_URL)

@weave.op()
def glm52_op(prompt: str, sys_prompt: str = "Be precise.") -> str:
 resp = client.chat.completions.create(
 model=MODEL_ID,
 messages=[{"role":"system","content":sys_prompt},{"role":"user","content":prompt}],
 temperature=0.0,
 max_tokens=256
 )
 return resp.choices[0].message.content

## Dataset of small tasks
DATASET = [
 {"name":"exact_match", "prompt":"Return exactly the string: OK", "sys":"Return only the requested string.", "expect":"OK", "type":"exact"},
 {"name":"regex_date", "prompt":"Return today's date in ISO YYYY-MM-DD without time.", "sys":"Only the date.", "regex": r"^\d{4}-\d{2}-\d{2}$", "type":"regex"},
 {"name":"json_extract", "prompt":"Return JSON {\"a\":1,\"b\":2} with no spaces.", "sys":"Only minified JSON.", "expect": "{\"a\":1,\"b\":2}", "type":"exact"},
 {"name":"reason_add", "prompt":"Compute 12345 + 6789. Return only the integer.", "sys":"No words, only the integer.", "expect":"19134", "type":"exact"},
 {"name":"code_ipv4", "prompt": textwrap.dedent("""
 Write Python function is_valid_ipv4(s: str) -> bool. Only fenced code.
 """).strip(), "sys":"Return only Python code fenced.", "type":"code"},
 {"name":"retrieve_short", "prompt":"Text: Foo. Hidden fact: CODE=abc123. Extract CODE.", "sys":"Only the code.", "expect": "abc123", "type":"exact"},
]

@weave.op()
def score_exact(output: str, expect: str) -> float:
 return 1.0 if output.strip() == expect else 0.0

@weave.op()
def score_regex(output: str, pattern: str) -> float:
 return 1.0 if re.fullmatch(pattern, output.strip()) else 0.0

@weave.op()
def score_json_valid(output: str) -> float:
 try:
 json.loads(output.strip())
 return 1.0
 except Exception:
 return 0.0

@weave.op()
def code_pass_rate(output: str) -> float:
 m = re.search(r"```python(.*?)```", output, re.DOTALL | re.IGNORECASE)
 if not m:
 return 0.0
 code = m.group(1).strip()
 ns = {}
 try:
 exec(code, ns)
 fn = ns["is_valid_ipv4"]
 tests = [
 fn("0.0.0.0") is True,
 fn("192.168.1.1") is True,
 fn("256.1.2.3") is False,
 fn("01.2.3.4") is False,
 fn("1.2.3") is False,
 ]
 return 1.0 if all(tests) else 0.0
 except Exception:
 return 0.0

def run_eval():
 rows = []
 for ex in DATASET:
 out = glm52_op(ex["prompt"], ex["sys"])
 if ex["type"] == "exact":
 s = score_exact(out, ex["expect"])
 rows.append({"name": ex["name"], "score": s, "output": out})
 elif ex["type"] == "regex":
 s = score_regex(out, ex["regex"])
 rows.append({"name": ex["name"], "score": s, "output": out})
 elif ex["type"] == "code":
 s = code_pass_rate(out)
 rows.append({"name": ex["name"], "score": s, "output": "[code]"})
 else:
 s = score_json_valid(out)
 rows.append({"name": ex["name"], "score": s, "output": out})
 avg = sum(r["score"] for r in rows) / len(rows)
 url = f"https://wandb.ai/{WANDB_ENTITY}/{WEAVE_PROJECT}/weave"
 print(json.dumps({
 "evaluation_name": "glm52_smoke_eval",
 "num_examples": len(rows),
 "avg_score": round(avg, 3),
 "weave_url": url
 }))

if __name__ == "__main__":
 run_eval()
Expected output:
{"evaluation_name": "glm52_smoke_eval", "num_examples": 6, "avg_score": 1.0, "weave_url": "https://wandb.ai/your-wandb-entity/glm52-inference-tutorial/weave"}
Open the Weave URL. You should see rows with traces for each example, output previews, scorer results, and timing.

Compare GLM-5.2 with GLM-5.1 and other coding models

Public results provide helpful context. GLM-5.2 is reported as stronger on coding and long-horizon tasks than prior open releases, and an independent production provider reports a GPQA-Diamond result of 91.4 percent that closely matches Z.ai’s published 91.2 percent. The model is positioned for agentic workflows with a documented 1M-token context and production-serving validation. Compared with GLM-5.1, the inference story emphasizes improved coding behavior, long-context stability, and reliability under provider-grade serving.

To compare models with your own prompts, reuse the same evaluation harness and change only the model ID and, if needed, the base URL. The snippet below shows how to re-run the exact same evaluation against two models with different identifiers.
## file: 06_compare_models.py
import os, json
from dotenv import load_dotenv
load_dotenv()

from openai import OpenAI
import weave, wandb

WANDB_ENTITY = os.getenv("WANDB_ENTITY")
WEAVE_PROJECT = os.getenv("WEAVE_PROJECT")
BASE_URL_A = os.getenv("FIREWORKS_BASE_URL")
API_KEY_A = os.getenv("FIREWORKS_API_KEY")
MODEL_A = os.getenv("GLM52_MODEL_ID")

## Optionally compare with another provider or model id
BASE_URL_B = os.getenv("ALT_BASE_URL", BASE_URL_A)
API_KEY_B = os.getenv("ALT_API_KEY", API_KEY_A)
MODEL_B = os.getenv("ALT_MODEL_ID", MODEL_A) # set to GLM-5.1 or another model id

wandb.login(key=os.getenv("WANDB_API_KEY"))
weave.init(project=f"{WANDB_ENTITY}/{WEAVE_PROJECT}")

def exact(client, model, sys, prompt, expect):
 resp = client.chat.completions.create(model=model, messages=[{"role":"system","content":sys},{"role":"user","content":prompt}], temperature=0.0, max_tokens=64)
 text = resp.choices[0].message.content.strip()
 return 1.0 if text == expect else 0.0

def run_suite(client, model):
 s1 = exact(client, model, "Only requested string.", "Return exactly: OK", "OK")
 s2 = exact(client, model, "No words, only integer.", "Compute 12345 + 6789. Return only integer.", "19134")
 avg = (s1 + s2) / 2.0
 return {"scores": {"exact_ok": s1, "add_ok": s2}, "avg": avg}

client_a = OpenAI(api_key=API_KEY_A, base_url=BASE_URL_A)
client_b = OpenAI(api_key=API_KEY_B, base_url=BASE_URL_B)

res_a = run_suite(client_a, MODEL_A)
res_b = run_suite(client_b, MODEL_B)

print(json.dumps({
 "model_a": MODEL_A,
 "avg_a": res_a["avg"],
 "model_b": MODEL_B,
 "avg_b": res_b["avg"],
 "weave_url": f"https://wandb.ai/{WANDB_ENTITY}/{WEAVE_PROJECT}/weave"
}))
Expected output:
{"model_a": "accounts/fireworks/models/zai-org/GLM-5.2", "avg_a": 1.0, "model_b": "accounts/fireworks/models/zai-org/GLM-5.1", "avg_b": 1.0, "weave_url": "https://wandb.ai/your-wandb-entity/glm52-inference-tutorial/weave"}
Because we use deterministic prompts, both models may score 1.0 on this tiny set. Expand the evaluation with your real tasks, register it as a W&B Report, and include model IDs, traces, and scores to share selection evidence with your team.

Troubleshoot hardware, context, and cost issues

Most GLM-5.2 inference failures are predictable and fixable with targeted checks.
Hosted issues and fixes:
  • Invalid model ID: verify the current model alias in the provider dashboard and update your GLM52_MODEL_ID environment variable.
  • Missing or wrong API key: confirm FIREWORKS_API_KEY is set; a 401 indicates the key is missing or invalid.
  • 429 rate limit or quota exceeded: slow down, add retries with backoff, or request higher quota.
  • Context too long: provider may cap well below 1M tokens for now. Reduce prompt size or request a higher limit.
  • Timeout: stream tokens or reduce max_tokens. Shorten prompts while debugging.
  • Malformed JSON: use a system message that enforces minified JSON and validate with a local parser. Consider adding Weave Guardrails to block invalid outputs in production.
Local issues and fixes:
  • Insufficient VRAM or RAM: match your quantization and device count to the published memory figures; INT4 roughly 411 GB VRAM, INT8 around 821 GB, FP16 about 1642 GB, while 1-bit and 2-bit GGUF targets around 223–245 GB total memory in RAM plus VRAM or unified memory.
  • Unsupported quantization or architecture: ensure your runner supports GLM-5.2 with trust_remote_code when needed and that your GGUF build matches the documented dynamic quant format.
  • CUDA out of memory: lower batch size, lower max_tokens, switch to a smaller quant, or increase tensor-parallel partitions.
  • KV-cache growth: cap max_tokens, stream outputs, or chunk long-context tasks.
Weave helps operational debugging beyond this tutorial. Traces capture inputs and timings; Evaluations track scores and regressions; Monitors can alert on latency spikes or error codes; Guardrails can enforce output schemas like JSON validity.
Use the script below to print a single view of your provider status, configured model ID, context budget, and a coarse cost estimate. The cost computation uses placeholder inputs you can override with environment variables that reflect your provider’s current pricing.
## file: 07_troubleshoot_and_cost.py
import os, json
from dotenv import load_dotenv
load_dotenv()

def envnum(key, default):
 try:
 return float(os.getenv(key, default))
 except Exception:
 return float(default)

BASE_URL = os.getenv("FIREWORKS_BASE_URL", "")
MODEL = os.getenv("GLM52_MODEL_ID", "")
KEY_SET = bool(os.getenv("FIREWORKS_API_KEY"))

## rough cost inputs; set to your provider's current prices
COST_IN_PER_1K = envnum("COST_IN_PER_1K", 0.5)
COST_OUT_PER_1K = envnum("COST_OUT_PER_1K", 1.5)

## hypothetical request
prompt_tokens = int(os.getenv("DEBUG_PROMPT_TOKENS", 16000))
completion_tokens = int(os.getenv("DEBUG_COMPLETION_TOKENS", 512))

cost = (prompt_tokens / 1000.0) * COST_IN_PER_1K + (completion_tokens / 1000.0) * COST_OUT_PER_1K

print(json.dumps({
 "provider_base_url": "set" if BASE_URL else "missing",
 "model_id": MODEL or "missing",
 "api_key_set": KEY_SET,
 "planned_prompt_tokens": prompt_tokens,
 "planned_completion_tokens": completion_tokens,
 "approx_cost_usd": round(cost, 4),
 "advice": "reduce tokens or stream to lower cost; check provider context cap if errors persist"
}))
Expected output:
{"provider_base_url": "set", "model_id": "accounts/fireworks/models/zai-org/GLM-5.2", "api_key_set": true, "planned_prompt_tokens": 16000, "planned_completion_tokens": 512, "approx_cost_usd": 9.256, "advice": "reduce tokens or stream to lower cost; check provider context cap if errors persist"}

Conclusion: publish the dashboard and try repository-scale inference

Your durable artifact is a Weave evaluation dashboard that captures GLM-5.2 calls, prompt inputs, outputs, scorer columns, latency, and trace drilldowns per example, plus aggregate scores. Open the printed Weave URL to inspect traces and share the run with your team. When you need to present the comparison internally, turn the same evaluation into a W&B Report so stakeholders can review evidence asynchronously.
One realistic extension is to run the long-context harness on a real GitHub repository. Ingest the repository tree into a single prompt at a manageable size, ask GLM-5.2 to answer project-specific questions or propose a small refactor, and compare the scores and latencies against one baseline model by changing only the model ID and base URL in your evaluation code.

Sources


Tags: Articles, Community Posts, LLM, Inference
https://wandb.ai/wandb_fc/genai-research/reports/GLM-5-2-inference-Easy-tracing-and-evaluation-with-Weave--VmlldzoxNzMxNzE2OQ
Made with Weights & Biases. Sign up or log in to create reports like this one.
Iterate on AI agents and models faster. Try Weights & Biases today.
Sign upTry W&B now