inference

Inference

Meta Llama 3.1 70B

Text

Model overview

Price

$0.80 - $0.80

Input - Output

Parameters

70B

Total

Context window

128K

Release date

Jul 2024

Llama 3.1 70B provides efficient multilingual conversational support ideal for applications where responsiveness and computational efficiency are critical. Effective for building chatbots, automated customer interactions, and applications needing fast yet reliable language understanding.

Meta

License: llama3.1

meta-llama/Llama-3.1-70B-Instruct

Supported features

Reasoning

JSON mode

Structured output

Tool calling

LoRA

Use this model

import openai
import weave
# Weave autopatches OpenAI to log LLM calls to W&B
weave.init("<team>/<project>")
client = openai.OpenAI(
    # The custom base URL points to W&B Inference
    base_url='https://api.inference.wandb.ai/v1',
    # Get your API key from https://wandb.ai/authorize
    # Consider setting it in the environment as OPENAI_API_KEY instead for safety
    api_key="<your-apikey>",
    # Optional: Team and project for usage tracking
    project="<team>/<project>",
)
response = client.chat.completions.create(
    model="meta-llama/Llama-3.1-70B-Instruct",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Tell me a joke."}
    ],
)
print(response.choices[0].message.content)

Enter to Rename, Shift+Enter to Preview