inference

Inference

Meta Llama 3.1 8B

Text

Model overview

Price

$0.22 - $0.22

Input - Output

Parameters

Total

Context window

128K

Release date

Jul 2024

Llama 3.1 8B provides efficient multilingual conversational support ideal for applications where responsiveness and computational efficiency are critical. Effective for building chatbots, automated customer interactions, and applications needing fast yet reliable language understanding.

Meta

License: llama3.1

meta-llama/Llama-3.1-8B-Instruct

Supported features

Reasoning

Structured output

Tool calling

Use this model

import openai
import weave
# Weave autopatches OpenAI to log LLM calls to W&B
weave.init("<team>/<project>")
client = openai.OpenAI(
    # The custom base URL points to W&B Inference
    base_url='https://api.inference.wandb.ai/v1',
    # Get your API key from https://wandb.ai/authorize
    # Consider setting it in the environment as OPENAI_API_KEY instead for safety
    api_key="<your-apikey>",
    # Optional: Team and project for usage tracking
    project="<team>/<project>",
)
response = client.chat.completions.create(
    model="meta-llama/Llama-3.1-8B-Instruct",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Tell me a joke."}
    ],
)
print(response.choices[0].message.content)

Enter to Rename, Shift+Enter to Preview