Welcome, AI Explained fans!

🧑‍💻 Simple Bench Evals Competition 🏅

Resources

Leaderboard
Starting code
Simple Bench docs
Weave docs

How to Participate

Setup - Refer to the starter code for installation instructions and environment setup.
Run the Benchmark:

Experiment with different models.
Adjust configurations like model_name, temperature, top_p, or max_tokens to see their impact on the evaluation results.
Modify the system_prompt.txt to customize the system's instruction, ensuring the prompt ends with: "Final Answer: X where X is one of the letters A, B, C, D, E, or F."

Run the submission block in the starter code.
Your results will be logged to Weave and appear on the leaderboard.
Follow the 🍩 link in the output to check your ranking.

Prizes

First Prize: Meta Ray-Bans + W&B Merch + Ultimate bragging rights 😎
Second Prize: Gift Card + W&B Merch
Third Prize: W&B Merch

Rules

If no 20/20 solutions are submitted, competition ends 31st Jan 2025 (incl).
Individual Submissions: Each participant must submit their own solution.
Fair Play: Organizers reserve the right to disqualify any submission suspected of cheating.
Supported Models: Only models specified in the starter code are allowed.
Ray Bans prize only available to participants located in US, Canada, Australia, France, Germany, Italy, Spain, UK due to shipping restrictions.

Introducing Weave from Weights & Biases

Evaluation comparison for MixEval hard benchmark

Log and debug language model inputs, outputs, and metadata
Build rigorous, apples-to-apples evaluations for language model use cases
Organize all information generated across the LLM workflow, from experimentation to evaluations to production

🧑‍💻 All you need are 3 lines of code

import weave
weave.init("ai_explained")

@weave.op()
def sum_nine(value_one: int):
    return value_one + 9

@weave.op()
def multiply_two(value_two: int):
    return value_two * 2

@weave.op()
def main():
    output = sum_nine(3)
    final_output = multiply_two(output)
    return final_output



♾️ Weave supports any use case

🤝 Weave lives where you work



🧑‍🎓 Explore Weave and build LLM apps with our free courses



Trusted by over 200,000 machine learning practitioners

Enterprises that trust Weights & Biases