xAI Launches Grok, a New LLM with Real-Time Knowledge
Elon Musk has entered the AI race with xAI and Grok-1.
Created on November 6|Last edited on November 6
Comment
xAI has introduced a beta version of their latest AI, Grok-1. Grok promises to not only provide answers to a wide range of questions but also to recommend which questions users should consider asking. Grok-1 is an autoregressive Transformer-based model pre-trained to perform next-token prediction. The model was then fine-tuned using extensive feedback from both humans and the early Grok-0 models. The initial Grok-1 has a context length of 8,192 tokens and has knowledge up to Q3 2023. It's unclear how many parameters Grok-1 has, however, Grok-0 was said to have 33 billion.
Real Time Knowledge Acquisition
Grok is built on the real-time knowledge acquisition capabilities of the X platform, ensuring it provides users with up-to-date responses. The AI's development path began with Grok-0 and has now progressed to Grok-1 after two months of additional training, showcasing remarkable improvements in its coding and reasoning abilities.
On the benchmarks, Grok-1 has demonstrated impressive performance, outperforming competitors in its computational category and standing just behind top-tier models like GPT-4, Palm 2, and Claude 2. Without knowing the true parameter count, it's difficult to judge the model just yet, however, outperforming models like LLaMa 2 70B is no easy feat.

Grok-1 Benchmarks
xAI underlines the importance of diverse applications for their AI, from acting as a research assistant to facilitating the pursuit of knowledge across disciplines. To maintain a reliable infrastructure for Grok, xAI has adopted technologies like Kubernetes, Rust, and JAX. Research at xAI is focused on ensuring that Grok can generate reliable information, with an emphasis on oversight, formal verification for safety, long-context understanding, adversarial robustness, and eventually, multimodal capabilities.
Beta Testing
For those in the United States eager to test Grok, xAI is extending an early access invitation to a select group, aiming to gather user feedback for further refinements.
As Grok moves from beta to more advanced versions, the company anticipates rolling out new features and capabilities, signaling a significant step forward in their long-term plans for AI development.
Announcement: https://x.ai/
Model Card: https://x.ai/model-card/
Add a comment
Tags: ML News
Iterate on AI agents and models faster. Try Weights & Biases today.