Falcon & DIDACT
The Open LLM Leaderboard's new rank #1 and DIDACT, a more capable code assistant.
Created on June 4|Last edited on June 4
Comment
Falcon
Falcon is yet another open-source LLM! It is currently the best open-source LLM according to the Open LLM Leaderboard.
The Technology Innovation Institute (TII) researchers behind Falcon released a family of Falcon models along with a dataset called RefinedWeb.
The authors describe Falcon to be just as good as GPT-3, matching the state-of-the-art models from Google, DeepMind, and Anthropic while being far more efficient.
DIDACT
DIDACT, or Dynamic Integrated Developer ACTivity, is a method for training LLMs on software development-based data. Its novelty lies in incorporating intermediate code data. That is, instead of just finished code, this method also leverages the intermediate steps of coding. They trained a model, which they called DIDACT, on a number of software development tasks like debugging, code review, and code editing.
DIDACT is a step towards a more general purpose code assistant. One that can not only autocomplete your code, but debug, review, comment, and more!
One interesting emergent property they found is called history augmentation which is the model's ability to estimate what the developer might do next based on a history of actions. The example they gave was adding a new argument to a function where DIDACT intuitively added the docstring automatically based on the addition of the new argument and the movement of the cursor. A similar property is edit prediction where the model will edit dependent code to be consistent with your updates on, say, upstream code.
References
Add a comment
Tags: ML News
Iterate on AI agents and models faster. Try Weights & Biases today.