Claude 2, Google's NotebookLM, LongNet, Quivr
Anthropic's new Claude model, LongNet & scaling to a billion tokens, and extracting information from unstructured data with Quivr!
Created on July 12|Last edited on July 12
Comment
Claude 2
Anthropic just released Claude 2 and can be tested here. It not only boasts substantially better performance than the original Claude model, but it also supports a max prompt length of 100k tokens. Another unique feature, from one of their demo videos, is their app's ability to upload files! Including files in the conversation is as seamless as dragging and dropping.
Check it out today!
NotebookLM
Google's new NotebookLM utilizes LLMs as your personal note-taking assistant. It can be used for summarization, Q&A, and generating ideas. Join the waitlist now!
LongNet
The paper "LongNet: Scaling Transformers to 1,000,000,000 Tokens" introduces dilated attention which, simply put, divides the Q, K, V matrices of shape in vanilla attention into equal length matrices. For each of these , an interval is selected to sparsely select certain rows of these matrices. The authors describe dilated attention as a drop-in replacement for standard attention, trading off the globality of the original attention mechanism for computational efficiency without sacrificing performance on shorter sequences. Unlike attention, dilated attention doesn't grow quadratically to the input sequence length, allowing for a huge scale up to, as the title puts it, a billion tokens.
Quivr
Quivr, a recently trending GitHub repository, leverages LLMs to extract information from unstructured data. They support a variety of modalities:
- Text
- Markdown
- PDF
- Powerpoint
- Excel (Not Yet)
- CSV
- Word
- Audio
- Video
References
Add a comment
Tags: ML News
Iterate on AI agents and models faster. Try Weights & Biases today.