NVIDIA Launches 'Chat With RTX'
A new tool for running local LLM's!
Created on February 13|Last edited on February 13
Comment
NVIDIA has launched a new tech demo, Chat with RTX, which enables users to create a personalized chatbot on Windows PCs equipped with an NVIDIA RTX GPU. This application allows for the integration of various types of files—including documents, notes, and video transcriptions—into a chatbot that users can interact with to get contextually relevant answers quickly. The technology behind Chat with RTX includes retrieval-augmented generation for accessing local files, along with the NVIDIA TensorRT-LLM software, offering a local solution for generative AI capabilities without the need for cloud computing.
Features and Requirements
Chat with RTX is designed to work on systems with specific requirements, such as Windows 11 OS, a minimum of 16GB of RAM, and an NVIDIA GeForce RTX 30 or 40 series GPU's with at least 8gb of VRAM, among other specifications. The application supports a range of file formats, making it versatile for different user needs. Additionally, users can query content from YouTube videos by incorporating video URLs, allowing the chatbot to access a broader range of information.
One of the key advantages of Chat with RTX is its local processing capability, ensuring fast response times and keeping the user's data secure on their device without requiring internet access or third-party data sharing. This aspect is particularly appealing for handling sensitive information.
Competition?
NVIDIA's unique position in the AI race is significantly bolstered by its direct control over both advanced hardware and specialized software, offering a compelling advantage in deploying new technologies like AI-driven chatbots, with the ability to deploy chatbots directly embedded into software updates. Adjacent to NVIDIA in terms of having large amounts of custom silicon currently deployed is Apple. Known for its strategic patience and emphasis on refining technology before release, Apple is likely observing NVIDIA's moves closely. Apple's strategy often involves waiting to see how technologies mature, then entering the market with a highly polished product that integrates seamlessly with its ecosystem. Given the growing importance of AI and machine learning, it's plausible to anticipate Apple's further investment in these areas, potentially through its M series processors. The M series' custom architecture already demonstrates Apple's commitment to optimizing hardware for specific performance and efficiency goals, making it a suitable platform for advanced AI capabilities.
Overall, Chat with RTX is just a small portion of NVIDIA's efforts to expand the applications of generative AI and accelerated computing to a wider audience, and this is likely only the beginning of NVIDIA's contributions to the world of AI.
Add a comment
Tags: ML News
Iterate on AI agents and models faster. Try Weights & Biases today.