Nvidia Card Owners Will be Able to Run GenAI Models on PCs

Nvidia presented an innovative tool for GeForce RTX 30 and 40 series users – Chat with RTX. It provides an AI-powered chatbot experience directly on your Windows PC. By gaining access to documents, files, and folders stored on the device, the system allows you to customize the GenAI model, following the example of the famous ChatGPT from OpenAI.

As Nvidia emphasizes, Chat with RTX eliminates the need to manually search through files or archives, allowing you to ask questions directly. Chat with RTX will monitor the local resources specified by the user and provide a relevant answer.

The smart tool not only supports the open-source model from AI Mistral but also other models, in particular Llama 2 from Meta. At the same time, its developers warn about the significant amount of memory required for its full operation – from 50 to 100 GB. In the current version, Chat with RTX works with formats such as PDF, DOC, DOCX, XML. Additionally, it is capable of downloading transcriptions of YouTube videos.

Nvidia's chatbot has a limitation. It lacks the ability to remember context. This means that it is impossible to take into account previous requests when formulating answers to new questions. For example, you decide to ask it about the results of the football match between Manchester United and Bayern in 2023. If immediately after that, you ask who scored the first goal, the chatbot will not understand that we are talking about a match between these two teams.

According to a report released at this year's World Economic Forum, devices that can run autonomously with GenAI models will gain popularity. The main reasons for the rapid growth are their advantages. Experts included a high level of confidentiality of data processing, lower response delay, and cost-effectiveness compared to cloud solutions.