HACKINTOSH.ORG | Macintosh discussion forums

NVIDIA’s new AI chatbot runs locally on your PC

<p>NVIDIA just released a <a data-i13n="elm:affiliate_link;sellerN:NVIDIA;elmt:;cpos:1;pos:1" href="https://shopping.yahoo.com/rdlw?merchantId=f52348f6-2304-415e-b16c-1fb252794066&amp;siteId=us-engadget&amp;pageId=1p-autolink&amp;featureId=text-link&amp;merchantName=NVIDIA&amp;custData=eyJzb3VyY2VOYW1lIjoiV2ViLURlc2t0b3AtVmVyaXpvbiIsImxhbmRpbmdVcmwiOiJodHRwczovL3d3dy5udmlkaWEuY29tL2VuLXVzL2FpLW9uLXJ0eC9jaGF0LXdpdGgtcnR4LWdlbmVyYXRpdmUtYWkvIiwiY29udGVudFV1aWQiOiJlM2UxNmQ5Yy02MmQ3LTRiMDAtYWIzZi0yNWY0Nzk5M2RlYjQifQ&amp;signature=AQAAAU8yikRt8Vx8N76LHQf67q7cO1CEF7nXPUYX_HRgkG4Y&amp;gcReferrer=https%3A%2F%2Fwww.nvidia.com%2Fen-us%2Fai-on-rtx%2Fchat-with-rtx-generative-ai%2F" class="rapid-with-clickid" data-original-link="https://www.nvidia.com/en-us/ai-on-rtx/chat-with-rtx-generative-ai/">free demo version[/url] of a chatbot that <a data-i13n="cpos:2;pos:1" href="https://blogs.nvidia.com/blog/chat-with-rtx-available-now/">runs locally on your PC.[/url] This is pretty neat, as it gives the chatbot access to your files and documents. You can feed Chat with RTX a selection of personal data and have it create summaries based on that information. You can also ask it questions, just like any chatbot, and dives into your data for answers.</p>
<p>The company says it allows users to “quickly, easily connect local files on a PC as a dataset to an open-source large language model like Mistral or Llama 2.” NVIDIA gives an example of a user asking the chatbot  about a restaurant their partner recommended while in Las Vegas. The software scans local files to find the answer. It supports a whole bunch of file formats, including .txt, .pdf, .doc/.docx and .xml. The company says it will load relevant files into its dataset “in seconds.”</p>
<span id="end-legacy-contents"></span><div id="161ecb99578b449ca26aa1997d4ede61"><iframe width="560" height="315" src="https://www.youtube.com/embed/gdsRJZT3IJw?si=gdB9_VJpc2bXH06o" title="YouTube video player" frameborder="0" allowfullscreen></iframe></div>
<p>Chat with RTX also integrates YouTube videos and playlists. You can add a video URL into the dataset and it will integrate the knowledge contained in the clip for contextual queries. NVIDIA says this will be useful when asking for travel recommendations “based on content from favorite influencer videos” or when looking for tutorials and summaries derived from educational resources.</p>
<p>The Verge <a data-i13n="cpos:3;pos:1" href="https://www.theverge.com/2024/2/13/24071645/nvidia-ai-chatbot-chat-with-rtx-tech-demo-hands-on">had some hands-on time[/url] with the chatbot and came away impressed, writing that they could see it as “a valuable part of data research for journalists or anyone who needs to analyze a collection of documents.”</p>
<p>This sounds like a big step toward something resembling an actual digital assistant that works within the contextual framework of your personal data. With most chatbots, the data is sent off to the cloud, but Chat with RTX “lets users process sensitive data on a local PC without the need to share it with a third party or have an internet connection.” So it’s safer and more contextually aware.</p>
<p>There are some limitations. This is a demo product so you should expect plenty of bugs, though NVIDIA should start squashing them once users begin issuing error reports and the like. There are also some strict hardware limitations here. Chat with RTX only works on Windows PCs with NVIDIA GeForce RTX 30 Series GPUs or higher and at least 8GB of VRAM.</p>
<p>NVIDIA has really been showing off its AI prowess lately, as the company just launched its next-generation of <a data-i13n="cpos:4;pos:1" href="https://www.engadget.com/nvidia-announces-its-next-generation-of-ai-supercomputer-chips-140004095.html">artificial intelligence supercomputer chips[/url]. It’s <a data-i13n="cpos:5;pos:1" href="https://www.engadget.com/nvidia-records-mega-profits-thanks-to-its-ai-chip-business-084552302.html">absolutely swimming in profits,[/url] due primarily to the company’s AI and data center segments.</p>This article originally appeared on Engadget at https://www.engadget.com/nvidias-new-ai-chatbot-runs-locally-on-your-pc-163406121.html?src=rss

Source: NVIDIA’s new AI chatbot runs locally on your PC (https://www.engadget.com/nvidias-new-ai-chatbot-runs-locally-on-your-pc-163406121.html?src=rss)