Open-Weight Models and Local AI Agents on NVIDIA GPUs

Glowing PC tower powering local AI agents with NVIDIA GPU on a desk

Developers can now run frontier-quality AI agents entirely on their own hardware, keeping source code, proprietary data and project context on a personal system instead of sending it to the cloud. Across August 2026, NVIDIA released and optimized a wave of open-weight models and local agent apps for its GPUs and personal AI systems, giving builders private local inference for coding, video generation, robotics and multistep agentic work. Many of these releases arrived with day-zero support on NVIDIA RTX GPUs, DGX Spark, DGX Station and Jetson.

What can you run locally right now?

The common thread across these releases is that capable models are being sized to run on a single consumer GPU or a compact personal AI system, with fast token generation and always-on operation. That means local coding agents, video and audio generation, private document processing and long-running autonomous tasks can execute on a desk rather than through a metered cloud service. The open-weight licensing behind most of these models also lets developers fine-tune them on private or specialized data while keeping that data on the device.

Perplexity Portable Computer runs a private agent on DGX Spark

Perplexity’s Portable Computer is a local agent app that runs a personal AI agent on NVIDIA DGX Spark, so everyday workflows stay private and do not count against token limits. It connects to widely used apps including Google Drive, Gmail, Slack and GitHub, and lets a user switch between local models for routine tasks and cloud models for more complex work. DGX Spark provides fast large language model inference and 24/7 always-on operation, which suits long-running autonomous agents. The app offers one-click local inference setup and ships with a specially post-trained Qwen 3.8 27B model, with a fine-tuned Nemotron 3.5 Lightning variant for faster responses planned. Support for GeForce RTX and RTX PRO GPUs, Windows and DGX Station is on the way.

How fast is Qwen3.8-27B on a single GPU?

Qwen3.8-27B is a 27-billion-parameter open model sized for a single GPU and built for responsive coding workflows that use local files, tools and project context. As the local companion to Qwen3.8-Max, it reaches 131 tokens per second on a single GeForce RTX 5090 when optimized with multi-token prediction. Day-zero support covers NVIDIA RTX GPUs including RTX PRO GPUs and the GeForce RTX 5090, and developers can start with llama.cpp, Ollama, Unsloth and LM Studio Bionic. The model also runs on DGX Spark and DGX Station, with NVFP4 optimizations coming, and NVIDIA Jetson brings the same day-zero support to edge devices.

New open-weight models available today

Several fresh models landed with NVIDIA GPU acceleration:

Cosmos 3 Edge is a 4-billion-parameter open world model for robotics, autonomous vehicles and vision AI. At a quarter the size of Cosmos 3 Nano, it runs on device on DGX Spark and Jetson.
MiniMax-H3 is a 33-billion-parameter open-weight model that generates video and natively synchronized stereo audio from text, images, video, audio or a mix, accessible through ComfyUI with checkpoints optimized for NVIDIA GPUs.
Poolside AI Laguna S 2.1 is a 118-billion-parameter open-weight agentic coding model built for hours-long tasks. An NVFP4 checkpoint runs it locally on a single DGX Spark with lower compute and memory needs.
DeepSeek-V4-Flash is a 284-billion-parameter mixture-of-experts model with 13 billion active parameters and a 1 million-token context window, runnable locally on a DGX Station via community-built GGUF versions.
Thinking Machines Lab Inkling-Small is a 276-billion-parameter open-weight multimodal model with native reasoning across text, images and audio and adjustable thinking effort. It activates 12 billion parameters per token and runs on a single DGX Station or two DGX Spark systems, with an NVFP4 checkpoint on Hugging Face.
Unsloth Desktop combines local model inference, image and video diffusion, fine-tuning, agent integrations, web research and code execution in one open-source desktop app, and is described as the first desktop app to both train and run AI models locally.
Alibaba Wan-Animate-2 is a 14-billion-parameter open-weight model that transfers motion and facial expressions from a driving video onto a static character image, whether human, cartoon, robot or animal. With day-zero ComfyUI support it generates up to 26x faster on RTX PRO 6000 Blackwell and 22x faster on an RTX 5090 compared with an Apple M3 Ultra.

Higher-quality local video with LTX-2.5

LTX-2.5 is an open-world video generation model that delivers higher-quality video, stronger prompt adherence and more consistent characters, scenes and voices across generations. New multishot support lets creators generate sequences that span multiple cuts while keeping continuity, and an upgraded diffusion video decoder improves fidelity and reduces artifacts. Its prompt enhancer uses Gemma4 E2B and a custom Gemma4 12B text encoder for tighter control over output. On an NVIDIA RTX 6000 PRO GPU, LTX-2.5 delivers up to 20% faster performance and 40% memory savings, and it is available through a ready-to-use ComfyUI workflow for text, image and video-to-video generation.

Meta’s Muse Glimmer for always-on local agents

Muse Glimmer is a 30-billion-parameter dense open-weight model with a 120K-plus context window, purpose-built for coding and local agentic AI. It delivers over 200 tokens per second on an RTX 5090 and runs on a single consumer GPU, supporting custom agents, private data processing, credential handling on the device, multistep tool calls and long-running workflows that resume with context intact. Developers can build agents with NemoClaw and fine-tune the model locally with NVIDIA NeMo Automodel, and run it through vLLM or llama.cpp, with llama.cpp supporting DFlash speculative decoding to speed generation.

Clustering DGX Spark systems and Nemotron 3.5 Lightning

Larger open models such as GLM 5.2 and DeepSeek V4 Flash can require multiple GPUs or DGX Spark systems working together. NVIDIA Sync, available for Windows and macOS, detects connected systems and provides private remote access through Tailscale, and its Cluster Assistant configures two or more DGX Spark systems into a high-speed cluster over NVIDIA ConnectX-7 ports, routing workloads and monitoring health. Later in August, DGX Spark also gains Google Chrome as a native ARM64 Linux build installable in one click, plus a Sync Resource Monitor for real-time and historical CPU and GPU usage. Rounding out the releases, NVIDIA expanded its Nemotron 3 family with Nemotron 3.5 Lightning, a customizable open 30B mixture-of-experts model for always-on agents that delivers up to 4x faster token generation and 30% faster time to completion compared with open models in its class, and can be fine-tuned on a user’s own examples.

FAQ

Which open-weight model runs local coding agents on a single GPU?

Qwen3.8-27B is a 27-billion-parameter open model sized for a single GPU. It reaches 131 tokens per second on a single GeForce RTX 5090 with multi-token prediction and works with llama.cpp, Ollama, Unsloth and LM Studio Bionic, plus DGX Spark, DGX Station and Jetson.

What is Perplexity Portable Computer?

It is a local agent app that runs a personal AI agent on NVIDIA DGX Spark, keeping workflows private and off token limits. It connects to Google Drive, Gmail, Slack and GitHub, and lets users switch between local models for everyday tasks and cloud models for complex work, using a specially post-trained Qwen 3.8 27B model.

How fast is Meta’s Muse Glimmer on local hardware?

Muse Glimmer is a 30-billion-parameter dense open-weight model with a 120K-plus context window that delivers over 200 tokens per second on an RTX 5090. It runs on a single consumer GPU and supports custom agents, private data processing, on-device credential handling and long-running multistep workflows.

Related coverage

This article summarizes reporting from blogs.nvidia.com. See our editorial disclaimer for how our articles are produced.

Open-Weight Models and Local AI Agents Accelerated by NVIDIA GPUs