The open project brings together VLM, video, OpenXR, and MCP for agents on XR glasses and devices. See architecture, use cases, and required controls.
Direct answer
NVIDIA XR AI is an open architecture for agents in AR glasses and XR devices that combines visual perception, voice, spatial context, and enterprise tools. The repository includes MCP servers for video, OpenXR, rendering, vectors, and transcriptions, allowing you to connect maintenance, digital twins, and other systems.
What architecture brings together
The project combines vision and language models, video analysis, transcription, rendering and OpenXR spatial information. MCP works as an integration layer to call tools and consult external systems depending on the task.
Why XR changes the agent
On a computer, the agent receives files and screens. With glasses, it also perceives the environment, direction of gaze and sequence of actions. This information can make the response contextual, but increases data sensitivity and the need for consent.
Enterprise applications
A technician can consult the maintenance order while observing the equipment; an engineer can relate the digital twin to the physical object; a researcher can retrieve protocols without interrupting activity. The answer should point to evidence and the current state of the system.
Essential controls
Camera and audio require minimization, capture indication and retention policy. Tools need per-action, logs, and confirmation permissions before impacting commands. Local or edge processing may reduce exposure, as appropriate.
Nexus Reading
The next enterprise assistant won't just be in a window: it will share the physical context of work. The advantage will come from integrating perception and data without losing privacy, traceability and ergonomics.
FAQ
What does MCP do in this architecture?
Standardizes agent connection to specific XR tools, data, and capabilities.
What data is sensitive?
Video, audio, location, pose, environment and content of business systems.
Should the agent act automatically?
Only in low-risk actions and within permissions; Critical changes require confirmation.
Essential guides to delve deeper into the decision
This editorial analysis was produced by Nexus from the official sources below, consulted on September 8, 2026. The text is original and interprets practical implications for companies.
- NVIDIA — Building AI Agents for AR Glasses and XR Devices with NVIDIA XR AI: architecture, components, repository and business cases.
- NVIDIA — XR AI repository: open code and components mentioned in the technical article.
Date reported by the main source: June 16, 2026.