An open-source platform for research and experimentation with intelligent virtual agents in Extended Reality (VR/AR) environments.
Creating virtual characters capable of natural interaction within immersive spaces has historically been a fragmented technical challenge. In Human-Computer Interaction (HCI) research, connecting language models, voice processing, and 3D rendering in Extended Reality headsets typically required custom, rigid developments that were difficult to replicate across laboratories. OVARP was born to transform this landscape by offering an open, flexible infrastructure that decouples agent intelligence from the deployment platform.
At its core, the platform operates through a fully modular artificial intelligence pipeline. This allows researchers to hot-swap between different language model, speech recognition, and speech synthesis providers in real time-without restarting the server or altering a single line of code. OVARP seamlessly supports both cloud-based services and local private engines to maintain strict control over latency and data privacy. The entire agent persona-including behavioral guardrails, vocal tone, and empathy calibration-is defined in declarative YAML profiles, enabling exact standardization and scientific replicability across experimental conditions.
A dual-transport architecture powered by WebSockets and ZeroMQ ensures that a single agent can simultaneously interact across web browsers, mobile devices, and immersive headsets. To streamline field research, OVARP incorporates an integrated Wizard-of-Oz control console, allowing researchers to monitor conversations live, intervene directly, trigger dynamic gestures or emotional states, and log timestamped event markers synchronized with real-time user spatial telemetry.
The ecosystem is complemented by open-source reference clients built in Unity for Meta Quest 3 headsets and XREAL smart glasses. In these environments, agents come to life with real-time lip synchronization, continuous facial expressions, procedural gestures, and responsive gaze tracking, proving the platform's versatility across both virtual reality and holographic augmented reality.
