WebXR is a web standard that enables immersive virtual reality (VR) and augmented reality (AR) experiences directly inside compatible browsers. It is standardized through the W3C’s Immersive Web Working Group as the successor to the earlier WebVR specification. Rather than functioning as a standalone rendering engine, WebXR provides an API layer that allows websites to:
- Request immersive sessions (immersive-vr or immersive-ar)
- Access headset tracking data
- Render stereoscopic views
- Anchor digital objects in 3D space
In practical terms, WebXR makes immersive experiences behave like web pages. When supported by a headset browser, an AR session can begin with a simple user gesture and permission grant. This browser-native model removes the dependency on app stores and proprietary distribution channels, which historically limited accessibility in immersive media.
How Passthrough AR Sessions Work
Passthrough AR on headsets relies on system-level camera feeds that display your physical surroundings in real time. WebXR does not directly control the headset’s cameras. Instead, it enables an immersive-ar session that renders digital content over the operating system’s passthrough layer.
When a user selects “Enter AR” in our player:
- The browser requests an immersive-ar session via the WebXR API.
- The headset activates its passthrough environment at the system level.
- A WebGL-rendered scene is layered over that passthrough view.
- Head tracking data continuously updates the scene to maintain spatial stability.
Because the rendered content responds to head movement with correct parallax and perspective, performers appear spatially anchored in the room rather than floating as flat overlays. The result is mixed reality video that integrates with the user’s environment instead of replacing it.
The Rendering Layer: Why Three.js Is Required
WebXR alone does not handle scene construction, rendering pipelines, or video compositing. It provides session control and spatial tracking. To build a complete AR video experience, a rendering engine is required.
At ARFreaks, we use Three.js, an open-source JavaScript 3D graphics library built on WebGL. In our implementation:
- WebXR manages the immersive session and tracking.
- Three.js constructs the 3D scene.
- The renderer outputs stereoscopic frames aligned with headset tracking.
- Video streams are mapped as textures onto 3D surfaces.
Three.js enables:
- Video textures applied to geometry
- Custom shader processing for transparency
- Stereo rendering for left and right eye views
- Performance optimization within browser constraints
Without a rendering engine like Three.js, WebXR cannot independently create spatial video experiences.
Video Processing: Chroma Key and Alpha Channel Workflows
Passthrough AR video requires transparency to blend naturally with the physical environment. Our player supports two primary workflows.
1. Chroma Key (Green Screen)
In chroma-key video, performers are filmed against a uniform background. During playback:
- A shader samples each video frame.
- Pixels within a defined color range are removed.
- Edge refinement and smoothing are applied to reduce halo artifacts.
Lighting consistency during filming significantly affects final quality. Uneven lighting can introduce noise at subject boundaries, requiring fine-tuned tolerance thresholds.
2. Alpha Channel Video
Alpha-channel video contains built-in transparency data per pixel. This allows:
- Cleaner edge definition
- More accurate hair and fine detail preservation
- Reduced need for runtime color-based removal
In these cases, the renderer reads transparency data directly and composites it over passthrough without real-time color extraction.
Both approaches are spatially positioned within a Three.js scene, allowing performers to appear at defined distances and orientations relative to the viewer.
How ARFreaks Combines These Layers
Our custom player integrates:
- WebXR session control
- Three.js rendering
- Optimized video texture processing
- Custom shader logic for transparency handling
The objective is spatial stability, clean compositing, and performance efficiency within current-generation mixed-reality hardware. Because the entire system runs in-browser, distribution is simplified. Users can access immersive AR experiences without installing dedicated applications, provided their device browser supports WebXR.
Development Priorities
Current development priorities for the ARFreaks player include:
- In-AR playback controls (play, pause, seek)
- Loop and repeat options
- Integrated volume adjustment inside AR mode
- Manual chroma-key tolerance and edge-smoothing sliders
- Selectable resolution tiers (including high-resolution streaming modes)
- One-click switching between passthrough AR and full VR environments
These improvements focus on usability refinement and deeper spatial integration rather than expanding beyond the browser-based architecture.
Why Passthrough AR Video Is Technically Challenging
Delivering stable, convincing passthrough AR video inside a browser environment presents several constraints:
Performance Limitations
Headset browsers operate under thermal and GPU constraints. Rendering stereoscopic frames, processing video textures, and applying real-time shaders must be optimized carefully to avoid frame drops.
Lighting Mismatch
A performer filmed under studio lighting may not visually match the lighting conditions in a user’s physical room. While true real-time relighting remains limited in browser-based environments, careful filming and neutral lighting design help mitigate this issue.
Edge Artifacts
Chroma key processing can introduce aliasing or shimmer along edges, particularly at high contrast boundaries. Advanced shader tuning and edge smoothing are necessary to preserve realism.
Bitrate vs Stability
Ultra-high-resolution video increases immersion but also raises decoding load and bandwidth requirements. Encoding decisions must balance clarity against playback stability within headset browsers.
These challenges explain why polished browser-based passthrough AR remains technically demanding despite broader mixed-reality experimentation across the industry.
Conclusion
WebXR enables immersive AR sessions inside the browser. Three.js provides the rendering infrastructure. Optimized video compositing makes passthrough transparency possible. Together, these layers create spatially anchored AR video without requiring proprietary apps or distribution platforms.
As mixed-reality hardware continues to mature, browser-native immersive media will likely become more capable. For now, delivering stable passthrough AR inside a web session requires careful architectural decisions, performance tuning, and transparency-aware rendering workflows. This is the foundation on which ARFreaks is built.
Last Updated: March 2026 | Written by Scott Camball, creator of ARFreaks and WebXR developer.