WebXR Anchors & Plane Detection

Building a WebXR App with Three.js

End-to-end guide to building a production WebXR application with Three.js and TypeScript — project setup, session lifecycle, controller input, teleportation, VR UI, and deployment.

WebXR Hand Tracking — From Joints to Gestures

A practical deep-dive into the WebXR Hand Tracking API — joint positions, pinch detection, custom gesture recognition, and interaction patterns for immersive VR applications.

One of the key challenges in XR is making virtual objects stay where you put them. Walk away, come back — is the object still there? This is where WebXR Anchors come in. Combined with plane detection, they form the foundation for spatial understanding in immersive web applications.

What Anchors Solve

Without anchors, objects are positioned relative to a reference space (typically local-floor or bounded-floor). Move through the space, and the objects move with you — they're not truly placed in the world.

An XRAnchor represents a fixed point in the user's environment. The runtime tracks this point across frames, adjusting its transform as the understanding of the environment improves. This means:

• World-locked persistence — objects stay in place as the user moves
• Dynamic correction — if the runtime refines its understanding of the room, the anchor updates
• Cross-session persistence — with anchor UUIDs, objects can survive app restarts

Creating Anchors

Anchors can be created from several sources:

From a Hit Test Result

The most common pattern — the user points at a surface, and an anchor is created at the intersection:

const session = await navigator.xr.requestSession('immersive-ar', {
  requiredFeatures: ['hit-test', 'anchors']
});

// Create hit test source
const hitTestSource = await session.requestHitTestSource({
  space: viewerReferenceSpace
});

// In frame loop
function onXRFrame(time: number, frame: XRFrame) {
  const hitResults = frame.getHitTestResults(hitTestSource);
  if (hitResults.length > 0) {
    const pose = hitResults[0].getPose(referenceSpace);
    if (pose) {
      const anchor = await frame.createAnchor(pose.transform, referenceSpace);
      // attachObjectToAnchor(anchor);
    }
  }
}

From Plane Intersection

If plane detection is active, you can create anchors directly from plane geometry:

// Get all detected planes
const detectedPlanes = frame.detectedPlanes;
for (const plane of detectedPlanes) {
  const centerPose = frame.getPose(plane.planeSpace, referenceSpace);
  if (centerPose && plane.semanticLabel === 'floor') {
    const anchor = await frame.createAnchor(centerPose.transform, referenceSpace);
    break;
  }
}

Free Placement (without hit test)

You can also create anchors at arbitrary positions in space:

const arbitraryTransform = new XRRigidTransform({
  x: 1.5, y: 0.5, z: -1.0  // 1.5m right, 0.5m up, 1m in front
});
const anchor = await frame.createAnchor(arbitraryTransform, referenceSpace);

Anchor Lifecycle

Understanding anchor states is critical for robust applications:

Creation → Tracking → (Loss) → (Recovery) → Deletion

// Check anchor tracking state each frame
function updateAnchor(anchor: XRAnchor, frame: XRFrame) {
  const anchorPose = frame.getPose(anchor.anchorSpace, referenceSpace);

  if (!anchorPose) {
    // Anchor is temporarily lost — occluded or out of tracking area
    // Option 1: Hide attached object
    anchorObject.visible = false;

    // Option 2: Use last known position as fallback
    anchorObject.position.copy(lastKnownPosition);
    return;
  }

  // Anchor is tracking — update object transform
  anchorObject.position.copy(anchorPose.transform.position);
  anchorObject.quaternion.copy(anchorPose.transform.orientation);
  lastKnownPosition.copy(anchorPose.transform.position);

  // Anchor is fully tracking
  anchorObject.visible = true;
}

Anchor loss happens when:

• The device loses tracking (low light, featureless surfaces)
• The user moves outside the anchor's tracking region
• The runtime garbage-collects anchors to free memory

Some anchors recover automatically when tracking is re-established. Others remain permanently lost — this is runtime-dependent.

Plane Detection

Plane detection identifies real-world surfaces (floors, walls, tables, ceilings) and exposes them as XRPlane objects.

Requesting Plane Detection

const session = await navigator.xr.requestSession('immersive-ar', {
  requiredFeatures: ['plane-detection']
});

Plane Updates

Plane detection is incremental — planes are added, removed, and refined over time:

session.addEventListener('plane-detection', (event: XRPlaneEvent) => {
  // New planes detected
  for (const plane of event.added) {
    visualizePlane(plane);
  }

  // Existing planes changed (polygon refined, classification updated)
  for (const plane of event.changed) {
    updatePlaneVisualization(plane);
  }

  // Planes removed (no longer visible or merged)
  for (const plane of event.removed) {
    removePlaneVisualization(plane);
  }
});

Plane Properties

Each XRPlane exposes:

interface XRPlane {
  planeSpace: XRSpace;            // Origin at plane center
  polygon: DOMPointReadOnly[];    // Boundary vertices
  lastChangedTime: number;         // Timestamp of last modification
  semanticLabel?: string;          // 'floor', 'wall', 'ceiling', 'table', etc.
  orientation: string;            // 'horizontal' | 'vertical'
}

Visualizing Planes

Create a mesh from the polygon vertices:

function createPlaneMesh(plane: XRPlane): THREE.Mesh {
  const shape = new THREE.Shape();
  const points = plane.polygon;

  shape.moveTo(points[0].x, points[0].z);
  for (let i = 1; i < points.length; i++) {
    shape.lineTo(points[i].x, points[i].z);
  }
  shape.closePath();

  const geometry = new THREE.ShapeGeometry(shape);
  const material = new THREE.MeshBasicMaterial({
    color: plane.semanticLabel === 'floor' ? 0x4488ff : 0xff8844,
    transparent: true,
    opacity: 0.3,
    side: THREE.DoubleSide
  });

  const mesh = new THREE.Mesh(geometry, material);
  // Rotate from XZ to XY (Three.js convention)
  mesh.rotation.x = -Math.PI / 2;
  return mesh;
}

Note: The polygon vertices are in the plane's local coordinate system (XZ plane), not world space. Attach the mesh to a THREE.Object3D parented to the planeSpace transform.

Semantic Labels

The semanticLabel property provides human-readable classification:

Not all runtimes provide semantic labels. Always check if the label exists before relying on it.

Persistent Anchors

Anchors can survive application restarts using their UUID:

// When anchor is created
const anchor = await frame.createAnchor(transform, referenceSpace);
const anchorId = anchor.uuid;
localStorage.setItem('mySavedAnchor', anchorId);

// On subsequent session — restore anchors
const savedAnchorId = localStorage.getItem('mySavedAnchor');
if (savedAnchorId) {
  // Session.requestAnchorInformation() returns anchor info by UUID
  const anchorInformation = await session.requestAnchorInformation(savedAnchorId);
  if (anchorInformation) {
    // Re-create scene objects at this anchor
  }
}

Important limitations:

• Not all runtimes support persistent anchors — check session.enabledFeatures
• Persistent anchors may fail if the environment has changed significantly
• Some runtimes limit the number of persistent anchors (e.g., 16 on Quest)
• Anchor UUIDs may be invalidated after a device reboot

Combining Anchors with Hit Testing for Placement UX

The most common UX pattern for object placement:

1. Preview phase: Ray from controller → hit test → ghost object snaps to surface
2. Confirm phase: User triggers select → create anchor at hit point
3. Post-placement: Object tracks the anchor; user can grab and adjust

class PlacementController {
  private previewObject: THREE.Mesh;
  private placedAnchors: XRAnchor[] = [];
  private hitTestSource: XRHitTestSource | null = null;

  async initialize(session: XRSession, refSpace: XRReferenceSpace) {
    this.hitTestSource = await session.requestHitTestSource({
      space: refSpace
    });
  }

  update(frame: XRFrame) {
    const results = frame.getHitTestResults(this.hitTestSource!);
    if (results.length > 0) {
      const pose = results[0].getPose(referenceSpace);
      if (pose) {
        // Move preview to hit surface
        this.previewObject.position.copy(pose.transform.position);
        this.previewObject.visible = true;
      }
    } else {
      this.previewObject.visible = false;
    }
  }

  async placeObject(frame: XRFrame) {
    const results = frame.getHitTestResults(this.hitTestSource!);
    if (results.length > 0) {
      const pose = results[0].getPose(referenceSpace);
      if (pose) {
        const anchor = await frame.createAnchor(pose.transform, referenceSpace);
        this.placedAnchors.push(anchor);
        // Instantiate final object at anchor position
      }
    }
  }
}

Performance Considerations

Series Cross-References

• Building a WebXR App with Three.js — Implementation guide that integrates anchors into a real application
• WebXR Hit Testing & Depth Sensing — Combine anchors with depth data for realistic occlusion
• WebXR Hand Tracking — Use hand gestures for anchor placement

References

• WebXR Anchor Module Spec (W3C)
• WebXR Plane Detection Spec (W3C)
• Meta Quest: Spatial Anchors