XR-Enhanced Collaboration: Self-Hosted Virtual Meeting Infrastructure

Executive Summary

Extended Reality (XR) technologies are transforming team collaboration, enabling immersive virtual meeting spaces that transcend geographical boundaries. This article explores how self-hosted XR platforms provide organizations with control over collaboration data, infrastructure sovereignty, and regulatory compliance. We present a practical framework for implementing XR-enhanced collaboration with custom avatar systems, shared 3D workspaces, and whiteboarding capabilities.

Key Takeaways:

• Self-hosted XR platforms ensure data sovereignty while enabling immersive collaboration

• HDF5-based collaboration engines enable real-time 3D canvas sharing and whiteboarding

• Custom avatar systems with WebGL, Three.js provide engaging, branded meeting experiences

• The total cost of ownership for self-hosted XR becomes competitive at scale (50+ users)

• Self-hosted XR platforms ensure data sovereignty while enabling immersive collaboration

• HDF5-based collaboration engines enable real-time 3D canvas sharing and whiteboarding

• Custom avatar systems with WebGL and Three.js provide engaging, branded meeting experiences

• The total cost of ownership for self-hosted XR becomes competitive at scale (50+ users)

The Immersive Collaboration Landscape

Why XR Matters for Team Collaboration

Traditional video conferencing limitations:

• 2D screens create distance, reducing engagement in distributed teams
• Limited spatial context (no shared physical space perception)
• Information channel overflow (simultaneous audio/video streams)
• Screen fatigue from continuous 2D consumption

XR-enhanced collaboration advantages:

• Spatial presence: Shared virtual space creates psychological proximity
• Natural interaction: 3D UI elements allow intuitive manipulation
• Enhanced memory retention: Spatial memory aids information recall
• Multisensory communication: Gestures, spatial audio, and visual cues

Market Trends and Adoption Drivers

Corporate adoption acceleration:

• Remote and hybrid workforce mandates (post-2020 pandemic)
• Digital transformation initiatives seeking competitive edge
• Sustainability goals (reducing travel carbon footprint)
• Talent acquisition (modern collaboration tools attract talent)

Industry-specific drivers:

Industry	XR Collaboration Use Cases	Key Benefits
Healthcare	Medical training, surgical planning, patient consultations	Reduced training costs, expertise sharing, patient outcomes
Education	Virtual classrooms, laboratory simulations, field trips	Experiential learning, resource optimization, global access
Manufacturing	Digital twins, facility planning, remote maintenance	Reduced design errors, faster time-to-market, safety
Architecture/Construction	Virtual site walkthroughs, client presentations, design reviews	Stakeholder alignment, reduced rework, cost savings
Finance	Interactive data visualization, portfolio reviews, scenario planning	Improved analysis, faster decisions, client engagement

Technology maturation drivers:

• WebXR standard enables browser-based XR without application downloads
• WebGL and Three.js ecosystem accelerate development
• Mobile device capabilities (ARKit, ARCore) democratize access
• Network improvements (5G, WebRTC) reduce latency for real-time collaboration
• Open-source XR platforms (Mozilla Hubs, WebXR Device API) reduce barriers

The Self-Hosting Imperative

Why self-hosted XR platforms?

Data Sovereignty:

• Meeting recordings, transcripts, and generated content stored under organizational control
• No third-party analysis or AI training on meeting data
• Audit trails for compliance requirements

Security:

• End-to-end encryption for all meeting communications
• Customizable authentication and authorization policies
• Network isolation for air-gapped deployments (government/defense sectors)

Customization:

• Branded avatar systems reflecting organizational identity
• Domain-specific collaboration tools (telehealth classrooms, manufacturing digital twins)
• Integration with internal systems (HR platforms, knowledge bases, project management)

Compliance:

• GDPR-compliant data handling (European hosting, data portability)
• HIPAA-ready architecture for healthcare applications
• SOC 2 Type II compliance for enterprise adoption

Technical Architecture: Self-Hosted XR Collaboration Platform

System Components Overview

Frontend Layer (WebXR Client)

Browser-based client using WebXR API and Three.js:

// WebXR initialization
const button = document.querySelector('#enter-xr');
button.addEventListener('click', async () => {
  const supported = await navigator.xr.isSessionSupported('immersive-vr');
  if (supported) {
    const xrSession = await navigator.xr.requestSession('immersive-vr', {
      optionalFeatures: ['local-floor', 'bounded-floor', 'hand-tracking']
    });
    xrSession.addEventListener('end', onSessionEnd);
    xrSession.updateRenderState({ baseLayer: baseLayer });
    await renderer.xr.setSession(xrSession);
  } else {
    Fallback: Show 2D interface for non-VR users
  }
});

// Three.js scene setup for virtual meeting room
const scene = new THREE.Scene();
const renderer = new THREE.WebGLRenderer({ antialias: true });
renderer.setSize(window.innerWidth, window.innerHeight);
renderer.xr.enabled = true;

// 3D avatar mesh
const avatarGeometry = new THREE.CapsuleGeometry(0.3, 1.8, 4, 8);
const avatarMaterial = new THREE.MeshStandardMaterial({ color: 0x3498db });
const avatar = new THREE.Mesh(avatarGeometry, avatarMaterial);
scene.add(avatar);

// Spatial audio for voice pickup from avatar position
const positionalAudio = new THREE.PositionalAudio(listener);
avatar.add(positionalAudio);
```text

**Key frontend capabilities**:

- **Avatar synchronization**: Real-time 3D position updates for all meeting participants
- **Spatial audio**: Voice emanates from avatar position in 3D space
- **Shared 3D canvas**: Interactive 3D workspaces for collaborative annotation
- **Whiteboarding**: 2D and 3D drawing surfaces with pencil-like input
- **Gesture recognition**: Hand tracking for natural 3D object manipulation

#### Backend Layer (HDF5 Collaboration Engine)

**HDF5 (Hierarchical Data Format)** as collaboration foundation:

```python
# HDF5 meeting state storage
import h5py
import numpy as np

# Open HDF5 file for meeting state
with h5py.File('meeting_state_20260402.h5', 'w') as f:
    # Create groups for meeting data
    avatars = f.create_group('avatars')
    canvas = f.create_group('canvas')
    whiteboard = f.create_group('whiteboard')

    # Avatar positions (timestamp, user_id, x, y, z)
    avatar_positions = avatars.create_dataset(
        'positions',
        shape=(0, 4),  # timestamp, user_id, x, y, z
        maxshape=(None, 4),
        dtype='float64'
    )

    # Canvas state (timestamp, object_id, x, y, z, rotation)
    canvas_objects = canvas.create_dataset(
        'objects',
        shape=(0, 6),  # timestamp, object_id, x, y, z, rotation
        maxshape=(None, 6),
        dtype='float64'
    )

    # Whiteboard strokes (timestamp, user_id, stroke_id, x, y, color, thickness)
    whiteboard_strokes = whiteboard.create_dataset(
        'strokes',
        shape=(0, 7),  # timestamp, user_id, stroke_id, x, y, color, thickness
        maxshape=(None, 7),
        dtype='float64'
    )

    # Meeting metadata
    f.attrs['meeting_id'] = 'meeting_20260402_091532'
    f.attrs['start_time'] = '2026-04-02T09:15:32Z'
    f.attrs['host_user_id'] = 'user_001'
```text

**Real-time collaboration via WebSocket**:

```python
# WebSocket server for real-time state synchronization
from fastapi import FastAPI, WebSocket
from fastapi.middleware.cors import CORSMiddleware
import h5py
import json
import asyncio

app = FastAPI()

# CORS configuration for WebXR client access
app.add_middleware(
    CORSMiddleware,
    allow_origins=["https://www.tobias-weiss.org"],
    allow_credentials=True,
    allow_methods=["*"],
    allow_headers=["*"],
)

# Open HDF5 file for concurrent access (read-write)
h5_file = h5py.File('meeting_state.h5', 'r+')

@app.websocket("/ws/meeting/{meeting_id}")
async def websocket_endpoint(websocket: WebSocket, meeting_id: str):
    await websocket.accept()

    while True:
        # Receive client updates
        data = await websocket.receive_json()

        # Update HDF5 state based on update type
        if data['type'] == 'avatar_position':
            h5_file['avatars']['positions'].resize((h5_file['avatars']['positions'].shape[0] + 1), axis=0)
            h5_file['avatars']['positions'][-1] = [data['timestamp'], data['user_id'], data['x'], data['y'], data['z']]
            h5_file.flush()  # Persist to disk
        elif data['type'] == 'whiteboard_stroke':
            h5_file['whiteboard']['strokes'].resize((h5_file['whiteboard']['strokes'].shape[0] + 1), axis=0)
            h5_file['whiteboard']['strokes'][-1] = [data['timestamp'], data['user_id'], data['stroke_id'], data['x'], data['y'], data['color'], data['thickness']]
            h5_file.flush()

        # Broadcast updated state to all connected clients
        updated_state = {
            'avatars': h5_file['avatars']['positions'][-10:].tolist(),  # Last 10 positions
            'whiteboard': h5_file['whiteboard']['strokes'][-10:].tolist()  # Last 10 strokes
        }
        await websocket.send_json(updated_state)
```text

#### Infrastructure Layer (Container Orchestration)

**Docker Compose configuration for XR platform**:

```yaml
version: '3.8'

services:
  # FastAPI backend for XR collaboration
  xr-backend:
    build: ./xr-backend
    ports:
      - "8000:8000"
    volumes:
      - ./hdf5-data:/hdf5-data
    environment:
      - HDF5_DATA_PATH=/hdf5-data/meeting_state.h5
      - CLOUD_STORAGE_API_KEY=your_cloud_storage_key
    networks:
      - xr-network
    restart: unless-stopped

  # Traefik reverse proxy for SSL termination
  traefik:
    image: traefik:latest
    command:
      - --api.insecure=true
      - --providers.docker=true
      - --entrypoints.web.address=:80
      - --entrypoints.websecure.address=:443
    ports:
      - "80:80"
      - "443:443"
      - "8080:8080"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - ./traefik/letsencrypt:/letsencrypt
    networks:
      - xr-network
    restart: unless-stopped

  # PostgreSQL for meeting metadata and user authentication
  postgres:
    image: postgres:15
    volumes:
      - postgres-data:/var/lib/postgresql/data
    environment:
      - POSTGRES_DB=xr_collaboration
      - POSTGRES_USER=xr_user
      - POSTGRES_PASSWORD=secure_password_here
    networks:
      - xr-network
    restart: unless-stopped

  # Redis for session management and real-time messaging
  redis:
    image: redis:7
    command: redis-server --appendonly yes
    volumes:
      - redis-data:/data
    networks:
      - xr-network
    restart: unless-stopped

volumes:
  hdf5-data:
  postgres-data:
  redis-data:

networks:
  xr-network:
    driver: bridge
```text

### Security and Compliance Design

#### End-to-End Encryption

**Media encryption for video/audio streams**:

```python
# WebRTC media encryption configuration
from aiortc import RTCPeerConnection, RTCSessionDescription

pc = RTCPeerConnection()

# Set prefered encryption method (end-to-end)
# Note: WebRTC encryption settings are limited by browser capabilities
configuration = RTCConfiguration(
    iceServers=[{
        "urls": "stun:stun.l.google.com:19302"
    }],
    iceTransportPolicy="relay"  # Force TURN for additional security
)

pc.setConfiguration(configuration)

# Add media tracks with RTCEncodingParameters for encryption hints
for transceiver in pc.getTransceivers():
    transceiver.sender.setParameters({
        "encodings": [{
            "maxBitrate": 500000,  # 500 kbps
            "scalabilityMode": "S1T3"  # Simulcast for adaptive quality
        }]
    })
```text

**HDF5 data encryption at rest**:

```python
# Encrypted HDF5 storage with cryptography
from cryptography.fernet import Fernet
import h5py
import hashlib

# Generate encryption key (in production, store securely)
key = Fernet.generate_key()
cipher = Fernet(key)

# Encrypt HDF5 file
def encrypt_hdf5(input_path, output_path):
    with open(input_path, 'rb') as f:
        encrypted_data = cipher.encrypt(f.read())

    with open(output_path, 'wb') as f:
        f.write(encrypted_data)

# Decrypt HDF5 file
def decrypt_hdf5(input_path, output_path):
    with open(input_path, 'rb') as f:
        decrypted_data = cipher.decrypt(f.read())

    with open(output_path, 'wb') as f:
        f.write(decrypted_data)

# Storage alternative: encrypted filesystem (LUKS on Linux)
# Recommended approach for production deployments
```text

#### Role-Based Access Control

```python
# FastAPI-based RBAC for XR platform
from fastapi.security import HTTPBearer, HTTPAuthorizationCredentials
from fastapi import Depends, HTTPException, status
import jwt

security = HTTPBearer()

# User roles and permissions
ROLE_PERMISSIONS = {
    "admin": [
        "create_meeting",
        "delete_meeting",
        "moderate_meeting",
        "export_meeting_data",
        "manage_users"
    ],
    "host": [
        "create_meeting",
        "moderate_meeting",
        "export_meeting_data"
    ],
    "participant": [
        "join_meeting",
        "mut_audio",
        "mut_video",
        "annotate_canvas",
        "draw_on_whiteboard"
    ],
    "guest": [
        "join_meeting",
        "view_content",
        "read_only_annotation"
    ]
}

# JWT token verification
def verify_token(credentials: HTTPAuthorizationCredentials = Depends(security)):
    token = credentials.credentials
    try:
        payload = jwt.decode(token, "your_secret_key", algorithms=["HS256"])
        return payload
    except jwt.ExpiredSignatureError:
        raise HTTPException(
            status_code=status.HTTP_401_UNAUTHORIZED,
            detail="Token expired"
        )
    except jwt.JWTError:
        raise HTTPException(
            status_code=status.HTTP_401_UNAUTHORIZED,
            detail="Invalid token"
        )

# Permission-based endpoint protection
@app.post("/meetings/{meeting_id}/canvas/update")
def update_canvas(
    meeting_id: str,
    update_data: CanvasUpdate,
    user: dict = Depends(verify_token)
):
    user_role = user.get("role", "guest")
    if "annotate_canvas" not in ROLE_PERMISSIONS[user_role]:
        raise HTTPException(
            status_code=status.HTTP_403_FORBIDDEN,
            detail="Permission denied"
        )

    # Update canvas in HDF5 file
    # ...

    return {"status": "success"}
```text

### Avatar System Implementation

#### Custom Avatar Mesh with Three.js

```javascript
// Avatar mesh configuration
class XRAvatar {
    constructor(config) {
        this.config = {
            height: 1.8,
            bodyWidth: 0.5,
            headRadius: 0.15,
            armLength: 0.6,
            handRadius: 0.08,
            legLength: 0.9,
color: config.color |  | 0x3498db,
            texture: config.texture,
            username: config.username,
            userType: config.userType // 'human' | 'bot'
        };

        this.mesh = new THREE.Group();
        this.createAvatarMesh();
    }

    createAvatarMesh() {
        // Body
        const bodyGeometry = new THREE.CapsuleGeometry(
            this.config.bodyWidth / 2,
            this.config.height * 0.6,
            8,
            16
        );
        const bodyMaterial = new THREE.MeshStandardMaterial({
            color: this.config.color,
            map: this.config.texture
        });
        const body = new THREE.Mesh(bodyGeometry, bodyMaterial);
        body.position.y = this.config.height * 0.5;
        this.mesh.add(body);

        // Head
        const headGeometry = new THREE.SphereGeometry(this.config.headRadius, 32, 32);
        const headMaterial = new THREE.MeshStandardMaterial({
            color: 0xffe0bd,  // Skin tone
            map: this.config.texture,
        });
        const head = new THREE.Mesh(headGeometry, headMaterial);
        head.position.y = this.config.height;
        this.mesh.add(head);

        // Arms
        const armGeometry = new THREE.CapsuleGeometry(
            this.config.handRadius,
            this.config.armLength,
            8,
            16
        );
        const armMaterial = new THREE.MeshStandardMaterial({
            color: 0xffe0bd,
            map: this.config.texture
        });

        const leftArm = new THREE.Mesh(armGeometry, armMaterial);
        leftArm.position.set(
            -this.config.bodyWidth / 2 - this.config.handRadius,
            this.config.height * 0.65,
            0
        );
        leftArm.rotation.z = Math.PI / 4;
        this.mesh.add(leftArm);

        const rightArm = new THREE.Mesh(armGeometry, armMaterial);
        rightArm.position.set(
            this.config.bodyWidth / 2 + this.config.handRadius,
            this.config.height * 0.65,
            0
        );
        rightArm.rotation.z = -Math.PI / 4;
        this.mesh.add(rightArm);

        // Hands
        const handGeometry = new THREE.SphereGeometry(this.config.handRadius, 16, 16);
        const handMaterial = new THREE.MeshStandardMaterial({
            color: 0xffe0bd,
            map: this.config.texture
        });

        const leftHand = new THREE.Mesh(handGeometry, handMaterial);
        leftHand.position.set(
            -this.config.bodyWidth / 2 - this.config.armLength * 0.7,
            this.config.height * 0.4,
            0.3
        );
        this.mesh.add(leftHand);

        const rightHand = new THREE.Mesh(handGeometry, handMaterial);
        rightHand.position.set(
            this.config.bodyWidth / 2 + this.config.armLength * 0.7,
            this.config.height * 0.4,
            0.3
        );
        this.mesh.add(rightHand);

        // Legs
        const legGeometry = new THREE.CapsuleGeometry(
            this.config.handRadius,
            this.config.legLength,
            8,
            16
        );
        const legMaterial = new THREE.MeshStandardMaterial({
            color: 0x34495e,
            map: this.config.texture
        });

        const leftLeg = new THREE.Mesh(legGeometry, legMaterial);
        leftLeg.position.set(-this.config.bodyWidth / 4, this.config.legLength / 2, 0);
        this.mesh.add(leftLeg);

        const rightLeg = new THREE.Mesh(legGeometry, legMaterial);
        rightLeg.position.set(this.config.bodyWidth / 4, this.config.legLength / 2, 0);
        this.mesh.add(rightLeg);

        // Name label (floating above head)
        const canvas = document.createElement('canvas');
        const context = canvas.getContext('2d');
        canvas.width = 256;
        canvas.height = 64;
        context.fillStyle = 'rgba(0,0,0,0.7)';
        context.fillRect(0, 0, canvas.width, canvas.height);
        context.fillStyle = 'white';
        context.font = '32px Arial';
        context.textAlign = 'center';
        context.fillText(this.config.username, 128, 48);

        const texture = new THREE.CanvasTexture(canvas);
        const labelGeometry = new THREE.PlaneGeometry(1.5, 0.4);
        const labelMaterial = new THREE.MeshBasicMaterial({
            map: texture,
            transparent: true,
            side: THREE.DoubleSide
        });
        const label = new THREE.Mesh(labelGeometry, labelMaterial);
        label.position.y = this.config.height + 0.3;
        label.rotation.x = 0.2; // Slight tilt towards viewer
        this.mesh.add(label);

        // User type indicator (different colors for bot vs. human)
        if (this.config.userType === 'bot') {
            bodyMaterial.color.setHex(0xe74c3c);  // Red for bots
        }
    }

    updatePosition(position) {
        this.mesh.position.set(position.x, position.y, position.z);
    }

    updateRotation(quaternion) {
        this.mesh.quaternion.set(quaternion.x, quaternion.y, quaternion.z, quaternion.w);
    }
}

// Create avatar from configuration
const avatarConfig = {
    username: 'Alice Johnson',
    color: 0x3498db,
    texture: null,
    userType: 'human'
};

const avatar = new XRAvatar(avatarConfig);
scene.add(avatar.mesh);
```text

#### Hand Tracking Integration

```javascript
// Hand tracking for natural interaction
const handLeft = renderer.xr.getController( 0 );
const handRight = renderer.xr.getController( 1 );

// Hand model loading
const handModelLeft = new XRHandModel( handLeft );

scene.add( handLeft );
scene.add( handModelLeft );
scene.add( handRight );

// Hand interaction: Grabbing objects
const raycaster = new THREE.Raycaster();
const grabDistance = 0.3;
let grabbedObject = null;

function handleHandInteraction(hand) {
    if (!hand.visible) return;

    const inputSource = hand.inputSource;
    const handPosition = hand.position.clone();
    const handRotation = hand.rotation.clone();

    // Check for intersection with 3D objects
    raycaster.set(handPosition, handRotation);
    const intersects = raycaster.intersectObjects(canvasObjects.children);

    // Grab detected objects
    if (inputSource.handedness === 'right' && inputSource.buttons[0].pressed) {
        if (intersects.length > 0 && intersects[0].distance < grabDistance) {
            grabbedObject = intersects[0].object;
            grabbedObject.position.copy(handPosition);
            grabbedObject.quaternion.copy(handRotation);
        }
    }

    // Update position if object is grabbed
    if (grabbedObject) {
        grabbedObject.position.copy(handPosition);
        grabbedObject.quaternion.copy(handRotation);

        // Release object if trigger released
        if (inputSource.handedness === 'right' && !inputSource.buttons[0].pressed) {
            grabbedObject = null;
        }
    }

    // Hand gesture: Index finger pointing for whiteboard annotation
    const thumb = handHand.getJointPoint('thumb-tip');
    const index = handHand.getJointPoint('index-finger-tip');

    if (inputSource.handedness === 'right' && inputSource.buttons[0].pressed) {
        const distance = thumb.position.distanceTo(index.position);

        // Pointing gesture (thumb and index fingers separated)
        if (distance > 0.1) {
            drawOnWhiteboard(index.position, handRotation);
        }
    }
}

// Whiteboard drawing function
function drawOnWhiteboard(position, rotation) {
    const raycaster = new THREE.Raycaster();
    raycaster.set(position, rotation);
    const intersects = raycaster.intersectObjects(whiteboardPlanes.children);

    if (intersects.length > 0) {
        const point = intersects[0].point;

        // Create 3D stroke on whiteboard
        const strokeGeometry = new THREE.TubeGeometry(
            new THREE.LineCurve3(
                new THREE.Vector3(lastPoint.x, lastPoint.y, lastPoint.z),
                new THREE.Vector3(point.x, point.y, point.z)
            ),
            1,
            0.005,
            8,
            false
        );
        const strokeMaterial = new THREE.MeshBasicMaterial({
            color: currentPenColor,
            transparent: true,
            opacity: 1
        });
        const stroke = new THREE.Mesh(strokeGeometry, strokeMaterial);
        whiteboardPlanes.add(stroke);

        // Send stroke data via WebSocket
        socket.send(JSON.stringify({
            type: 'whiteboard_stroke',
            point: { x: point.x, y: point.y, z: point.z },
            color: currentPenColor,
            thickness: currentPenThickness
        }));

        lastPoint = { x: point.x, y: point.y, z: point.z };
    }
}

// Register hand interaction callback
renderer.setAnimationLoop(() => {
    handleHandInteraction(handLeft);
    handleHandInteraction(handRight);
    renderer.render(scene, camera);
});
```text

### 3D Canvas Collaboration

#### Shared Whiteboard Implementation

```javascript
// 3D whiteboard plane
const whiteboardWidth = 4;
const whiteboardHeight = 3;
const whiteboardGeometry = new THREE.PlaneGeometry(whiteboardWidth, whiteboardHeight);
const whiteboardMaterial = new THREE.MeshPhysicalMaterial({
    color: 0xffffff,
    side: THREE.DoubleSide,
    roughness: 0.2,
    metalness: 0.1
});
const whiteboard = new THREE.Mesh(whiteboardGeometry, whiteboardMaterial);
whiteboard.position.set(0, 1.5, -2);
scene.add(whiteboard);

// Strokes on whiteboard (stored as separate meshes)
const whiteboardStrokes = new THREE.Group();
whiteboard.add(whiteboardStrokes);

// Whiteboard tools
const whiteboardTools = {
    currentColor: 0x000000,
    currentThickness: 0.02,
    eraserMode: false
};

// Whiteboard UI (in XR space)
const uiGeometry = new THREE.PlaneGeometry(0.8, 0.3);
const uiMaterial = new THREE.MeshBasicMaterial({
    color: 0x222222,
    transparent: true,
    opacity: 0.8
});
const uiPanel = new THREE.Mesh(uiGeometry, uiMaterial);
uiPanel.position.set(1.5, 1.8, -0.5);
scene.add(uiPanel);

// Add color picker buttons to UI
const colors = [0x000000, 0xff0000, 0x00ff00, 0x0000ff, 0xffff00, 0xff00ff, 0x00ffff];
const colorButtons = [];

colors.forEach((color, index) => {
    const buttonGeometry = new THREE.CircleGeometry(0.03, 32);
    const buttonMaterial = new THREE.MeshBasicMaterial({ color: color });
    const button = new THREE.Mesh(buttonGeometry, buttonMaterial);
    button.position.set(1.2 + index * 0.1, 1.9, -0.4);
    button.userData = { color: color };
    scene.add(button);
    colorButtons.push(button);
});

// Color selection interaction
const colorRaycaster = new THREE.Raycaster();

function handleColorSelection(handPosition, handRotation) {
    colorRaycaster.set(handPosition, handRotation);
    const intersects = colorRaycaster.intersectObjects(colorButtons);

    if (intersects.length > 0 && hand.inputSource.buttons[0].pressed) {
        whiteboardTools.currentColor = intersects[0].object.userData.color;
        whiteboardTools.eraserMode = false;
    }
}

// Eraser tool toggle
const eraserButtonGeometry = new THREE.CircleGeometry(0.05, 32);
const eraserButtonMaterial = new THREE.MeshBasicMaterial({ color: 0xffffff });
const eraserButton = new THREE.Mesh(eraserButtonGeometry, eraserButtonMaterial);
eraserButton.position.set(1.8, 1.75, -0.4);
scene.add(eraserButton);

// Eraser selection interaction
function handleEraserSelection(handPosition, handRotation) {
    colorRaycaster.set(handPosition, handRotation);
    const intersects = colorRaycaster.intersectObject(eraserButton);

    if (intersects.length > 0 && hand.inputSource.buttons[0].pressed) {
        whiteboardTools.eraserMode = !whiteboardTools.eraserMode;
        eraserButton.material.color.setHex(
            whiteboardTools.eraserMode ? 0xff0000 : 0xffffff
        );
    }
}

// Clear whiteboard button
const clearButtonGeometry = new THREE.PlaneGeometry(0.4, 0.1);
const clearButtonMaterial = new THREE.MeshBasicMaterial({ color: 0x888888 });
const clearButton = new THREE.Mesh(clearButtonGeometry, clearButtonMaterial);
clearButton.position.set(1.5, 1.6, -0.4);
scene.add(clearButton);

function handleClearWhiteboard(handPosition, handRotation) {
    colorRaycaster.set(handPosition, handRotation);
    const intersects = colorRaycaster.intersectObject(clearButton);

    if (intersects.length > 0 && hand.inputSource.buttons[0].pressed) {
        // Remove all strokes
        while (whiteboardStrokes.children.length > 0) {
            whiteboardStrokes.remove(whiteboardStrokes.children[0]);
        }

        // Send clear command via WebSocket
        socket.send(JSON.stringify({
            type: 'whiteboard_clear',
            meetingId: meetingId
        }));
    }
}

// Register UI interaction callbacks
renderer.setAnimationLoop(() => {
    handleColorSelection(handLeft.position, handLeft.rotation);
    handleEraserSelection(handLeft.position, handLeft.rotation);
    handleClearWhiteboard(handLeft.position, handLeft.rotation);

    renderer.render(scene, camera);
});
```text

#### 3D Object Manipulation Canvas

```javascript
// 3D canvas for shared object manipulation
const canvas3DGeometry = new THREE.BoxGeometry(3, 2, 0.5);
const canvas3DMaterial = new THREE.MeshPhysicalMaterial({
    color: 0xf0f0f0,
    side: THREE.DoubleSide,
    transparent: true,
    opacity: 0.9,
    roughness: 0.3,
    metalness: 0.1
});
const canvas3D = new THREE.Mesh(canvas3DGeometry, canvas3DMaterial);
canvas3D.position.set(0, 1.5, -1);
scene.add(canvas3D);

// Ground plane for positioning objects
const groundGeometry = new THREE.PlaneGeometry(5, 5);
const groundMaterial = new THREE.MeshStandardMaterial({
    color: 0xeeeeee,
    side: THREE.DoubleSide
});
const ground = new THREE.Mesh(groundGeometry, groundMaterial);
ground.rotation.x = -Math.PI / 2;
ground.position.y = 0;
scene.add(ground);

// Object palette (spawnable objects)
const objectPalette = [
    { type: 'box', geometry: new THREE.BoxGeometry(0.5, 0.5, 0.5) },
    { type: 'sphere', geometry: new THREE.SphereGeometry(0.3, 32, 32) },
    { type: 'cylinder', geometry: new THREE.CylinderGeometry(0.2, 0.2, 0.5, 32) },
    { type: 'cone', geometry: new THREE.ConeGeometry(0.3, 0.5, 32) }
];

const paletteIcons = [];
objectPalette.forEach((obj, index) => {
    const icon = new THREE.Mesh(obj.geometry, new THREE.MeshStandardMaterial({ color: 0x3498db }));
    icon.position.set(2.5 + index * 0.7, 0.25, -0.5);
    icon.userData = { objectType: obj.type };
    scene.add(icon);
    paletteIcons.push(icon);
});

// Object spawning
function spawnObject(objectType, position) {
    const paletteObj = objectPalette.find(obj => obj.type === objectType);
    if (!paletteObj) return;

    const material = new THREE.MeshStandardMaterial({ color: 0x3498db });
    const spawnedObject = new THREE.Mesh(paletteObj.geometry.clone(), material);
    spawnedObject.position.copy(position);
    spawnedObject.position.y = 0.3; // Start above ground
    canvas3D.add(spawnedObject);

    // Send object spawn via WebSocket
    socket.send(JSON.stringify({
        type: 'object_spawn',
        objectType: objectType,
        position: spawnedObject.position.toArray(),
        rotation: spawnedObject.quaternion.toArray()
    }));

    return spawnedObject;
}

// Object manipulation (grabbing and moving)
const grabbedObject = null;
const grabRaycaster = new THREE.Raycaster();

function handleObjectManipulation(handPosition, handRotation) {
    if (!hand.visible) return;

    grabRaycaster.set(handPosition, handRotation);

    // Check for palette object interaction (spawning)
    const paletteIntersects = grabRaycaster.intersectObjects(paletteIcons);
    if (paletteIntersects.length > 0 && paletteIntersects[0].distance < 0.5) {
        if (hand.inputSource.buttons[0].pressed) {
            spawnObject(
                paletteIntersects[0].object.userData.objectType,
                handPosition.clone()
            );
        }
    }

    // Check for spawned object interaction (grabbing)
    if (!grabbedObject) {
        const objectIntersects = grabRaycaster.intersectObjects(canvas3D.children);
        if (objectIntersects.length > 0 && objectIntersects[0].distance < 1.0) {
            if (hand.inputSource.buttons[0].pressed) {
                grabbedObject = objectIntersects[0].object;
                grabbedObject.material.color.setHex(0xe74c3c); // Change to red when grabbed
            }
        }
    }

    // Move grabbed object
    if (grabbedObject) {
        grabbedObject.position.copy(handPosition);

        // Update server
        socket.send(JSON.stringify({
            type: 'object_move',
            objectId: grabbedObject.uuid,
            position: grabbedObject.position.toArray(),
            rotation: grabbedObject.quaternion.toArray()
        }));

        // Release object if trigger released
        if (!hand.inputSource.buttons[0].pressed) {
            grabbedObject.material.color.setHex(0x3498db); // Reset to blue
            grabbedObject = null;
        }
    }
}

// Register object manipulation callback
renderer.setAnimationLoop(() => {
    handleObjectManipulation(handLeft.position, handLeft.rotation);
    handleObjectManipulation(handRight.position, handRight.rotation);

    renderer.render(scene, camera);
});
```text

## Use Case Specific Implementations

### Healthcare: Telehealth Classroom

**Adaptations for medical training**:

```javascript
// Medical imaging texture loader for XR whiteboard
function loadMedicalImage(imageUrl) {
    const textureLoader = new THREE.TextureLoader();
    textureLoader.load(imageUrl, (texture) => {
        const imageGeometry = new THREE.PlaneGeometry(1.5, 1.0);
        const imageMaterial = new THREE.MeshBasicMaterial({ map: texture });
        const medicalImage = new THREE.Mesh(imageGeometry, imageMaterial);
        medicalImage.position.set(-1.5, 1.8, -1.98);
        medicalImage.name = `medical_${Date.now()}`;
        whiteboard.add(medicalImage);

        // Allow manipulation (pan, zoom) of medical images
        makeInteractable(medicalImage);
    });
}

// Surgical planning annotations
const surgicalAnnotationTool = {
    toolType: 'annotation',
    allowedShapes: ['point', 'line', 'arrow', 'text'],
    colors: {
        incision: 0xff0000,
        landmark: 0x00ff00,
        pathology: 0x0000ff,
        note: 0xffff00
    }
};

// VRFDA-compatible DICOM viewer integration
// (DICOM = Digital Imaging and Communications in Medicine)
function initializeDICOMViewer() {
    const dicomViewer = new DICOMViewer({
        baseTexture: whiteboard.material.map,
        overlayTexture: annotationTexture
    });

    // Add DICOM slices to whiteboard
    dicomViewer.loadSeries('patient123', 'mri_brain')
        .then(() => {
            console.log('MRI series loaded');
        });
}
```text

**Regulatory compliance (HIPAA)**:

```python
# HIPAA-compliant data handling
from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes
import os

def encrypt_meeting_audio(audio_data):
    """Encrypt meeting audio with AES-256-GCM (NIST-approved)"""
    key = get_encryption_key()  # From secure KMS or hardware token
    iv = os.urandom(12)

    cipher = Cipher(
        algorithms.AES(key),
        modes.GCM(iv)
    )
    encryptor = cipher.encryptor()

    encrypted_audio = encryptor.update(audio_data) + encryptor.finalize()
    auth_tag = encryptor.tag

    return {
        'iv': iv,
        'auth_tag': auth_tag,
        'ciphertext': encrypted_audio
    }

def audit_meeting_access(meeting_id, user_id, action):
    """Log all meeting access for HIPAA audit requirements"""
    audit_entry = {
        'meeting_id': meeting_id,
        'user_id': user_id,
        'action': action,  # 'join', 'speak', 'annotate', 'export'
        'timestamp': datetime.utcnow().isoformat(),
        'ip_address': get_client_ip()
    }

    # Store in audit log (immutable storage required)
    append_to_audit_trail(audit_entry)
```text

### Education: Virtual Science Laboratories

**Interactive lab equipment simulation**:

```javascript
// Beaker simulation for chemistry labs
function createBeaker(position, volume, liquidColor) {
    const beakerGroup = new THREE.Group();

    // Glass beaker
    const beakerGeometry = new THREE.CylinderGeometry(0.2, 0.2, 0.3, 32, 1, true);
    const glassMaterial = new THREE.MeshPhysicalMaterial({
        color: 0xffffff,
        transparent: true,
        opacity: 0.4,
        roughness: 0.1,
        metalness: 0.2,
        side: THREE.DoubleSide
    });
    const beaker = new THREE.Mesh(beakerGeometry, glassMaterial);
    beakerGroup.add(beaker);

    // Liquid inside beaker
    const liquidHeight = volume / 100 * 0.25;  // Max 100mL, beaker height 0.25
    const liquidGeometry = new THREE.CylinderGeometry(0.18, 0.18, liquidHeight, 32);
    const liquidMaterial = new THREE.MeshPhysicalMaterial({
        color: liquidColor,
        transparent: true,
        opacity: 0.8,
        roughness: 0.2,
        metalness: 0.1
    });
    const liquid = new THREE.Mesh(liquidGeometry, liquidMaterial);
    liquid.position.y = -0.1 + liquidHeight / 2;
    beakerGroup.add(liquid);

    beakerGroup.position.copy(position);
    beakerGroup.userData = {
        type: 'beaker',
        volume: volume,
        liquidColor: liquidColor
    };

    canvas3D.add(beakerGroup);
    return beakerGroup;
}

// Chemical reaction simulation (physics engine needed)
function simulateChemicalReaction(beaker1, beaker2) {
    const reaction = {
        reagents: [
            { beaker: beaker1, volume: 50 },
            { beaker: beaker2, volume: 50 }
        ],
        products: [],
        reactionType: 'exothermic'  // or 'endothermic'
    };

    // Determine reaction based on types
    // (simplified for demo)
    if (isAcidBaseReaction(beaker1, beaker2)) {
        reaction.products = [{
            type: 'salt',
            volume: 100,
            color: 0xeeeeee,  // White precipitate
            heat: 50  // Temperature rise in Kelvin
        }];
    }

    return reaction;
}

// Physics engine integration for container spills
function simulateFluidDynamics(container, spillVolume) {
    // Use physics engine (e.g., Ammo.js, Cannon.js)
    // for realistic liquid simulation
    const physicsEngine = new PhysicsEngine();
    const fluidParticles = createFluidParticles(spillVolume);

    physicsEngine.addBodies(fluidParticles);
    physicsEngine.simulate();  // Simulate for X seconds

    return physicsEngine.getResult();
}
```text

**Student assessment and feedback**:

```java
// Java backend for student assessment
@RestController
@RequestMapping("/api/labs")
public class VRAssessmentController {

    @PostMapping("/submit")
    public LabResult submitLabResult(@RequestBody LabSubmission submission) {
        // Validate lab protocol execution
        boolean protocolFollowed = validateProtocol(
            submission.getSteps(),
            submission.getResults()
        );

        // Grade based on experimental accuracy
        double accuracy = calculateAccuracy(
            submission.getMeasurements(),
            getExpectedMeasurements(submission.getLabId())
        );

        // Generate AI-powered feedback
        String feedback = generateFeedback(
            submission,
            protocolFollowed,
            accuracy
        );

        LabResult result = new LabResult();
        result.setProtocolFollowed(protocolFollowed);
        result.setAccuracy(accuracy);
        result.setFeedback(feedback);
        result.setTimestamp(Instant.now());

        return result;
    }
}
```text

### Manufacturing: Digital Twins for Facility Planning

**Facility 3D model integration**:

```javascript
// Load facility BIM model into XR workspace
function loadFacilityModel(modelUrl) {
    const gltfLoader = new GLTFLoader();
    const dracoLoader = new DRACOLoader();

    dracoLoader.setDecoderPath('/draco/');  // Path to Draco decoder
    gltfLoader.setDRACOLoader(dracoLoader);

    gltfLoader.load(modelUrl, (gltf) => {
        const facilityScene = gltf.scene;

        // Scale down to fit in XR canvas
        const boundingBox = new THREE.Box3().setFromObject(facilityScene);
        const size = boundingBox.getSize(new THREE.Vector3());
        const maxDim = Math.max(size.x, size.y, size.z);
        const scale = 3 / maxDim;  // Fit in 3-unit space
        facilityScene.scale.setScalar(scale);

        // Position in front of user
        facilityScene.position.set(0, 1.5 - boundingBox.min.y * scale, -2);
        facilityScene.rotation.y = -Math.PI / 2;  // Rotate for better viewing

        // Enable interaction with individual facility components
        facilityScene.traverse((child) => {
            if (child.isMesh) {
                child.userData = {
                    type: 'facility_component',
                    componentId: child.name,
                    draggable: true
                };
            }
        });

        canvas3D.add(facilityScene);
    });
}

// Facility optimization analysis (simulated)
function analyzeFacilityLayout(facilityModel) {
    const analysis = {
        walkwayCongestion: [],
        equipmentEfficiency: [],
        spaceUtilization: 0,
        recommendations: []
    };

    // Analyze walkway traffic patterns
    // (would use BIM metadata + RFID data from real facility)
    facilityModel.traverse((child) => {
        if (child.userData?.componentId?.startsWith('walkway_')) {
            const congestionScore = calculateWalkwayCongestion(
                child.userData.componentId
            );
            analysis.walkwayCongestion.push({
                componentId: child.userData.componentId,
                congestionScore: congestionScore
            });
        }
    });

    // Calculate overall space utilization
    analysis.spaceUtilization = calculateTotalUtilization(facilityModel);

    // Generate recommendations
    analysis.recommendations = generateOptimizationRecommendations(
        analysis.walkwayCongestion,
        analysis.equipmentEfficiency
    );

    return analysis;
}

// Equipment heat map visualization
function visualizeEquipmentEfficiency(facilityModel, efficiencyData) {
    // Color-code equipment based on efficiency
    const heatmapColormap = [
        { threshold: 0.5, color: 0xff0000 },      // Red for low efficiency
        { threshold: 0.75, color: 0xffff00 },     // Yellow for medium
        { threshold: 1.0, color: 0x00ff00 }       // Green for high
    ];

    facilityModel.traverse((child) => {
        if (child.userData?.componentId?.startsWith('equipment_')) {
            const equipmentId = child.userData.componentId;
const efficiency = efficiencyData[equipmentId] |  | 0.5;

            // Find color based on efficiency
            const colormapEntry = heatmapColormap.find(
                entry => efficiency <= entry.threshold
) |  | heatmapColormap[heatmapColormap.length - 1];

            child.material.color.setHex(colormapEntry.color);

            // Add tooltip with efficiency data
            const tooltip = createTooltip3D(
                `${efficiency * 100}% efficient`,
                child.position.clone().add(new THREE.Vector3(0, 0.5, 0))
            );
            canvas3D.add(tooltip);
        }
    });
}
```text

## Scaling and Performance Optimization

### Horizontal Scaling Strategy

**Multi-server deployment with session affinity**:

```yaml
# Docker Compose HA configuration
version: '3.8'

services:
  # Load balancer with session affinity
  xr-lb:
    image: traefik:latest
    command:
      - --api.insecure=true
      - --providers.docker=true
      - --entrypoints.web.address=:80
      - --entrypoints.websecure.address=:443
      - --providers.loadbalancer.stickiness=true
      - --providers.loadbalancer.stickiness.cookieName=XR_SESSION
    ports:
      - "80:80"
      - "443:443"
      - "8080:8080"
    networks:
      - xr-network
    restart: unless-stopped

  # XR backend instance 1
  xr-backend-1:
    build: ./xr-backend
    ports:
      - "8001:8000"
    volumes:
      - ./hdf5-data:/hdf5-data
    environment:
      - INSTANCE_ID=1
      - REDIS_URL=redis://redis:6379
    deploy:
      replicas: 1
      placement:
        constraints:
          - node.labels.xr-pool == primary
    networks:
      - xr-network
      - redis-network
    restart: unless-stopped

  # XR backend instance 2
  xr-backend-2:
    build: ./xr-backend
    ports:
      - "8002:8000"
    volumes:
      - ./hdf5-data:/hdf5-data
    environment:
      - INSTANCE_ID=2
      - REDIS_URL=redis://redis:6379
    deploy:
      replicas: 1
      placement:
        constraints:
          - node.labels.xr-pool == secondary
    networks:
      - xr-network
      - redis-network
    restart: unless-stopped

  # Redis for distributed session management
  redis:
    image: redis:7
    command: redis-server --cluster-enabled yes
    networks:
      - redis-network
    restart: unless-stopped

networks:
  xr-network:
    driver: bridge
  redis-network:
    driver: bridge
```text

### Media Optimization

**Adaptive bitrate streaming for video feeds**:

```javascript
// Adaptive bitrate selector
class AdaptiveBitrateSelector {
    constructor() {
        this.currentBitrate = 2000;  // Start at 2 Mbps
        this.bitrateLevels = [500, 1000, 2000, 4000, 8000];
        this.currentIndex = 2;  // Index for 2000 kbps
        this.lastQualityChange = Date.now();
        this.minQualityChangeInterval = 5000;  // 5 seconds
    }

    evaluateNetworkConditions(networkMetrics) {
        const now = Date.now();
        const timeSinceLastChange = now - this.lastQualityChange;

        if (timeSinceLastChange < this.minQualityChangeInterval) {
            return;  // Don't change quality too quickly
        }

        const bandwidth = networkMetrics.bandwidth;
        const latency = networkMetrics.latency;
        const packetLoss = networkMetrics.packetLoss;

        if (bandwidth > 4000 && latency < 100 && packetLoss < 0.01) {
            // Excellent network: can increase bitrate
            this.increaseBitrate();
} else if (bandwidth < 1000 |  | latency > 300 |  | packetLoss > 0.05) {
            // Poor network: decrease bitrate
            this.decreaseBitrate();
        }

        this.lastQualityChange = now;
        return this.currentBitrate;
    }

    increaseBitrate() {
        const nextIndex = Math.min(
            this.currentIndex + 1,
            this.bitrateLevels.length - 1
        );

        if (this.currentIndex !== nextIndex) {
            this.currentIndex = nextIndex;
            this.currentBitrate = this.bitrateLevels[nextIndex];
            console.log(`Increased bitrate to ${this.currentBitrate} kbps`);
        }
    }

    decreaseBitrate() {
        const previousIndex = Math.max(
            this.currentIndex - 1,
            0
        );

        if (this.currentIndex !== previousIndex) {
            this.currentIndex = previousIndex;
            this.currentBitrate = this.bitrateLevels[previousIndex];
            console.log(`Decreased bitrate to ${this.currentBitrate} kbps`);
        }
    }
}

// Video stream player with adaptive bitrate
function createAdaptiveVideoPlayer(videoElement, streamUrl) {
    const player = new VideoPlayer(videoElement, {
        streaming: {
            enableAutoAdaptation: true,
            initialBitrate: 2000,
            adaptiveBitrateSelector: new AdaptiveBitrateSelector()
        }
    });

    player.load(streamUrl);

    // Monitor network metrics
    setInterval(() => {
        const networkMetrics = {
            bandwidth: player.getBandwidthEstimate(),
            latency: player.getRtt(),
            packetLoss: player.getPacketLossRate()
        };

        const newBitrate = player.options.streaming.adaptiveBitrateSelector.evaluateNetworkConditions(
            networkMetrics
        );

        if (newBitrate !== player.currentBitrate) {
            player.switchBitrate(newBitrate);
        }
    }, 1000);  // Evaluate every second

    return player;
}
```text

### HDF5 Optimization

**Chunked storage for high-frequency updates**:

```python
# Optimized HDF5 chunking strategy
import h5py
import numpy as np

# Create HDF5 file with optimized chunking
with h5py.File('meeting_state_optimized.h5', 'w') as f:
    # Avatar positions chunked by time (optimal for append-only writes)
    avatar_positions = f.create_dataset(
        'avatars/positions',
        shape=(0, 5),  # timestamp, user_id, x, y, z
        maxshape=(None, 5),
        dtype='float64',
        chunks=(1000, 5),  # Optimal for time-series data
        compression='gzip',
        compression_opts=4
    )

    # Whiteboard strokes chunked by stroke_id (optimal for retrieval by stroke)
    whiteboard_strokes = f.create_dataset(
        'whiteboard/strokes',
        shape=(0, 8),  # timestamp, user_id, stroke_id, x, y, color, thickness, line_width
        maxshape=(None, 8),
        dtype='float64',
        chunks=(1000, 8),  # Optimized for stroke writes
        compression='lz4',  # Faster compression for high-frequency data
        shuffle=True,  # Improve compression ratio
        fletcher32=True  # Integrity checking
    )

    # Create index for fast stroke lookup
    f.create_dataset(
        'whiteboard/stroke_index',
        shape=(0, 2),
        maxshape=(None, 2),
        dtype='int64',
        chunks=(100, 2)  # Smaller chunks for index data
    )
```text

## Conclusion: The Immersive Collaboration Future

Self-hosted XR collaboration platforms offer organizations unprecedented control over immersive meeting spaces while ensuring data sovereignty and regulatory compliance. By leveraging open-source standards (WebXR, HDF5, Three.js), organizations can deploy scalable, secure virtual meeting infrastructure without vendor lock-in.

The journey begins with foundational infrastructure (Docker Swarm, Traefik, authentication) and expands to domain-specific implementations (healthcare telehealth, education labs, manufacturing digital twins). The return on investment increases rapidly as user count grows, making self-hosted XR economically competitive for organizations with 50+ regular users.

Organizations that invest in self-hosted XR collaboration today enjoy advantages in:

- **Data Sovereignty**: Complete control over meeting data, IP protection
- **Customization**: Branded avatar systems, domain-specific collaboration tools
- **Compliance**: Built-in GDPR, HIPAA, and regulatory support
- **Innovation**: Freedom to integrate with existing systems and customize workflows

The immersive collaboration revolution is underway—will your organization lead or follow?

---

*This article is part of the XR Collaboration Series on tobias-weiss.org, exploring immersive meeting technologies and self-hosted implementations.*