XR-Enhanced Collaboration: Self-Hosted Virtual Meeting Infrastructure
~5 min read
XR/VRXR CollaborationSelf-Hosted Virtual MeetingsVR MeetingsDigital TwinHealthcare XREducation XRHDF5 CollaborationVirtual RealityAugmented Reality3D Visualization
Save products you love by clicking the heart icon.
Extended Reality (XR) technologies are transforming team collaboration, enabling immersive virtual meeting spaces that transcend geographical boundaries. This article explores how self-hosted XR platforms provide organizations with control over collaboration data, infrastructure sovereignty, and regulatory compliance. We present a practical framework for implementing XR-enhanced collaboration with custom avatar systems, shared 3D workspaces, and whiteboarding capabilities.
Key Takeaways:
Self-hosted XR platforms ensure data sovereignty while enabling immersive collaboration
HDF5-based collaboration engines enable real-time 3D canvas sharing and whiteboarding
Custom avatar systems with WebGL, Three.js provide engaging, branded meeting experiences
The total cost of ownership for self-hosted XR becomes competitive at scale (50+ users)
Self-hosted XR platforms ensure data sovereignty while enabling immersive collaboration
HDF5-based collaboration engines enable real-time 3D canvas sharing and whiteboarding
Custom avatar systems with WebGL and Three.js provide engaging, branded meeting experiences
The total cost of ownership for self-hosted XR becomes competitive at scale (50+ users)
Traditional video conferencing limitations:
XR-enhanced collaboration advantages:
Corporate adoption acceleration:
Industry-specific drivers:
| Industry | XR Collaboration Use Cases | Key Benefits |
|---|---|---|
| Healthcare | Medical training, surgical planning, patient consultations | Reduced training costs, expertise sharing, patient outcomes |
| Education | Virtual classrooms, laboratory simulations, field trips | Experiential learning, resource optimization, global access |
| Manufacturing | Digital twins, facility planning, remote maintenance | Reduced design errors, faster time-to-market, safety |
| Architecture/Construction | Virtual site walkthroughs, client presentations, design reviews | Stakeholder alignment, reduced rework, cost savings |
| Finance | Interactive data visualization, portfolio reviews, scenario planning | Improved analysis, faster decisions, client engagement |
Technology maturation drivers:
Why self-hosted XR platforms?
Data Sovereignty:
Security:
Customization:
Compliance:
Browser-based client using WebXR API and Three.js:
// WebXR initialization
const button = document.querySelector('#enter-xr');
button.addEventListener('click', async () => {
const supported = await navigator.xr.isSessionSupported('immersive-vr');
if (supported) {
const xrSession = await navigator.xr.requestSession('immersive-vr', {
optionalFeatures: ['local-floor', 'bounded-floor', 'hand-tracking']
});
xrSession.addEventListener('end', onSessionEnd);
xrSession.updateRenderState({ baseLayer: baseLayer });
await renderer.xr.setSession(xrSession);
} else {
Fallback: Show 2D interface for non-VR users
}
});
// Three.js scene setup for virtual meeting room
const scene = new THREE.Scene();
const renderer = new THREE.WebGLRenderer({ antialias: true });
renderer.setSize(window.innerWidth, window.innerHeight);
renderer.xr.enabled = true;
// 3D avatar mesh
const avatarGeometry = new THREE.CapsuleGeometry(0.3, 1.8, 4, 8);
const avatarMaterial = new THREE.MeshStandardMaterial({ color: 0x3498db });
const avatar = new THREE.Mesh(avatarGeometry, avatarMaterial);
scene.add(avatar);
// Spatial audio for voice pickup from avatar position
const positionalAudio = new THREE.PositionalAudio(listener);
avatar.add(positionalAudio);
```text
**Key frontend capabilities**:
- **Avatar synchronization**: Real-time 3D position updates for all meeting participants
- **Spatial audio**: Voice emanates from avatar position in 3D space
- **Shared 3D canvas**: Interactive 3D workspaces for collaborative annotation
- **Whiteboarding**: 2D and 3D drawing surfaces with pencil-like input
- **Gesture recognition**: Hand tracking for natural 3D object manipulation
#### Backend Layer (HDF5 Collaboration Engine)
**HDF5 (Hierarchical Data Format)** as collaboration foundation:
```python
import h5py
import numpy as np
# Open HDF5 file for meeting state
with h5py.File('meeting_state_20260402.h5', 'w') as f:
# Create groups for meeting data
avatars = f.create_group('avatars')
canvas = f.create_group('canvas')
whiteboard = f.create_group('whiteboard')
# Avatar positions (timestamp, user_id, x, y, z)
avatar_positions = avatars.create_dataset(
'positions',
shape=(0, 4), # timestamp, user_id, x, y, z
maxshape=(None, 4),
dtype='float64'
)
# Canvas state (timestamp, object_id, x, y, z, rotation)
canvas_objects = canvas.create_dataset(
'objects',
shape=(0, 6), # timestamp, object_id, x, y, z, rotation
maxshape=(None, 6),
dtype='float64'
)
# Whiteboard strokes (timestamp, user_id, stroke_id, x, y, color, thickness)
whiteboard_strokes = whiteboard.create_dataset(
'strokes',
shape=(0, 7), # timestamp, user_id, stroke_id, x, y, color, thickness
maxshape=(None, 7),
dtype='float64'
)
# Meeting metadata
f.attrs['meeting_id'] = 'meeting_20260402_091532'
f.attrs['start_time'] = '2026-04-02T09:15:32Z'
f.attrs['host_user_id'] = 'user_001'
```text
**Real-time collaboration via WebSocket**:
```python
# WebSocket server for real-time state synchronization
from fastapi import FastAPI, WebSocket
from fastapi.middleware.cors import CORSMiddleware
import h5py
import json
import asyncio
app = FastAPI()
# CORS configuration for WebXR client access
app.add_middleware(
CORSMiddleware,
allow_origins=["https://www.tobias-weiss.org"],
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
# Open HDF5 file for concurrent access (read-write)
h5_file = h5py.File('meeting_state.h5', 'r+')
@app.websocket("/ws/meeting/{meeting_id}")
async def websocket_endpoint(websocket: WebSocket, meeting_id: str):
await websocket.accept()
while True:
# Receive client updates
data = await websocket.receive_json()
# Update HDF5 state based on update type
if data['type'] == 'avatar_position':
h5_file['avatars']['positions'].resize((h5_file['avatars']['positions'].shape[0] + 1), axis=0)
h5_file['avatars']['positions'][-1] = [data['timestamp'], data['user_id'], data['x'], data['y'], data['z']]
h5_file.flush() # Persist to disk
elif data['type'] == 'whiteboard_stroke':
h5_file['whiteboard']['strokes'].resize((h5_file['whiteboard']['strokes'].shape[0] + 1), axis=0)
h5_file['whiteboard']['strokes'][-1] = [data['timestamp'], data['user_id'], data['stroke_id'], data['x'], data['y'], data['color'], data['thickness']]
h5_file.flush()
# Broadcast updated state to all connected clients
updated_state = {
'avatars': h5_file['avatars']['positions'][-10:].tolist(), # Last 10 positions
'whiteboard': h5_file['whiteboard']['strokes'][-10:].tolist() # Last 10 strokes
}
await websocket.send_json(updated_state)
```text
#### Infrastructure Layer (Container Orchestration)
**Docker Compose configuration for XR platform**:
```yaml
version: '3.8'
services:
# FastAPI backend for XR collaboration
xr-backend:
build: ./xr-backend
ports:
- "8000:8000"
volumes:
- ./hdf5-data:/hdf5-data
environment:
- HDF5_DATA_PATH=/hdf5-data/meeting_state.h5
- CLOUD_STORAGE_API_KEY=your_cloud_storage_key
networks:
- xr-network
restart: unless-stopped
# Traefik reverse proxy for SSL termination
traefik:
image: traefik:latest
command:
- --api.insecure=true
- --providers.docker=true
- --entrypoints.web.address=:80
- --entrypoints.websecure.address=:443
ports:
- "80:80"
- "443:443"
- "8080:8080"
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
- ./traefik/letsencrypt:/letsencrypt
networks:
- xr-network
restart: unless-stopped
# PostgreSQL for meeting metadata and user authentication
postgres:
image: postgres:15
volumes:
- postgres-data:/var/lib/postgresql/data
environment:
- POSTGRES_DB=xr_collaboration
- POSTGRES_USER=xr_user
- POSTGRES_PASSWORD=secure_password_here
networks:
- xr-network
restart: unless-stopped
# Redis for session management and real-time messaging
redis:
image: redis:7
command: redis-server --appendonly yes
volumes:
- redis-data:/data
networks:
- xr-network
restart: unless-stopped
volumes:
hdf5-data:
postgres-data:
redis-data:
networks:
xr-network:
driver: bridge
```text
### Security and Compliance Design
#### End-to-End Encryption
**Media encryption for video/audio streams**:
```python
# WebRTC media encryption configuration
from aiortc import RTCPeerConnection, RTCSessionDescription
pc = RTCPeerConnection()
# Set prefered encryption method (end-to-end)
# Note: WebRTC encryption settings are limited by browser capabilities
configuration = RTCConfiguration(
iceServers=[{
"urls": "stun:stun.l.google.com:19302"
}],
iceTransportPolicy="relay" # Force TURN for additional security
)
pc.setConfiguration(configuration)
# Add media tracks with RTCEncodingParameters for encryption hints
for transceiver in pc.getTransceivers():
transceiver.sender.setParameters({
"encodings": [{
"maxBitrate": 500000, # 500 kbps
"scalabilityMode": "S1T3" # Simulcast for adaptive quality
}]
})
```text
**HDF5 data encryption at rest**:
```python
# Encrypted HDF5 storage with cryptography
from cryptography.fernet import Fernet
import h5py
import hashlib
# Generate encryption key (in production, store securely)
key = Fernet.generate_key()
cipher = Fernet(key)
# Encrypt HDF5 file
def encrypt_hdf5(input_path, output_path):
with open(input_path, 'rb') as f:
encrypted_data = cipher.encrypt(f.read())
with open(output_path, 'wb') as f:
f.write(encrypted_data)
# Decrypt HDF5 file
def decrypt_hdf5(input_path, output_path):
with open(input_path, 'rb') as f:
decrypted_data = cipher.decrypt(f.read())
with open(output_path, 'wb') as f:
f.write(decrypted_data)
# Storage alternative: encrypted filesystem (LUKS on Linux)
# Recommended approach for production deployments
```text
#### Role-Based Access Control
```python
# FastAPI-based RBAC for XR platform
from fastapi.security import HTTPBearer, HTTPAuthorizationCredentials
from fastapi import Depends, HTTPException, status
import jwt
security = HTTPBearer()
# User roles and permissions
ROLE_PERMISSIONS = {
"admin": [
"create_meeting",
"delete_meeting",
"moderate_meeting",
"export_meeting_data",
"manage_users"
],
"host": [
"create_meeting",
"moderate_meeting",
"export_meeting_data"
],
"participant": [
"join_meeting",
"mut_audio",
"mut_video",
"annotate_canvas",
"draw_on_whiteboard"
],
"guest": [
"join_meeting",
"view_content",
"read_only_annotation"
]
}
# JWT token verification
def verify_token(credentials: HTTPAuthorizationCredentials = Depends(security)):
token = credentials.credentials
try:
payload = jwt.decode(token, "your_secret_key", algorithms=["HS256"])
return payload
except jwt.ExpiredSignatureError:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Token expired"
)
except jwt.JWTError:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Invalid token"
)
# Permission-based endpoint protection
@app.post("/meetings/{meeting_id}/canvas/update")
def update_canvas(
meeting_id: str,
update_data: CanvasUpdate,
user: dict = Depends(verify_token)
):
user_role = user.get("role", "guest")
if "annotate_canvas" not in ROLE_PERMISSIONS[user_role]:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Permission denied"
)
# Update canvas in HDF5 file
# ...
return {"status": "success"}
```text
### Avatar System Implementation
#### Custom Avatar Mesh with Three.js
```javascript
// Avatar mesh configuration
class XRAvatar {
constructor(config) {
this.config = {
height: 1.8,
bodyWidth: 0.5,
headRadius: 0.15,
armLength: 0.6,
handRadius: 0.08,
legLength: 0.9,
color: config.color | | 0x3498db,
texture: config.texture,
username: config.username,
userType: config.userType // 'human' | 'bot'
};
this.mesh = new THREE.Group();
this.createAvatarMesh();
}
createAvatarMesh() {
// Body
const bodyGeometry = new THREE.CapsuleGeometry(
this.config.bodyWidth / 2,
this.config.height * 0.6,
8,
16
);
const bodyMaterial = new THREE.MeshStandardMaterial({
color: this.config.color,
map: this.config.texture
});
const body = new THREE.Mesh(bodyGeometry, bodyMaterial);
body.position.y = this.config.height * 0.5;
this.mesh.add(body);
// Head
const headGeometry = new THREE.SphereGeometry(this.config.headRadius, 32, 32);
const headMaterial = new THREE.MeshStandardMaterial({
color: 0xffe0bd, // Skin tone
map: this.config.texture,
});
const head = new THREE.Mesh(headGeometry, headMaterial);
head.position.y = this.config.height;
this.mesh.add(head);
// Arms
const armGeometry = new THREE.CapsuleGeometry(
this.config.handRadius,
this.config.armLength,
8,
16
);
const armMaterial = new THREE.MeshStandardMaterial({
color: 0xffe0bd,
map: this.config.texture
});
const leftArm = new THREE.Mesh(armGeometry, armMaterial);
leftArm.position.set(
-this.config.bodyWidth / 2 - this.config.handRadius,
this.config.height * 0.65,
0
);
leftArm.rotation.z = Math.PI / 4;
this.mesh.add(leftArm);
const rightArm = new THREE.Mesh(armGeometry, armMaterial);
rightArm.position.set(
this.config.bodyWidth / 2 + this.config.handRadius,
this.config.height * 0.65,
0
);
rightArm.rotation.z = -Math.PI / 4;
this.mesh.add(rightArm);
// Hands
const handGeometry = new THREE.SphereGeometry(this.config.handRadius, 16, 16);
const handMaterial = new THREE.MeshStandardMaterial({
color: 0xffe0bd,
map: this.config.texture
});
const leftHand = new THREE.Mesh(handGeometry, handMaterial);
leftHand.position.set(
-this.config.bodyWidth / 2 - this.config.armLength * 0.7,
this.config.height * 0.4,
0.3
);
this.mesh.add(leftHand);
const rightHand = new THREE.Mesh(handGeometry, handMaterial);
rightHand.position.set(
this.config.bodyWidth / 2 + this.config.armLength * 0.7,
this.config.height * 0.4,
0.3
);
this.mesh.add(rightHand);
// Legs
const legGeometry = new THREE.CapsuleGeometry(
this.config.handRadius,
this.config.legLength,
8,
16
);
const legMaterial = new THREE.MeshStandardMaterial({
color: 0x34495e,
map: this.config.texture
});
const leftLeg = new THREE.Mesh(legGeometry, legMaterial);
leftLeg.position.set(-this.config.bodyWidth / 4, this.config.legLength / 2, 0);
this.mesh.add(leftLeg);
const rightLeg = new THREE.Mesh(legGeometry, legMaterial);
rightLeg.position.set(this.config.bodyWidth / 4, this.config.legLength / 2, 0);
this.mesh.add(rightLeg);
// Name label (floating above head)
const canvas = document.createElement('canvas');
const context = canvas.getContext('2d');
canvas.width = 256;
canvas.height = 64;
context.fillStyle = 'rgba(0,0,0,0.7)';
context.fillRect(0, 0, canvas.width, canvas.height);
context.fillStyle = 'white';
context.font = '32px Arial';
context.textAlign = 'center';
context.fillText(this.config.username, 128, 48);
const texture = new THREE.CanvasTexture(canvas);
const labelGeometry = new THREE.PlaneGeometry(1.5, 0.4);
const labelMaterial = new THREE.MeshBasicMaterial({
map: texture,
transparent: true,
side: THREE.DoubleSide
});
const label = new THREE.Mesh(labelGeometry, labelMaterial);
label.position.y = this.config.height + 0.3;
label.rotation.x = 0.2; // Slight tilt towards viewer
this.mesh.add(label);
// User type indicator (different colors for bot vs. human)
if (this.config.userType === 'bot') {
bodyMaterial.color.setHex(0xe74c3c); // Red for bots
}
}
updatePosition(position) {
this.mesh.position.set(position.x, position.y, position.z);
}
updateRotation(quaternion) {
this.mesh.quaternion.set(quaternion.x, quaternion.y, quaternion.z, quaternion.w);
}
}
// Create avatar from configuration
const avatarConfig = {
username: 'Alice Johnson',
color: 0x3498db,
texture: null,
userType: 'human'
};
const avatar = new XRAvatar(avatarConfig);
scene.add(avatar.mesh);
```text
#### Hand Tracking Integration
```javascript
// Hand tracking for natural interaction
const handLeft = renderer.xr.getController( 0 );
const handRight = renderer.xr.getController( 1 );
// Hand model loading
const handModelLeft = new XRHandModel( handLeft );
scene.add( handLeft );
scene.add( handModelLeft );
scene.add( handRight );
// Hand interaction: Grabbing objects
const raycaster = new THREE.Raycaster();
const grabDistance = 0.3;
let grabbedObject = null;
function handleHandInteraction(hand) {
if (!hand.visible) return;
const inputSource = hand.inputSource;
const handPosition = hand.position.clone();
const handRotation = hand.rotation.clone();
// Check for intersection with 3D objects
raycaster.set(handPosition, handRotation);
const intersects = raycaster.intersectObjects(canvasObjects.children);
// Grab detected objects
if (inputSource.handedness === 'right' && inputSource.buttons[0].pressed) {
if (intersects.length > 0 && intersects[0].distance < grabDistance) {
grabbedObject = intersects[0].object;
grabbedObject.position.copy(handPosition);
grabbedObject.quaternion.copy(handRotation);
}
}
// Update position if object is grabbed
if (grabbedObject) {
grabbedObject.position.copy(handPosition);
grabbedObject.quaternion.copy(handRotation);
// Release object if trigger released
if (inputSource.handedness === 'right' && !inputSource.buttons[0].pressed) {
grabbedObject = null;
}
}
// Hand gesture: Index finger pointing for whiteboard annotation
const thumb = handHand.getJointPoint('thumb-tip');
const index = handHand.getJointPoint('index-finger-tip');
if (inputSource.handedness === 'right' && inputSource.buttons[0].pressed) {
const distance = thumb.position.distanceTo(index.position);
// Pointing gesture (thumb and index fingers separated)
if (distance > 0.1) {
drawOnWhiteboard(index.position, handRotation);
}
}
}
// Whiteboard drawing function
function drawOnWhiteboard(position, rotation) {
const raycaster = new THREE.Raycaster();
raycaster.set(position, rotation);
const intersects = raycaster.intersectObjects(whiteboardPlanes.children);
if (intersects.length > 0) {
const point = intersects[0].point;
// Create 3D stroke on whiteboard
const strokeGeometry = new THREE.TubeGeometry(
new THREE.LineCurve3(
new THREE.Vector3(lastPoint.x, lastPoint.y, lastPoint.z),
new THREE.Vector3(point.x, point.y, point.z)
),
1,
0.005,
8,
false
);
const strokeMaterial = new THREE.MeshBasicMaterial({
color: currentPenColor,
transparent: true,
opacity: 1
});
const stroke = new THREE.Mesh(strokeGeometry, strokeMaterial);
whiteboardPlanes.add(stroke);
// Send stroke data via WebSocket
socket.send(JSON.stringify({
type: 'whiteboard_stroke',
point: { x: point.x, y: point.y, z: point.z },
color: currentPenColor,
thickness: currentPenThickness
}));
lastPoint = { x: point.x, y: point.y, z: point.z };
}
}
// Register hand interaction callback
renderer.setAnimationLoop(() => {
handleHandInteraction(handLeft);
handleHandInteraction(handRight);
renderer.render(scene, camera);
});
```text
### 3D Canvas Collaboration
#### Shared Whiteboard Implementation
```javascript
// 3D whiteboard plane
const whiteboardWidth = 4;
const whiteboardHeight = 3;
const whiteboardGeometry = new THREE.PlaneGeometry(whiteboardWidth, whiteboardHeight);
const whiteboardMaterial = new THREE.MeshPhysicalMaterial({
color: 0xffffff,
side: THREE.DoubleSide,
roughness: 0.2,
metalness: 0.1
});
const whiteboard = new THREE.Mesh(whiteboardGeometry, whiteboardMaterial);
whiteboard.position.set(0, 1.5, -2);
scene.add(whiteboard);
// Strokes on whiteboard (stored as separate meshes)
const whiteboardStrokes = new THREE.Group();
whiteboard.add(whiteboardStrokes);
// Whiteboard tools
const whiteboardTools = {
currentColor: 0x000000,
currentThickness: 0.02,
eraserMode: false
};
// Whiteboard UI (in XR space)
const uiGeometry = new THREE.PlaneGeometry(0.8, 0.3);
const uiMaterial = new THREE.MeshBasicMaterial({
color: 0x222222,
transparent: true,
opacity: 0.8
});
const uiPanel = new THREE.Mesh(uiGeometry, uiMaterial);
uiPanel.position.set(1.5, 1.8, -0.5);
scene.add(uiPanel);
// Add color picker buttons to UI
const colors = [0x000000, 0xff0000, 0x00ff00, 0x0000ff, 0xffff00, 0xff00ff, 0x00ffff];
const colorButtons = [];
colors.forEach((color, index) => {
const buttonGeometry = new THREE.CircleGeometry(0.03, 32);
const buttonMaterial = new THREE.MeshBasicMaterial({ color: color });
const button = new THREE.Mesh(buttonGeometry, buttonMaterial);
button.position.set(1.2 + index * 0.1, 1.9, -0.4);
button.userData = { color: color };
scene.add(button);
colorButtons.push(button);
});
// Color selection interaction
const colorRaycaster = new THREE.Raycaster();
function handleColorSelection(handPosition, handRotation) {
colorRaycaster.set(handPosition, handRotation);
const intersects = colorRaycaster.intersectObjects(colorButtons);
if (intersects.length > 0 && hand.inputSource.buttons[0].pressed) {
whiteboardTools.currentColor = intersects[0].object.userData.color;
whiteboardTools.eraserMode = false;
}
}
// Eraser tool toggle
const eraserButtonGeometry = new THREE.CircleGeometry(0.05, 32);
const eraserButtonMaterial = new THREE.MeshBasicMaterial({ color: 0xffffff });
const eraserButton = new THREE.Mesh(eraserButtonGeometry, eraserButtonMaterial);
eraserButton.position.set(1.8, 1.75, -0.4);
scene.add(eraserButton);
// Eraser selection interaction
function handleEraserSelection(handPosition, handRotation) {
colorRaycaster.set(handPosition, handRotation);
const intersects = colorRaycaster.intersectObject(eraserButton);
if (intersects.length > 0 && hand.inputSource.buttons[0].pressed) {
whiteboardTools.eraserMode = !whiteboardTools.eraserMode;
eraserButton.material.color.setHex(
whiteboardTools.eraserMode ? 0xff0000 : 0xffffff
);
}
}
// Clear whiteboard button
const clearButtonGeometry = new THREE.PlaneGeometry(0.4, 0.1);
const clearButtonMaterial = new THREE.MeshBasicMaterial({ color: 0x888888 });
const clearButton = new THREE.Mesh(clearButtonGeometry, clearButtonMaterial);
clearButton.position.set(1.5, 1.6, -0.4);
scene.add(clearButton);
function handleClearWhiteboard(handPosition, handRotation) {
colorRaycaster.set(handPosition, handRotation);
const intersects = colorRaycaster.intersectObject(clearButton);
if (intersects.length > 0 && hand.inputSource.buttons[0].pressed) {
// Remove all strokes
while (whiteboardStrokes.children.length > 0) {
whiteboardStrokes.remove(whiteboardStrokes.children[0]);
}
// Send clear command via WebSocket
socket.send(JSON.stringify({
type: 'whiteboard_clear',
meetingId: meetingId
}));
}
}
// Register UI interaction callbacks
renderer.setAnimationLoop(() => {
handleColorSelection(handLeft.position, handLeft.rotation);
handleEraserSelection(handLeft.position, handLeft.rotation);
handleClearWhiteboard(handLeft.position, handLeft.rotation);
renderer.render(scene, camera);
});
```text
#### 3D Object Manipulation Canvas
```javascript
// 3D canvas for shared object manipulation
const canvas3DGeometry = new THREE.BoxGeometry(3, 2, 0.5);
const canvas3DMaterial = new THREE.MeshPhysicalMaterial({
color: 0xf0f0f0,
side: THREE.DoubleSide,
transparent: true,
opacity: 0.9,
roughness: 0.3,
metalness: 0.1
});
const canvas3D = new THREE.Mesh(canvas3DGeometry, canvas3DMaterial);
canvas3D.position.set(0, 1.5, -1);
scene.add(canvas3D);
// Ground plane for positioning objects
const groundGeometry = new THREE.PlaneGeometry(5, 5);
const groundMaterial = new THREE.MeshStandardMaterial({
color: 0xeeeeee,
side: THREE.DoubleSide
});
const ground = new THREE.Mesh(groundGeometry, groundMaterial);
ground.rotation.x = -Math.PI / 2;
ground.position.y = 0;
scene.add(ground);
// Object palette (spawnable objects)
const objectPalette = [
{ type: 'box', geometry: new THREE.BoxGeometry(0.5, 0.5, 0.5) },
{ type: 'sphere', geometry: new THREE.SphereGeometry(0.3, 32, 32) },
{ type: 'cylinder', geometry: new THREE.CylinderGeometry(0.2, 0.2, 0.5, 32) },
{ type: 'cone', geometry: new THREE.ConeGeometry(0.3, 0.5, 32) }
];
const paletteIcons = [];
objectPalette.forEach((obj, index) => {
const icon = new THREE.Mesh(obj.geometry, new THREE.MeshStandardMaterial({ color: 0x3498db }));
icon.position.set(2.5 + index * 0.7, 0.25, -0.5);
icon.userData = { objectType: obj.type };
scene.add(icon);
paletteIcons.push(icon);
});
// Object spawning
function spawnObject(objectType, position) {
const paletteObj = objectPalette.find(obj => obj.type === objectType);
if (!paletteObj) return;
const material = new THREE.MeshStandardMaterial({ color: 0x3498db });
const spawnedObject = new THREE.Mesh(paletteObj.geometry.clone(), material);
spawnedObject.position.copy(position);
spawnedObject.position.y = 0.3; // Start above ground
canvas3D.add(spawnedObject);
// Send object spawn via WebSocket
socket.send(JSON.stringify({
type: 'object_spawn',
objectType: objectType,
position: spawnedObject.position.toArray(),
rotation: spawnedObject.quaternion.toArray()
}));
return spawnedObject;
}
// Object manipulation (grabbing and moving)
const grabbedObject = null;
const grabRaycaster = new THREE.Raycaster();
function handleObjectManipulation(handPosition, handRotation) {
if (!hand.visible) return;
grabRaycaster.set(handPosition, handRotation);
// Check for palette object interaction (spawning)
const paletteIntersects = grabRaycaster.intersectObjects(paletteIcons);
if (paletteIntersects.length > 0 && paletteIntersects[0].distance < 0.5) {
if (hand.inputSource.buttons[0].pressed) {
spawnObject(
paletteIntersects[0].object.userData.objectType,
handPosition.clone()
);
}
}
// Check for spawned object interaction (grabbing)
if (!grabbedObject) {
const objectIntersects = grabRaycaster.intersectObjects(canvas3D.children);
if (objectIntersects.length > 0 && objectIntersects[0].distance < 1.0) {
if (hand.inputSource.buttons[0].pressed) {
grabbedObject = objectIntersects[0].object;
grabbedObject.material.color.setHex(0xe74c3c); // Change to red when grabbed
}
}
}
// Move grabbed object
if (grabbedObject) {
grabbedObject.position.copy(handPosition);
// Update server
socket.send(JSON.stringify({
type: 'object_move',
objectId: grabbedObject.uuid,
position: grabbedObject.position.toArray(),
rotation: grabbedObject.quaternion.toArray()
}));
// Release object if trigger released
if (!hand.inputSource.buttons[0].pressed) {
grabbedObject.material.color.setHex(0x3498db); // Reset to blue
grabbedObject = null;
}
}
}
// Register object manipulation callback
renderer.setAnimationLoop(() => {
handleObjectManipulation(handLeft.position, handLeft.rotation);
handleObjectManipulation(handRight.position, handRight.rotation);
renderer.render(scene, camera);
});
```text
## Use Case Specific Implementations
### Healthcare: Telehealth Classroom
**Adaptations for medical training**:
```javascript
// Medical imaging texture loader for XR whiteboard
function loadMedicalImage(imageUrl) {
const textureLoader = new THREE.TextureLoader();
textureLoader.load(imageUrl, (texture) => {
const imageGeometry = new THREE.PlaneGeometry(1.5, 1.0);
const imageMaterial = new THREE.MeshBasicMaterial({ map: texture });
const medicalImage = new THREE.Mesh(imageGeometry, imageMaterial);
medicalImage.position.set(-1.5, 1.8, -1.98);
medicalImage.name = `medical_${Date.now()}`;
whiteboard.add(medicalImage);
// Allow manipulation (pan, zoom) of medical images
makeInteractable(medicalImage);
});
}
// Surgical planning annotations
const surgicalAnnotationTool = {
toolType: 'annotation',
allowedShapes: ['point', 'line', 'arrow', 'text'],
colors: {
incision: 0xff0000,
landmark: 0x00ff00,
pathology: 0x0000ff,
note: 0xffff00
}
};
// VRFDA-compatible DICOM viewer integration
// (DICOM = Digital Imaging and Communications in Medicine)
function initializeDICOMViewer() {
const dicomViewer = new DICOMViewer({
baseTexture: whiteboard.material.map,
overlayTexture: annotationTexture
});
// Add DICOM slices to whiteboard
dicomViewer.loadSeries('patient123', 'mri_brain')
.then(() => {
console.log('MRI series loaded');
});
}
```text
**Regulatory compliance (HIPAA)**:
```python
# HIPAA-compliant data handling
from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes
import os
def encrypt_meeting_audio(audio_data):
"""Encrypt meeting audio with AES-256-GCM (NIST-approved)"""
key = get_encryption_key() # From secure KMS or hardware token
iv = os.urandom(12)
cipher = Cipher(
algorithms.AES(key),
modes.GCM(iv)
)
encryptor = cipher.encryptor()
encrypted_audio = encryptor.update(audio_data) + encryptor.finalize()
auth_tag = encryptor.tag
return {
'iv': iv,
'auth_tag': auth_tag,
'ciphertext': encrypted_audio
}
def audit_meeting_access(meeting_id, user_id, action):
"""Log all meeting access for HIPAA audit requirements"""
audit_entry = {
'meeting_id': meeting_id,
'user_id': user_id,
'action': action, # 'join', 'speak', 'annotate', 'export'
'timestamp': datetime.utcnow().isoformat(),
'ip_address': get_client_ip()
}
# Store in audit log (immutable storage required)
append_to_audit_trail(audit_entry)
```text
### Education: Virtual Science Laboratories
**Interactive lab equipment simulation**:
```javascript
// Beaker simulation for chemistry labs
function createBeaker(position, volume, liquidColor) {
const beakerGroup = new THREE.Group();
// Glass beaker
const beakerGeometry = new THREE.CylinderGeometry(0.2, 0.2, 0.3, 32, 1, true);
const glassMaterial = new THREE.MeshPhysicalMaterial({
color: 0xffffff,
transparent: true,
opacity: 0.4,
roughness: 0.1,
metalness: 0.2,
side: THREE.DoubleSide
});
const beaker = new THREE.Mesh(beakerGeometry, glassMaterial);
beakerGroup.add(beaker);
// Liquid inside beaker
const liquidHeight = volume / 100 * 0.25; // Max 100mL, beaker height 0.25
const liquidGeometry = new THREE.CylinderGeometry(0.18, 0.18, liquidHeight, 32);
const liquidMaterial = new THREE.MeshPhysicalMaterial({
color: liquidColor,
transparent: true,
opacity: 0.8,
roughness: 0.2,
metalness: 0.1
});
const liquid = new THREE.Mesh(liquidGeometry, liquidMaterial);
liquid.position.y = -0.1 + liquidHeight / 2;
beakerGroup.add(liquid);
beakerGroup.position.copy(position);
beakerGroup.userData = {
type: 'beaker',
volume: volume,
liquidColor: liquidColor
};
canvas3D.add(beakerGroup);
return beakerGroup;
}
// Chemical reaction simulation (physics engine needed)
function simulateChemicalReaction(beaker1, beaker2) {
const reaction = {
reagents: [
{ beaker: beaker1, volume: 50 },
{ beaker: beaker2, volume: 50 }
],
products: [],
reactionType: 'exothermic' // or 'endothermic'
};
// Determine reaction based on types
// (simplified for demo)
if (isAcidBaseReaction(beaker1, beaker2)) {
reaction.products = [{
type: 'salt',
volume: 100,
color: 0xeeeeee, // White precipitate
heat: 50 // Temperature rise in Kelvin
}];
}
return reaction;
}
// Physics engine integration for container spills
function simulateFluidDynamics(container, spillVolume) {
// Use physics engine (e.g., Ammo.js, Cannon.js)
// for realistic liquid simulation
const physicsEngine = new PhysicsEngine();
const fluidParticles = createFluidParticles(spillVolume);
physicsEngine.addBodies(fluidParticles);
physicsEngine.simulate(); // Simulate for X seconds
return physicsEngine.getResult();
}
```text
**Student assessment and feedback**:
```java
// Java backend for student assessment
@RestController
@RequestMapping("/api/labs")
public class VRAssessmentController {
@PostMapping("/submit")
public LabResult submitLabResult(@RequestBody LabSubmission submission) {
// Validate lab protocol execution
boolean protocolFollowed = validateProtocol(
submission.getSteps(),
submission.getResults()
);
// Grade based on experimental accuracy
double accuracy = calculateAccuracy(
submission.getMeasurements(),
getExpectedMeasurements(submission.getLabId())
);
// Generate AI-powered feedback
String feedback = generateFeedback(
submission,
protocolFollowed,
accuracy
);
LabResult result = new LabResult();
result.setProtocolFollowed(protocolFollowed);
result.setAccuracy(accuracy);
result.setFeedback(feedback);
result.setTimestamp(Instant.now());
return result;
}
}
```text
### Manufacturing: Digital Twins for Facility Planning
**Facility 3D model integration**:
```javascript
// Load facility BIM model into XR workspace
function loadFacilityModel(modelUrl) {
const gltfLoader = new GLTFLoader();
const dracoLoader = new DRACOLoader();
dracoLoader.setDecoderPath('/draco/'); // Path to Draco decoder
gltfLoader.setDRACOLoader(dracoLoader);
gltfLoader.load(modelUrl, (gltf) => {
const facilityScene = gltf.scene;
// Scale down to fit in XR canvas
const boundingBox = new THREE.Box3().setFromObject(facilityScene);
const size = boundingBox.getSize(new THREE.Vector3());
const maxDim = Math.max(size.x, size.y, size.z);
const scale = 3 / maxDim; // Fit in 3-unit space
facilityScene.scale.setScalar(scale);
// Position in front of user
facilityScene.position.set(0, 1.5 - boundingBox.min.y * scale, -2);
facilityScene.rotation.y = -Math.PI / 2; // Rotate for better viewing
// Enable interaction with individual facility components
facilityScene.traverse((child) => {
if (child.isMesh) {
child.userData = {
type: 'facility_component',
componentId: child.name,
draggable: true
};
}
});
canvas3D.add(facilityScene);
});
}
// Facility optimization analysis (simulated)
function analyzeFacilityLayout(facilityModel) {
const analysis = {
walkwayCongestion: [],
equipmentEfficiency: [],
spaceUtilization: 0,
recommendations: []
};
// Analyze walkway traffic patterns
// (would use BIM metadata + RFID data from real facility)
facilityModel.traverse((child) => {
if (child.userData?.componentId?.startsWith('walkway_')) {
const congestionScore = calculateWalkwayCongestion(
child.userData.componentId
);
analysis.walkwayCongestion.push({
componentId: child.userData.componentId,
congestionScore: congestionScore
});
}
});
// Calculate overall space utilization
analysis.spaceUtilization = calculateTotalUtilization(facilityModel);
// Generate recommendations
analysis.recommendations = generateOptimizationRecommendations(
analysis.walkwayCongestion,
analysis.equipmentEfficiency
);
return analysis;
}
// Equipment heat map visualization
function visualizeEquipmentEfficiency(facilityModel, efficiencyData) {
// Color-code equipment based on efficiency
const heatmapColormap = [
{ threshold: 0.5, color: 0xff0000 }, // Red for low efficiency
{ threshold: 0.75, color: 0xffff00 }, // Yellow for medium
{ threshold: 1.0, color: 0x00ff00 } // Green for high
];
facilityModel.traverse((child) => {
if (child.userData?.componentId?.startsWith('equipment_')) {
const equipmentId = child.userData.componentId;
const efficiency = efficiencyData[equipmentId] | | 0.5;
// Find color based on efficiency
const colormapEntry = heatmapColormap.find(
entry => efficiency <= entry.threshold
) | | heatmapColormap[heatmapColormap.length - 1];
child.material.color.setHex(colormapEntry.color);
// Add tooltip with efficiency data
const tooltip = createTooltip3D(
`${efficiency * 100}% efficient`,
child.position.clone().add(new THREE.Vector3(0, 0.5, 0))
);
canvas3D.add(tooltip);
}
});
}
```text
## Scaling and Performance Optimization
### Horizontal Scaling Strategy
**Multi-server deployment with session affinity**:
```yaml
# Docker Compose HA configuration
version: '3.8'
services:
# Load balancer with session affinity
xr-lb:
image: traefik:latest
command:
- --api.insecure=true
- --providers.docker=true
- --entrypoints.web.address=:80
- --entrypoints.websecure.address=:443
- --providers.loadbalancer.stickiness=true
- --providers.loadbalancer.stickiness.cookieName=XR_SESSION
ports:
- "80:80"
- "443:443"
- "8080:8080"
networks:
- xr-network
restart: unless-stopped
# XR backend instance 1
xr-backend-1:
build: ./xr-backend
ports:
- "8001:8000"
volumes:
- ./hdf5-data:/hdf5-data
environment:
- INSTANCE_ID=1
- REDIS_URL=redis://redis:6379
deploy:
replicas: 1
placement:
constraints:
- node.labels.xr-pool == primary
networks:
- xr-network
- redis-network
restart: unless-stopped
# XR backend instance 2
xr-backend-2:
build: ./xr-backend
ports:
- "8002:8000"
volumes:
- ./hdf5-data:/hdf5-data
environment:
- INSTANCE_ID=2
- REDIS_URL=redis://redis:6379
deploy:
replicas: 1
placement:
constraints:
- node.labels.xr-pool == secondary
networks:
- xr-network
- redis-network
restart: unless-stopped
# Redis for distributed session management
redis:
image: redis:7
command: redis-server --cluster-enabled yes
networks:
- redis-network
restart: unless-stopped
networks:
xr-network:
driver: bridge
redis-network:
driver: bridge
```text
### Media Optimization
**Adaptive bitrate streaming for video feeds**:
```javascript
// Adaptive bitrate selector
class AdaptiveBitrateSelector {
constructor() {
this.currentBitrate = 2000; // Start at 2 Mbps
this.bitrateLevels = [500, 1000, 2000, 4000, 8000];
this.currentIndex = 2; // Index for 2000 kbps
this.lastQualityChange = Date.now();
this.minQualityChangeInterval = 5000; // 5 seconds
}
evaluateNetworkConditions(networkMetrics) {
const now = Date.now();
const timeSinceLastChange = now - this.lastQualityChange;
if (timeSinceLastChange < this.minQualityChangeInterval) {
return; // Don't change quality too quickly
}
const bandwidth = networkMetrics.bandwidth;
const latency = networkMetrics.latency;
const packetLoss = networkMetrics.packetLoss;
if (bandwidth > 4000 && latency < 100 && packetLoss < 0.01) {
// Excellent network: can increase bitrate
this.increaseBitrate();
} else if (bandwidth < 1000 | | latency > 300 | | packetLoss > 0.05) {
// Poor network: decrease bitrate
this.decreaseBitrate();
}
this.lastQualityChange = now;
return this.currentBitrate;
}
increaseBitrate() {
const nextIndex = Math.min(
this.currentIndex + 1,
this.bitrateLevels.length - 1
);
if (this.currentIndex !== nextIndex) {
this.currentIndex = nextIndex;
this.currentBitrate = this.bitrateLevels[nextIndex];
console.log(`Increased bitrate to ${this.currentBitrate} kbps`);
}
}
decreaseBitrate() {
const previousIndex = Math.max(
this.currentIndex - 1,
0
);
if (this.currentIndex !== previousIndex) {
this.currentIndex = previousIndex;
this.currentBitrate = this.bitrateLevels[previousIndex];
console.log(`Decreased bitrate to ${this.currentBitrate} kbps`);
}
}
}
// Video stream player with adaptive bitrate
function createAdaptiveVideoPlayer(videoElement, streamUrl) {
const player = new VideoPlayer(videoElement, {
streaming: {
enableAutoAdaptation: true,
initialBitrate: 2000,
adaptiveBitrateSelector: new AdaptiveBitrateSelector()
}
});
player.load(streamUrl);
// Monitor network metrics
setInterval(() => {
const networkMetrics = {
bandwidth: player.getBandwidthEstimate(),
latency: player.getRtt(),
packetLoss: player.getPacketLossRate()
};
const newBitrate = player.options.streaming.adaptiveBitrateSelector.evaluateNetworkConditions(
networkMetrics
);
if (newBitrate !== player.currentBitrate) {
player.switchBitrate(newBitrate);
}
}, 1000); // Evaluate every second
return player;
}
```text
### HDF5 Optimization
**Chunked storage for high-frequency updates**:
```python
# Optimized HDF5 chunking strategy
import h5py
import numpy as np
# Create HDF5 file with optimized chunking
with h5py.File('meeting_state_optimized.h5', 'w') as f:
# Avatar positions chunked by time (optimal for append-only writes)
avatar_positions = f.create_dataset(
'avatars/positions',
shape=(0, 5), # timestamp, user_id, x, y, z
maxshape=(None, 5),
dtype='float64',
chunks=(1000, 5), # Optimal for time-series data
compression='gzip',
compression_opts=4
)
# Whiteboard strokes chunked by stroke_id (optimal for retrieval by stroke)
whiteboard_strokes = f.create_dataset(
'whiteboard/strokes',
shape=(0, 8), # timestamp, user_id, stroke_id, x, y, color, thickness, line_width
maxshape=(None, 8),
dtype='float64',
chunks=(1000, 8), # Optimized for stroke writes
compression='lz4', # Faster compression for high-frequency data
shuffle=True, # Improve compression ratio
fletcher32=True # Integrity checking
)
# Create index for fast stroke lookup
f.create_dataset(
'whiteboard/stroke_index',
shape=(0, 2),
maxshape=(None, 2),
dtype='int64',
chunks=(100, 2) # Smaller chunks for index data
)
```text
## Conclusion: The Immersive Collaboration Future
Self-hosted XR collaboration platforms offer organizations unprecedented control over immersive meeting spaces while ensuring data sovereignty and regulatory compliance. By leveraging open-source standards (WebXR, HDF5, Three.js), organizations can deploy scalable, secure virtual meeting infrastructure without vendor lock-in.
The journey begins with foundational infrastructure (Docker Swarm, Traefik, authentication) and expands to domain-specific implementations (healthcare telehealth, education labs, manufacturing digital twins). The return on investment increases rapidly as user count grows, making self-hosted XR economically competitive for organizations with 50+ regular users.
Organizations that invest in self-hosted XR collaboration today enjoy advantages in:
- **Data Sovereignty**: Complete control over meeting data, IP protection
- **Customization**: Branded avatar systems, domain-specific collaboration tools
- **Compliance**: Built-in GDPR, HIPAA, and regulatory support
- **Innovation**: Freedom to integrate with existing systems and customize workflows
The immersive collaboration revolution is underway—will your organization lead or follow?
---
*This article is part of the XR Collaboration Series on tobias-weiss.org, exploring immersive meeting technologies and self-hosted implementations.*