# WebXR Meshing API Level 1

## A Collection of Interesting Ideas, 1 January 1970

Previous Versions:
Editors:
  Rik Cabanier (Magic Leap)

Copyright © 1970 W3C® (MIT, ERCIM, Keio, Beihang). W3C liability, trademark and permissive document license rules apply.

## Abstract

This specification describes support for accessing the geometry of real world objects during a WebXR session.

## Status of this document

## 1. Introduction

Meshing is a technique that uses a device’s sensors to build a 3 dimensional representation of the world.

The mesh consists of a large collection of geometry of medium complexity that changes slowly over time (aka the world mesh) and a very small collection of complex quickly changing content that represents the geometry close to the user (aka the near mesh).

A user agent can have support for no, one or both types of geometry and the website author can request one or both types of geometry data. Requesting the geometry data is an expensive process so an author should only request it if they will do something with the information.

Typically mesh data is not used to determine things like hand gestures or the recognition of real world objects. (However a UA could choose to infer this through postprocessing.)

The most common use cases for the world mesh are:

1. physics: interaction of virtual objects with the real world

2. occlusion: blocking part or the whole virtual object by a real world one

The most common use cases for the near mesh are:

1. a visual representation of objects near the viewer.

2. occlusion

### 1.1. Terminology

### 1.2. Application flow

Most applications using the WebXR Meshing API will follow a similar usage pattern:

- During the creation of a WebXR session, pass a XRFeatureInit object with parameters for world and near mesh.

- For each XRFrame, get the requested mesh data and apply it to the scene.

## 2. Initialization

### 2.1. XRMeshQuality

    enum XRMeshQuality {
       "low",
       "medium",
       "high"
    };

`XRMeshQuality` defines the quality of the mesh. A higher quality means that the mesh will be finer but also more resource intensive. It is up to the UA to define the quality level.

### 2.2. XRWorldMeshFeature

The "worldmesh" feature is used to request world meshing.

    dictionary XRWorldMeshFeature: XRFeatureInit {
        XRMeshQuality quality = "medium";
        double width = 10.0;
        double height = 10.0;
        double breadth = 10.0;
    };

The `width`, `height` and `breadth` attributes define the distance, in meters, of the width, height and breadth of the area around the observer that should be meshed. The `quality` attribute defines the UA dependent quality of the mesh.

The following code attempts to create an `immersive-ar` `XRSession` with world meshing.

    let xrSession;

    navigator.xr.requestSession("immersive-ar", {
                                requiredFeature: {
                                    name: "XRWorldMeshFeature",
                                    quality: "medium"}}).then((session) => {
        xrSession = session;
    });

should the world mesh follow the observer or should it stay relative to the original position or should it be configurable?

should the world mesh have an XRSpace?

### 2.3. XRNearMeshFeature

The "nearmesh" feature is used to request meshing near the observer. It is UA dependent what the scanned area for the near mesh is but it MUST not overlap with the world mesh.

    dictionary XRNearMeshFeature: XRFeatureInit {
        XRMeshQuality quality = "medium";
    };

The `quality` attribute defines the UA dependent quality of the mesh.

The following code attempts to create an `immersive-ar` `XRSession` with near meshing.

    let xrSession;

    navigator.xr.requestSession("immersive-vr", {
                                requiredFeature: {
                                    name: "XRNearMeshFeature",
                                    quality: "low"}}).then((session) => {
        xrSession = session;
    });

should the near mesh have an XRSpace?

## 3. Frame Loop

### 3.1. XRMesh structures

    dictionary XRMeshBlock {
        required Float32Array vertices;
        required Uint16Array indices;
        Float32Array normals;
    };

A `XRMeshBlock` contains the geometry data of a single mesh instance.

`vertices` contains a buffer with points in 3D space. Each point consists of 3 floats.

Each value in `indices` points to an offset into a point inside `vertices` and defines the corner of a triangle. The set of triangles creates a polygon mesh. NOTE: the offset of each point is found by taking the index value and multiplying by 3.

`XRMeshBlock` can contain an optional `normals` which defines a buffer with the normal of each vertex. The size of `normals` must be the same as `vertices`.

is normals needed? If so, should it be requested during initialisation?

    interface XRNearMesh {
        readonly setlike<XRMeshBlock>;
    };

    interface XRWorldMesh {
        readonly maplike<DOMString, XRMeshBlock>;
    };

    dictionary XRMetadata {
        XRWorldMesh worldMesh;
        XRNearMesh nearMesh;
    };

The `worldMesh` attribute contains updates to the world mesh. If any key in the dictionary was a previously provided provided XRWorldMesh this new value must replace the previous XRMeshBlock. If the value of a new world XRMeshBlock contains no vertices, the existing XRMeshBlock must be deleted.

The `nearMesh` attribute contains a new near mesh which will replace the previous near mesh (if any). If nearMesh contains no new XRMeshBlock object, there is no near mesh.

### 3.2. XRFrame

    partial interface XRFrame {
        readonly attribute XRMetadata metaData;
    };

Each XRFrame contains a `metaData` attribute with new or updated world or near mesh data.

should the mesh data persist per browser session or per xr session?

## 4. Security and Privacy Considerations

The WebXR Meshing API is a powerful feature with that carries significant privacy risks. A UA MUST ask permission from the user during session creation before meshing data is returned to the page.

An inline session must NOT have access to mesh data.

Additionally, mesh data MUST be constructed from the geometry of the real world. It MUST not reveal writing, colors, pictures or other visual content.

clarify this section

## Index

### Terms defined by this specification

- breadth
  - attribute for XRWorldMeshFeature/breadth, in §2.2
  - dict-member for XRWorldMeshFeature, in §2.2
- height
  - attribute for XRWorldMeshFeature/height, in §2.2
  - dict-member for XRWorldMeshFeature, in §2.2
- "high", in §2.1
- indices, in §3.1
- "low", in §2.1
- "medium", in §2.1
- metaData
  - attribute for XRFrame, in §3.2
  - dict-member for XRFrame, in §3.2
- nearMesh, in §3.1
- normals, in §3.1
- quality
  - attribute for XRNearMeshFeature/quality, in §2.3
  - attribute for XRWorldMeshFeature/quality, in §2.2
  - dict-member for XRNearMeshFeature, in §2.3
  - dict-member for XRWorldMeshFeature, in §2.2
- vertices, in §3.1
- width
  - attribute for XRWorldMeshFeature/width, in §2.2
  - dict-member for XRWorldMeshFeature, in §2.2
- worldMesh, in §3.1
- XRMeshBlock, in §3.1
- XRMeshQuality, in §2.1
- XRMetadata, in §3.1
- XRNearMesh, in §3.1
- XRNearMeshFeature, in §2.3
- XRWorldMesh, in §3.1
- XRWorldMeshFeature, in §2.2

https://heycam.github.io/webidl/#idl-DOMString Referenced in:

- 3.1. XRMesh structures

https://heycam.github.io/webidl/#idl-Float32Array Referenced in:

- 3.1. XRMesh structures (2)

https://heycam.github.io/webidl/#idl-Uint16Array Referenced in:

- 3.1. XRMesh structures

https://heycam.github.io/webidl/#idl-double Referenced in:

- 2.2. XRWorldMeshFeature (2) (3)

### Terms defined by reference

- [WebIDL] defines the following terms:
  - DOMString
  - Float32Array
  - Uint16Array
  - double

## References

### Normative References

[WebIDL]
  Boris Zbarsky. Web IDL. 15 December 2016. ED. URL: https://heycam.github.io/webidl/

## IDL Index

    enum XRMeshQuality {
       "low",
       "medium",
       "high"
    };

    dictionary XRWorldMeshFeature: XRFeatureInit {
        XRMeshQuality quality = "medium";
        double width = 10.0;
        double height = 10.0;
        double breadth = 10.0;
    };

    dictionary XRNearMeshFeature: XRFeatureInit {
        XRMeshQuality quality = "medium";
    };

    dictionary XRMeshBlock {
        required Float32Array vertices;
        required Uint16Array indices;
        Float32Array normals;
    };

    interface XRNearMesh {
        readonly setlike<XRMeshBlock>;
    };

    interface XRWorldMesh {
        readonly maplike<DOMString, XRMeshBlock>;
    };

    dictionary XRMetadata {
        XRWorldMesh worldMesh;
        XRNearMesh nearMesh;
    };

    partial interface XRFrame {
        readonly attribute XRMetadata metaData;
    };

## Issues Index

should the world mesh follow the observer or should it stay relative to the original position or should it be configurable? ↵
should the world mesh have an XRSpace? ↵
should the near mesh have an XRSpace? ↵
is normals needed? If so, should it be requested during initialisation? ↵
should the mesh data persist per browser session or per xr session? ↵
clarify this section ↵
#enumdef-xrmeshquality Referenced in:

- 2.1. XRMeshQuality
- 2.2. XRWorldMeshFeature
- 2.3. XRNearMeshFeature

#dictdef-xrmeshblock Referenced in:

- 3.1. XRMesh structures (2) (3) (4)

#dom-xrmeshblock-vertices Referenced in:

- 3.1. XRMesh structures (2) (3)

#dom-xrmeshblock-indices Referenced in:

- 3.1. XRMesh structures

#dom-xrmeshblock-normals Referenced in:

- 3.1. XRMesh structures (2)

#xrnearmesh Referenced in:

- 3.1. XRMesh structures

#xrworldmesh Referenced in:

- 3.1. XRMesh structures

#dictdef-xrmetadata Referenced in:

- 3.2. XRFrame

#dom-xrmetadata-worldmesh Referenced in:

- 3.1. XRMesh structures

#dom-xrmetadata-nearmesh Referenced in:

- 3.1. XRMesh structures