Example app and hardware integrations for AR in a web browser with 3 degrees of freedom

Live Demo

System Requirements
Installation
Usage
How It Works
Student, Researcher, and OEM Info
Calibration and Performance
Manual Build and Deploy

*Experimental Android setup uses the device's integrated IMU instead of headset drivers, and therefore doesn't require installation. If your use case is mobile-only, see spidgets-mobile for a simpler repo and instructions for Termux.

Setting up new drivers

1. Clone this repo and open it in a terminal

2. Run chmod u+x ./bin/ar-server

3. Place the headset upright on a flat surface

4. Run sudo ./bin/ar-server and wait for calibration

Optional: Follow the udev step here for non-su

1. Clone this repo and open it in a powershell terminal

2. Place the headset upright on a flat surface

3. Run bin/ar-server and wait for calibration (first time setup)

Skip to the manual build

Android skip to the webserver step in the manual build.

Optional: Some third party websites (e.g. YouTube) need this extension in order to bypass restrictions caused by iframe headers. This may violate third party TOS or void warranties. Use at your own risk. On Android, extensions seem to only be available with Kiwi or Yandex browsers.

Soon: When stable, we can host an easier download/install process on a CDN + option to run in background on startup

Run bin/ar-server in a terminal

Browse to http://localhost:8000

Experiment with simple HUDs by editing webroot/index.html. Stay tuned for more interactive examples and docs.

Options:

• Change port with the --port flag e.g. bin/ar-server --port 3000
• Pre-record 10 seconds of motion data to allow testing without a headset using the --record flag. See webroot/compat for mock data example.
• Calibrate for drift with the --cal flag. Refer to the calibration section.

1. Rust driver euler_60 reads the raw sensor data from the glasses and outputs euler angles (i.e. roll, pitch, and yaw) at 60Hz. Source

2. Server-side JS process ar-server manages the connection, corrects for yaw drift, and exposes the euler angles on a socket.io connection. Also exposes functions like re-centering and power save mode

3. Front-end in webroot/index.html listens to the websocket and uses spidgets-core to position the widgets and convert euler angles to matrix3d calculations to simulate 3D space

Front-end in webroot/index.html uses spidgets-core to position the widgets and convert deviceorientation events in the browser to matrix3d calculations to simulate 3D space

Connect new IMU devices by replacing bin/euler_60 driver with your own stdout program binary

Expected format:

...
   0.03188  -0.48346  -0.00605
   0.03174  -0.48465  -0.00596
   0.03166  -0.48591  -0.00588
   0.03196  -0.48720  -0.00600
...

i.e.

{whitespace}{pitch radians}{whitespace}{yaw radians}{whitespace}{roll radians}

Printed at or above target refresh rate

Then go to installation. For dev support, open an issue or contact: [email protected]

Windows, Linux: see if the euler angles from your glasses are printing to the console by running the below command. If not, build headset-utils and copy the new euler_60 binary to the same location.

bin/euler_60

MacOS do the same with this instead:

bin/euler_60_apple_silicon

Android doesn't need euler_60.

Then get the webserver ready. Requires Bun or NodeJS

Start ar-server.js. You may need sudo for the second step:

bun install
bun ar-server.js

then browse to http://localhost:8000

Create the main executable for end users:

bun build ./ar-server.js --compile --outfile bin/ar-server

Calibration is a one-minute process that measures yaw drift rate and saves it for later. This process will run automatically during first time setup. It can also be run manually like so:

bin/ar-server --cal

The headset should be placed upright on a flat stable surface for the full 60 seconds. Console messages will indicate when the process is complete and the UI is ready.

At this time we don't store config profiles per-headset, so calibration will need to be re-run if multiple headsets are used

Optional: If the headset is usually used at a different angle than upright (e.g. in a recliner), it would be more accurate to wear it during calibration. Measurement starts 5 seconds after the command to allow time to adjust. The user can move freely during the test but should be in roughly the same orientation at T+5 and T+60.

• With Rokid glasses, use a 120Hz display mode (either 1920x1080 or x1200 is okay). 60Hz creates input lag
• iframes of third party websites can be very GPU-intensive especially if they have a lot of divs. See webroot/widgets for ideas for building performant widgets. HTML custom elements are the cleanest pattern.
• If you have a discrete GPU, you may want to change your OS settings to use it for the browser. Most browsers will use the iGPU by default