RotoTextwiki

RotoText

From the RotoText wiki

RotoText is a web application that uses eye tracking and geometric projection to let players align their gaze to solve a Line of Sight puzzle game on their device screen. It detects the user's face and iris from a camera, estimates 3D eye position, and projects that position and game elements onto the screen to create the illusion of depth and motion parallax as the player moves their head. The application supports both desktop and mobile devices by switching between high and low resolution camera modes and adapting settings like device orientation and focal length.

flowchart LR
  n1[["Device configuration"]]
  n2[["Eye tracking"]]
  n3[["Game engine"]]
  n4[["Geometric projection"]]
  n5[["Interaction UI"]]
  n6[["Project infrastructure"]]
  n7[["Scene rendering"]]
  n5 -->|"12 calls, 10 imports"| n4
  n3 -->|"8 calls, 9 imports"| n4
  n5 -->|"11 calls, 5 imports"| n1
  n5 -->|"10 calls, 6 imports"| n3
  n7 -->|"7 calls, 7 imports"| n4
  n5 -->|"8 calls, 5 imports"| n2
  n4 -->|"7 calls, 3 imports"| n3
  n1 -->|"4 calls, 4 imports"| n4
  n5 -->|"6 calls, 2 imports"| n7
  n2 -->|"2 calls, 2 imports"| n4
  n4 -->|"3 calls, 1 import"| n7
  n1 -->|"1 call, 1 import"| n2
  n3 -->|"1 call, 1 import"| n7
  n4 -->|"1 call, 1 import"| n1
Each box is a feature; an arrow points from the feature whose code calls or imports to the one it uses.

Purpose and features

Players solve a Line of Sight puzzle by moving their eye to align pieces on the screen; the game progresses through 7 levels with different shapes. (see Game engine) The application displays a 3D scene with ring targets at varying depths inside a box, rendered in SVG and projected from the tracked eye position to show motion parallax. (see Scene rendering) Users can calibrate the camera and adjust settings like iris diameter, smoothing parameters, and device presets through the interaction UI and debug panel.[1] (see Interaction UI, Device configuration) The system applies one-euro filtering to reduce jitter in eye position and depth data collected from camera input. (see Device configuration)

Layers

  • The eye-tracking layer detects facial landmarks from video using MediaPipe and performs geometric calculations to triangulate 3D eye position from iris size and midpoint, with support for both 1280×720 desktop and 640×480 mobile camera resolutions. (see Eye tracking)
  • The device configuration layer stores hardware presets for MacBooks and iPhones with their camera positions and lens angles, maintains user-adjustable settings like iris diameter and smoothing parameters, and applies filtering to smooth eye-tracking signals. (see Device configuration)
  • The geometric projection layer converts 3D eye and scene positions into 2D screen coordinates through perspective transformations, billboarding, and viewport mappings, handling correction for viewing angle and constant apparent size across distances. (see Geometric projection)
  • The game engine layer maintains game state across 7 levels, advances play in response to player input, and computes visual properties of pieces including their on-screen alignment and color based on distance from the solution position. (see Game engine)
  • The scene rendering layer builds and displays the 3D box and target scene in SVG by projecting world geometry onto the screen from the tracked eye position, showing eight ring targets at varying depths with grid-lined walls. (see Scene rendering)
  • The interaction UI layer displays the user interface and handles input across modes for viewing text, target discovery, and the Line of Sight puzzle game, coordinating camera calibration, game progress display, and mouse-mode eye control. (see Interaction UI)

Request paths

  • Scene rendering constructs the backdrop through buildBoxGrid() and buildTargetsScene(), which call rectCenter() and viewportRectMm() to position elements in world space, then createSvgRenderer() projects them by calling projectPoint() for each primitive.[2][3]
  • On each video frame, openCamera() provides a frame to Eye tracking's FaceTracker.detect(), which returns facial landmarks; eyeFromIris() and irisMidpoint() extract iris data and compute 3D eye position in camera space.[4]
  • Eye position is smoothed through OneEuroFilter3, then at render time projectPoint() projects it and game pieces onto the screen using the ScreenFrame, and cameraToScreen() and layoutToScreenRect() map those coordinates to viewport pixels.[5]
  • The game loop calls stepSession() to advance the game state, gameView() to compute piece positions, buildLevel() to generate scene geometry, and misalignmentPx() to determine alignment status for rendering.[6]
  • Device settings flow through buildScreenFrame() to create a ScreenFrame that encodes camera position, focal length, and viewport dimensions for all coordinate transformations.[7]

Feature dependencies

  • Interaction UI imports and calls functions from Geometric projection (computeCorrection(), cameraToScreen(), layoutToScreenRect()), Device configuration (findPreset(), guessPreset(), settingsFromPreset()), and Game engine (stepSession(), gameView(), LEVELS, buildLevel()).[8]
  • Interaction UI also imports and calls functions from Eye tracking (openCamera(), eyeFromIris(), irisMidpoint()) and Scene rendering (createSvgRenderer(), buildBoxGrid(), buildTargetsScene()).[9]
  • Game engine imports from Geometric projection (projectPoint(), screenMmToViewport()) and Scene rendering (rectCenter(), ViewportRectMm).[10]
  • Scene rendering imports from Geometric projection (projectPoint(), ScreenFrame) to project 3D scene primitives onto the screen.[3]
  • Eye tracking imports vec3 from Geometric projection and uses it to construct 3D eye positions from 2D iris data.[11]
  • Device configuration imports from Geometric projection to build screen frames from device settings.[12]

References

  1. ^ src/ui/debugPanel.ts:L4-13@dbf2b8f (NumericKey)
  2. ^ src/main.ts:L14-15@dbf2b8f
  3. ^ a b src/scene/svgRenderer.ts:L1-2@dbf2b8f
  4. ^ src/main.ts:L17-18@dbf2b8f
  5. ^ src/main.ts:L10-11@dbf2b8f
  6. ^ src/main.ts:L2-5@dbf2b8f
  7. ^ src/main.ts:L13@dbf2b8f
  8. ^ src/main.ts:L9-13@dbf2b8f
  9. ^ src/main.ts:L14-18@dbf2b8f
  10. ^ src/game/alignment.ts:L1-2@dbf2b8f
  11. ^ src/tracking/eyeEstimator.ts:L1@dbf2b8f
  12. ^ src/settings.ts:L2@dbf2b8f

This page was last edited on 1 October 2026, at commit dbf2b8f.