← back to robot-framework

# 📱 phone-head — the Pixel taped to Dash's head is a sensor pod

This is the one Jake wanted. A Pixel zip-tied to the robot's head, streaming its rear camera + every browser-reachable sensor over wifi to the Mac, so Claude runs the OODA loop from a first-person view with a real compass. Built same-night from Jake's 6/11 voice memo. Source of truth: `phone-head/server.mjs` + `phone-head/README.md` + `phone-head/CLAUDE-LOOP.md`.

## What it does (sensors UP — the opposite of the Android remote)

Jake's 6/11 brief: "make an android app that uses all sensors android has … assume I'm gonna tape the phone to the head of the robot and do similar kinds of ooda exploration. phone needs to talk to local computer over wifi, needs to be wireless. claude is in control of ooda."

Why it matters: the first-person camera + an absolute compass give the rig its first ground-truth turn measurement — snapshot heading before a turn, again after, instead of photo-guessing the angle from the old top-down webcam.

## The phone page (rendered from phone.html — not a screen-grab)

No real screenshot exists in the repo. The mockup below is drawn faithfully from `phone-head/phone.html` — the single page the Pixel opens in Chrome. To replace it with a real photo, run the server and grab a frame on the phone (see "How to run it").

STREAMING · 8788 BATT 78%
phone-head
Rear cam + sensors → Mac over wifi.
Camera (rear, ~1.4 fps)
first-person frame.jpg
Live sensors (5 Hz)
compass 213° accel xyz 0.02 0.01 9.79 gyro−0.4 0.1 0.0 light 142 lux mic rms 0.06
START STREAMING

Open `https://<mac-ip>:8788/` in Chrome → allow camera+mic+location → tap START. Wake-lock keeps the screen on.

## What it is

the build | Not a native app (yet) — a single web page served from the Mac over LAN, opened in Chrome on the Pixel. Zero install, zero npm deps, wireless, works tonight. HTTPS is mandatory (Chrome gates camera/mic/motion/geo behind secure contexts); the server auto-generates a self-signed cert into `certs/` on first run. The WebSocket server is hand-rolled RFC 6455 inside `server.mjs`.
senses | Rear camera (getUserMedia → canvas → JPEG ~1.4 fps) + 5 Hz packets: accel/gyro (DeviceMotion), absolute compass (DeviceOrientation), magnetometer, ambient light (or camera mean-luma fallback), mic RMS level, geolocation, battery. `state.caps` reports which sensors actually came alive on this phone.
claude's loop | OBSERVE `curl /state` + `/frame.jpg` (check `stateAgeMs`/`frameAgeMs` < 3000 first) → ORIENT (fuse compass + IMU + frame) → DECIDE one verb + a written prediction → ACT `POST :5174/command` → check. Full contract: `phone-head/CLAUDE-LOOP.md`.
HTTP surface | `GET /` phone page · `/watch` Mac dashboard · `/health` · `/state` · `/frame.jpg` · `/history?n=50` (ring buffer, ~2 min @ 5 Hz — "did it shake when it hit the wall?") · `WS /uplink` (phone → server).
how to run it | `node phone-head/server.mjs` on the Mac (prints `https://<mac-lan-ip>:8788/`) → Pixel on same wifi opens that URL in Chrome → Advanced → Proceed (self-signed) → tape to Dash's head, camera out → START STREAMING. Smoke-tested end-to-end with `phone-head/test-uplink.mjs` (fake phone over WS).
limits | Pixels have no lidar; rear ToF/depth + barometer are NOT browser-reachable. Tab throttles when backgrounded, screen must stay on, mic is RMS level not raw audio. Those are native-path items.

## Native-Android upgrade path (when the browser ceiling is hit)

① Termux + termux-api (~1 hr) | `pkg install termux-api` → `termux-sensor -a` dumps EVERY hardware sensor as JSON (barometer, proximity, step counter, raw uncalibrated IMU). Pipe to `curl`/websocat. No app build.
② eject the Expo app (~1 evening) | `expo prebuild` the existing `../android/` app, add `react-native-sensors` + `react-native-vision-camera`. One codebase, both directions (sensors up, commands down).
③ bare Kotlin service | SensorManager (all ~25 sensor types incl. TYPE_PRESSURE, TYPE_PROXIMITY), Camera2/CameraX with DEPTH16 ToF where hardware exists, AudioRecord raw PCM, runs screen-off, survives Doze. The "phone is permanently the robot's head" build.

Direction note: phone-head is sensors UP (Pixel → Mac → claude). The sibling Android.html ("Dash Remote") is the opposite — commands DOWN (phone → laptop → robot). Different jobs, same robot.