📱 phone-head — the Pixel taped to Dash's head is a sensor pod
This is the one Jake wanted. A Pixel zip-tied to the robot's head, streaming its rear camera + every browser-reachable sensor over wifi to the Mac, so Claude runs the OODA loop from a first-person view with a real compass. Built same-night from Jake's 6/11 voice memo. Source of truth: phone-head/server.mjs + phone-head/README.md + phone-head/CLAUDE-LOOP.md.
What it does (sensors UP — the opposite of the Android remote)
Jake's 6/11 brief: "make an android app that uses all sensors android has … assume I'm gonna tape the phone to the head of the robot and do similar kinds of ooda exploration. phone needs to talk to local computer over wifi, needs to be wireless. claude is in control of ooda."
The phone page (rendered from phone.html — not a screen-grab)
No real screenshot exists in the repo. The mockup below is drawn faithfully from phone-head/phone.html — the single page the Pixel opens in Chrome. To replace it with a real photo, run the server and grab a frame on the phone (see "How to run it").
Open https://<mac-ip>:8788/ in Chrome → allow camera+mic+location → tap START. Wake-lock keeps the screen on.
What it is
| the build | Not a native app (yet) — a single web page served from the Mac over LAN, opened in Chrome on the Pixel. Zero install, zero npm deps, wireless, works tonight. HTTPS is mandatory (Chrome gates camera/mic/motion/geo behind secure contexts); the server auto-generates a self-signed cert into certs/ on first run. The WebSocket server is hand-rolled RFC 6455 inside server.mjs. |
|---|---|
| senses | Rear camera (getUserMedia → canvas → JPEG ~1.4 fps) + 5 Hz packets: accel/gyro (DeviceMotion), absolute compass (DeviceOrientation), magnetometer, ambient light (or camera mean-luma fallback), mic RMS level, geolocation, battery. state.caps reports which sensors actually came alive on this phone. |
| claude's loop | OBSERVE curl /state + /frame.jpg (check stateAgeMs/frameAgeMs < 3000 first) → ORIENT (fuse compass + IMU + frame) → DECIDE one verb + a written prediction → ACT POST :5174/command → check. Full contract: phone-head/CLAUDE-LOOP.md. |
| HTTP surface | GET / phone page · /watch Mac dashboard · /health · /state · /frame.jpg · /history?n=50 (ring buffer, ~2 min @ 5 Hz — "did it shake when it hit the wall?") · WS /uplink (phone → server). |
| how to run it | node phone-head/server.mjs on the Mac (prints https://<mac-lan-ip>:8788/) → Pixel on same wifi opens that URL in Chrome → Advanced → Proceed (self-signed) → tape to Dash's head, camera out → START STREAMING. Smoke-tested end-to-end with phone-head/test-uplink.mjs (fake phone over WS). |
| limits | Pixels have no lidar; rear ToF/depth + barometer are NOT browser-reachable. Tab throttles when backgrounded, screen must stay on, mic is RMS level not raw audio. Those are native-path items. |
Native-Android upgrade path (when the browser ceiling is hit)
| ① Termux + termux-api (~1 hr) | pkg install termux-api → termux-sensor -a dumps EVERY hardware sensor as JSON (barometer, proximity, step counter, raw uncalibrated IMU). Pipe to curl/websocat. No app build. |
|---|---|
| ② eject the Expo app (~1 evening) | expo prebuild the existing ../android/ app, add react-native-sensors + react-native-vision-camera. One codebase, both directions (sensors up, commands down). |
| ③ bare Kotlin service | SensorManager (all ~25 sensor types incl. TYPE_PRESSURE, TYPE_PROXIMITY), Camera2/CameraX with DEPTH16 ToF where hardware exists, AudioRecord raw PCM, runs screen-off, survives Doze. The "phone is permanently the robot's head" build. |