Commit 96f813
2026-08-07 11:31:46 admin: init| /dev/null .. 04-user-journeys.md | |
| @@ 0,0 1,107 @@ | |
| + | --- |
| + | title: User Journeys |
| + | aliases: [Journeys, User Flows] |
| + | tags: [3netra/wiki, ux, journeys, diagrams] |
| + | product: 3netra Meta Android App |
| + | source_repository: 3Netra-ai/3n_meta_android |
| + | source_ref: origin/prod |
| + | source_revision: b48aada8fd5427b5db0c02945569963c4ae543c8 |
| + | last_verified: 2026-08-06 |
| + | up: "[[Home]]" |
| + | --- |
| + | |
| + | # User Journeys |
| + | |
| + | ## Journey Map (Aiden's campus day) |
| + | |
| + | ```mermaid |
| + | journey |
| + | title Aiden's campus day with 3netra |
| + | section Morning arrival |
| + | Onboard & pair glasses: 5: Aiden, Instructor |
| + | Campus navigation cue: 4: Aiden |
| + | section Class |
| + | What do I see (scene): 5: Aiden |
| + | Read lecture slides/whiteboard (OCR): 5: Aiden |
| + | Classroom/lecture mode + PDF notes: 4: Aiden, Instructor |
| + | Who's here (face ID): 5: Aiden |
| + | Feelings (emotion): 4: Aiden |
| + | section Campus life |
| + | Library/quad description: 4: Aiden |
| + | section Afternoon |
| + | Lecture notes + AI summary: 4: Aiden |
| + | Today's Story recap: 5: Aiden |
| + | ``` |
| + | |
| + | ## Glasses Gesture Map (product specification, not hardware verification) |
| + | |
| + | | Gesture | Action | Mode | Status | |
| + | |---|---|---|---| |
| + | | Single tap (right temple) | "What do I see?" | Scene | 🟡 callable method; Home invokes it, but no DAT callback registration was found | |
| + | | Double tap (right temple) | "Read what's written" | OCR/Read | 🟡 callable method only | |
| + | | Hold 2s (right temple) | "Who is in front of me?" / voice command | Person ID / mic | 🟡 callable method starts Android recognition; no DAT callback found | |
| + | | Single tap (left temple) | "What is happening here?" | Activity | ⏳ | |
| + | | Double tap (left temple) | "How does that person feel?" | Emotion | 🟡 callable method only | |
| + | | Voice "Hey Netra" | Wake word activation | — | ⏳ planned (service removed) | |
| + | |
| + | ## Journey 1 — First-Time Setup (~3 min) |
| + | |
| + | 1. Student/disability support coordinator installs app → Login/Signup (email+password, OTP, or Google) |
| + | 2. Permission rationale screen (BT, location, camera, mic, notifications) with FERPA/COPPA banner |
| + | 3. Onboarding: Netra welcome → academic level selection (tunes vocabulary for high school/college) → glasses pairing (Meta View app must be paired first; auto-connect scans for "Meta"/"RB Meta" devices) |
| + | 4. (Optional) companion selection + nickname — see [[05-Avatars-Companions]] |
| + | 5. Home: "👋 Hey, {name}! Ready to explore your campus?" |
| + | |
| + | ## Journey 2 — Student core loop (hands-free) |
| + | |
| + | 1. Gesture or voice triggers `BluetoothService` headless pipeline (no UI needed) |
| + | 2. When hardware capture succeeds, the app invokes Supabase `chat-vision`; the Android client currently discards its answer and substitutes fixed success text |
| + | 3. Scene UI speaks through Android TTS. Speaker routing and Sarvam behavior are not device-tested |
| + | 4. Every result logged → feeds My Story |
| + | |
| + | ## Journey 3 — "What do I see?" (UI path) |
| + | |
| + | Home → 🔍 → `scene_description/Scene` → the student taps a mode chip → capture attempt → fixed `"AI response received"` on a successful Edge Function call → Android TTS + attempted session log. First-load auto-capture is commented out. Mode chips let the student select Scene/Read/Emotions/People. |
| + | |
| + | ## Journey 4 — Reading the board (Classroom mode) |
| + | |
| + | Home exposes a Classroom quick action, but `SceneDescriptionScreen` has no Classroom chip/prompt branch and falls back to generic scene behavior. `BluetoothService` contains a Classroom/PDF branch, yet no verified UI or gesture route reaches it. The PDF helper uses legacy public external-storage writes and is not a reliable modern-Android export flow. |
| + | |
| + | ## Journey 5 — Recognizing a classmate or professor |
| + | |
| + | Instructor/disability support coordinator (PIN-gated) registers classmates & faculty: Friends tab → + → Admin PIN → name → CameraX capture → FaceNet embedding → Supabase. |
| + | Student: 👤 Who's here? → FaceNet calculates an embedding and fetches registered embeddings from Supabase for cosine comparison (>0.5). A match is read aloud locally; otherwise the code invokes the Edge Function. It is neither local-only nor verified as instant, and successful Edge calls still resolve to fixed success text in this client. |
| + | |
| + | > [!warning] Privacy caveat |
| + | > The active enrollment path stores embeddings in Supabase (`face_embeddings`), while the encrypted on-device store exists but is secondary in this flow — see [[11-Security-Privacy-Compliance#Critical gaps]]. |
| + | |
| + | ## Journey 6 — "Be My Eyes" chat |
| + | |
| + | Chat tab → type question and/or attach gallery image or video → `askAssistant` samples/compresses media → Supabase `chat-vision` → fixed success text on client-side success. Trash clears in-memory history. No response-time target has implementation or test evidence. |
| + | |
| + | ## Journey 7 — Lecture notes |
| + | |
| + | Home → 📝 Notes → Start Recording (device speech recognizer) → live transcript pane → Summarize Now → `SupabaseAiService.summarizeLecture`; the current client returns fixed success text rather than decoding the model summary. |
| + | |
| + | ## Journey 8 — End-of-day "Today's Story" |
| + | |
| + | My Story tab renders the day's sessions as an academic timeline (mode emoji, time, narration). **🔊 Read My Day Out Loud** compiles "Here is your day so far… That's everything I saw today!" and speaks it — designed for review or sharing. |
| + | |
| + | ## Journey 9 — Educator & Disability Support Specialist |
| + | |
| + | - **Face library management** (PIN-protected): register/remove classmates & professors |
| + | - **Settings:** some preferences persist locally; server settings synchronization is attempted. The high-contrast and reduce-motion preferences are explicitly local-only. |
| + | - **Dashboard layer** (`SupabaseDashboardRepository`): code exposes settings/audit calls, but no active educator surface or authorization workflow is verified. |
| + | - **Planned:** web Educator Portal, Disability Support Portal summaries, weekly academic reports |
| + | |
| + | ## Journey 10 — Emergency SOS (scaffolded) |
| + | |
| + | Intended: hold temple 2s → `EmergencyAlertManager` (`TRIGGER_SOS` intent) → payload `{student_id, GPS, recent_summaries, status:"active"}` → Supabase `emergency_alerts` → emergency contact notified. |
| + | |
| + | > [!warning] Current state |
| + | > Service exists, GPS + summaries are TODO stubs, and no UI/gesture triggers it yet. Do not represent as operational — see [[12-Risks-Decisions]]. |
| + | |
| + | --- |
| + | |
| + | > [!tip] Navigation |
| + | > ⬅️ [[03-Frontend-Navigation|Frontend & Navigation]] · 🏠 [[Home]] · ➡️ [[05-Avatars-Companions|Avatars & Companions]] |