Blame
|
1 | --- |
||||||
| 2 | title: User Journeys |
|||||||
| 3 | aliases: [Journeys, User Flows] |
|||||||
| 4 | tags: [3netra/wiki, ux, journeys, diagrams] |
|||||||
| 5 | product: 3netra Meta Android App |
|||||||
| 6 | source_repository: 3Netra-ai/3n_meta_android |
|||||||
| 7 | source_ref: origin/prod |
|||||||
| 8 | source_revision: b48aada8fd5427b5db0c02945569963c4ae543c8 |
|||||||
| 9 | last_verified: 2026-08-06 |
|||||||
| 10 | up: "[[Home]]" |
|||||||
| 11 | --- |
|||||||
| 12 | ||||||||
| 13 | # User Journeys |
|||||||
| 14 | ||||||||
| 15 | ## Journey Map (Aiden's campus day) |
|||||||
| 16 | ||||||||
| 17 | ```mermaid |
|||||||
| 18 | journey |
|||||||
| 19 | title Aiden's campus day with 3netra |
|||||||
| 20 | section Morning arrival |
|||||||
| 21 | Onboard & pair glasses: 5: Aiden, Instructor |
|||||||
| 22 | Campus navigation cue: 4: Aiden |
|||||||
| 23 | section Class |
|||||||
| 24 | What do I see (scene): 5: Aiden |
|||||||
| 25 | Read lecture slides/whiteboard (OCR): 5: Aiden |
|||||||
| 26 | Classroom/lecture mode + PDF notes: 4: Aiden, Instructor |
|||||||
| 27 | Who's here (face ID): 5: Aiden |
|||||||
| 28 | Feelings (emotion): 4: Aiden |
|||||||
| 29 | section Campus life |
|||||||
| 30 | Library/quad description: 4: Aiden |
|||||||
| 31 | section Afternoon |
|||||||
| 32 | Lecture notes + AI summary: 4: Aiden |
|||||||
| 33 | Today's Story recap: 5: Aiden |
|||||||
| 34 | ``` |
|||||||
| 35 | ||||||||
| 36 | ## Glasses Gesture Map (product specification, not hardware verification) |
|||||||
| 37 | ||||||||
| 38 | | Gesture | Action | Mode | Status | |
|||||||
| 39 | |---|---|---|---| |
|||||||
| 40 | | Single tap (right temple) | "What do I see?" | Scene | π‘ callable method; Home invokes it, but no DAT callback registration was found | |
|||||||
| 41 | | Double tap (right temple) | "Read what's written" | OCR/Read | π‘ callable method only | |
|||||||
| 42 | | Hold 2s (right temple) | "Who is in front of me?" / voice command | Person ID / mic | π‘ callable method starts Android recognition; no DAT callback found | |
|||||||
| 43 | | Single tap (left temple) | "What is happening here?" | Activity | β³ | |
|||||||
| 44 | | Double tap (left temple) | "How does that person feel?" | Emotion | π‘ callable method only | |
|||||||
| 45 | | Voice "Hey Netra" | Wake word activation | β | β³ planned (service removed) | |
|||||||
| 46 | ||||||||
| 47 | ## Journey 1 β First-Time Setup (~3 min) |
|||||||
| 48 | ||||||||
| 49 | 1. Student/disability support coordinator installs app β Login/Signup (email+password, OTP, or Google) |
|||||||
| 50 | 2. Permission rationale screen (BT, location, camera, mic, notifications) with FERPA/COPPA banner |
|||||||
| 51 | 3. Onboarding: Netra welcome β academic level selection (tunes vocabulary for high school/college) β glasses pairing (Meta View app must be paired first; auto-connect scans for "Meta"/"RB Meta" devices) |
|||||||
| 52 | 4. (Optional) companion selection + nickname β see [[05-Avatars-Companions]] |
|||||||
| 53 | 5. Home: "π Hey, {name}! Ready to explore your campus?" |
|||||||
| 54 | ||||||||
| 55 | ## Journey 2 β Student core loop (hands-free) |
|||||||
| 56 | ||||||||
| 57 | 1. Gesture or voice triggers `BluetoothService` headless pipeline (no UI needed) |
|||||||
| 58 | 2. When hardware capture succeeds, the app invokes Supabase `chat-vision`; the Android client currently discards its answer and substitutes fixed success text |
|||||||
| 59 | 3. Scene UI speaks through Android TTS. Speaker routing and Sarvam behavior are not device-tested |
|||||||
| 60 | 4. Every result logged β feeds My Story |
|||||||
| 61 | ||||||||
| 62 | ## Journey 3 β "What do I see?" (UI path) |
|||||||
| 63 | ||||||||
| 64 | Home β π β `scene_description/Scene` β the student taps a mode chip β capture attempt β fixed `"AI response received"` on a successful Edge Function call β Android TTS + attempted session log. First-load auto-capture is commented out. Mode chips let the student select Scene/Read/Emotions/People. |
|||||||
| 65 | ||||||||
| 66 | ## Journey 4 β Reading the board (Classroom mode) |
|||||||
| 67 | ||||||||
| 68 | Home exposes a Classroom quick action, but `SceneDescriptionScreen` has no Classroom chip/prompt branch and falls back to generic scene behavior. `BluetoothService` contains a Classroom/PDF branch, yet no verified UI or gesture route reaches it. The PDF helper uses legacy public external-storage writes and is not a reliable modern-Android export flow. |
|||||||
| 69 | ||||||||
| 70 | ## Journey 5 β Recognizing a classmate or professor |
|||||||
| 71 | ||||||||
| 72 | Instructor/disability support coordinator (PIN-gated) registers classmates & faculty: Friends tab β + β Admin PIN β name β CameraX capture β FaceNet embedding β Supabase. |
|||||||
| 73 | Student: π€ Who's here? β FaceNet calculates an embedding and fetches registered embeddings from Supabase for cosine comparison (>0.5). A match is read aloud locally; otherwise the code invokes the Edge Function. It is neither local-only nor verified as instant, and successful Edge calls still resolve to fixed success text in this client. |
|||||||
| 74 | ||||||||
| 75 | > [!warning] Privacy caveat |
|||||||
| 76 | > The active enrollment path stores embeddings in Supabase (`face_embeddings`), while the encrypted on-device store exists but is secondary in this flow β see [[11-Security-Privacy-Compliance#Critical gaps]]. |
|||||||
| 77 | ||||||||
| 78 | ## Journey 6 β "Be My Eyes" chat |
|||||||
| 79 | ||||||||
| 80 | Chat tab β type question and/or attach gallery image or video β `askAssistant` samples/compresses media β Supabase `chat-vision` β fixed success text on client-side success. Trash clears in-memory history. No response-time target has implementation or test evidence. |
|||||||
| 81 | ||||||||
| 82 | ## Journey 7 β Lecture notes |
|||||||
| 83 | ||||||||
| 84 | Home β π Notes β Start Recording (device speech recognizer) β live transcript pane β Summarize Now β `SupabaseAiService.summarizeLecture`; the current client returns fixed success text rather than decoding the model summary. |
|||||||
| 85 | ||||||||
| 86 | ## Journey 8 β End-of-day "Today's Story" |
|||||||
| 87 | ||||||||
| 88 | My Story tab renders the day's sessions as an academic timeline (mode emoji, time, narration). **π Read My Day Out Loud** compiles "Here is your day so farβ¦ That's everything I saw today!" and speaks it β designed for review or sharing. |
|||||||
| 89 | ||||||||
| 90 | ## Journey 9 β Educator & Disability Support Specialist |
|||||||
| 91 | ||||||||
| 92 | - **Face library management** (PIN-protected): register/remove classmates & professors |
|||||||
| 93 | - **Settings:** some preferences persist locally; server settings synchronization is attempted. The high-contrast and reduce-motion preferences are explicitly local-only. |
|||||||
| 94 | - **Dashboard layer** (`SupabaseDashboardRepository`): code exposes settings/audit calls, but no active educator surface or authorization workflow is verified. |
|||||||
| 95 | - **Planned:** web Educator Portal, Disability Support Portal summaries, weekly academic reports |
|||||||
| 96 | ||||||||
| 97 | ## Journey 10 β Emergency SOS (scaffolded) |
|||||||
| 98 | ||||||||
| 99 | Intended: hold temple 2s β `EmergencyAlertManager` (`TRIGGER_SOS` intent) β payload `{student_id, GPS, recent_summaries, status:"active"}` β Supabase `emergency_alerts` β emergency contact notified. |
|||||||
| 100 | ||||||||
| 101 | > [!warning] Current state |
|||||||
| 102 | > Service exists, GPS + summaries are TODO stubs, and no UI/gesture triggers it yet. Do not represent as operational β see [[12-Risks-Decisions]]. |
|||||||
| 103 | ||||||||
| 104 | --- |
|||||||
| 105 | ||||||||
| 106 | > [!tip] Navigation |
|||||||
| 107 | > β¬
οΈ [[03-Frontend-Navigation|Frontend & Navigation]] Β· π [[Home]] Β· β‘οΈ [[05-Avatars-Companions|Avatars & Companions]] |
|||||||
