Home A - Z Create page
Page Index

An Otter Wiki

Blame

eec37f Anonymous 2026-08-07 11:27:07
init
1 
---
2
title: System Architecture
3
aliases: [3netra Architecture, Meta App Architecture]
4
tags: [3netra/wiki, architecture, diagrams]
5
product: 3netra Meta Android App
6
source_repository: 3Netra-ai/3n_meta_android
7
source_ref: origin/prod
8
source_revision: b48aada8fd5427b5db0c02945569963c4ae543c8
9
last_verified: 2026-08-06
10
up: "[[Home]]"
11
---
12
13
# System Architecture
14
15
## High-Level System Diagram
16
17
```mermaid
18
flowchart TB
19
subgraph HW["Meta Ray-Ban Smart Glasses"]
20
CAM[12MP Camera]
21
MIC[Mic Array]
22
SPK[Open-ear Speakers]
23
TAP[Temple Touch Controls]
24
end
25
26
subgraph PHONE["3netra Companion App (Kotlin/Compose)"]
27
UI[UI Layer — Jetpack Compose]
28
BT[BluetoothService — orchestrator]
29
WCS[WearableCameraService — Meta MWDAT]
30
VLM[Vision Pipeline — SupabaseAiService]
31
FACE[FaceNet TFLite + Supabase face_embeddings]
32
TTS[TTS — Android TTS; Sarvam path conditional]
33
VOICE[VoiceCommandService / AudioTranscriptionService]
34
end
35
36
subgraph CLOUD["Cloud"]
37
SB[(Supabase — Auth/Postgres/Realtime)]
38
EDGE[Supabase Edge Function chat-vision]
39
OAI[OpenAI gpt-4o]
40
SARVAM[Sarvam.ai — translate/TTS/STT]
41
OLLAMA[Ollama LAN — moondream/llava — dev only]
42
end
43
44
CAM -->|MWDAT camera session| WCS
45
TAP -.gesture callback bridge not found.- BT
46
MIC -->|BT SCO| VOICE
47
BT --> WCS --> VLM
48
VLM -->|Functions invoke| EDGE --> OAI
49
BT --> TTS --> SARVAM
50
SPK -.audio.- TTS
51
UI --> SB
52
FACE --> SB
53
VLM -.legacy/dev only.- OLLAMA
54
```
55
56
## Service-Oriented App Architecture
57
58
```mermaid
59
flowchart LR
60
subgraph UI["UI Layer (Compose screens)"]
61
S[Screens + ViewModels]
62
end
63
subgraph SVC["Service Layer (singletons)"]
64
SUP[SupabaseService — auth+DB]
65
BLS[BluetoothService — gestures/pipeline/chat]
66
AI[SupabaseAiService — Edge Function client]
67
SAR[SarvamService — Indic voice]
68
WCSV[WearableCameraService — Meta camera]
69
FNM[FaceNetModel + Supabase face embeddings]
70
EMS[EmergencyAlertManager — SOS]
71
MISC[Hallway/Recess/Emotion services — prompt wrappers]
72
end
73
subgraph EXT["External"]
74
SB[Supabase]
75
EDGE[Supabase Edge Function]
76
SA[Sarvam API]
77
META[Meta MWDAT SDK / BLE]
78
AND[Android TTS / SpeechRecognizer / CameraX]
79
end
80
S --> SVC
81
SUP --> SB
82
BLS --> META
83
AI --> EDGE
84
SAR --> SA
85
WCSV --> META
86
EMS --> SB
87
MISC --> AWSS
88
BLS --> AND
89
```
90
91
## Repository Layout (polyglot)
92
93
```
94
3nApp-Mobile/
95
├── src/frontend/ # ★ PRIMARY — native Kotlin/Compose app (own Gradle build)
96
│ └── app/src/main/kotlin/com/threenetra/app/
97
│ ├── MainActivity.kt # entry: Meta SDK init, auth deep links, start destination
98
│ ├── MainViewModel.kt # Supabase auth StateFlow
99
│ ├── services/ (18) # Supabase, Bluetooth, AwsBackend, Sarvam, WearableCamera,
100
│ │ # VoiceCommand, AudioTranscription, Emotion, Hallway, Recess,
101
│ │ # EmergencyAlert, PinManager, SmartCapture, VisionService(Ollama)…
102
│ ├── ml/FaceNetModel.kt # TFLite FaceNet embeddings (facenet.tflite asset)
103
│ ├── ui/ # Navigation, auth, onboarding, student screens, NetraTheme
104
│ └── utils/ # EncryptedFaceStorageManager, MediaCompressor, PdfExportManager
105
├── src/backend/ # AWS SAM prototype — not referenced by the active app path
106
├── android/ # Expo/React Native generated shell (vestigial; no JS sources)
107
├── app/ # root-level Kotlin prototype (camera stub, returns null)
108
├── functions/ # Firebase Cloud Functions — Resend welcome/notification emails (legacy)
109
├── api/send-welcome.js # Vercel serverless — Resend welcome email (legacy)
110
├── supabase/ # Supabase CLI local dev config (no migrations/seed committed)
111
├── docs/ # 16-file doc suite (architecture, API, VLM, setup, debugging)
112
├── .agent_docs/ # PRD (md+pdf), architecture/design prompts, assets
113
└── skills/ # agent skills: prod-local-deploy, sync-agent-skills, vlm-providers
114
```
115
116
> [!warning] Two Android codebases coexist
117
> The maintained native app lives in `src/frontend/`; the Expo/RN shell at root/`android/` is missing its JS entry files (RN build would fail). All feature work happens in `src/frontend/`. See ADR-001 in [[12-Risks-Decisions]].
118
119
## Backends
120
121
### 1. Supabase (primary data/auth)
122
123
GoTrue auth (email/password, email OTP, Google IDToken), PostgREST, Realtime channel on `student_settings`. Kotlin SDK `io.github.jan-tennert.supabase` BOM 2.5.0 + Ktor. See [[08-Data-Model-Supabase]].
124
125
### 2. Supabase Edge Function (active app call target)
126
127
`SupabaseAiService` invokes `chat-vision` through the Supabase Functions client. The deployed function source calls OpenAI `gpt-4o`, but the Android client discards the function response and returns the literal success text `"AI response received"`. Thus capture/chat/summary UI wiring is present, but meaningful model narration is **not** verified at this ref.
128
129
### 3. AWS SAM "Routing Logic" prototype (not active)
130
131
One Lambda (`nodejs20.x`, 30s/512MB) behind API Gateway catch-all `/{proxy+}` is implemented, including routes below, but `AwsBackendService` has no production call sites (only its minimal test).
132
133
| Route | Function |
134
|---|---|
135
| `POST /api/chat` | OpenAI proxy — `{prompt, mediaType: NONE/IMAGE/VIDEO, frames[]}`; **gpt-4o** for video, **gpt-4o-mini** otherwise; max_tokens 300; system prompt "helpful and concise assistant for a visually impaired student" |
136
| `POST /api/vision` | Rekognition `DetectLabels` (max 10, min conf 75) |
137
| `POST /api/upload` | base64 image → S3 `uploads/{ts}.jpg` (no lifecycle rule is configured; a comment merely suggests one) |
138
| `POST /api/auth` | Supabase token verification |
139
140
### 4. Legacy email paths
141
142
Firebase Cloud Functions (`sendWelcomeEmail` on auth.user onCreate, `sendNotificationEmail` callable) and Vercel `api/send-welcome.js` — all via **Resend**. No `firebase.json`; Supabase replaced Firebase auth/data.
143
144
## AI Pipeline (capture → speech)
145
146
1. Trigger: UI button, glasses gesture, or voice command
147
2. `WearableCameraService.capturePhoto()` — MWDAT device session → 15fps LOW stream → still photo (Bitmap/HEIC → bytes; mutex-guarded)
148
3. Optional on-device FaceNet calculation followed by Supabase embedding lookup (People/Emotions modes)
149
4. `SupabaseAiService` invokes `chat-vision`; at this ref it replaces the response with a fixed success string
150
5. `logLocalSession()` → in-memory history and an attempted Supabase `sessions` insert
151
6. Speech is Android TTS in `SceneDescriptionScreen`; `BluetoothService.speak()` has a conditional Sarvam path. End-to-end voice/vendor behavior is untested.
152
153
See the sequence diagram in [[03-Frontend-Navigation#Frontend Flow Chart (core capture loop)]].
154
155
## Tech Stack
156
157
| Layer | Technology |
158
|---|---|
159
| UI | Jetpack Compose (BOM 2024.10.00) + Material3 + Navigation-Compose 2.7.7 |
160
| Language/Async | Kotlin 2.1.10, Coroutines, StateFlow, kotlinx-serialization |
161
| Auth/DB | Supabase (gotrue-kt, postgrest-kt, realtime-kt) + Ktor client |
162
| Identity | Google Credential Manager + play-services-auth 21.0.0 |
163
| Glasses | Meta MWDAT SDK 0.7.0 (GitHub Packages Maven) |
164
| Vision AI | Supabase `chat-vision` Edge Function → OpenAI gpt-4o; AWS and Ollama clients are unreferenced/legacy paths |
165
| Face ID | LiteRT 1.4.0 FaceNet; active flow fetches/stores embeddings in Supabase |
166
| Voice | Sarvam.ai + Android TextToSpeech/SpeechRecognizer |
167
| Camera (phone) | CameraX 1.4.2 (face registration) |
168
| Security | androidx.security-crypto (EncryptedSharedPreferences), jbcrypt |
169
| Build/CI | Gradle 8.x, GitHub Actions debug-APK build on pushes to `prod`; it does not run tests or lint |
170
171
---
172
173
> [!tip] Navigation
174
> ⬅️ [[01-Product-Overview|Product Overview]] · 🏠 [[Home]] · ➡️ [[03-Frontend-Navigation|Frontend & Navigation]]