app/plans/app-state-machine-architecture.md

Ponderants App State Machine Architecture

Executive Summary

This document outlines the complete hierarchical state machine architecture for Ponderants, integrating the recently-completed Voice Mode with the full app experience: Conversation → Edit → Galaxy visualization.

Current Status: Voice Mode state machine is complete and tested (✅) Remaining Work: 3 major phases across ~15-20 implementation tasks

---

1. Current State (What We Have)

✅ Completed

• Voice Mode State Machine (lib/voice-machine.ts)

  • States: idle, checkingForGreeting, listening, userSpeaking, timingOut, submittingUser, waitingForAI, generatingTTS, playingTTS
  • Fully tested with development controls
  • Clean XState v5 implementation

• Chat Interface (app/chat/page.tsx)

  • Text input with AI responses
  • Voice mode integration
  • Initial greeting message
  • User menu with logout

• Authentication (OAuth with Bluesky/ATproto)

• AI Integration (Vercel AI SDK with Gemini)

• TTS (Deepgram API)

❌ Missing

• Node creation/extraction from conversation
• Node editing interface
• 3D galaxy visualization
• App-level state management
• Persistent navigation UI
• ATproto publishing
• Vector embeddings & linking

---

2. Hierarchical State Machine Architecture

Level 1: App Machine (Top Level)

┌─────────────────────────────────────────────────────────┐
│                     APP MACHINE                         │
│                                                         │
│  ┌─────────┐      ┌─────────┐      ┌─────────┐       │
│  │  Convo  │ ←──→ │  Edit   │ ←──→ │ Galaxy  │       │
│  │         │      │         │      │         │       │
│  └────┬────┘      └─────────┘      └─────────┘       │
│       │                                                │
│       └─── Manages: voiceMode / textMode              │
└─────────────────────────────────────────────────────────┘

States:

• convo: Active conversation (voice or text)
• edit: Editing a node
• galaxy: 3D visualization of node graph

Context:

{
  currentNodeId: string | null;
  pendingNodeDraft: NodeDraft | null;
  nodes: Node[];
  mode: 'mobile' | 'desktop';
}

Events:

• EDIT_NODE (from conversation or save button)
• VIEW_GALAXY (from nav button)
• RETURN_TO_CONVO (from nav button)
• PUBLISH_NODE (from edit mode)
• CANCEL_EDIT

Level 2: Conversation Machine (Child of App.Convo)

┌─────────────────────────────────────────────────────────┐
│               CONVERSATION MACHINE                      │
│                                                         │
│  ┌─────────┐                  ┌─────────┐             │
│  │  Voice  │ ←──────────────→ │  Text   │             │
│  │         │                  │         │             │
│  └────┬────┘                  └─────────┘             │
│       │                                                │
│       └─── Embeds: voiceMachine (from lib/voice...)   │
└─────────────────────────────────────────────────────────┘

States:

• voice: Voice conversation mode (invokes voiceMachine)
• text: Text-only conversation mode

Context:

{
  messages: Message[];
  suggestedNodes: NodeSuggestion[];
}

Events:

• TOGGLE_VOICE
• TOGGLE_TEXT
• SUGGEST_NODE (from AI)
• CREATE_NODE (user confirms suggestion)

Level 3: Voice Machine (Existing - Child of Conversation.Voice)

Already implemented in lib/voice-machine.ts. No changes needed.

---

3. Data Model

Node Schema

interface Node {
  id: string; // ATproto record URI
  title: string;
  content: string; // Markdown
  embedding: number[]; // gemini-embedding-001 (768 dims)
  links: {
    to: string; // Node ID
    strength: number; // 0-1, from vector similarity
    userApproved: boolean;
  }[];
  position3D: { x: number; y: number; z: number }; // UMAP coords
  createdAt: Date;
  updatedAt: Date;
  published: boolean; // Published to ATproto PDS
}

interface NodeDraft {
  title: string;
  content: string;
  conversationContext: Message[]; // Last N messages
}

interface NodeSuggestion {
  draft: NodeDraft;
  confidence: number; // AI's confidence in suggestion
}

---

4. UI Architecture

Responsive Navigation

Mobile (< 768px)

┌─────────────────────────────────────┐
│         App Content                 │
│                                     │
│                                     │
│                                     │
│                                     │
├─────────────────────────────────────┤
│  [Convo] [Edit] [Galaxy]           │  ← Bottom Bar
└─────────────────────────────────────┘

Desktop (≥ 768px)

┌─────┬──────────────────────────────┐
│     │                              │
│  C  │                              │
│  o  │      App Content             │
│  n  │                              │
│  v  │                              │
│  o  │                              │
│     │                              │
│  E  │                              │
│  d  │                              │
│  i  │                              │
│  t  │                              │
│     │                              │
│  G  │                              │
│  a  │                              │
│  l  │                              │
│  a  │                              │
│  x  │                              │
│  y  │                              │
├─────┴──────────────────────────────┤
│   User Menu                        │
└────────────────────────────────────┘

Component Structure

app/
├── layout.tsx (with AppShell from Mantine)
├── page.tsx (redirects to /chat)
└── chat/
    └── page.tsx

components/
├── AppStateMachine.tsx (Provides app-level state context)
├── Navigation/
│   ├── MobileBottomBar.tsx
│   └── DesktopSidebar.tsx
├── Conversation/
│   ├── ConversationView.tsx (existing chat UI)
│   ├── VoiceControls.tsx (extracted from page)
│   ├── TextInput.tsx (extracted from page)
│   └── NodeSuggestionCard.tsx (NEW - shows AI suggestion)
├── Edit/
│   ├── NodeEditor.tsx (NEW - Mantine form with RTE)
│   └── LinkSuggestions.tsx (NEW - shows related nodes)
└── Galaxy/
    ├── GalaxyView.tsx (NEW - R3F canvas)
    ├── NodeMesh.tsx (NEW - 3D node representation)
    └── ConnectionLines.tsx (NEW - edges between nodes)

lib/
├── app-machine.ts (NEW - top-level state machine)
├── conversation-machine.ts (NEW - voice/text toggle)
└── voice-machine.ts (EXISTING ✅)

hooks/
├── useAppMachine.ts (NEW)
├── useConversationMode.ts (NEW)
└── useVoiceMode.ts (EXISTING ✅)

---

5. Implementation Phases

Phase 1: App State Machine Foundation (Est: 2-3 hours)

Tasks:

1. Create lib/app-machine.ts with states: convo, edit, galaxy
2. Create components/AppStateMachine.tsx provider
3. Update app/layout.tsx to wrap with provider
4. Create hooks/useAppMachine.ts

Acceptance Criteria:

• Can transition between convo/edit/galaxy states
• State persists across page navigations
• Development panel shows current app state

---

Phase 2: Navigation UI (Est: 2-3 hours)

Tasks:

1. Create components/Navigation/MobileBottomBar.tsx
   • 3 buttons: Convo, Edit, Galaxy
   • Highlights active mode
   • Fixed position at bottom
2. Create components/Navigation/DesktopSidebar.tsx
   • Vertical layout
   • Icons + labels
   • Mantine NavLink components
3. Update app/layout.tsx with responsive navigation
4. Add Mantine AppShell for layout management

Acceptance Criteria:

• Navigation shows on all pages
• Active state highlights correctly
• Clicking nav triggers state machine events
• Responsive (bottom bar mobile, sidebar desktop)

---

Phase 3: Node Creation Flow (Est: 4-5 hours)

Tasks:

1. Update AI system prompt to suggest nodes
2. Create components/Conversation/NodeSuggestionCard.tsx
   • Shows AI-suggested node title/content
   • "Save to Edit" and "Dismiss" buttons
3. Update conversation-machine.ts to handle:
   • SUGGEST_NODE event from AI response
   • CREATE_NODE event from user action
4. Implement node suggestion detection in AI response
5. Wire up "Save to Edit" → transitions to Edit mode

Acceptance Criteria:

• AI can suggest creating a node during conversation
• Suggestion appears as card in chat
• Clicking "Save to Edit" transitions to edit mode with draft
• Draft includes conversation context

---

Phase 4: Node Editor (Est: 3-4 hours)

Tasks:

1. Create components/Edit/NodeEditor.tsx
   • Title input (Mantine TextInput)
   • Content editor (Mantine RichTextEditor or Textarea with markdown preview)
   • "Publish" and "Cancel" buttons
   • "Continue Conversation" button
2. Create hooks/useNodeEditor.ts (Mantine form)
3. Implement publish flow:
   • Generate embedding (gemini-embedding-001)
   • Write to ATproto PDS
   • Cache in SurrealDB
4. Create components/Edit/LinkSuggestions.tsx
   • Vector search for similar nodes
   • User can approve/reject links

Acceptance Criteria:

• Can edit node title and content
• Markdown preview works
• "Publish" writes to ATproto + SurrealDB
• "Cancel" discards changes, returns to convo
• "Continue Conversation" saves draft, returns to convo
• Link suggestions appear based on embeddings

---

Phase 5: Galaxy Visualization (Est: 5-6 hours)

Tasks:

1. Implement UMAP dimensionality reduction for nodes
2. Create components/Galaxy/GalaxyView.tsx
   • R3F Canvas with OrbitControls
   • Dark space background
   • Camera setup
3. Create components/Galaxy/NodeMesh.tsx
   • Sphere for each node
   • Size based on node importance
   • Color based on node age or category
   • On hover: show tooltip with title
   • On click: transition to Edit mode
4. Create components/Galaxy/ConnectionLines.tsx
   • Lines between linked nodes
   • Opacity based on link strength
5. Optimize rendering for 100+ nodes

Acceptance Criteria:

• Nodes render in 3D space
• Can orbit/zoom camera
• Clicking node opens it in Edit mode
• Links visible between related nodes
• Smooth performance with 100+ nodes
• Responsive (works on mobile)

---

Phase 6: Conversation Machine (Est: 2-3 hours)

Tasks:

1. Create lib/conversation-machine.ts
   • States: voice, text
   • Invokes voiceMachine in voice state
2. Create hooks/useConversationMode.ts
3. Refactor app/chat/page.tsx to use conversation machine
4. Add voice/text toggle button

Acceptance Criteria:

• Can toggle between voice and text modes
• Voice mode properly invokes existing voiceMachine
• State transitions are clean
• Toggle button shows current mode

---

6. Remaining Work Breakdown

By Feature Area

Feature	Tasks	Est. Hours	Priority
App State Machine	4	2-3	P0 (Foundation)
Navigation UI	4	2-3	P0 (Foundation)
Node Creation	5	4-5	P1 (Core Flow)
Node Editor	4	3-4	P1 (Core Flow)
Galaxy Viz	5	5-6	P1 (Core Flow)
Conversation Machine	4	2-3	P2 (Enhancement)
Testing	6	3-4	P0 (Ongoing)
ATproto Integration	3	2-3	P1 (Core Flow)
Vector Search	2	2-3	P1 (Core Flow)

Total Estimation

• Core Features: 18-22 hours
• Testing & Polish: 3-4 hours
• Total: ~21-26 hours of focused development

By Priority

• P0 (Must Have): App machine, Navigation, Testing infrastructure
• P1 (Core Value): Node creation, Editor, Galaxy, ATproto, Vector search
• P2 (Enhancement): Conversation machine (voice/text toggle)

---

7. Technical Considerations

State Persistence

• Use localStorage for app state persistence
• Restore state on page reload
• Clear state on logout

Performance

• Lazy load Galaxy view (code splitting)
• Virtualize node list in large graphs
• Debounce vector search queries
• Memoize UMAP calculations

Error Handling

• Graceful fallback if ATproto write fails
• Retry logic for network errors
• User-friendly error messages
• Rollback SurrealDB cache if PDS write fails

Accessibility

• Keyboard navigation for all UI
• ARIA labels for state machine controls
• Focus management on state transitions
• Screen reader announcements

---

8. Success Metrics

Must Pass Before Launch

• All magnitude tests pass
• Full user flow works: Convo → Node suggestion → Edit → Publish → Galaxy → View node
• No TypeScript errors
• Mobile and desktop layouts work
• Data writes to ATproto PDS successfully
• Vector search returns relevant results

Quality Bars

• State transitions are instant (< 100ms)
• Galaxy renders smoothly (60fps with 100 nodes)
• Voice mode integration doesn't break
• No console errors or warnings

---

9. Next Steps

Immediate (Start Here)

1. Review this plan with user - Confirm priorities and scope
2. Create app-machine.ts - Foundation for everything
3. Build navigation UI - Visual feedback for state changes
4. Implement node suggestion detection - Start extracting value from conversations

Short Term (This Week)

• Complete Phase 1 & 2 (App machine + Navigation)
• Begin Phase 3 (Node creation flow)
• Write magnitude tests for new flows

Medium Term (Next Week)

• Complete Phase 4 (Editor)
• Complete Phase 5 (Galaxy)
• Integration testing

---

10. Risk Assessment

Low Risk ✅

• App state machine (similar to voice machine)
• Navigation UI (standard Mantine components)
• Node editor (forms and RTE)

Medium Risk ⚠️

• ATproto publishing (OAuth flow works, but write API untested)
• Vector embeddings (API calls should work, but scale unknown)
• UMAP dimensionality reduction (library integration)

High Risk 🔴

• Galaxy performance on mobile (R3F can be heavy)
• Node suggestion detection from AI (prompt engineering needed)
• Link suggestion accuracy (depends on embedding quality)

Mitigation Strategies

• Galaxy: Start with simple spheres, add detail later. Implement LOD.
• Node Detection: Use structured output from Gemini if freeform fails
• Links: Allow manual link creation as fallback

---

Appendix A: File Tree (Post-Implementation)

app/
├── layout.tsx (AppShell + AppStateMachine provider)
├── page.tsx (redirect to /chat)
├── chat/page.tsx (Conversation view)
├── edit/page.tsx (Node editor view)
└── galaxy/page.tsx (3D visualization view)

components/
├── AppStateMachine.tsx
├── Navigation/
│   ├── MobileBottomBar.tsx
│   └── DesktopSidebar.tsx
├── Conversation/
│   ├── ConversationView.tsx
│   ├── VoiceControls.tsx
│   ├── TextInput.tsx
│   └── NodeSuggestionCard.tsx
├── Edit/
│   ├── NodeEditor.tsx
│   └── LinkSuggestions.tsx
└── Galaxy/
    ├── GalaxyView.tsx
    ├── NodeMesh.tsx
    └── ConnectionLines.tsx

lib/
├── app-machine.ts (NEW)
├── conversation-machine.ts (NEW)
└── voice-machine.ts (EXISTING ✅)

hooks/
├── useAppMachine.ts (NEW)
├── useConversationMode.ts (NEW)
└── useVoiceMode.ts (EXISTING ✅)

api/
├── nodes/route.ts (CRUD for nodes)
├── embeddings/route.ts (Generate embeddings)
└── links/route.ts (Vector search for suggestions)